# Workspace voice does not confirm the selected permit

Investigated October 21, 2026. Deployment has not been performed from this chat.

The reported browser connects to LiveKit, waits for a worker to publish the selected permit and state, then disconnects after 20 seconds. The exact message comes from `src/lib/voice/permit-confirmation.ts`. The microphone remains closed until both `workspace.permit_id` and `workspace.state_code` match. Receiving a join token or connecting to LiveKit does not establish that a voice worker joined the room.

The supplied server log shows workers registering with `mode=synchron`, `dispatch_name=[automatic]`, and an empty agent name. It does not show the dedicated Workspace worker. Missing Workspace activation after the server move is the leading hypothesis; current worker registration and session metadata must confirm it. An outdated worker, an incorrect dispatch name/project, or failed session loading can also prevent acknowledgement. The earlier Gemini `CONSUMER_SUSPENDED` failures are a separate issue.

## Existing implementation

Brain commit `89e367a` saves the selected permit in the session and publishes its confirmation from the worker. Commit `d9b4ef7` targets a separate named worker, default `hha-workspace-logistics-v1`, so older automatic workers cannot receive these sessions. Both are present in the locally available Brain `origin/main` history at `20f5f39`; that does not prove the active server process includes them. The local Brain checkout is older than that reference and was not switched or updated.

Workspace tests passed: 9 tests across `voice-permit-confirmation`, `voice-permit-token`, and `chat-permit-selection`. No application code, database migration, or production process was changed in this investigation.

## Downloaded worker log evidence

The subsequently supplied `logistics_agent.log` contains 6,836 lines (886,214 bytes). Its newest structured timestamp is October 21, 2026 at 06:56:34 UTC / 12:26:34 PM IST. This precedes the reported 12:36 PM browser screenshot, so this file does not capture that failing attempt.

- All 19 structured worker registrations have an empty `agent_name`. The final startup explicitly says `mode=synchron, dispatch_name=[automatic]`.
- No line contains `order-bot-workspace-` or `hha-workspace-logistics-v1`. The eight distinct room names in structured events are ordinary Synchron rooms. This establishes the contents of this file, not the absence of a separate worker or log elsewhere.
- All 68 structured `worker failed` events report port 8082 already in use. The latest is October 20 at 12:05:00 UTC; later starts use a random port and register successfully.
- Gemini `CONSUMER_SUSPENDED` exceptions run from October 20 at 13:11:41 through 14:11:33 UTC. A later ordinary Synchron session reaches speaking state. These exceptions cannot establish the current Workspace failure.
- This is a worker log. It does not include the current Workspace API request/response or a dedicated worker receiving the failing room.

The leading explanation remains missing or misconfigured named Workspace dispatch. To distinguish a missing worker, a name/project mismatch, and session-loading failure, correlate the same fresh attempt across the API token event, the dedicated worker, and its selected-permit acknowledgement.

Capture the following after reproducing once on the active Brain host:

```bash
cd /home/heavyhaulagent-brain/htdocs/brain.heavyhaulagent.com || exit 1
date -u
sudo systemctl list-units --all 'heavyhaul-brain*workspace*.service' --no-pager
ls -l logs/workspace_logistics_agent.log logs/trip_ai_chat_copilot.jsonl
grep -E 'mode=workspace|dispatch_name|registered worker|Received job request|No logistics sessions document' \
  logs/workspace_logistics_agent.log | tail -n 30
tail -n 80 logs/trip_ai_chat_copilot.jsonl
```

The structured Trip AI log defaults to the file above; if the host sets `TRIP_AI_LOG_FILE`, use that configured private path without printing environment contents. In the default log, `voice_token_completed` means the API issued a session, `voice_worker_session_loaded` shows a worker found it, and `voice_worker_session_lookup_failed` locates a session-loading failure. Only a matching participant acknowledgement establishes that the browser can open its microphone. Share sanitized operational records without join tokens, API keys, or customer transcripts.

## Check the active Brain host

### New session logs and connection-code comparison

The later `heavyhaul_detailed.log` (43 lines), `order_bot_debug.log` (42 lines), and `llm_interactions.log` (28 lines) show successful Gemini text responses and four Workspace `LiveKit session prepared` events. The later whole-trip attempts are recorded at 07:19:18, 07:20:11, and 07:20:45 on October 21; these log timestamps do not specify a timezone. The first prepared session is permit-scoped, while the later sessions have no permit suffix. This confirms that both kinds of Workspace voice request reach Brain's Mongo session creation. It does not by itself prove that the subsequent JWT signing/response or worker dispatch succeeded. These files contain no worker room-join or selected-permit acknowledgement.

The connection code explains the working-test-bot / failing-Workspace split:

| Stage | Brain test bot | Workspace |
|---|---|---|
| Token endpoint | `/api/livekit/token` | `/api/workspace/logistics-voice/token` |
| Room prefix | `order-bot-` | `order-bot-workspace-` |
| LiveKit project credentials | `LIVEKIT_*_ORDER` | Same `LIVEKIT_*_ORDER` variables |
| Dispatch | Automatic worker, empty agent name | JWT `RoomConfiguration.agents` targets `workspace_voice_agent_name()` |
| Worker process mode | Defaults to `synchron` | Must be started explicitly as `workspace` |
| Selected-permit microphone | Not part of test-bot protocol | Opens only after exact worker permit/state acknowledgement |

Workspace chat and voice both use the same `callFlask` base URL and authentication configuration in `src/lib/adapters/flask.ts`. The voice adapter sends the Workspace trip, user, language, and selected permit to Brain. The stored session and successful chat indicate that changing the Workspace base URL or API credential is not supported by the supplied evidence. The leading remaining hypothesis is a missing or mismatched named Workspace worker; an API token response and a matching worker registration are needed for confirmation.

Four isolated routing-contract checks passed against the existing Brain `origin/main` dispatch module: default Synchron behavior, named Workspace room filtering, custom agent-name selection, and rejection of invalid mode/name configuration. These checks do not start a worker or contact production.

### Environment settings

No `.env` file was changed. No Workspace `.env` change is required by the inspected connection code or new logs.

| Setting | Where to set it | Required action |
|---|---|---|
| `LOGISTICS_AGENT_MODE=workspace` | Dedicated Workspace worker command or systemd unit only | Required to register the named Workspace worker. Do not put this in the shared Brain `.env`, which is also read by the working Synchron worker. |
| `LOGISTICS_AGENT_PORT=0` | Dedicated Workspace worker command or systemd unit | Use an OS-assigned loopback health port on this host. The code default 8083 conflicts with an existing listener; the interview worker also defaults to 8083. LiveKit dispatch does not require a fixed health port. |
| `WORKSPACE_LOGISTICS_AGENT_NAME` | Both API and Workspace-worker environments if customized | Optional: leave unset to use `hha-workspace-logistics-v1`. If set, the exact value must match both processes; an empty value is invalid. |
| `LIVEKIT_URL_ORDER`, `LIVEKIT_API_KEY_ORDER`, `LIVEKIT_API_SECRET_ORDER` | Existing Brain configuration | Retain the working project's values; the API and dedicated worker must load the same project. |
| `HHA_API_BASE_URL`, `HHA_API_CLIENT_SECRET`, `HHA_API_CLIENT_ID` | Workspace configuration | Retain the working chat configuration. |

The local Brain root `.env` has the three order-specific LiveKit variables configured and leaves the mode, port, and custom dispatch name unset. The local `logistics_bot/.env` supplies no alternative order-specific LiveKit values. This is a local comparison, not verification of the active server environment. An unset worker mode defaults to Synchron; it does not launch a second Workspace worker automatically.

Run these commands in the existing server shell:

```bash
cd /home/heavyhaulagent-brain/htdocs/brain.heavyhaulagent.com || exit 1
git rev-parse --short HEAD
test -f logistics_bot/voice_dispatch.py && echo 'Named dispatch module present'
grep -n 'workspace.permit_id' logistics_bot/livekit_agent.py
grep -n 'RoomAgentDispatch' routes/workspace_updates.py
sudo systemctl list-units --all 'heavyhaul-brain*workspace*.service' --no-pager
pgrep -af 'workspace_logistics_agent.log' || true
ss -lntp | grep ':8083' || true
```

If the source checks fail, deploy the existing named-dispatch/permit-selection implementation into this active server checkout before starting a worker. Preserve server changes; a broad pull or reset is not part of these instructions.

If a dedicated Workspace service already exists, inspect/restart that service rather than starting another copy. The old automatic Synchron service is a separate process and its restart does not activate Workspace mode.

## Start the missing Workspace worker once

Use this only when the source checks pass, no dedicated worker is already running. The OS chooses a free health-listener port; no fixed port must be freed. This command uses the correct CloudPanel path and the existing agent Python environment:

```bash
cd /home/heavyhaulagent-brain/htdocs/brain.heavyhaulagent.com || exit 1
mkdir -p logs
nohup env LOGISTICS_AGENT_MODE=workspace LOGISTICS_AGENT_PORT=0 \
  ./venv_agent/bin/python -u scripts/run_with_rotating_log.py \
  --log-file logs/workspace_logistics_agent.log -- \
  ./venv_agent/bin/python -u logistics_bot/livekit_agent.py start \
  > logs/workspace_logistics_agent_bootstrap.log 2>&1 &
```

Check the new log for `mode=workspace`, a dispatch name, and `registered worker` with the same nonempty `agent_name`. The default name is `hha-workspace-logistics-v1`; if `WORKSPACE_LOGISTICS_AGENT_NAME` is configured, the API and worker must use the same value and LiveKit project.

```bash
grep -E 'mode=workspace|dispatch_name|registered worker' logs/workspace_logistics_agent.log | tail -n 10
```

Once registration succeeds, restart only the main API if its token builder needs to reload the deployed code:

```bash
sudo systemctl restart heavyhaul-brain-main-api.service
```

The one-off `nohup` launch is not a persistent systemd installation. After successful verification, configure a dedicated Workspace unit using the host's existing service conventions, ownership, restart policy, and credential handling.

## Confirmed startup failure and corrected port

The operator's first Workspace launch exited before registration with `OSError: [Errno 98]` binding `127.0.0.1:8083`. The supplied listener output shows PID 198013 already owning port 8083 on IPv4 and IPv6. Its process identity has not been verified, but `ai_interview/agent_worker.py` also defaults to 8083. The subsequent LiveKit shutdown `AssertionError` follows the failed initialization; the bind error is the first failure to address.

The earlier command recommended 8083 without accounting for this host's existing listener. It is corrected above to `LOGISTICS_AGENT_PORT=0`, which passes zero into the worker listener so the OS allocates a free port. This listener serves worker health; LiveKit job routing uses the named outbound registration. Do not terminate the process occupying 8083 merely to free the port. No `.env` edit is needed. The corrected launch has not yet been verified on the server.

## Verify the failing flow

Refresh Workspace, select a permit, and reconnect once. In Network inspect only the voice-token response's `agent_name` and `permit_scope`; do not share the token. The named worker must receive the resulting `order-bot-workspace-...` room and publish matching permit/state attributes. Success means the selected-permit greeting is audible and a question about that permit receives the correct response. Registration or passing unit tests alone does not establish success.

If it still fails, capture the dedicated Workspace worker log for that attempt, rather than only `logs/logistics_agent.log`. Redact credentials and customer details before sharing. Do not remove the browser confirmation guard to make the connection appear successful.

## Operator verification: named worker activated

On October 21, 2026 at 07:42:28 UTC / 13:12:28 IST, the operator supplied logs showing registration as `hha-workspace-logistics-v1` on loopback port 8084. The worker received and accepted a Workspace-prefixed room, connected to it, initialized MongoDB, loaded the Workspace trip snapshot, initialized the agent, and transitioned from listening to speaking at 07:42:31 UTC. This confirms production worker registration, named dispatch, room acceptance, trip loading, and entry into speaking state. The reconnect warnings were immediately followed by successful Mongo initialization and snapshot loading.

The working launch uses per-process `LOGISTICS_AGENT_MODE=workspace LOGISTICS_AGENT_PORT=8084`; no `.env` edit was reported. The earlier port-zero command remains an alternative when a fixed health port is unnecessary. Browser audio playback, selected-permit acknowledgement on a scoped session, a spoken question/answer round trip, and persistence across service/server restarts remain unverified. The one-off worker has not been converted into a managed service by this chat. No application code or migration was changed.
