# FluidVoice + Company OS

FluidVoice is the Company OS voice-input layer for AgentChat. It uses the
in-browser mic when available, falls back to server speech-to-text for browsers
without native recognition, and can also pair with macOS FreeFlow as an external
dictation app.

[FreeFlow](https://github.com/zachlatta/freeflow) is a free macOS dictation app (Wispr Flow alternative): hold-to-talk, context-aware cleanup, custom vocabulary, and OpenAI-compatible LLM providers.

Company OS integrates FreeFlow in two ways:

1. **FluidVoice in-console dictation** — mic button in the operator agent-chat composer (`WorkspaceRightRail`).
2. **macOS FreeFlow setup** — point FreeFlow’s LLM cleanup at your local Ollama stack and import Company OS vocabulary.

It also includes a **Voice-to-Verified-Task** workspace surface at `/workspace/voice`.
That path treats speech as reviewed intent: the operator sees the raw transcript, cleaned command,
selected action, and evidence note before anything is persisted. It is not a mutation bypass.
Voice can create a task, add a task note, start a reviewed run, or record approval intent, but it
cannot apply patches, promote changes, or approve mutations on its own. FreeFlow remains optional;
manual paste and the in-browser mic path are first-class fallbacks.

---

## FluidVoice in-console dictation (browser)

**Flow:** browser STT (Web Speech API) or local Moondream Parakeet Redux → optional Ollama cleanup (FreeFlow-style prompt) → append to composer.

| Step | Component |
|------|-----------|
| Mic UI | `web/company-console/app/components/console/WorkspaceRightRail.tsx` |
| Client hook | `web/company-console/app/lib/use-operator-dictation.ts` |
| API | `POST /api/operator-dictation` |
| Cleanup | `web/company-console/app/lib/operator-dictation-server.ts` → Ollama `/api/chat` |

Install `ffmpeg` and the optional Python dependency in the runtime's Python environment:

```bash
python3 -m pip install -r web/company-console/scripts/requirements-parakeet-redux.txt
```

The first local inference downloads about 178 MB of weights from Hugging Face.
Use `HSM_PARKEET_PYTHON=/path/to/python` to select another Python environment
or `HSM_PARKEET_DEVICE=cpu|mps|cuda` to select a device.

**Env (Next / `.env`):**

```bash
HSM_DICTATION_STT_PROVIDER=parakeet-redux

# Defaults: cleanup on, Ollama at OLLAMA_URL, model = HSM_AGENT_CHAT_MODEL
HSM_DICTATION_CLEANUP=1
HSM_DICTATION_CLEANUP_MODEL=<explicit-installed-local-model>
HSM_DICTATION_STT_MODEL=whisper-large-v3
HSM_DICTATION_GROQ_BASE_URL=https://api.groq.com/openai/v1
# Optional hosted fallback: set HSM_DICTATION_STT_PROVIDER=groq and GROQ_API_KEY
```

With Company OS up (`bash scripts/company-os-up.sh`), click **Dictate with FluidVoice** in agent-chat, speak, click again to stop. Cleaned text lands in the message box; send as usual.

---

## macOS FreeFlow + Company Ollama

FreeFlow has **no server** — audio/transcripts go to your configured STT + LLM providers only.

### 1. Install FreeFlow

Download from [FreeFlow releases](https://github.com/zachlatta/freeflow/releases/latest/download/FreeFlow.dmg).

### 2. LLM cleanup → local Ollama

In **FreeFlow Settings → LLM**:

| Field | Value |
|-------|--------|
| API URL | `http://127.0.0.1:11434/v1` |
| API key | `ollama` (any non-empty string) |
| Model | An explicitly installed local model for cleanup only |

Start Ollama and pull the chat model:

```bash
bash scripts/ollama-pull-chat-model.sh
```

Use the same stack as agent-chat: `config/qwen3-ollama.env`.

### 3. STT (FreeFlow or local Company OS)

FreeFlow itself may use its configured provider. Company Console's server audio path defaults to **Parakeet Redux**, a local ASR model run through Moondream Photon. It is transcription only; AgentChat remains responsible for conversation and task execution. The model supports 25 languages and is CC-BY-4.0. For details, see [the model card](https://huggingface.co/moondream/parakeet-redux).

The Company Console server needs `ffmpeg` and `moondream>=2.4.0`. Install the optional dependency with `python3 -m pip install -r web/company-console/scripts/requirements-parakeet-redux.txt`. The first audio request downloads about 178 MB of model weights into the Hugging Face cache. `HSM_PARKEET_DEVICE` may be `cpu`, `mps`, or `cuda`; leave it unset for Photon device selection. To use Groq instead, set `HSM_DICTATION_STT_PROVIDER=groq` and `GROQ_API_KEY`.

### 4. Import Company OS vocabulary

Export terms (personas, skills, HSM jargon) for FreeFlow **Custom vocabulary**:

```bash
bash scripts/company-os-freeflow-vocab.sh [company_id]
```

Or from a running console (with operator auth):

```bash
curl -sS "http://127.0.0.1:3050/api/operator-dictation/vocabulary?companyId=YOUR_COMPANY_ID&format=text"
```

Paste one term per line into **FreeFlow Settings → Custom vocabulary**.

### 5. Dictate into Company Console

Focus the agent-chat textarea in Chrome/Safari at `http://127.0.0.1:3050`, use FreeFlow’s global shortcut (default: hold **Fn**), or use the in-console mic.

FreeFlow reads nearby app context so task titles, persona names, and visible UI text improve spelling during cleanup.

---

## API reference

### `POST /api/operator-dictation`

**JSON** (browser STT):

```json
{
  "companyId": "…",
  "transcript": "hey um assign this to the cto",
  "persona": "operator",
  "taskTitle": "Ship harness smoke",
  "context": "optional surrounding text"
}
```

**Multipart** (server STT via Parakeet Redux by default, or Groq when configured):

- `companyId`, `audio` (webm), optional `persona`, `taskTitle`, `context`

**Response:**

```json
{
  "transcript": "Assign this to the CTO.",
  "rawTranscript": "hey um assign this to the cto",
  "cleaned": true,
  "sttProvider": "browser"
}
```

### `GET /api/operator-dictation/vocabulary`

Query: `companyId` (required), `format=text` (optional plain list).

---

## Privacy

- **FreeFlow (macOS):** no FreeFlow server; only your STT/LLM providers see audio/text.
- **In-console dictation:** browser-recognized transcripts stay in-browser. MediaRecorder audio is sent to the Company Console server, normalized locally with ffmpeg, transcribed by local Parakeet Redux by default (or Groq if configured), and removed from its temporary directory after processing. Cleanup calls local Ollama when enabled. The first Parakeet Redux use fetches model weights from Hugging Face.

---

## License note

Cleanup prompt adapted from [zachlatta/freeflow](https://github.com/zachlatta/freeflow) (MIT). Parakeet Redux weights are [CC-BY-4.0](https://huggingface.co/moondream/parakeet-redux). Company OS integration code is part of this repo.