Gemini Transcribe or Live: Choose the Correct Endpoint

Choose by the state of the audio: send a completed recording to Gemini 3.5 Transcribe, but use Gemini 3.5 Transcribe Live when text must appear while audio is still arriving. The first path processes a file through the Interactions API; the second keeps a Live API connection open for incremental transcription.

Then check the final artifact your application must retain. Speaker labels and word-level timings require non-streaming Transcribe, while Live provides interim and finalized text without those annotations. A product that needs immediate captions and an enriched archive therefore needs a recorded source and a second, post-session request.

Choose the endpoint before designing storage

For prerecorded input, Google’s Transcribe documentation instructs developers to upload an audio file and pass its URI and MIME type to gemini-3.5-transcribe. The model supports automatic language detection, language hints, custom vocabulary, speaker diarization, word-level timestamps, and verbatim or smart output. The same documentation sets several important boundaries: custom vocabulary cannot be combined with diarization or word timestamps; smart mode is also incompatible with those annotations; diarization supports up to eight speakers, with attribution for three or more marked experimental; and enabling word timestamps may reduce transcription accuracy.

For a microphone, call, broadcast, or another unfinished source, connect to gemini-3.5-transcribe-live. Uploading a succession of tiny files is not equivalent: it lacks Live’s interim and finalized event types and introduces artificial boundaries that the application would have to reconcile.

Audio is complete: choose Transcribe.
Text is needed before the audio ends: choose Transcribe Live.
Live captions plus speaker or word annotations: stream for immediacy, retain the recording, and transcribe that file afterward.

Minimal flow for a prerecorded file

Gemini 3.5 Transcribe processes a completed recording into a transcript with speaker labels and word-level timing data.

Upload the completed recording through the Files API and retain the returned URI and MIME type.
Create an interaction with gemini-3.5-transcribe and supply the uploaded audio as input.
Provide supported BCP-47 language codes when the languages are known, or omit the list for automatic detection.
Choose verbatim mode for a close spoken record and for diarization or word timestamps. Choose smart mode for cleaned, structured prose.
Read the complete transcript from the interaction output. If annotations are enabled, extract the word information attached to the returned content.

Decide which representation is authoritative before making the request. Smart mode removes disfluencies, resolves spoken corrections, and restructures content for readability, so it should not stand in for an exact spoken record. If an archive needs both readable copy and an auditable timed transcript, use separate requests or produce the readable derivative in a controlled post-processing stage.

Vocabulary creates another branch in the request plan. A specialized term list can bias recognition, but a file request using it cannot simultaneously request speaker diarization or word timestamps. Products needing both terminology support and annotations must prioritize one configuration, run separate requests, or redesign the downstream reconciliation step.

Minimal flow for a live stream

Gemini 3.5 Transcribe Live revises an interim caption while preserving an earlier finalized utterance.

Open a Live API connection with gemini-3.5-transcribe-live and request text responses.
Configure language detection or hints, custom vocabulary, transcription mode, and voice-activity behavior when creating the session.
Convert incoming audio to raw 16-bit PCM and send successive chunks over the active connection.
Render interim events as replaceable interface state. Append only finalized events to durable transcript history.
Signal the end of the audio stream and close the connection cleanly.

The Live transcription guide specifies mono, little-endian raw 16-bit PCM at 16 kHz and 100-millisecond chunks. It distinguishes speculative interim hypotheses from finalized utterances, limits continuous sessions to 10 minutes, permits up to 1,000 vocabulary terms while noting that results are typically best with up to 100, and confirms that Live offers utterance-level timing rather than speaker diarization or word-level timestamps.

Those event semantics should be reflected in storage. The caption surface can replace the current interim line, while the durable writer appends finalized segments only. If a source may outlast the session limit, the application needs an explicit connection-rotation policy and its own sequence numbers or source-audio offsets; do not assume separate sessions will form one continuous transcript automatically.

Capability matrix: combinations that change the pipeline

Prerecorded file: Transcribe through the Interactions API.
Continuous audio: Transcribe Live through the Live API.
Interim and finalized events: Live only.
Speaker diarization: non-streaming verbatim mode; unavailable in Live.
Word-level timestamps: non-streaming verbatim mode; unavailable in Live.
Diarization plus word timestamps: supported together in one non-streaming verbatim request.
Custom vocabulary plus either annotation: rejected on the non-streaming path.
Smart output plus either annotation: incompatible.
Custom vocabulary and smart output: available on both paths, subject to each endpoint’s other restrictions.

The endpoint and output mode answer different questions. Transcribe versus Live determines whether the input is a completed file or an active stream; verbatim versus smart determines whether the output preserves speech or favors readability. Annotation and vocabulary requirements can then rule out combinations that otherwise appear valid.

Validate the request contract before ingestion

Convert those combinations into configuration checks. Reject smart output with diarization or word timestamps. Reject custom vocabulary with either annotation on file jobs. Reject a Live configuration if the product promises speaker-attributed or word-timed output unless recording and post-session enrichment are enabled.

Confirm that completed-file jobs use the Interactions API and ongoing streams use the Live API.
Validate uploaded MIME types separately from Live PCM format, channel, and sample-rate requirements.
Normalize vocabulary lists and validate every supplied language hint against the supported codes.
Keep interim text outside committed transcript history.
For two-stage transcripts, preserve the source recording and a stable identifier that links provisional and enriched outputs.

The Gemini 3.5 Audio model card lists audio and text input context windows of up to 96K tokens and text output of up to 32K tokens for both transcription models. It also warns that they may exhibit foundation-model failures such as hallucinations, as well as occasional slowness or timeouts. Token ceilings belong in request validation, and downstream systems should not treat generated transcripts as infallible records.

Make failures stop at recoverable boundaries

For file jobs, persist the upload identifier, interaction identifier, requested configuration, and job state. Retry transient failures with backoff, but enforce idempotency in the application queue so a client-side timeout does not produce duplicate transcript records. Send permanent format or configuration failures for correction instead of resubmitting them indefinitely.

For Live, store finalized segments with increasing sequence numbers and retain the last committed source offset. After a disconnection, discard or explicitly mark the unresolved interim hypothesis. If buffered audio can be replayed, restart from a controlled boundary and reconcile duplicates; if it cannot, preserve a visible gap rather than joining text across the interruption.

The resulting design is straightforward: Transcribe owns completed recordings and enriched annotations; Transcribe Live owns low-latency text from active audio. When a product requires both outcomes, treat the live transcript as provisional and the post-session transcript as a separate authoritative artifact.

Gemini Transcribe or Live? Choose by File vs Stream, Then Check Limits