Skip to content

[Bug]: Antigravity ACP cannot natively read local audio/video files from disk paths (needed more than ever with Gemini 4 Neon) #39234

Open

Description

@AlonResearch

Before submitting

  • I searched existing issues and did not find an open issue covering this.
  • I included enough detail to reproduce or investigate the problem.

Why this is not a duplicate of #25292 or #38504

Earlier reports describing this exact capability loss (#24969 and #36955) were closed as duplicates of PR #25292. However, PR #25292 fixed a different problem: it only made client_view_file return an error on binary files instead of decoding raw bytes as garbled UTF-8 text. It did not restore or enable native multimodal reading of local audio or video files from disk.

The only related open issue (#38504) also tracks only the remaining view_file binary-text bug on MP4 files, not delivering local audio or video files natively to Gemini models.

Area

apps/server

Problem description

When a user gives an Antigravity thread a file path on disk to a local audio or video file (such as .mp3, .wav, or .mp4), the Gemini model cannot actually listen to the audio or watch the video:

Why this matters more than ever with Gemini 4 Neon

With Gemini 4 Neon now available in T3 Code's Antigravity provider, native audio and video understanding is one of the strongest reasons to pick Gemini models over text/image-only coding models. Real workflows frequently involve audio or video files that already live on disk (for example long lecture recordings, meeting audio, screen recordings, or large media folders that should not be copied into a git repository or manually re-uploaded through the composer). Having to fall back to external Gemini API scripts, Whisper-only text, or FFmpeg frame extraction defeats the main advantage of running Gemini 4 Neon in T3 Code.

Steps to reproduce

  1. Open T3 Code with the antigravity provider and gemini-4-neon-medium (or another multimodal Gemini model).
  2. Point the agent to a local .mp3, .wav, or .mp4 file by file path and ask it to listen to the audio or inspect the video natively (without external ASR or frame-extraction scripts).
  3. Observe the tools available to the agent (view_file, client_view_file, run_command).

Expected behavior

Antigravity ACP exposes a native media-reading path (or handles audio/video in its file-reading tool) so Gemini models such as Gemini 4 Neon can natively ingest local audio and video files referenced by path, matching the standalone agy CLI capability. Native MP4 handling should also work on Windows, not only macOS and Linux.

Actual behavior

The agent has no tool that passes a local audio or video file on disk into Gemini as native multimodal input, so it cannot listen to audio or watch video referenced by path.

Impact

Major degradation or frequent failure

Version or commit

T3 Code (Beta) 0.1.10

Environment

Windows 11 (10.0.26200.0), T3 Code Desktop 0.1.10, provider antigravity, model gemini-4-neon-medium.

Workaround

Manually splitting/uploading audio files as composer attachments (up to 50 MiB per file), or calling the Gemini API directly via an external Python script with a separate API key. Neither works seamlessly for local media workflows or Windows native MP4 video.

Activity

  1. kuelshammer commented on Nov 21, 2026

    @kuelshammer

    Adding a data point for images — the title says audio/video, but the same gap covers reading image files from the workspace.

    Setup: T3 Code server from 2026-10-16 (pre-#25292), Antigravity managed runtime 1.3.0, oauth-personal. Workflow: ask the agent to look at a PNG/JPG inside the project ("open diagrams/foo.png and tell me what the diagram shows"). The agent calls its file-read tool, T3 answers fs/read_text_file with the raw bytes decoded as text, and Gemini reports it perceives nothing. On a post-#25292 build the same read should at least fail with "File 'x' is not a UTF-8 text file." instead of returning garbled bytes — better, but the workflow is still impossible; the image never reaches the model either way.

    Two observations that might help scoping:

    • Prompt attachments work: in the [Bug]: Antigravity reads images, PDFs, audio and video as UTF-8 text (view_file / client_view_file), so Gemini models never see media files #24969 testing, a blind prompt-attachment test passed with a synthetic PNG carrying pixel-encoded secrets. So the gap is specifically the file-path read path, not image transport in general.
    • ACP v1 only defines fs/read_text_file and fs/write_text_file (packages/effect-acp/src/_generated/schema-v1.gen.ts), and the Antigravity CLI maps view_file / client_view_file onto those. As long as that holds, no client-side change in T3 can deliver the bytes as media — this needs a runtime-side (Google) media-read capability, plus T3-side plumbing to advertise and handle it.

    Until then the workarounds are composer attachments or falling back to the Antigravity CLI directly, which handles image paths natively.

    Comment written with AI assistance, posted by the account owner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions