Pack · #365 trending skills repo · ↗ +1 stars this week
Token Goat
DFKHelper/token-goat
Auto AI improver. Cost and token burn reducer, turn saver, and focus keeper for Claude Code, Codex, Copilot, Gemini, and more: surgical read hints, PDF/Office/CSV/md file interception, 255+ filter & interception rules, compact manifest injection, image shrinking, cache and compact skills, cache MCPs, prompt injection protection, and much more.
- Stars
- 146
- This week
- +1
- Forks
- 20
- Open issues
- 0
- Latest release
v2.9.34
Get started
Install the skills
npx skills add DFKHelper/token-goatInstalls into Claude Code, Codex, Cursor, OpenCode and the other agents it finds on your machine.
git clone --depth 1 https://github.com/DFKHelper/token-goat.git /tmp/token-goat
cp -r /tmp/token-goat ~/.claude/skills/token-goatClaude Code reads ~/.claude/skills/ (or .claude/skills/ in a project). Other agents read the same SKILL.md format from their own skills folder.
Documentation
README
Token-Goat
Give the model what it needs, not everything you have.
85% smaller reads · 49% of first-read bytes withheld · 97.4% image compression · leaner prompt cache · 205+ filter & interception rules · 94–99% skill overhead cut · compaction memory · prompt injection guard · 3.7 GB never reached the model · 1.1 Gt tokens saved
Saves AI turns, context and tokens, and improves its focus. Fully automated, always online.
Also defends against prompt injection. Every fetched page, tool result, and extracted document is wrapped in an untrusted-content fence before hitting the model, whether or not it matched an attack pattern, and the scan only decides what the label says. One config line to disable.
Your AI re-reads the same file three times. Every whole-file dump is paid for again on every turn after it. Every compaction causes amnesia. Every build log buries the one line that matters. You pay for all of it. Token-Goat fixes all of it — automatically.
Token-Goat sits silently between your AI and your tools. Re-read a file? It gets a one-line hint and a narrow-slice suggestion instead of the full file again. Grab a screenshot? A 100 KB copy reaches the model instead of 10 MB. Run pytest, npm install, docker build, or cargo? The thousands of progress bars and passing-test names are stripped to the failures before the output even reaches the context window. Open a PDF, a large Markdown doc, or a CSV? The hook intercepts it — heading tree, page count, or column preview — so the model never pays for the full file. Run gh run watch or next dev a second time? Prior output is recalled rather than re-run. Compact a long session? It gets a clean structured manifest of edited files and key symbols so nothing important is forgotten. Sessions drop 40–90%+ in cost. You change nothing about how you work.
Each of these also costs a turn. A model that pages a file ten lines at a time, or runs the same command again to see output it already has, spends round trips as well as tokens. Token-Goat aims to answer the next question as well as the current one. These features save turns today:
token-goat brief "file::symbol"returns the symbol body, its callers and its containing doc section in one call.token-goat answer "<question>"answers a plain-English question from the index alone, or refuses with a reason.token-goat read "file::symbol"returns one function in place of paging through the file.token-goat bash-output <id>andtoken-goat web-output <id>recall captured output instead of running the command or fetch again.- A repeat read of a file the session already has gets a "You've already read this file" hint in place of the whole file.
- Before compaction, a session manifest is injected for the summary to use.
The symbol index also understands native Nginx, Caddyfile, and Apache HTTP Server configuration blocks, so token-goat read "file::symbol" can jump directly to them.
Works with Claude Code, Gemini CLI, Qwen Code, Codex CLI, Aider, Cursor, Cline, Windsurf, Copilot CLI, VS Code (Copilot agent), Visual Studio (Copilot agent, MCP tools and instructions only), Zed (MCP tools only), Kimi Code, Grok CLI (xAI Grok Build), Antigravity CLI (agy), and OpenCode, plus pi (pi-coding-agent).
Ask your AI to install it fully (give it this GitHub link), or install in one command:
npm install -g token-goat && token-goat install
Restart your AI sessions. Run token-goat stats a couple of minutes after your next session to see the massive savings. It also doubles as a great tracker of your work. Welcome to token efficiency.
Built and continually improved, free, by one person. If it saves you tokens, drop a ⭐️ at the top of this page. One click. Makes my day. Also, if you'd like anything added, drop me a line.
Install · CLI · Architecture Q&A · What gets installed? · Stats · Security & uninstall
Same requirements, smarter input: fewer input tokens, shorter answers, and context that stops compounding across rounds
Stats display — gradient bars, sparklines, and a calendar heatmap in 24-bit color
The problem
AIs read auth.py. Then reads it again. And again. Then a third time after compaction wipes the session. Then it can't find what it wanted and searches other lines and files. You pay for every token and most of it is waste.
Long sessions accumulate waste six ways. Screenshots cross the model at full resolution. A single PNG can land at 10+ MB. The agent re-reads files it already parsed earlier in the same conversation. When a session compacts, the summary LLM doesn't know which files were edited or which symbols mattered, so it preserves the wrong things. Every pytest, npm install, docker build, or git log dumps thousands of lines of progress bars, deprecation warnings, and passing-test names that bury the one line that actually matters.
The fifth waste is skills. A single large skill injects 10k–65k tokens every time. Run a five-iteration /improve loop and you've paid for five full copies of the same rules. Token-Goat now blocks repeat skill loads before they happen: a PreToolUse hook intercepts the second invocation, serves the cached compact (~400 tokens) instead, and only allows a reload when compaction may have evicted the skill from context. It also intercepts direct reads of skill files and ensures the compaction manifest carries the full skill index — so nothing is forgotten and the full body never re-enters context unnecessarily.
The sixth waste is what a large read costs after the turn it lands in. The prompt cache makes it cheaper, not free: Anthropic bills the first cache write at 1.25x the base input price (2x for the one-hour cache), then about 0.1x for every later turn that reads it back, until the session compacts. A 2,000-line file read to check a three-line helper is carried, and paid for, on every one of those turns. It also brings compaction closer, and the turn after a compaction writes the new prefix at the write price. Token-Goat's surgical reads add only the requested symbol or slice, so every later turn carries less.
The fastest way to reduce AI token costs is fixing these six, not writing shorter prompts. Each one is preventable. Token-Goat intercepts all six, automatically.
What changes
| Without Token-Goat | With Token-Goat |
|---|---|
| A full-file read is written to the prompt cache once, then read back on every later turn until compaction | Surgical reads add only the lines asked for, so every later turn reads back less and compaction comes later |
| 3.3 MB screenshot lands in model context | 84 KB compressed copy, 97.4% smaller |
| Agent re-reads files from earlier in the session | "Already read this" reminder with narrow slice suggestion |
| Read tool asks for lines the session was already given | Answered with a pointer at the copy already delivered instead of the file, when the text this read would return matches the text already served for it whole line for whole line. Proof rather than a read count, so it also covers a file inside the recent-read protection window; a changed file, a wider range, and reread_deny = false all pass through |
Same unchanged file read again through the shell (cat, head, tail, sed -n '1,40p') |
Byte-identical repeat replaced with a one-line pointer instead of the whole body: 4,031 bytes to 189 on this project. A later read wholly inside an earlier one collapses the same way |
A shell read of a file overlaps an earlier one without sitting inside it (sed -n '100,140p' after sed -n '120,160p') |
Only the already-shown stretch is withheld, replaced in place by one line naming its line numbers and the command that returns it; every line not shown before arrives untouched. 56% of the second read on a real pair over this project's README. Line numbers come from the command's own ranges, so a tail, whose start the command does not state, is left alone. Full text stays available via token-goat bash-output <id> |
Part of a file re-read under a different command (head -40 F then sed -n '1,30p' F) |
Repeat replaced with a pointer when every line coming back was already served for that same file this session. Matched on the text, on whole-line boundaries, so a read that adds any new lines is left whole; the record is dropped when the file is edited or the conversation is compacted. A file read with the Read tool counts as served too, limited to the lines that read actually handed over, and a read that came back truncated counts for nothing |
| Read tool asks for a range that overlaps one already delivered | Only the overlapping stretch is withheld, replaced by one line naming the line numbers and the recall command; the new lines come through with their own numbers and spacing, byte for byte. A stretch is kept whenever dropping it would save less than the line replacing it costs, and a result carrying anything the redactor would strip, a truncated result, or a file changed on disk all pass through whole. Set elide_served_lines = false to disable |
Shell command run with colour on (git, npm, npx, token-goat) |
Terminal escape sequences removed, leaving byte-for-byte the same output with colour off: 13.2% smaller on a real git diff here, and about a quarter of the bytes on the calls it fires for. Nothing is summarized or withheld, so there is no recall pointer; it runs on failed commands too, and only when the escapes are worth a rewrite |
| Agent re-reads a file edited mid-session | Unified diff injected as a hint — full Read avoided when the diff covers the change. Docs and source/style/data files (.md/.ts/.css/.json/…) by default; set serve_diff_on_reread = false to disable source diffs |
| Compaction forgets which files were edited | Structured session manifest injected before compact |
Same files re-read from scratch after /compact |
Recovery hint at SessionStart lists cached snapshot + bash + WebFetch IDs |
| Loaded skill body summarized away by compaction | ### Active Skills manifest section + **Skills**: recovery block list every loaded skill; full body recoverable via token-goat skill-body <name> without re-invoking |
| Large skill bodies re-injected each turn (6 active skills = 65k+ tokens) | <!-- COMPACT_END --> marker: everything above the marker is the compact form; token-goat detects it on load, caches the compact slice, and injects only that — typically ~400 tokens vs. 10k+ |
| Model reads a skill SKILL.md file directly mid-session (burning the full 10k–65k tokens again) | Pre-Read hook intercepts */.claude/skills/<name>/SKILL.md paths; if the skill is already cached this session it emits a token-goat skill-body <name> hint instead |
| Same large skill invoked twice in a session | PreToolUse hook blocks the reload; serves cached compact (~400 tokens) via additionalContext instead of the full 40–65k body. Allows the reload if compaction fired since the last load |
Skill invoked with first_load_compact=true and <!-- COMPACT_END --> present |
First load also blocked; only the curated compact section is served. Full body available via token-goat skill-body <name> on demand |
| Same docs URL fetched twice in a session with the same question | Re-fetch blocked at any context pressure; cached body available via token-goat web-output <id>. Keyed on the URL and the prompt together, since a WebFetch answer is specific to the question asked, so the same page fetched with a different question is left alone |
cat src/auth.py or Get-Content module.py run via Bash |
Pre-Bash hook detects whole-file reads of indexed source files and suggests token-goat read "file::Symbol", skeleton, or section — covers cat, bat, type, PowerShell Get-Content/gc |
rg pattern src/ or grep -rn run via Bash (first time) |
Pre-Bash hook suggests token-goat symbol <name> and token-goat semantic "<query>" as indexed alternatives to a full directory walk |
rg "^def" src/file.py or grep "class " module.ts — structural search on a single source file |
Pre-Bash hook redirects to token-goat skeleton "file" or outline "file" — all symbols with line numbers, no full-file read |
rg or grep run twice with the same pattern |
Pre-Bash dedup hint fires on repeated rg/grep/ag calls the same way it fires on the native Grep tool; repeat searches return a cached match-count hint instead of re-running |
Read tool targets tool-results/<id>.txt or tasks/<id>.output |
Pre-Read hook suggests token-goat bash-output <id> --tail N / --grep PATTERN / --section H; the filename stem is the output ID |
Repeated monitoring command run again (gh run watch, next dev, vitest, docker logs) |
Pre-bash recall hint: when a prior run is cached and its output exceeds 2 KB, a pointer to token-goat bash-output <id> --grep PATTERN is injected instead of re-running the command. Cache is keyed on the base command, so re-running with a different trailing pipe (e.g., ` |
pnpm/yarn/bun install or build dumps full output |
pnpm, yarn, and bun compress filters now strip install noise and build logs the same way npm does; pnpm run/yarn run route through their own filter |
| Surgical-read command returns a 10k-line symbol or a full section dump | Capped at ~25k tokens; marker names the truncation ratio and narrowing command (symbol → file::Class.method; section → sub-heading; cached → --grep/--tail) |
| Full file read for one function or section | token-goat read file::symbol, about 85% smaller |
pytest dumps 150 PASSED lines + dots + tracebacks |
Failures-first view, 80 to 97% smaller |
npm install floods deprecation warnings + spinner |
Errors kept; warnings collapsed by package, ~90% smaller |
docker build emits sha256 digests + transfer progress |
Step headers + errors kept; noise dropped, ~75% smaller |
ruff / eslint / mypy repeat the same rule 50 times |
Grouped by rule with first 3 examples, ~80% smaller |
Same pytest / cargo / git log re-run mid-session |
Small prior outputs (≤8 KB) served inline on first repeat; larger outputs get a hint pointing at token-goat bash-output <id> |
Same Grep pattern re-run with hundreds of matches |
Pre-Grep dedup hint quotes the prior match count |
Grep in content mode repeats the same file path on every match line |
Post-Grep hook folds matches under one path header per file (lossless, path/line survive verbatim) |
| Same docs URL fetched twice with the same prompt | Re-fetch denied (redirects to token-goat web-output <id>) once the cached body clears web_dedup_min_bytes, at every context pressure. A repeat with a different prompt passes through: the cached answer was written for the earlier question |
token-goat section pyproject.toml::tool.ruff |
One TOML table extracted instead of the whole config; same for .yaml/.yml/.json/.ini/.cfg/.env/Dockerfile |
Typoed token-goat symbol getUserr |
symbol matches on exact name; a miss returns No matches for 'getUserr' (no fuzzy/auto-redirect) — use token-goat find getUserr for a typo-tolerant name lookup, or token-goat semantic "<what it does>" when you don't know the name at all |
grep/rg returns 50+ match lines |
File-level summary: top 20 files by match count; full result cached, ~80% smaller |
| Same "already read" hint fires on every re-read | Suppressed after first injection; SHA-256 fingerprinting prevents the same nag twice per session |
| A file already read in another session is requested unchanged | With hints.cross_session_read_dedup = true, a project-scoped SHA-256 evidence record adds a compact advisory instead of replaying content. Changed files are named at startup for a fresh surgical read; cached bodies are never injected |
token-goat semantic finds no indexed source match |
It falls back to up to 100 recent, project-scoped, secret-redacted workspace-evidence entries. Vectors are created only for this explicit command and cached locally; results are labeled as cached evidence rather than live source |
| Same bash command runs 3+ times in one session | Escalating warning: "ran 2×" on repeat, "WARNING: ran N×" by the third; output always cached |
| Agent starts cold with no git context in a dirty repo | Branch, change counts, and 5 recent commits injected at startup (~50 tokens) |
| Re-read hint shows only the line range | Hint includes previously-accessed symbol names: [symbols: login, refresh, …] |
| Manifest too large or unstructured after compaction | Manifest gains ### MUST_PRESERVE sealed block, ### What Worked (last 2 green test runs), inline git diffs, and ### TODOs from TaskList |
| CSV/JSON/JSONL/log file re-read when only structure changed | Pre-Read hint for structured files (CSV headers, JSON keys, log format), ~70% smaller than full read |
| Index-only files (lockfiles, source maps, bundles) read on every session | Pre-Read suppression for read-only files (package-lock.json, *.map, dist/), skipped unless explicitly edited |
| Large markdown file read in full (README.md, CHANGELOG.md, CLAUDE.md ≥8 KB) | Heading tree intercepted instead — H1–H3 with #2/#3 disambiguation; token-goat section shortcuts listed for well-known files; post-edit injects a re-read suggestion rather than the full file |
| Subagent reads a ≥30 KB markdown file whole, on its first look at it | Off by default. Set subagent_markdown_first_read_deny = true under [hints] to block that read and answer with the heading tree instead. Only fires in a subagent, only on a first, un-ranged read of a .md/.mdx/.markdown file with at least three headings; a read that already asks for a line range, and every main-session read, pass through untouched |
Grep that looks up one declaration (def parse_args, function load) |
Off by default. Set inline_symbol_body = true under [hints] (or TOKEN_GOAT_INLINE_SYMBOL_BODY=1) to refuse the Grep and answer with that symbol's body, so the next turn is not spent on token-goat read. Acts only on a plain content search where the fresh index holds the name exactly once in scope; every other case gets the usual hint |
Claude Code spawns an agent without subagent_type while tools-restricted agent definitions exist |
Off by default. Set agent_scoped_spawn_deny = true under [hints] (or TOKEN_GOAT_AGENT_SCOPED_SPAWN_DENY=1) to refuse the first such spawn in a session and name the restricted agents, so the retry can pick one. An untyped spawn runs as general-purpose, whose context starts with every tool and MCP schema on the machine. Re-issuing the same call runs it: the refusal happens once per session. A spawn that names any subagent_type, general-purpose included, is never refused, and Copilot CLI and VS Code are left alone |
Grep for one declaration (def parse_args, class Config, function load) or one bare name (parse_args, \bparse_args\b), in one file, a directory or the working directory |
Off by default. Set inline_symbol_body = true under [hints] (or TOKEN_GOAT_INLINE_SYMBOL_BODY=1) to refuse that Grep and put the symbol's body in the refusal, so the next turn is not spent running token-goat read. A project's own .token-goat.toml cannot turn it on. It fires only when the pattern is nothing but the declaration word and one name, or just the name, the fresh index holds that name exactly once in the searched scope, and the refusal (every matching line plus the body) is under 12,000 bytes; anything else keeps the hint that names the command. A directory is answered only when token-goat can read every file Grep would search: git's list of tracked and not-ignored files, at most 200 files and 2,000,000 bytes, all indexed, none a link, binary or non-UTF-8, in a git work tree, with no RIPGREP_CONFIG_PATH. A bare name's refusal lists every matching line as path:line:text before the body. The Grep must be a plain content search: output_mode is content, with no -i, context, glob, type or head_limit. The pattern must also match exactly one line, and that line must sit inside the symbol. The body is the exact text token-goat read prints, fenced as untrusted file content and redacted. The same symbol is refused once per session; repeating the Grep gets the ordinary hint |
| An agent ends its turn after editing source files that no test, lint, typecheck or build run has covered since | A one-time note at the Stop event names the unverified files and the checks that would cover them. It is about edits inside the session's project root; scratch files outside the root and under the OS temp dir never trigger it (judged by where the path really leads, so a link or junction there into the project is still a project edit), while a sibling package, another repo or a sibling worktree still does. Besides test, lint, typecheck and build runners, a passing validator script (run_*tests*.py, validate*.py, check_*.py, or any script run with --self-test) counts as a check when it carries only path or selector arguments (a .md argument never counts, since the script may be producing that document): a help, listing, version or preview run (--help, --list, --dry-run, any other flag) does not. A git commit or git push never counts, whatever hook it ran. It is an advisory, never a block, because a block at Stop can trap a session in a loop. Claude Code and Codex CLI show it; Copilot CLI's and VS Code's stop hooks accept only a block, so it stays silent there. Turn it off with stop_verification_advisory = false under [hints] (or TOKEN_GOAT_STOP_VERIFICATION_ADVISORY=0) |
A subagent finishes with a failed last check, source edited after its last passing check, a report that claims a change it never made, or a report that claims a change when git status did not finish |
Up to three lines are appended to the report the parent receives, saying which of those happened. When git status does not finish in time, the line says so ("git status did not finish, so this report's change claim was not checked"); that line takes the place of the claimed-a-change-never-made line, so the limit of three still holds. A subagent with nothing to flag gets nothing added. On Copilot CLI the lines ride on the subagent report rewrite; the report part of that rewrite is the secret-redacted original, so a secret token-goat redacted stays redacted. Copilot's tool events carry no agent id, so there only the claimed-a-change-never-made check applies to a subagent (it reads the git tree), not the failed-check or unverified-edit flags. On Codex the lines ride on the result of the multi-agent v1 wait_agent tool, keyed on the subagent's thread id; Codex's v2 multi-agent tools return no report, so nothing is delivered there. VS Code Chat gets no receipt, because its SubagentStop payload carries no report text, and Visual Studio and JetBrains have no agent hooks. Turn it off with agent_receipt = false under [hints] (or TOKEN_GOAT_AGENT_RECEIPT=0) |
| Claude Code Bash command on Windows that contains two backslashes in a row | Run from a temp file when your permission rules already allow the command, otherwise refused before it runs, with the reason and what to do instead. Git Bash halves every run of two or more backslashes in the command it is handed, so an escaped path, a regular expression or a printf format string would run as different text with no error. A command that needs no prompt is written, as typed, to a file that deletes itself on its first line, and run from there; one that would prompt, or that uses $, a backtick, eval, rm or a protected path, is refused, because the prompt would show a file name and not the command. Then write the text to a file with the Write tool and run it by path, use the PowerShell tool, or use forward slashes in paths. A command over 7,000 characters with no such backslashes is left alone. Claude Code on Windows only; set deny_bash_double_backslash = false under [hints] (or TOKEN_GOAT_DENY_BASH_DOUBLE_BACKSLASH=0) to turn it off |
| A source file is read whole for the first time, with no earlier copy to compare it against | Keeps each function's declaration and its first eight lines, replacing the rest with one line naming the symbol, the exact range removed, and the command that returns it. Set fold_code_bodies = false under [hints] to turn it off. Comment blocks of twelve lines or more fold to their first two on the same terms. Imports, signatures, type declarations, classes and interfaces are never folded, and a file whose index no longer matches what is on disk keeps its bodies. Measured on this repository's own source: 229 of 256 files fold, and 43% of what a read of them delivers is removed. A read that already names an offset or a limit, and any file holding something that looks like a secret, pass through untouched |
| A source or markdown file of 50 KB or more is read whole for the first time, and token-goat has indexed its symbols or headings | The read goes through with a note naming the file's size, its symbol or heading count and the read, outline or skeleton command that returns just the part you need. Set first_read_symbol_policy = "deny" under [hints] (or TOKEN_GOAT_FIRST_READ_SYMBOL_POLICY=deny) to refuse the whole-file read instead, or "off" to say nothing. first_read_symbol_bytes sets the size (50000 by default, 0 turns the check off). Both can only be set in your global config or the environment: a repository's .token-goat.toml cannot set them |
A Write, Edit, MultiEdit or Codex apply_patch puts a placeholder such as // ... existing code ... or [rest of file unchanged] into a file, or a whole-file Write to a file that was only partly read deletes five or more lines the session never saw |
Refused before it runs, with the placeholder or the unseen line ranges named and a token-goat read command for each range. Repeating the identical call goes through. Codex's pre-tool hook can refuse an apply_patch, and VS Code's apply_patch and multi_replace_string_in_file are checked; VS Code's insert_edit_into_file edits are not covered. Set write_blind_drop = "warn" under [hints] (or TOKEN_GOAT_WRITE_BLIND_DROP=warn) to add a note instead of refusing, or "off" to say nothing. A note is silenced during quiet hours, a refusal is not |
A Write creates a new file named like a copy of an existing one in the same directory (solver_v2.py, solver_fixed.py, solver.py.bak) and repeats most of that file |
A note, once per original per session, naming how much of the original it repeats and suggesting Edit, token-goat replace or token-goat outline on the original. It needs the original to reach write_rewrite_min_lines (40) and write_rewrite_unchanged_pct (75) under [hints]. Files under migrations, versions, fixtures, __snapshots__ and any vN directory below the project root are skipped, write_sibling_copy_allow adds path globs to skip, and write_sibling_copy = false (or TOKEN_GOAT_WRITE_SIBLING_COPY=0) turns it off |
A Read, Grep, Edit or Write names a path that does not exist, or a shell command fails with No such file or directory |
The failed call is answered with up to three existing paths it probably meant (a stray @ or :12 suffix, a curly apostrophe, a one or two letter typo, a case difference, the same name in another directory), once per path per session. A credential-style file name is never offered. The read commands print the same list under Did you mean: |
An Edit fails because its old_string was one invisible character off (trailing space, curly quote, indentation) |
The reply quotes the exact matching lines as they are on disk, so the retry copies them instead of re-reading the file. A credential-style file name gets the line range only |
| A shell command fails that an earlier session fixed twice (an install, build or test command from a short allowlist) | One line offers the fix again, marked not verified. Only known runners and paths are kept, and no other part of the command is stored on disk, only digests |
| A whole-file re-read is refused and the agent asks again, or tries the same file at other line ranges | The whole-file refusal is given at most once per file per compaction, then the read goes through with a short note. A refused line range locks later line-range reads on that file until it changes, and the refusal names the read "path@START-END" or skeleton command to use instead |
| PDF opened via Read | Full read denied; PDF shows page count and outline (token-goat pdf-extract pulls the actual text, optionally paged/sliced, when the outline isn't enough) |
| Excel/PowerPoint/Word file (.xlsx/.pptx/.docx) opened via Read | Full read denied; redirects to the matching narrow-slice command family (xlsx-sheets/xlsx-columns/xlsx-head/xlsx-range/xlsx-query, pptx-outline/pptx-slide/pptx-notes/pptx-text, docx-outline/docx-tables/docx-text) instead of extracting the whole document as text |
| SQLite database (.db, .sqlite, .sqlite3, .db3) opened via Read | Full read denied; redirects to the SQLite narrow-slice family (sqlite-tables/sqlite-schema/sqlite-query) instead of reading binary bytes into context |
| Parquet file (.parquet) opened via Read | Full read denied; redirects to DuckDB for SQL querying instead of reading binary columnar data |
| Large JSON or YAML file (≥20 KB) read in full | Top-level keys or array summary previewed; redirects to json-outline/json-query or yaml-outline/yaml-query instead of blowing context on multi-megabyte payloads |
A file read through inline Python, Node or PowerShell (python -c "...open('r.json')...") |
Runs capped at 2,000 tokens with no other filtering, so a one-key projection comes back whole; a whole-file dump is cut, ends with a recall id, and names json-outline/json-query for JSON and YAML. Still refused where the wrapper cannot run: a pipeline or chain, a background &, compression turned off |
| Findings you want to survive a compaction | token-goat note set <key> "<finding>" stores one per project; notes print at every session start and in the compaction manifest. Add --anchor src/auth.ts::login, or name a file::symbol in the finding, and the note is marked "(changed since note)" once that symbol changes. If a slow start misses its 3-second deadline on Copilot CLI or VS Code, the session-start text and notes arrive with the first tool call |
| Large JSON Lines file (.jsonl ≥20 KB) read in full | Record count and first-record schema shown; suggests slicing records with offset/limit instead of loading millions of tokens |
| Other Office binary (.odt, .ods, .ott, .odp) opened via Read | Full read denied; redirects to pandoc for text extraction (no dedicated reader for these formats yet) |
| Large CSV or TSV file (≥10 KB) read in full | Column headers, row count, and 3 sample rows shown; token-goat csv-query projects columns and/or filters rows instead of a full read; duckdb query suggestion for very large tabular data |
| WebFetch returns a page's full raw HTML | HTML-to-text extraction strips markup/scripts/styles before the model ever sees it — readable prose instead of a wall of tags |
| Large WebVTT/SRT transcript (≥10 KB) read in full | Duration, cue count, and detected speakers shown; token-goat transcript-outline gives a skimmable speaker/time overview and token-goat transcript slices by speaker/time range/pattern instead of a full read |
| Large TXT or log file (≥20 KB) read in full | Line count + first/last 5 lines shown; .log/.out files bias toward --tail 100 --grep; general catch-all for any file ≥100 KB |
| Subagent reads a 47–86 KB recon dump (or greps a 73 KB transcript) and overflows its window | pre_read denies a full Read at or above large_read_redirect_bytes |
…
Releases