◆Solutions — Finance

In finance, a confident wrong answer is the failure mode.

Financial documents are dense, numeric, and genuinely multimodal. The number you need lives in a table cell, a chart, or an XBRL tag; a generic stack retrieves the fluent paragraph beside it and answers with a figure that reads well and is wrong. In this domain that is a liability, not a glitch. Below is a field log of problems we’ve actually worked through for finance teams, including a new way we use AgentGrad to make an agent catch its own wrong answers.

Talk to us Read the field log

Field logFinance domain

Agentic grounding

A semantic question has to resolve to the exact XBRL tag.

71% → 93%tool-calling accuracy on XBRL tag selection

Public filings come with a gift for grounding: every 10-K and 10-Q is annotated with XBRL tags that anchor a number to its exact location in the filing. A fintech client wanted LLM answers grounded through them. The catch: mapping a semantic question to the right tag among thousands of candidates is nothing like keyword lookup, and the agent doing the mapping picked the correct tag only 71% of the time.

We built a query-to-tag tool and let AgentGrad optimize its definition and few-shot examples against real failures, lifting tool-calling accuracy to 93%. The optimization had a second effect: it closed the gap between Fable and Sonnet 5 on this task, so orchestration moved to the smaller model and inference cost dropped 5x.

Query → XBRL tag · tool call

“How fast did subscription revenue grow in FY25?”

AgentGrad-optimized tool

·us-gaap:Revenues

✓us-gaap:RevenueFromContractWithCustomer…anchor

·us-gaap:DeferredRevenueCurrent

·us-gaap:CostOfRevenue

+ thousands more

One question, thousands of candidate tags; the optimized tool definition picks the anchor. Illustrative tags.

Multimodal retrieval

The corpus is half tables and charts. The fix wasn’t a better VLM.

+15%recall@k, at 60% lower latency and cost

What separates finance from legal is how multimodal the documents are: internal financial documents lean on figures and tables as heavily as prose. Following the literature, a client had built search on VLM-based embeddings (A), and recall@k, the metric their product lived on, had plateaued.

We took the opposite route (B) with PreciseSearch: parse every figure and table into a detailed textual description with a frontier LLM once, offline, then serve hybrid search with lightweight fine-tuned text embeddings plus BM25. Recall@k rose 15%, and with no VLM left in the serving path, latency dropped 60% and inference cost fell with it. The VLM-retrieval literature rarely optimizes this simple baseline; owning the whole stack vertically is what surfaces it.

One page · two retrieval routes

acme-10k-fy25.pdfp.14

The following table sets forth net revenue by segment as a percentage of total net revenue:

SegmentFY23FY24FY25

Cloud28%33%41%

Retail31%27%22%

Services26%25%24%

Other15%15%13%

(A)

VLMembeds page pixels

[ dense vector ]

(B)

frontier-LLMparses the page

text description“cloud grew from 28% to 41% of revenue; retail fell to 22%; …”

text-embed

BM25

[ dense + sparse ]

The parse route runs once offline and serves with no VLM in the path → +15% recall@k, 60% lower latency.

Agentic reliability · new

An agent that runs a DCF has to produce the same answer every time.

k = 3reruns to flag an incorrect answer at test time

A hedge fund’s agent drove 20+ tools, writing code in a sandbox and building spreadsheets, to set up standard analyses like a DCF. The blocker wasn’t capability but reliability: with growth and discount assumptions pinned, five runs of the same workflow still produced five subtly different fair values. The drift was pipeline error, not modeling judgment: one run missed a quarter’s restated revenue, sitting in a 10-K filed three years later after a segment reclassification.

We pointed AgentGrad at the problem in a new way: auto-discover the sources of inconsistency, then write specialized skill files for each until five runs of the agent converge on one answer. Self-consistency is a classic self-supervised training signal; here it becomes a reliability objective with artifacts you can read. And an agent trained to agree with itself makes disagreement diagnostic: at test time, as few as k = 3 reruns flag an incorrect answer before it ships.

DCF fair value · five runs · fixed assumptions

before

run 1$4.2B

run 2$3.9B

run 3$4.6B

run 4$4.2B

run 5$3.1B

with skill files

run 1$4.2B

run 2$4.2B

run 3$4.2B

run 4$4.2B

run 5$4.2B

Same inputs every run; trained until five runs agree. At test time k = 3 reruns flag a wrong answer. Illustrative values.

What you own

The assets a competitor can’t replicate.

Across every engagement, what we deliver is yours: proprietary assets calibrated to your corpus and your workflows, not a licensed configuration of someone else's APIs that a rival could buy off the same shelf.

◆Query-to-XBRL tool harness — AgentGrad-optimized tool definitions and few-shot examples.

◆Reliability skill files — interpretable skill-file artifacts, regenerated as workflows change.

◆Fine-tuned sparse + dense weights — lightweight BM25 and embedding adapters tuned to your corpus.

◆Multimodal parsing pipeline — frontier-LLM descriptions of every figure and table, computed offline.

◆Self-consistency harness — the k-run detector that flags a wrong answer before a client sees it.

◆Eval sets built from production failures — the labeled traces every optimization above ran against.

Who we work with

Built for teams shipping financial AI to production.

Organizations

Asset managers & hedge funds building research copilots
Banks with in-house analyst-facing assistants
Fintech & data vendors shipping document QA to customers

Roles

Heads of AI / ML at financial institutions
Quant & research engineering leads
Product leads accountable for a customer-facing accuracy KPI

Tell us your hardest finance-AI problem.

Answers that must ground to the exact filing, a corpus that is half tables and charts, an agent that won't produce the same number twice: send us the one you're stuck on.

Writing behind the finance work.

From From the Field, Search That Works, and GenAI Economics.

Thesis

The stack is the product.

The anchor essay: why assembled stacks stall, and the end-to-end loop that fixes them.

11 min read · Aug 2026

Search That Works

Component benchmarks lie. Tune the stack.

Why single-component leaderboards are the wrong place to choose a retrieval component, and what closes the gap once the stack is tuned together.

9 min read · Jun 2026

Optimizing Agents

Why we built AgentGrad.

The library behind our agent-optimization engagements, out of stealth: a PyTorch-shaped training loop for agent runtimes.

11 min read · Aug 2026