Before an earnings call, the system predicts the questions analysts will ask and who will ask them. I built it as a prep desk, not a chat demo: an evidence pack of industry data, trends, past transcripts, filings, and sell-side notes — never the target call's own Q&A — then a ranked slate, then a grade against the real transcript. Across 18 held-out calls the slates covered roughly 75% of what analysts actually asked, under a partial-credit rubric, not exact-match accuracy. It is being offered to the CFOs of two major insurers.
Investor-relations and CFO prep for a public insurer is still tribal. A week before the call, a small group sits with last quarter's deck, one sell-side preview, and whatever someone remembers from the last five Q&As. They guess. The guesses become a briefing book. Management rehearses the book. Then the first analyst on the line asks about spread compression, remaining buyback authorization, or uptake of a new annuity product that never made the list. That miss is public — on the transcript, in the follow-up notes, and often in the afternoon tape.
The archive is sitting there. Every past call for the ticker, every peer call in the same quarter, every 10-Q and 8-K, every sell-side note is the actual distribution of questions. Themes recur: net investment income and spread, capital return, statutory capital, new-product sales, mortality or lapse experience, reserve adequacy, hedging P&L. Individual analysts have signatures. One coverage person always opens on capital. Another never lets a new product go without a persistency follow-up. Nobody had turned that archive into a ranked slate, an asker prediction, and a scoreboard after the call. That is the product.
I treated it as a production prep system, not a chat window.
The evidence store is versioned per ticker and quarter: management presentations, prior-quarter Q&A threads, peer-insurer transcripts, 10-Q and 8-K text, and sell-side notes. The pack builder is the load-bearing piece. It assembles what a human IR team would have the morning of the call and refuses the target call's own Q&A. That leakage firewall is unit-tested. If a pack includes a sentence from the held-out transcript, the build fails. Packs are immutable once locked. A rerun against the same ticker-quarter hash has to emit the same pack, or the tests fail. I do not ship a slate I cannot defend.
A LangGraph run sits on the pack. A coverage panel of independent agents reads it and proposes questions across the theme map — spread, buybacks, new products, capital, experience, hedging — so one agent's blind spot cannot drop a topic. A persona desk models the named analysts who actually cover the name, from their historical ask patterns, not invented personalities. A synthesizer merges both desks with a deterministic merge, so coverage the panel already proved cannot be silently trimmed to make the slate look tidy. Model calls route through LiteLLM so the graph is not welded to one provider. If an agent fails, the web app shows the failure. There is no canned fallback that invents a clean result.
The slate streams into a React app over SSE as NDJSON events — rank, question, likely asker, evidence chips, confidence — so IR can watch the desk work and then lock a briefing book.
I hold out the target call. After it happens, a grader scores the locked slate against the real Q&A. Coverage is not exact-match accuracy. It is a partial-credit rubric: a predicted question scores if it is the same topic and the same operational ask, even when the wording differs. "Walk us through spread compression" hits "can you unpack NII and the drivers of spread this quarter." A near-miss on the same theme gets partial credit. A miss on buybacks or a new product is a zero for that actual question.
The judge is an LLM-as-judge with that rubric, plus a human spot-check on a sample of calls so the judge cannot drift. I also run a single-shot prompt baseline on the same packs. The graph has to beat it, or the extra agents are theater. Traces land in LangSmith. Promptfoo holds the pack-builder prompts and the judge rubric as regressions. Eighteen held-out calls, life and annuity names, is the current book. Roughly 75% coverage under that rubric. I say it that way on purpose.
The output IR actually uses is the ranked slate, the asker column, and the scoreboard after the call. The night before, they lock a ticker-quarter, read who is likely to open, which follow-up will come if management hedges, and which theme is still uncovered. After the call it is a grade. Misses go back into the next quarter's pack as open threads.
It is being offered to the CFOs of two major insurers. Not sold, not signed. Offered. That is the honest status.
Python, FastAPI, LangGraph, LiteLLM, LangSmith, Promptfoo, React, SSE / NDJSON streaming. Postgres for the versioned evidence store. pytest on the leakage firewall.