Presented at AI Engineer.

Wall Street still doesn’t trust AI, and for good reason. In finance, a hallucinated number is not a quirk. It’s a wrong answer that someone has to put their name behind. Trust there means having someone to blame, and nobody wants to sign off on a number a model made up.

This 18-minute talk covers where wrong numbers come from and how we built Kepler so the model never writes one.

Where Wrong Numbers Come From

Wrong numbers have three causes:

  1. The wrong document. The system pulls the wrong filing, period, or company.
  2. The wrong number from the right document. The source is correct, but the extraction isn’t.
  3. The model doing math. An LLM handles arithmetic it was never built to do reliably.

Each one needs its own fix, and better prompting doesn’t fix any of them.

Lookups Can’t Get It Wrong

At Kepler, code extracts every fact ahead of time and gives each one an ID. The model plans the analysis and points to those IDs; it never writes a number itself. A lookup either resolves to a real, sourced fact or it doesn’t, so the model has nothing to invent.

The talk also covers:

  • Searching wide, then narrowing, so retrieval finds the right document before anything is extracted.
  • Scoped document access, so firms never see each other’s data.
  • Verified, corrected, or refused: every answer is checked, and anything that can’t be resolved is corrected or refused rather than passed through.
  • Results from production, and the questions to ask any AI vendor about how they verify their output.

Questions about verifiable AI or the architecture behind Kepler? Feel free to reach out.