This week, we’re detailing exactly how Carta built an agent to solve one of its most pressing accounting problems.

The Dynamic Context Problem: How Carta’s Internal AI Agents Save Thousands of Hours of Back-and-Forth Work

Jayant Tikmani, Director of Machine Learning at Carta, finds the best problems for AI to solve based on this principle: deliver a differentiated customer experience and enhance service quality rather than reduce cost.

He found a worthy problem in the company’s fund administration business.

This service-based org of 600 people helps 2,500 customers handle complex accounting, reporting and compliance work. There’s a wide set of tasks this team can do for clients, so when a problem arises, it takes a lot of time to figure out what’s happening, what to do and how to fix it. Scaling is bottlenecked by context-gathering, which CPO Vrushali Paunikar says is an industrial engineering challenge.

So Tikmani worked with domain experts in the fund admin business and engineers to pinpoint workflows where agents could understand a task, build relevant context and suggest or complete next steps.

The result was the creation of an agent to handle cash reconciliation — turning an 11-minute task into one that could be completed in seconds. There are 20,000-25,000 of these tasks per month, which saved Carta’s internal team over 3,500 hours per month. Here’s how he and the team did it:

• Use a focused, low-investment PoC to assess risk, feasibility and answer key questions. Could the agent retrieve the right context from an internal tool via an API or database query? How should they structure and present the agent-generated diagnosis and recommendations to internal teams? Did users actually find the output useful and accurate enough to act on?
• Give the agent the right context for specific tasks and tools. Domain experts diagrammed the full workflow in Lucidchart, which acted as the source code for the agent’s system prompt. With that input, they created a set of gold-standard prompting techniques relying heavily on in-context learning: provide examples, the expected output structure and task framing.
• Lightweight design meant to evolve through iteration. Prototype and get a product in the hands of users to move as quickly as possible. To do this, they decoupled product performance (which is based on model behavior) from UX and workflow integration. They could evolve the agent’s reasoning and output quality without needing to constantly rebuild the product surface or its backend systems.
• Evaluating on one metric: helpfulness to users. Internal experts drove the agent’s feedback process. They provided a helpfulness rating via the product’s interface, compared normal task resolution speed to that of the agent and tracked reduction in back-and-forth between teams by monitoring escalations and handoffs.

This essay is a detailed look into how Carta identified a problem, set the right parameters for solving it and actually built the tool to do it. It’s an excellent example of AI in production driving meaningful business impact.

Thanks, as always, for reading and sharing,

-The Review Editors