Two Engineers, Three Months, 70% AI-Generated Code: The Real Story Behind OpenAI's Data Agent
Emma Tang leaned forward, her voice carrying the weight of someone who'd seen the future and wasn't impressed. 'We built this entire system with two engineers in three months. Seventy percent of the code was written by AI.' She paused, letting that sink in. 'And now 4,000 employees use it every day.'
The numbers are staggering. Six hundred petabytes of data across 70,000 datasets. A plain-English interface that returns charts, dashboards, and long-form analytical reports. But the real story isn't the scale—it's the architecture that makes it possible.
Aris leaned back, coughing over a glass of cheap bourbon. 'I spent six years trying to solve thermal throttling on the 10nm node only for marketing to call it a feature,' he growled. 'This is just a fancy heater.'
The Six-Layer Context Stack That Actually Works
Most enterprise AI agents fail because they drown in context. Tang's team discovered the opposite: less context produces better results. The agent uses six context layers, but the magic isn't in having more—it's in having the right ones.
The foundation layer is basic schema metadata and curated expert descriptions. Simple, right? But layer two is where things get interesting: institutional knowledge pulled from Slack, Google Docs, and Notion. This isn't just documentation—it's the tribal knowledge that exists in every company but never makes it into formal systems.
Layer three is Codex Enrichment. This is the technical guts of the system. A daily asynchronous process where Codex examines important data tables, analyzes the underlying pipeline code, and determines each table's upstream and downstream dependencies, ownership, granularity, join keys, and similar tables.
'We give it a prompt, have Codex look at the code and respond with what we need, and then persist that to the database,' Tang explained. 'When a user later asks about revenue, the agent searches a vector database to find which tables Codex has already mapped to that concept.'
The remaining layers include a learning memory that stores corrections from previous conversations and live queries against the data warehouse when no prior information exists. But here's the kicker: the team discovered that 'all query history is everybody's 'select star, limit 10.' It's not really helpful.' So they tier historical query patterns, flagging canonical dashboards and executive reports as 'source of truth' while deprioritizing everything else.
This architecture mirrors the principles we saw in Gemini Live Search's latency nightmare—where too much context creates decision paralysis rather than better outcomes.
Why Codex Became the Ultimate Data Engineer
Codex serves triple duty in the system. Users access the data agent through Codex via MCP. The team used Codex to generate more than 70% of the agent's own code. But the third role is the most technically fascinating: the daily enrichment process.
Finding the right table among 70,000 datasets is, by Tang's own admission, the single hardest technical challenge. 'That's the biggest problem with this agent,' she said. 'And it's where Codex plays its most inventive role.'
The physics of this approach are compelling. Instead of brute-force searching through all 70,000 datasets, Codex builds a semantic map of the data ecosystem. It understands not just what data exists, but how it relates, who owns it, and what questions it can answer. This is fundamentally different from traditional data catalogs that rely on manual tagging and metadata.
The agent also streams its intermediate reasoning to users in real time, exposes which tables it selected and why, and links directly to underlying query results. Users can interrupt the agent mid-analysis to redirect it. The system also checkpoints its progress, enabling it to resume after failures.
This streaming architecture is similar to what we analyzed in OpenAI's DoD amendment physics—where real-time feedback loops create more reliable systems than batch processing ever could.
The Overconfidence Problem: Why AI Agents Lie
Tang was remarkably candid about the agent's biggest behavioral flaw: overconfidence. 'It's a really big problem, because what the model often does is feel overconfident,' she said. 'It'll say, 'This is the right table,' and just go forth and start doing analysis. That's actually the wrong approach.'
The fix came through prompt engineering that forces the agent to linger in a discovery phase. 'We found that the more time it spends gathering possible scenarios and comparing which table to use—just spending more time in the discovery phase—the better the results,' she said.
The prompt reads almost like coaching a junior analyst: 'Before you run ahead with this, I really want you to do more validation on whether this is the right table. So please check more sources before you go and create actual data.'
This discovery phase is critical because it addresses a fundamental limitation of large language models: they're trained to be helpful, not accurate. Without explicit constraints, they'll confidently generate plausible-sounding but incorrect answers.
The team also learned, through rigorous evaluation, that less context can produce better results. 'It's very easy to dump everything in and just expect it to do better,' Tang said. 'From our evals, we actually found the opposite. The fewer things you give it, and the more curated and accurate the context is, the better the results.'
This mirrors what we saw in The Ergonomics Arms Race—where more features don't necessarily mean better performance.
Guardrails That Are Deliberately Simple
When it comes to safety, Tang took a pragmatic approach that may surprise enterprises expecting sophisticated AI alignment techniques.
'I think you just have to have even more dumb guardrails,' she said. 'We have really strong access control. It's always using your personal token, so whatever you have access to is only what you have access to.'
The agent operates purely as an interface layer, inheriting the same permissions that govern OpenAI's data. It never appears in public channels—only in private channels or a user's own interface. Write access is restricted to a temporary test schema that gets wiped periodically and can't be shared.
Longer term, Tang said, the plan is to move toward a multi-agent architecture where specialized agents monitor and assist each other. 'We're moving towards that eventually, but right now, even as it is, we've gotten pretty far.'
This approach to guardrails is surprisingly similar to what we saw in Video KYC's human-assisted identity shield—where simple, enforceable rules often outperform complex AI alignment techniques.
Why OpenAI Won't Sell This Tool—But Wants You to Build Your Own
Despite the obvious commercial potential, OpenAI told VentureBeat that the company has no plans to productize its internal data agent. The strategy is to provide building blocks and let enterprises construct their own.
'We use all the same APIs that are available externally,' Tang said. 'The Responses API, the Evals API. We don't have a fine-tuned model. We just use 5.2. So you can definitely build this.'
That message aligns with OpenAI's broader enterprise push. The company launched OpenAI Frontier in early February, an end-to-end platform for enterprises to build and manage AI agents. It has since enlisted McKinsey, Boston Consulting Group, Accenture, and Capgemini to help sell and implement the platform.
According to information shared with VentureBeat by OpenAI, Codex is now used by 95% of engineers at OpenAI and reviews all pull requests before they're merged. Its global weekly active user base has tripled since the start of the year, surpassing one million. Overall usage has grown more than fivefold.
Tang described a shift in how employees use Codex that transcends coding entirely. 'Codex isn't even a coding tool anymore. It's much more than that,' she said. 'I see non-technical teams use it to organize thoughts and create slides and to create daily summaries.'
This transformation of Codex from a coding tool to an organizational assistant is reminiscent of what we saw in Nintendo's audio innovation—where tools designed for one purpose found entirely new applications.
The Unsexy Prerequisite That Will Determine Who Wins the AI Agent Race
When asked what other enterprises should take away from OpenAI's experience, Tang didn't point to model capabilities or clever prompt engineering. She pointed to something far more mundane.
'This is not sexy, but data governance is really important for data agents to work well,' she said. 'Your data needs to be clean enough and annotated enough, and there needs to be a source of truth somewhere for the agent to crawl.'
The underlying infrastructure—storage, compute, orchestration, and business intelligence layers—hasn't been replaced by the agent. It still needs all of those tools to do its job. But it serves as a fundamentally new entry point for data intelligence, one that is more autonomous and accessible than anything that came before it.
Tang closed the interview with a warning for companies that hesitate. 'Companies that adopt this are going to see the benefits very rapidly,' she said. 'And companies that don't are going to fall behind. It's going to pull apart. The companies who use it are going to advance very, very quickly.'
Asked whether that acceleration worried her own colleagues—especially after a wave of recent layoffs at companies like Block—Tang paused. 'How much we're able to do as a company has accelerated,' she said, 'but it still doesn't match our ambitions, not even one bit.'
Industry Insights: #IndustrialTech #HardwareEngineering #NextCore #SmartManufacturing #TechAnalysis
Bringing you the latest in technology and innovation.