Databricks Omnigent: How We Built a Procurement Analytics Pipeline 40-60% Faster

Author: Mayank Mishra: VP – Delivery & Solutions 

Key takeaways:

• Most enterprises already run agents across several models, frameworks, and coding tools; the risk is not tool diversity itself, but the lack of a common governance layer across them.
  
• A meta-harness like Omnigent sits above those agents as one control layer, so teams get portability and central control without a rebuild and without lock-in.
• Omnigent is open source and usable today. The managed Databricks version, which inherits Unity AI Gateway governance, is in beta, so the immediate opportunity is to adopt early, prove the control layer, and prepare for governed scale.

A question I hear often from engineering leaders is some version of, “Which agent framework should we standardize on?” It is the wrong question to lead with. By the time a leader asks it, their teams are usually already running several: Claude Code in one place, Codex or Cursor in another, a couple of custom agents, and a framework or two underneath. That is not a mistake to correct. It is how capable teams work, choosing the best tool for each job. The real problem is quieter and more urgent: there is no common way to operate, secure, or control all those agents at once. Databricks’ Omnigent, released this year as an open-source meta-harness, is one of the clearest signs that the answer is not to pick one agent, but to govern the layer above all of them.

At KPI Partners, we see Omnigent as more than another agent framework announcement. It reflects the direction our Databricks work is already taking: Helping enterprises turn scattered AI experimentation into governed, secure, and scalable agent operations. This point of view explains why that shift matters now, why Databricks is well positioned to support it, and where KPI Partners can help platform and engineering leaders make the move pragmatically.

The Multi-Agent Enterprise Is Already Here

The direction is not anecdotal. Gartner expects agentic AI to be built into 33 percent of enterprise software applications by 2028, up from less than 1 percent in 2024. That curve will pull more agents into more corners of the business every quarter. Add the coding agents and frameworks teams already layer on top, and a typical engineering organization is no longer running one agent stack; it is running several in parallel. Each choice may be reasonable in isolation. Together, without a shared control layer, they create an operating problem: different interfaces, different policies, different bills, and no single place to see what any of them is doing.

The Real Exposure Is Not the Framework; it is the Gaps Between Them

When I look at where multi-agent programs get challenging, it is rarely the model or the framework. It is the space between them. Three gaps show up every time.

• There is no unified control, so no one can say with confidence what every agent is allowed to do.
• There is no spend visibility, so token costs accumulate across tools with no way to attribute or cap them.
• There is no shared security posture, so each agent handles data and secrets on its own terms.
  

The cost of those gaps is measurable. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. On the security side, IBM's 2025 Cost of a Data Breach report found that 97 percent of organizations hit by an artificial intelligence (AI) related breach lacked proper AI access controls. Neither of those is a framework problem. Both are what happens when many agents run with nothing governing them together.

What a Meta-Harness Actually Changes

A meta-harness is a layer that sits above your agents and gives them one common interface and one set of controls. Omnigent provides that layer over Claude Code, Codex, Cursor, Pi, the open-source frameworks many teams already use, and the custom agents they write themselves. Teams define an agent once in a short definition file, then change its harness or model with a one-line edit while its tools, prompts, skills, and policies stay the same.

Policies and an operating-system sandbox are enforced at that layer rather than left to each agent: You can, for example, keep an agent from ever seeing a sensitive credential and inject it only on approved outbound requests. Because Omnigent is open source under a permissive license, teams can adopt and extend it without waiting on a vendor. The shift it represents is the point: You stop trying to standardize the agents and start governing the layer above them.

Five Agents, One Harness: A Procurement Pipeline Delivered Faster

This is not theoretical for us. When our team recently built a new procurement analytics pipeline from SAP data, we ran the work through Omnigent, and the harness coordinated a team of specialized agents rather than a single general-purpose one:

• An architecture agent inspected the requirements and designed the medallion data model.
• A coding agent built the ingestion pipelines, the Delta tables, and the transformations.
• A testing agent generated the data-quality checks alongside the build.
  
• A review agent examined security, performance, and coding standards.
• A documentation agent produced the deployment and operational documentation.

Five agents, each doing what it does best, all governed and coordinated on one harness. The result was a meaningfully faster delivery, in the range of 40 to 60 percent faster for the initial build, and it did not come by cutting the corners that usually get cut under time pressure. Testing and documentation were produced with the implementation rather than bolted on afterward, and the review cycle shortened because standards were checked as the work happened.

Two things stand out to me about that result.

1. Speed did not come at the expense of quality; If anything, test coverage and documentation improved, because they were part of the flow rather than an afterthought.
2. None of it would have been governable without the harness. Five agents acting independently would have been five things to secure, bill, and audit. On one harness, they were one.

Why Databricks Is the Governed Path

Open source solves portability. Governance is where the managed version matters. Omnigent on Databricks, currently in beta, runs the same meta-harness but wires it into Unity AI Gateway and Unity Catalog, so policies, spend controls, and observability apply centrally across every model, Model Context Protocol (MCP) tool, and agent, right next to the data they act on.

That is what turns a portability layer into a control plane: One place to set what agents may do, one place to see and cap what they spend, and one audit trail of what they did. Because the managed version is still in beta, I would treat it as an early-adopter move rather than a finished production standard, and pair it with the open-source harness that is available now. The direction is unambiguous, and it is a Databricks-grounded one.

It Sits Alongside What You Already Run

The objection I expect from any team already invested in a framework is fair: “We are mid-flight, and we are not rebuilding.” They do not have to. Omnigent is a drop-in wrapper, not a migration. Existing agent definitions, MCP tools, and skills keep working as they are, and adoption can be incremental. That is what makes consolidation onto one governed layer realistic rather than aspirational. Teams add control above the agents they already run, not replace them. A practical first move is to bring one noisy, high-spend agent under the harness, prove the control and visibility, and expand from there.

Where KPI Partners Comes In

Consolidating scattered agents onto one governed harness is less about the harness itself and more about the decisions that make it enterprise-ready: which agents to bring under control first, what policies and spend limits to set, how to wire the managed version into Unity AI Gateway and Unity Catalog, and how to do all of this without disrupting teams already shipping with their current tools. That is where KPI Partners brings practical value: translating Databricks’ agent governance direction into rollout plans, control models, and adoption paths that work in real enterprise environments.

Through our agentic AI practice, KPI Partners helps enterprises design that governed, secure control layer and fold their existing agents into it, and the KPI Partners and Databricks partnership is where the open-source harness and the governed managed version come together on one foundation. For a platform owner, the outcome is simple to state: One place to see, secure, and control every agent, whoever built it and whatever it runs on.

What to Do Now

You do not need to wait for the managed version to leave beta to get ahead of this. I would start in four places:

1. Inventory the agents, models, and frameworks your teams are actually running, because the number is usually higher than anyone expects.
2. Adopt the open-source harness to standardize the control layer now, without committing to a single model or vendor.
3. Pilot the managed version on Databricks for one workload where central governance and spend control matter most.
4. Set your policies and spend limits centrally from the start, since retrofitting control onto a live fleet of agents is far harder than building it in early.

Every one of these pays off immediately and compounds as the number of agents grows.

Stop Optimizing the Agents. Start Governing the Layer Above Them.

The multi-agent enterprise is not coming; it is already how your best teams work. The leaders who pull ahead will not be the ones who force everyone onto a single framework. They will be the ones who stop optimizing individual agents and start governing the layer above them all, getting portability and safety at the same time. A meta-harness is how that becomes real, and it is a move worth starting now.

Related reading: KPI Partners agentic AI practice

Ready to Transform Your Data Strategy? Talk to our experts and discover how KPI Partners can accelerate your data and analytics initiatives.

Talk to Our Experts