Governance

In a post I wrote last December, I discussed the production triangle: the constraint that governs every production system, including software. You can optimize for two of three dimensions, time, quantity, and quality, but never all three. Push velocity while adding features, and quality absorbs the cost. Every time, without exception. This constraint is not new. Kahneman mapped the cognitive mechanism in Thinking, Fast and Slow: under time pressure, we disengage System 2, the slow, deliberate reasoning that catches the things that will hurt us later, and default to fast, intuitive pattern-matching. Kuutila et al.’s 2020 systematic review of the software engineering literature confirmed the outcome: increased throughput, decreased quality, and a root cause that almost always traces back to the pressure itself being a product of bad estimation. The triangle is not a theory. It is physics. ...

Gartner published a prediction this week: by 2027, 40% of enterprises will demote or decommission autonomous AI agents due to governance gaps that are only identified after production incidents. The root cause they name at Level 3 of their autonomy framework is approval fatigue. Agents execute actions, writing data, sending communications, and modifying configurations, but only after explicit human approval. Under time pressure, that approval becomes reflexive. The control degrades, and the risk compounds in silence. ...

I recently argued on the JFrog blog that trusted AI requires more than model quality. It requires visibility, provenance, governance, and a real system of control around the things models consume, build, and ship. That is the foundation. This post is about what you build on top of it. Because visibility is necessary. Without it, you cannot govern anything. If you cannot see which models are running, where they came from, how they behave, and what they touch, you do not have a governance posture. You have hope dressed up as architecture. ...