https://molsen.ca/tags/ai-governance/Recent content in AI Governance on Mike OlsenHugo -- gohugo.ioen-caSun, 08 Feb 2026 00:00:00 +0000https://molsen.ca/writing/update-february-2026/Sun, 08 Feb 2026 00:00:00 +0000https://molsen.ca/writing/update-february-2026/Since I formulated the oracle/assistant distinction in late 2024, reasoning models with chain-of-thought, retrieval-augmented generation, agentic tool use, structured outputs, and determinism mechanisms have advanced significantly. I commissioned systematic research to assess whether those advances had made the framework obsolete. The distinction remains fundamentally valid: none of those advances resolves the core governance challenges. The research also identified a threat I hadn’t anticipated: agents can now make the routing decision that the framework assigns to humans.https://molsen.ca/writing/answers-to-critics/Sun, 08 Feb 2026 00:00:00 +0000https://molsen.ca/writing/answers-to-critics/An adversarial review identified weaknesses that cannot be answered by saying the critic misread the framework. Some require refinement. Others concern precision without changing the substance, and some concern tone rather than argument. The AI-First / EKA Tension AI-First Software says the system can “dynamically select algorithms and approaches” and that “the user describes outcomes; the system determines approach.” Executable Knowledge Architecture insists that expert judgment stays in the method, and the AI translates that method into execution.https://molsen.ca/writing/eka-in-practice/Fri, 02 Jan 2026 00:00:00 +0000https://molsen.ca/writing/eka-in-practice/We’ve deployed Executable Knowledge Architecture at a professional services firm with significant liability exposure. The implementation details are competitive advantage and can’t be shared publicly, but the pattern is transferable. Orientation, Controls, and Methodology Records New users experience the oracle pattern’s failure mode firsthand by role playing during the orientation. They are prompted to ask naive questions and receive plausible (and possibly correct!) answers but then discuss how those answers cannot be verified without reproducing the effort to obtain them independently.https://molsen.ca/writing/universal-interface-thesis/Fri, 02 Jan 2026 00:00:00 +0000https://molsen.ca/writing/universal-interface-thesis/Agentic AI as Universal Interface describes how code generation connects expertise to systems for software, analysis, and business logic. The same approach can apply beyond knowledge work. Code generation can provide an interface to any system that accepts commands, including manufacturing equipment, building automation, medical devices, networks, and enterprise software. The user needs to know the desired outcome. The Current Model Manufacturers or third parties design equipment-specific screens, menus, and workflows based on their expectations of user needs.https://molsen.ca/writing/return-of-the-assistant/Thu, 01 Jan 2026 00:00:00 +0000https://molsen.ca/writing/return-of-the-assistant/In the 1950s and 1960s, every executive had a secretary. Typing pools handled document production. Stenographers transcribed dictation. The professional focused on content; the secretary handled execution. Executives delegated formatting, typing, and filing because their time was more valuable elsewhere. Secretaries’ shorthand, typing speed, and knowledge of office systems made execution faster and more reliable than if executives did it themselves. The Liberation of DIY Personal computers gave professionals direct control over office tasks.https://molsen.ca/writing/agentic-ai-universal-interface/Mon, 30 Jun 2025 00:00:00 +0000https://molsen.ca/writing/agentic-ai-universal-interface/Software encodes expertise as instructions a machine can execute. When an engineer encodes a structural calculation into a spreadsheet, that spreadsheet embeds what the engineer knows. When a business analyst creates formulas to process financial data, those formulas distill how the business works. When a software engineer encodes a business process into an ERP system, that system inscribes the process designer’s expertise into reusable automation. The calculation runs without the engineer present.https://molsen.ca/writing/ontology-generation/Mon, 30 Jun 2025 00:00:00 +0000https://molsen.ca/writing/ontology-generation/An ontology represents an organization’s concepts, their properties, and their relationships in a structured form. Palantir’s Foundry platform uses this approach to model organizational entities and operations, making objects, properties, relationships, and actions explicit and queryable. Structured organizational knowledge lets people act systematically. Without it, expertise often remains with individuals, process documentation diverges from practice, and departures cause knowledge loss. The problem has always been construction. Building an ontology is expensive.https://molsen.ca/writing/automating-expertise-gets-easier/Mon, 30 Jun 2025 00:00:00 +0000https://molsen.ca/writing/automating-expertise-gets-easier/Agentic AI as Universal Interface removes the barriers between domain expertise and system access. This has consequences for what we call “software.” Applications and Integration Software Most software serves needs created by those technical barriers. Applications sit between users and data because users couldn’t directly express what they wanted from the data. Integration software connects systems because connecting systems required specialized knowledge. Business software encodes business logic because encoding required coding skill.https://molsen.ca/writing/executable-knowledge-architecture/Tue, 07 Jan 2025 00:00:00 +0000https://molsen.ca/writing/executable-knowledge-architecture/AI explanations can be unfaithful. Answers to Critics explains the need for independent verification. Anyone can now ask ChatGPT questions they once needed experts to answer. That access puts pressure on the value of expertise and on how professionals use AI. Oracle use means asking AI questions and using its answers. Verifying those answers requires the expertise to produce them yourself. This verification paradox prevents professional accountability in the oracle pattern.https://molsen.ca/writing/what-benchmarks-arent-measuring/Tue, 31 Dec 2024 00:00:00 +0000https://molsen.ca/writing/what-benchmarks-arent-measuring/We do not know how much valuable professional work AI can competently perform. Existing benchmarks do not provide that measure. Tame Problems OpenAI’s GDPval benchmark represents substantial investment in evaluation. It recruits credentialed lawyers, accountants, engineers, and physicians to compare AI outputs with human work in blind evaluations of realistic tasks. It is carefully designed and transparently documented, improving on simpler metrics while measuring the wrong things for this purpose.https://molsen.ca/writing/capability-governance/Fri, 27 Dec 2024 00:00:00 +0000https://molsen.ca/writing/capability-governance/An artifact is an output someone else will act on, such as a report, calculation, recommendation, design, or analysis. Generation is how an artifact was produced: written by a human, drafted by AI, or produced through some combination of review, refinement, and uncritical approval. Defensibility is whether and to what degree the artifact can be defended when challenged. Defensibility is established through one of two routes: exposed methodology that can be inspected, or empirical track record that demonstrates reliability.https://molsen.ca/writing/strong-oracle-trap/Tue, 26 Nov 2024 00:00:00 +0000https://molsen.ca/writing/strong-oracle-trap/When I describe oracle pattern as ungovernable, people often push back. “That’s not how I use it,” they say. “I don’t just ask a question and accept the answer. I have a conversation. I probe. I challenge. The AI asks me clarifying questions. We iterate.” I call this “strong oracle pattern.” It improves answer quality but leaves the verification problem unresolved. The Oracle Spectrum Oracle use ranges from accepting a single answer to examining a problem through extended dialogue.https://molsen.ca/writing/ai-oracle-vs-assistant/Tue, 05 Nov 2024 00:00:00 +0000https://molsen.ca/writing/ai-oracle-vs-assistant/There are two fundamentally different ways to deploy AI in professional work. The choice between them determines whether you can govern what you’re doing. Oracle pattern: You ask the AI a question. It gives you an answer. You use that answer. Assistant pattern: You tell the AI what you want to accomplish. It produces artifacts that help you accomplish it. You verify those artifacts before using them. Most people default to Oracle pattern.https://molsen.ca/writing/llm-stochasticity-determinism/Fri, 01 Nov 2024 00:00:00 +0000https://molsen.ca/writing/llm-stochasticity-determinism/The criticism that LLMs are “too stochastic for production use” misunderstands their nature. I ran a systematic experiment to demonstrate this. This experiment demonstrates the assistant-pattern mechanism on one task. For broader evidence across reasoning models, retrieval-augmented generation systems, and production deployments, see Update for February 2026. The Experiment Hypothesis: LLM “unreliability” is a usage problem, not a tool limitation. Seven models tested: GPT-4, GPT-4.1, plus mini variants of each Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku 100 iterations per model.https://molsen.ca/writing/ai-first-software/Thu, 04 Apr 2024 00:00:00 +0000https://molsen.ca/writing/ai-first-software/For the relationship between interface design and governance, see Update for February 2026 and Answers to Critics. “These systems have unequivocally solved language. They possess what experts call formal competence.”: Andrej Karpathy The debate about artificial general intelligence distracts from what generative AI already makes possible. AI can interpret and act on natural language reliably and at scale. That capability is sufficient to change how we design software, whether or not AI ever achieves general intelligence.https://molsen.ca/writing/ai-table-extraction-comparison/Tue, 20 Feb 2024 00:00:00 +0000https://molsen.ca/writing/ai-table-extraction-comparison/I ran identical documents through Docling, ScaleDP, and Claude-based sub-agents. Title-detection accuracy ranged from 11% to 100%, with consequences for whether extraction saved work or created more cleanup. The Test: Real Proposals, Real Complexity I used consulting proposals with project cost summaries, budget breakdowns with multi-level headers, milestone schedules, and staffing matrices. The tables included merged cells, currency formatting, task hierarchies, and formatting quirks from working documents. The BHP proposal contained a seemingly simple five-row cost summary and a complex 17-by-24 budget breakdown spanning multiple staff categories and task hierarchies.https://molsen.ca/writing/parrots-are-all-you-need/Sun, 15 Jan 2023 00:00:00 +0000https://molsen.ca/writing/parrots-are-all-you-need/Original publication, January 2023. The title predates the widespread use of “is all you need”; I would choose a different title now. CIOs either already consider AI an urgent priority, will soon recognize the need to do so, or will be out of work in two years. The June 2017 paper “Attention Is All You Need” changed how natural language processing models were trained. ChatGPT launched on November 30, 2022 and immediately became the fastest-growing platform in Internet history.