Coda Is a Workbench. It Is Not a Skill Runtime

Developers Central
1

MCP can read the page. That does not make the page a skill harness.

Coda is a world-class chef whose station is already set.

Every knife is sharp and in its slot. Every spice is labeled and measured. The recipe cards are beautifully organized in a drawer, and an assistant (MCP) can even read the cards for you.

The kitchen is not the restaurant

The moment you leave the recipe cards in the drawer, call the kitchen “the dining experience”, and expect the meal to appear on the table for customers, you’ve confused the place where you prepare with the place where the work is actually served.

The kitchen remains magnificent. It just isn’t meant to be the runtime.

This will sound ungrateful coming from me. I spent years trying to prove the opposite.

And that proof held, as long as the document was also the application. It may still hold when you use integrated or deeply embedded models alongside page and table-based skills. Increasingly, though, people are building where agentic abstractions are the norm. Grok Bot is a good example.

I helped build this idea

Promptology, around the 2023 AI at Work Hackathon, treated early Coda AI as a prompt workbench. Role, task, goal, steps, rules. Outcomes. Analytics. Cross-doc reuse. Hyper-prompts you could save, restore, and measure. We did it in tables, even before Coda AI existed as a product, through OpenAI Packs.

It worked. Coda was, and still is, an unusually good place to construct a skill. You can see the logic. You can version it. You can hand a button to a human who does not want to live in a terminal. Most helpful, you can build late-binding logic. That part is still ideal.

Doug Loud took the next honest step. His memo, “The Power of Coda – Coda AI Can Have Skills like Claude,” treats a Coda document as the place you stash Claude-like Skills. A prompts table. Practical buttons. Inspiration from Max and from Christiaan, the Wizard of Ghent. The thread was enthusiastic, and it should have been. He was standing on a real lineage, not a vibe.

I am not here to walk that back. I am here to draw a line we keep refusing to draw.

Coda is an excellent authoring surface for prompts and skills, not a runtime for agentic skills.

Those are not the same job. We keep stuffing them into the same page because the page is convenient, and convenience is how architectures die.

First, the argument I need to put in its place

You will hear, including from an earlier me, that progressive loading is the reason this fails. That version is too broad.

Coda’s progressive loading is a client-side browser behavior. Open a doc in the live UI and Coda lights the page you are on first. The rest hydrates in the background. During that window the doc is only partly interactive. Buttons miss. Formulas are still evaluating. Cross-page data is not in the DOM yet.

A computer-use agent lives in that window. Browser automation. Vision. DOM clicks. It “sees” the page the way a human does, which means it inherits the race. Incomplete context. Skills that fire before their tables exist. A demo that works when someone is staring at the doc and lies when the tab is still waking up.

That is a real trap. It is a browser trap. It is not the MCP trap.

Coda API and Coda MCP talk to the backend data model. They ask for a page, a table, a row, a content type, by ID. They get Markdown or JSON. They do not sit through the UI load sequence. A ReadPage call does not wait for aisle 47 to light up. Size and performance limits still exist on huge docs or ones with complex formulas. The characteristic “partially interactive until fully loaded” problem does not, for the most part.

What still shows up in the community is cached data. API and MCP can hand you yesterday. That is not progressive loading, but it is a Coda-shaped trap.

If your skill substrate is consumed by a pure API or MCP agent, progressive loading is largely risk-free. The downside is material when the agent is driving a real browser. That happens more often than the architecture diagrams admit. When MCP or the API runs out of road, computer use is the automatic next move. Coda has several of those gaps, as we all know.

The other bills I am about to name do not care how you connect.

The bills that do not care how you connect

The real risk is dependency. We are hanging agentic workflows on a system designed as a collaborative, formula-driven document, not as a file-based skill harness.

Formulas on a canvas re-evaluate when the page renders. Doug flagged this. Open the page, and the AI function runs again. Tokens burn. API costs inflate. Downstream outputs get overwritten by a human who only meant to look. MCP does not save you if the skill itself is a canvas formula with a side effect. In a file-based harness, opening a file does not execute it. In Coda, looking can be a verb with a bill attached.

Canvas pages also carry UI metadata, rich-text wrappers, layout constructs. Pass that blob into a context window and you have paid for the chrome. A skill.md with a name, a description, and the instructions you actually need is cheap on purpose. Modern agents use progressive disclosure to load meaning. Coda pages, used as the payload, disclose widgets.

Then there is time. Coda’s calculation graph is asynchronous. A multi-step skill that writes intermediate state to a row and then reads it back is racing the doc, API or not. If the graph has not finished, step two runs on last Tuesday’s values. File systems are boring here. Boring is the point.

Governance is worse than people admit. A Coda doc is a room with one lock on the front door. There is no strict page-level sandbox for system instructions. The collaborator who can edit the project plan can also “improve” the skill that runs your close process. They will not mean to. They will. A Markdown file in a skills directory can be permissioned like code. A canvas cannot.

Scale compounds it. A “skills doc” that collects more pages, more prompt tables, more cross-references, and more supporting data gets heavier to calculate and easier to damage. Reusability was the point of Promptology. A single living document as the runtime works against that.

None of this requires a dark aisle in the browser. It requires us to notice that a document is not an operating system.

Why this feels like a betrayal

Because the kitchen was so good, we asked it to be the restaurant.

Promptology still holds. Build the skill in Coda. Test the language. Argue about the steps. Measure the outcomes. That is knowledge work, and Coda is still one of the best rooms I know for it.

Runtime is a different room. Agent harnesses expect files that can expose frontmatter and top-of-document metadata that a Coda page cannot. They expect the same bytes at 2 a.m. that you saw at 2 p.m. They expect no hidden calculation graph, no canvas formula that fires because someone opened a tab, no collaborator who renamed a heading and took the system prompt with it.

mcpOS was built with this weakness in mind. A constitution. Predictable file structures. Data that lives outside the documents that describe it.

If the agent is a browser, they also expect a page that has finished becoming a page. That is the progressive-loading tax, and it is real, and it is narrow, but it is a risk.

Ignore the wider set, and you build dependency into a system that was never designed for agentic workflows or real-time requests. You get a demo that sings while the author is on the page, and a production skill that forgets its own name the moment the side effects silently occur.

Workarounds exist. Keep every skill on one page. Never put a formula with a side effect on a canvas. Mirror the prompts out to Markdown and call them through MCP. Those workarounds are the tell. They are how you admit the elegant idea, “just stash it in a Coda page,” is already leaking.

The pertinent truth

I still want Coda in the mix. I still will not let it be the floor the agent stands on. Author in Coda. Run from files.

If your skill cannot survive a collaborator, a recalc, or a computer-use session that arrived too early, it was never a skill. It was a document posing as one.

Technology generally solves more than it breaks, as long as we stop calling the kitchen the dining experience.

2

I am traveling today, so I won’t have time to make a detailed response. But here is a short interim response from my own (quite extensive) experience…

The issues that @Bill_French highlights are real. And a naive maker who uses Coda AI columns as if they were just another type of formula will run headlong into all these issues.

That is why we developed the “Agentix Framework” that I have been writing about here.

We have encountered each of those issues, and found a way to overcome them.
Our agents run reliably when required in an event-driven regime such that they never see the race conditions that Bill highlights, and everything is orchestrated so the AI inferences only execute when the data is available and the doc is stable.

At this point, we have deployed hundreds of such agents and have audit logs for thousands of runs that show they can be reliable enough to be used in a 6-sigma quality-controlled workflow (<3 defects per million runs).

But only at the cost of a lot of research, trial and error, and a strict TDD (test-driven development) paradigm that tracks and measures reliability.

But these issues can be addressed by the SH GO AI team to make it safe for Coda makers to use SuperDocs as a “skills runtime engine”. (They have been resolved by the Notion team.) So it is a question of more development work, not (i.m.h.o) a question of “the wrong paradigm”.

We are not there yet. The Agentix Framework does allow ordinary non-technical makers to build and run Agentic Workflows. But without such guardrails, those makers would run into the problems Bill illuminates.

But I am asking the SH GO team to implement those few changes needed to address the limitations presented by Bill, and then we will have the MOST powerful platform for building and running business automations.

(Grok Bot is amazing, and I am loving it, my new favorite lobster! But my clients need to harness the investment they have made in Coda expertise and existing automations - and the SH GO paradigm is the best way for them to do that.)

Bill has articulated so much more clearly than I could, all the reasons why we had to engineer our ‘Agentix Framework’ the way we did. Thanks Bill.

Respect
:lobster:Max

3

Merci @Max_OBrien for your clairity, it is indeed essential that non technical users can confidently create agents in Coda.

@Henrique_Cruz1 , this may interest you:

Cheers, Christiaan

4

My observation:
Downstream validation + repair is almost always necessary even with the most advanced frontier models.

And I think there’s some progress on this front. Per Superhuman …

  • In the initial LLM API call, we already specify that the response should be in JSON, which is not sufficient.
  • Once we receive the response, we use a JSON parser to check whether the JSON is valid (not whether it exactly matches the one specified in the user prompt). This step helps catch trailing commas, missing commas, and other issues that make the JSON structurally invalid, but it does not verify that the output matches the required schema or exact prompt-specified structure.
  • If the JSON is not valid, we have a repair process where we feed the LLM the user prompt, an example of a previous value (with the correct output), the example of the incorrect JSON, and ask the LLM to correct the failing response.

I think this is a good start, but there’s more.

5

I measure AI progress in ‘dog years’. A year is almost like a decade. A month is like a year.

So several ‘decades ago’ (2 years approx) I was dead against using JSON with LLMs.

Back then, it was a good position to take; JSON needs mucho tokens (a forest of punctuation symbols) and has nested semantics (structures within structures).

And LLMs got their underwear in a bunch so easily with stuff like that.

So instead, we focused on using YAML (same capabilities, less semantic noise, similar to MD).
And users loved that because YAML is so much easier for humans to read.

But it turns out that YAML is more confusing for LLMs than JSON due to the invisibility of the indentations that are so essential to the structure of YAML.

In fact, we found pure MD to work much better (# vs ## vs ### and hyphen-bullets, etc).
The LLM gets better hints and had WAY more training data for this.

But NOW things are different. All the labs have invested a lot of training (pre- and post-) into handling JSON in their models. OpenAI has a specific API for enforcing JSON thinking.

JSON is so important to the world of IT that making LLMs good at it is as importent as making them good at JS etc.

So TODAY we use JSON for the parts that humans do not need to read or write.
We use a cross between MD and YAML for the schemas that user write (call it YAML-ish).

But the latest models EXCELL at JSON, and now that Superdocs AI has a specific output type for JSON, this has become our best mechanism for reliable output and memory.

The good old ‘secret’ formula _Merge(A,B) suddenly becomes our most powerful piece of CFL because it ‘merges’ the two JSON objects in a deterministic way - a full depth-ordered CRUD mechanism for keeping track of the cumulative outputs from all the inferences in an agentic workflow.

Many thanks to @Henrique_Cruz1 and his team for responding to our request for this feature so quickly. Day into night impact!

So I would suggest that @Bill_French’s position against the use of JSON in AI may have been superceesed by recent developments?

:lobster:Max