Skip to content
Atomic WaxBook an AssessmentBook

← Field Notes

The CMS said 25 fields. The table had 27. Nobody could save the homepage.

Peter Benes ·

On August 28 nobody could save an edit to the homepage of sourcedpr.com, our PR and sourcing service. Not the field being edited, not anything. Every autosave in the EmDash admin failed with the same message, naming two fields:

operation_friends_heading: unknown field on collection 'homepage'

and the same for operation_friends_body.

Those fields belonged to a section removed about two weeks earlier. The section was gone from the page. The admin was still choking on it.

Two sources of truth

EmDash, the CMS we build Atomic Wax sites on, keeps two records of every collection. One is the physical table in Cloudflare D1, with one column per field. The other is the field registry, _emdash_fields, which lists the fields the admin will accept.

On August 13 a session removed that section. It deleted the two field rows from the registry. It never dropped the two columns from the table, ec_homepage.

The editor loads an entry by reading the whole row, then posts every key back when it saves. The server checks each key against the registry, finds two it doesn’t recognize, and rejects the entire write. The registry said 25 fields. The table had 27 content columns. The difference was exactly the two orphans.

EmDash’s schema sync is additive. It adds columns when you add fields. It does not drop them when you delete fields. That is a reasonable default, since dropping a column destroys data, but it makes removal a two-step job, and only one step had happened.

What it was not

The obvious suspects were a stale browser tab or an old draft. We ruled both out with evidence rather than a hunch. The only saved revision dated from July 25 and did not mention the old section at all. The keys were coming from the live row.

The fix

Both columns held empty strings, and nothing else in the table depended on them: no index, no unique constraint, no foreign key. So we dropped them with two ALTER TABLE ... DROP COLUMN statements. Afterward there were zero orphan columns, all 14 indexes were unchanged, the live routes still returned 200, and the homepage saved again.

The second orphan, which we left alone

A sweep across all ten collections found one more pair: business and website on the contact messages collection. Same failure waiting to happen, but it needed the opposite fix.

That collection holds real form submissions, and those columns may contain what people typed into the form. Dropping them could delete real data. The right remedy is to register the two fields, so the registry and the table agree.

The rule we wrote down: drop the column when the field was retired; register the field when the column holds real data.

The same family, four days earlier

On a client build on August 24 we hit a cousin of this bug. A custom field named published_at collided with a column EmDash already uses for itself. Seeding the content returned HTTP 200 and quietly stopped partway through. Renaming the field to post_date fixed it. Different trigger, same lesson: the schema has rules that a success message will not tell you about.

Also found

While we were in there, we noticed the one stored revision of the homepage was stale. Restoring it from the admin would have silently reverted three content changes that were live. Nothing was broken yet. It is the kind of button someone presses in a hurry.

The leave-behind

Column-to-field parity is now a check, not a task. One SQL query per EmDash site compares each table’s columns with the registry and lists anything present in one but not the other. It costs nothing to run, and the fix it points to depends on which side is wrong.

It is the same lesson as our deploy-drift check on kpixies.com. We had already stopped trusting status files over probes. We were still trusting a completed migration over the schema it actually left behind.

What this means if you hire us

A site that runs itself only works if its editor works, so our checks cover the plumbing behind the admin, not just the pages in front of it. When we build on EmDash and Cloudflare D1, schema drift is something we look for on purpose. How it works shows where those checks sit in a build.