Field Notes · IIoT · Industrial controls
Dispatches from the intersection of industrial control systems, IIoT architecture, and biolab experiments. These are the notes I wish existed when I was debugging SCADA at 2 am — written close to the metal, after the hardware answers back.
Field Notes from a Bioreactor — read the series in order →
-
The Recovery That Wasn’t
A settled culture fooled the sensor. The OD ‘recovery’ from 0.0084 to 0.0137 turned out to be stir-burst transients and an air bubble, not growth. The real density is flat and low, the cooling fix is holding, and…
-
Watching the Watchdog: Nine Months of UPS Data, a Third of It Missing
The UPS was fine. The grid was fine. The monitoring was the thing that kept dying — a third of nine months of telemetry simply missing, a firewall that only let one host’s replies through, and a shutdown…
-
Back in the Band — the Cooling Crisis Is Resolved
It turns out the vessel didn’t need a thermoelectric cooler or an HVAC retrofit — it needed a $20 fan. The afternoon peak fell from 32.1°C to 24.1°C overnight, the culture is finally living inside its 19–22°C safe…
-
The Control That Wouldn’t Flash
A positive control built to prove the camera could see a real dinoflagellate flash instead proved something worse: it can’t. The gate that separates flash from noise held at 0.288 against a required 0.5 — even with a…
-
Two Thresholds — The Cells Survive the Heat, the Glow Does Not
Eighteen days of sustained heat looked like a slow-motion disaster — until the data split it into two thresholds seven degrees apart. The cells are surviving. The bioluminescence isn’t. That reframes what the cooling fix is actually for.
-
The Letterbox Is an Attack Surface: Going Paperless with an AI Mail Clerk
Scanning is a tenth of going paperless. The rest: OCR baked into the PDFs rather than an index, a search service that was quietly eating a third of the NAS, and an AI that reads the mail —…
-
The /usr/bin/bash Cooling Fix Worked
The Pi came out from under the vial, a fan went across the assembly, and the overnight floor dropped three degrees — from a stubborn 27–28°C to a stable 24.5°C. A zero-cost fix, the culture still isn’t in…
-
The Pioreactor Rebooted and Broke the Circadian Clock
A hardware watchdog silently rebooted the Pioreactor mid-afternoon, killing the grow light for fifteen minutes and nearly re-anchoring the culture’s circadian clock eight hours off schedule. Here’s the forensics on a 5.20V under-voltage fault, the restore that almost…
-
The Pioreactor Was Cooking Its Own Culture
For a month I called the Pioreactor’s heat an external given — the garage in August, a reactor with no cooling side. A 96-hour diagnostic proved me wrong: the vessel’s overnight floor never falls, the evening peak lands…
-
The Culture Refused to Stay Dead — OD Rose Again
I called the culture dead last week from four days of flat OD. It wasn’t flat — it was climbing all along, +16% over ten days under lethal heat. Here’s how I tested whether that rise is real…
New posts are infrequent — when something worth writing happens in the lab or the rack room.