I wrote an entry a while back about a service that locks my car when I leave it parked at home. It has been running quietly since, doing its job, and I had more or less stopped thinking about it. Then I came back to the car at a clinic car park and found it sitting there unlocked.
Nothing happened. Nothing was taken, the car was fine, and the whole thing cost me only the specific unpleasant feeling of having built something that had let me down without telling me. But it is the first time one of my failures has had a consequence outside the house, and I have been chewing on it for a few days.
What the bug was
The service listens for a few different signals that mean I have arrived and stopped: the car reporting itself parked, the location saying I am home, and my phone dropping its Bluetooth connection to the car, which in practice is the earliest and most reliable sign that I have got out and walked away.
Those signals arrive close together, and early on I had a problem where a single arrival would trigger the whole chain several times over. So I did the obvious thing and added a debounce: once we have handled one of these, ignore anything else for two minutes.
Here is what that actually does. I drive off. Somewhere mid-drive a location check runs, sees the car is not at home, concludes correctly that there is nothing to do, and completes. That check starts the two minute timer. Then I arrive, park, get out, and the Bluetooth disconnect fires, which is the one signal in the whole system that actually means I have walked away from the car. It lands inside the timer and gets dropped on the floor.
So the debounce ate the only event that mattered, and it did so because I had written it to treat every signal as interchangeable noise. They are not interchangeable. One of them is a report that nothing needs doing and the others are a report that something does.
The fix, and the shape of the mistake
The debounce is gone. What replaced it treats a burst of signals as one arrival and resolves them together, rather than taking the first one and deafening itself to the rest. Same protection against the duplicate-fire problem, without discarding the evidence.
The reason I am writing this one up is that the bug is entirely characteristic and I expect to make it again in a different costume. I built a filter to suppress noise, at a moment when the problem in front of me was noise, and I never revisited whether everything it was suppressing was actually noise. It sat there being correct for weeks. The failure needed a specific ordering of events to appear, so it did not appear, until one day it did and the outcome was a car sitting unlocked in a public car park.
The rule I broke is one I already knew, and it is the same one from the entry about the front door. Anything with real consequences deserves either software I trust far more than my own, or a lot more paranoia than I gave this. I am keeping the car service, because the alternative is remembering to do it myself and I demonstrably do not, but it now shouts at me through the alert light when it cannot confirm a lock. It failing quietly is the part I am not willing to keep.