Why I made it
A research lab at a Swiss university studies how cells deal with bacteria. They film cells under a microscope for about two hours, one picture every 30 seconds, with bacteria that glow. Each time a cell takes in a bacterium, they want to know when it went in, and whether the cell broke it down or still had it at the end of the film.
Until now, someone sat with each film and scored it by hand, frame by frame, in a spreadsheet. One film is a few hundred frames with hundreds of cells, so that takes hours. It looked like work a computer should do, so I built it.
How it works
You open a film and press Run Analysis. The app goes through four steps:
- Detect the bacteria in every frame, by their bright core.
- Follow them from frame to frame, through short gaps and crossings.
- Find the cells, and keep track of each one for the whole film.
- Call the events. A bacterium that goes inside a cell starts an event. If it disappears, that's a kill, timed from the frame it went in. If it's still there when the film ends, it's a failed kill.
Then a researcher goes through the events, starting with the clearest ones. They can confirm, reject, move the start or end frame, split one event in two, or add one the app missed. Their review is saved next to the film, and the result is an Excel file with every event.
Under the hood it's Python. The cells are found by a deep learning model (Cellpose). Running it on every frame took too long on a normal Mac, so it runs on every third frame and the frames in between reuse the nearest result: the frames are 30 seconds apart and a cell only moves a couple of pixels in that time. The bacteria are found with classic image processing and linked into tracks with trackpy. A set of rules decides what counts as an event, for example when two bacteria are in the same cell and you can't tell which one was killed.
Working with the researchers
The app only helps if it fits the way the lab already works. The Excel file has the same columns as the sheet they used to fill in by hand, plus the outcome as a number they can count and average. The settings use their words, like "failed kill" and "shared cells", not mine.
Most of the work came from their feedback. They sent films and told me where the app was wrong: an event split in two because a bacterium went out of focus for a moment, a failed kill missed in a crowded part of the film, a bacterium that only passed by a cell counted as eaten. Each of those became a rule or a setting, and a new version for them to try.
| Event | Frame start | Frame end | Duration (frames) | Duration (min) | Outcome | Outcome code |
|---|---|---|---|---|---|---|
| 1 | 12 | 27 | 15 | 7.5 | Killed | 1 |
| 2 | 31 | 241 | 210 | 105.0 | Failed kill | 0 |
| 3 | 48 | 59 | 11 | 5.5 | Killed | 1 |
Measuring, not guessing
With image analysis it's easy to change a setting, look at one frame and decide it looks better. I wanted numbers instead. I made a reference set of frames with every cell outlined, and every version of the app is scored against it: how many cells it finds, how many it invents, and how well the outlines match.
That changed the decisions. My first approach used classic image processing, and tuning it with the reference set raised its score from 0.61 to 0.78. Measuring also showed why it still went wrong: two touching cells were often seen as one, because one of them never got a starting point. Fixing that halved the number of merged cells. In the end the numbers showed that a small deep learning model was clearly better, and fast enough when it only runs on every third frame. That is what the app uses now.
Every Excel file also records which version of the app made it, so any result can be traced back to the exact code that counted it.
Shipping it properly
The researchers aren't programmers, so it had to install like any other Mac app. It's signed and notarised by Apple, so it opens without a security warning, and there are builds for both Apple silicon and the older Intel Macs the lab actually has.
Everything runs on their own Mac. No film, image or result ever leaves it, which matters for unpublished research. The app checks its licence offline when it starts, and a small server on Cloudflare only hands out renewed licences, so if it's down nobody is blocked. The only thing it sends is anonymous usage counts, and the product site says exactly what and why.
The lab is now using it on their own films.