About AquaMind

AquaMind is a methodological proof of concept that tests whether computer vision, geometry-based rules and machine learning can together track zebrafish consistently in a cluttered side-view tank and turn their movements into measured behaviour.

It is built as a chain, one stage per link: detect, track, keep identity, measure, classify, deploy. See the objectives.

Aqua, for the water. Mind, for what I wanted to place on the other side of the glass: an observer that learns, records every single move, and never blinks, tires, or looks away.

For years, that observer was me, and I blinked.

AquaMind in action, following zebrafish in a planted side-view tank. Each fish keeps its own ID and colour, glass reflections are ignored, the status panel flags crossings and potential identity swaps, and the chasing classifier scores every pair of fish.

Why I built it

During my PhD in behavioural ecology, watching coral reef fish took countless hours: eyes glued to a small tank, notebook balanced on my knee, four mechanical clickers in hand, counting every turn and strike. It demanded an almost meditative focus for days, weeks, and months on end, knowing a single second of distraction could introduce noise into the data or ruin an entire trial. Even at night, when I closed my eyes, I still saw them darting around like little Pac-Men.

Much of the field I trained in rests on that relentless manual observation. One of my own studies showed that underfed fish in slightly warmer water stop reacting to the scent of danger (Lienart et al. 2014), an answer that, like so many others, came from someone sitting in a chair with clickers in hand. I couldn’t stop thinking that a computer should be doing this: scoring every trial the same way, never tiring, and following every fish frame by frame in more detail than the human eye can catch. Back then, getting a program to reliably spot even one small reef fish, half hidden behind a piece of coral or its shelter, seemed out of reach.

Govinda Lienart leaning over a row of open experimental fish tanks under a shade roof at a field station on Lizard Island, looking down into the water. — FigWatching fish by eye during my PhD (James Cook University). Left: checking experimental tanks on Lizard Island, on the Great Barrier Reef. Top right: replicate tanks seen side-on, a grid on each front glass for scoring movement. Bottom right: observing from behind a blackout curtain, so the fish do not react to the observer.

Two rows of identical glass experimental tanks seen from the side under a shade roof, each with a black backing and a grid drawn on the front glass, fed by water pipes from above. — FigWatching fish by eye during my PhD (James Cook University). Left: checking experimental tanks on Lizard Island, on the Great Barrier Reef. Top right: replicate tanks seen side-on, a grid on each front glass for scoring movement. Bottom right: observing from behind a blackout curtain, so the fish do not react to the observer.

After my PhD, I stepped away from that kind of observation for over a decade. I kept working on animal behaviour and field surveys, but counting fish in the wild is a world away from tracking their intimate micro-behaviours up close. Then large language models (LLMs) arrived. Fascinated by their potential, I wanted to understand what was happening under the hood and dive deeper into machine learning. In 2025, I enrolled in a postgraduate AI programme in Brussels, but it was mostly theoretical, and I missed real hands-on application. So in 2026, I bought a tank, filled it with zebrafish, a small freshwater fish that is one of the most widely used animals in lab research, and started filming.

AquaMind became my practical training ground. I picked a notoriously difficult setup on purpose: a side-view tank. While the side perspective is widely used in fish research, revealing swimming depth, posture, and feeding strikes, it also introduces complex reflections, hiding spots behind plants, and moments where multiple fish merge into a single silhouette as they cross paths. On top of that, zebrafish are so identical they could be twins. Could computer vision, machine learning, and geometry-based rules follow every individual, maintain identity, and reliably flag complex interactions like chases or strikes?

AquaMind is not a new scientific discovery; it is a proof of concept and a methodological framework. I wanted to see if someone with a biology background could build the entire pipeline from raw video to quantitative behaviour, tailored to this exact environment. With open-source libraries, pretrained models, and an LLM acting as a coding mentor, that level of custom engineering has finally come within reach for biologists.

In my experience, LLM-assisted coding can be both a blessing and a curse. Generate too much code too quickly, and a multi-step computer vision pipeline turns into Frankenstein’s monster: a system you built but no longer understand, where silent errors spread across modules and technical debt piles up. Used deliberately, however, an LLM acts as a patient teacher for everything from high-level network architecture down to a single missing bracket. That is how I built AquaMind. I first used it as a machete to hack through a dense jungle of possibilities. Then I cleared a clean, structured path myself: reviewing every line of code, rewriting it by hand, changing whatever did not fit, and testing it until I could explain and defend each decision.

I hope AquaMind serves as more than just a portfolio piece. If it inspires another biologist to take the leap into computer vision and machine learning, it will have achieved its goal. The code is public, and this site explains the reasoning behind every step, including what did not work. The aims follow below.

Objectives

The objectives follow the chain, one per link, from raw video to a deployed classifier:

Detect every fish in every frame, and tell fish from their reflections.
Track each fish through occlusion and crossings without breaking its path.
Keep identity of each fish after paths cross, even between look-alikes.
Measure activity, depth and distance between fish from the tracks.
Classify chases and feeding strikes.
Deploy a trained classifier as a service that others can call.

Throughout, each link is evaluated against hand-labelled data, so errors are measured, not assumed, and every result is traceable to the data, code and model that produced it. Each stage records what its method contributes and where it falls short.

The stages

Stage	What it does
1 · Frame extraction	Samples each video at one frame per second into MySQL
2 · Annotation	Hand-drawn boxes in Label Studio, stored as relational rows
3 · Object detection	Fine-tunes YOLOv8; DVC for data, MLflow for models
4 · Custom tracker	Gives each fish a persistent ID through occlusion and crossings
5 · Improving the detector	Relabels the frames the tracker struggles with and retrains
6 · Re-identification	Learns an appearance embedding for each fish
7 · Chasing detection	Per-fish and pairwise kinematics, then classifiers
8 · Feeding-strike detection	Kinematic baselines versus a CNN-LSTM on image crops
9 · Pipeline & deployment	Packages a trained classifier behind an API

The Architecture shows how the stages connect.