Rotifers, Microbeads and Algae

By Jord Liu and The Exploratorium

Background

This is the Machine Learning half of a larger project at the Exploratorium's Biology Lab called Seeing Scientifically, which is a research project that investigates how to use machine learning and other exhibit technology to best teach visitors in an informal learning context like the Exploratorium.

In this iteration of the project, we train an ML model to detect microscopic animals called rotifers, parts of their body (e.g. head, gut, jaw), and microbeads and algae in real time. This model is then integrated into a museum exhibit kiosk prototype that is deployed live on the Exploratorium's museum floor, and visitor research is collected on the efficacy of the exhibit.

Data and Model

The images used here are captured directly from a microscope feed and then labelled by Exploratorium employees and volunteers. Some include up to hundreds of microbeads or algae, some are brightfield and some are darkfield. They show rotifers in multiple poses, including some where the tails are not readily visible. There is relatively little variance in the images here as the environment is highly controlled. We use tiled data of multiple sizes mixed in with the full images.

We use YOLOv4, though future work includes retraining with YOLO-R, YOLO-v7, and other SOTA models. We also experimented with KeypointRCNN for pose estimation but found that the performance did not exceed our baseline of using YOLOv4 and treating the keypoints as objects.

Current performance by class is: class_id = 0, name = algae, ap = 64.29% (TP = 176, FP = 79) class_id = 1, name = bead, ap = 77.01% (TP = 251, FP = 41) class_id = 2, name = bigbead, ap = 82.46% (TP = 36, FP = 5) class_id = 3, name = egg, ap = 95.51% (TP = 16, FP = 4) class_id = 4, name = gut, ap = 82.55% (TP = 70, FP = 13) class_id = 5, name = head, ap = 78.38% (TP = 59, FP = 3) class_id = 6, name = mastics, ap = 86.82% (TP = 49, FP = 6) class_id = 7, name = poop, ap = 56.27% (TP = 34, FP = 15) class_id = 8, name = rotifer, ap = 72.60% (TP = 83, FP = 17) class_id = 9, name = tail, ap = 46.14% (TP = 27, FP = 7)

Examples

Screen captures from our exhibit as of July 2022.