MADA-driver-assistant-redu Computer Vision Project

MADA (Multimedia Agentic Driver Assistant) project's goal is having a minimum, but as functional as possible, driver assistant that works outdoor in real-time. Next link shows the MADA blocks.

MADA's Object Detector is based on a yolo model trained with COCO 2017 and fine-tuned with a dataset containing 23 classes from 4 categories:

• Vehicles: 'bicycle', 'bus', 'car', 'motorcycle', 'truck'
• People: 'person'
• Traffic Signs: 'construction', 'cycles crossing', 'dead end street', 'give way', 'go left', 'go right', 'no entry', 'no left turn', 'no overtaking', 'no priority', 'no right turn', 'pedestrian crossing', 'roundabout', 'school crossing', 'speed limit', 'stop'
• Traffic Lights: 'traffic light'

The images and initial annotations were selected from several datasets:

• Microsoft's COCO (Common Objects in COntext): includes more than 80 general image types, from which a few are useful for MADA: person, car, bus, bicycle, truck, motorcycle.
• GTSDB (German Traffic Sign Detection Benchmark): composed of images and annotations of more than 40 traffic sign classes, and some of them have been selected for MADA: speed limits, stop, give way, roundabout, pedestrian crossing...
• DFG-TSD (DFG Traffic Sign Dataset; DFG is a Slovenian company): includes more than 200 traffic sign classes, some of which have been selected to complement those less represented in GTSDB or not included: some speed limits, dead end street, no left turn, no right turn, no priority.
• S2TLD (SJTU Small Traffic Light Detection; SJTU is Shanghai Jiao Tong University): provides traffic light images and annotations, with separate types for red, green and yellow lights (finally I decided to merge them in a single traffic light class).

Since images from each of the datasets might have non-annotated instances of classes considered only in another dataset (for instance, the images in the S2TLD dataset have only annotations of traffic lights, but there are also non-annotated cars, crossing people and traffic signs), I had to review the whole dataset to add annotations (labels and bounding boxes) for the non-annotated instances. The merging of images and annotations (and the additions) from different datasets was done in the Roboflow platform.

References

COCO dataset:

• official website
• paper:

- @article{lin2014microsoft,
  title={Microsoft COCO: Common Objects in Context},
  author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C. Lawrence},
  journal={arXiv preprint arXiv:1405.0312},
  year={2014}
}

GTSDB dataset:

@inproceedings{Houben-IJCNN-2013,
   author = {Sebastian Houben and Johannes Stallkamp and Jan Salmen and Marc Schlipsing and Christian Igel},
   booktitle = {International Joint Conference on Neural Networks},
   title = {Detection of Traffic Signs in Real-World Images: The {G}erman {T}raffic {S}ign {D}etection {B}enchmark},
   number = {1288},
   year = {2013},
}

DFG-TSD dataset:

@article{Tabernik2019ITS,
   title = {{Deep Learning for Large-Scale Traffic-Sign Detection and Recognition}},   
   author = {Tabernik, Domen and Sko{\v{c}}aj, Danijel},
   journal = {IEEE Transactions on Intelligent Transportation Systems},
   year = {2019},
   doi={10.1109/TITS.2019.2913588}, 
   ISSN={1524-9050}
}

S2TLD dataset:

@article{yang2022scrdet++,
  title={Scrdet++: Detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing},
  author={Yang, Xue and Yan, Junchi and Liao, Wenlong and Yang, Xiaokang and Tang, Jin and He, Tao},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022},
  publisher={IEEE}
}

More details and code in MADA's GitHub repo.