Project information

• Category: Artificial Intelligence, Vision
• Project Tools: Python, OpenCV, Label Studio, GIS
• Project date: March, 2023
• Project URL: Github Repository

Project description

This project is my bachelor's thesis, and I defended it with a 20/20 grade. This project aims to develop a smart system to manage the traffic signs in a street. To be more precise, this system takes continuous images from the road, then detects traffic signs in the picture and calculates the distance between the detected sign and the camera (which is used to take pictures). Now, with the GPS information for each image (same as the camera's GPS data) and the distance between detected signs and the camera, the system estimates the approximate GPS location of traffic signs. Consequently, we can demonstrate the location of each type of traffic sign on a map, enabling municipalities to manage city traffic signs.

Different parts of the project are as below:

- Collect a dataset: We collected about 6000 images from the Tehran streets and labeled them using Label Studio. You can find the dataset in Kaggle.
- Fine-tune pre-trained model: We decided on Faster-RCNN as our object detection model and fine-tuned it using our dataset. To do so, we split our dataset into train and valid parts with an 80% to 20% ratio. We used GPUs provided by Google Colab to train our model.
- Camera Calibration: To estimate the distance between an object in an image and the camera, we need to calibrate our camera. We used the chessboard pattern of the OpenCV library to calibrate the camera.
- Distance Estimation: We used Solve-PnP (one of the PnP distance estimation algorithms) to estimate the distance between the object and the camera. The methodology is to match the two-dimensional coordinates of a point with the three-dimensional coordinates of the same point, and in this way, calculate the distance between the object and the camera. Solve-PnP does this for four points and outputs rotation and transition matrices.
- Determining the geographic coordinates of detected objects: We need to convert the distance in metric scale to the geographic scale. We used the Haversine formula to reach this goal.
- Visualize the results: The last step is to visualize detected traffic signs on a map. We used Google's map in GIS software and visualized the system output.