A proof-of-concept solution based on the Computer Vision model for automatic number plate recognition in real-time. It is useful for automating control of vehicle entry on the company’s territory, paying for toll roads or parking, and increasing road safety.

Challenge

A license plate recognition field provides us with a wide range of capabilities, including automatic car license plate recognition. Such software is widely used not only by traffic police for road safety. It also could be seamlessly integrated into car park operators’ workflow and help automatically track car entry and exit in areas, manage parking lots, and prevent fraudulent behavior. As a bonus, it is also convenient for drivers — they don’t have to use tickets for parking anymore.

Hypothesis

We’ve developed an automatic license plate recognition PoC allowing us to automate car parking and reduce operational costs by decreasing the number of employees who monitor it. It reads a license plate number on the movement and even if the car is turned to the camera at an angle detecting a vehicle, recognizing the license plate on a car, and determining the text on a license plate.

Throughout the project, we used a DIET approach. It helps us to give clients valuable outcomes in every step of the project which could be a result of research, the main working feature, or a feasible ready version of the product.

Hypothesis research

Discovery

We started with analyses of the main task — recognition of the car license plate on the go and in real-time, considering that the car won’t be in the frontal position most frequently. Then we divided it into three subtasks: car detection, identification of the license plate, and text recognition.

We analyzed the existing approaches and chose pre-trained neural networks as the most optimal and well-known solution for these subtasks. Also, we specified the conditions in which the AI license plate recognition should work, including clarity and quality of shooting in various weather conditions and camera positioning.

Ideation

At the Ideation stage, we analyzed open source datasets for training the license plate detection model, as well as ready-made solutions to perform tasks at each step of process: car detection, identification of the license plate, and text recognition. We concluded that most of existing approaches are not suitable for us because it focused on a specific license plate region (e.g., European, US, Brazilian, Taiwanese, etc.), frequently explore datasets containing approximately frontal images, and detect a license plate only when a car isn’t moving.

After research, we were ready for the development of the a proof-of-concept (PoC) solution. During the work, we faced a challenge related to possible license plate distortion due to camera angle. In order to avoid it, we used Warped Planar Object Detection Network allowing us to transform a detected lisence plate into a rectangle as if looking at a frontal view.

Also, we had to deal with the low video quality and errors in the license text recognition. So, for better accuracy, we designed and used special functions to do an elementary check of the detected text and remove bounding box jumps in video license plate recognition.

By the Experiment stage, our solution is prepared for working with customer data and particular requirements. Also, we would make model accuracy improvements and the development of algorithms evaluating the confidenсе level, e.g., the probability the model relates the recognized letter or number to some class “a,” “b,”…”z,” 0,1,2,3..9, etc.

The Transformation stage would provide a customer with the implementation of the AI car number plate recognition model into the existing program environment, which further drives for monitoring automation and optimization of business processes.