ANPR/ALPR on the UP Embedded Vision Starter kit

While there's nothing new about Automatic Number Plate Recognition it has long been restricted to those applications where cost is no object. What is new is doing it reliably (if somewhat less quickly) on low-cost hardware, there are many many more applications for this tech if it can be delivered at a low enough cost. The Up Embedded Vision Starter Kit could be the ideal platform for such a system.

Project State

Public Project

Licences

Software Licence: GPL v.3
Hardware Licence: Project has no hardware

Project Tags

Admins

fish-beetle

Does this project pique your interest?

Login or register to join or follow this project.

I've already developed a reliable algorithm to read UK number plates, it's coded in python3 and uses openCV2 to do the heavy lifting. I have a working system based on an UPSquared board linked to an IP camera, but can it be made to work on an UP with a Basler usb camera?

As a hardware engineer and part-time python programmer I was not prepared to start coding in C or C++, so for the starter kit to be of any use I needed to get pypylon working. Pypylon is Basler's python wrapper for their pylon C++ camera API. To avoid a steep C++ learning curve and have any chance of success I had to get python3, openCV and pypylon working on the UP board:

From zero to Pypylon...

Download the ubilinux iso image from the up-community web site

Note ubuntu was not available at time of writing

Download rufus image-writer software

Use rufus to wite the iso image to a usb stick in UEFI / DD mode

Boot the up-board with the usb stick

Allow the os installation to proceed

Up board powers-down after installation

Once powered-down, remove the usb stick & re-start with the little white button

Note default pwd is ubilinux, change as required

$sudo apt-get update
$sudo apt-get upgrade

Reboot as required

Install Basler's pylon s/w direct from their web site:

Note goo.gl url in UP's paper documentation is broken

Download the tar ball and un-pack it

Follow the instructions in the README & INSTALL files

I needed a reboot at this stage

Now plug in the camera

Run Basler's PylonViewApp (find it in /opt/pylon5/bin)

Camera should appear under the USB devices list

Double click to connect to it

Grab a frame to test the camera

Use continuous grab mode to set-up the lens's focus

Close the application once test is complete

$sudo apt-get install pip3 # get python3's pip package installer
$sudo pip3 install opencv-python # also installs numpy

At github.com/basler/pypylon...

From the releases page download pypylon wheel for py3.5 on x86-64 linux

In my case filename was pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl

Also, from samples download or copy opencv.py

$sudo pip3 install pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl
$sudo apt-get install idle3
$idle3  # test the pypylon install

file->open opencv.py

F5 to run

The camera's live image appears on-screen, success!

Testing ANPR

OK, perfect, I have an openCV python3 development platform with a live image source from the Basler dart usb camera. Now to test the UKRegOCR number-plate identification & ocr code - this was developed earlier for an UPSquared board, but how well will it work on the lower-performance UP board?

Starting with the opencv.py code from github I added some lines to make it easier to work on as I like to edit on my Windows PC. Next I downloaded the UKRegOCR package from: http://www.marvellconsultants.co.uk/ANPR/UKRegOCR.zip and unpacked the files to the same directory. I simply added an import for the UKRegOCR module to opencv.py and a call to the lookForPlate() function on each frame grabbed from the camera. This function uses a well-trained Haar Cascade classifier to spot and locate number-plate-like objects in an image. Doing this slowed the frame-rate down to about 1 per second - much slower than on an UPSquared. Calling the ReadPlate() function to do the actual OCR process (on just the portion of the frame that has a plate in it) as expected slowed things down further to around 5-6 seconds per frame, about three times slower than an UPSquared but still faster than a RPi3. Oh well, that will maybe limit the range of applications that the UP platform could service but there's always the UPSquared for those jobs - the UP's advantage is its low cost.

See BaslCam1.py in Code section

After checking processor usage during the operation of the above code it became clear that the OCR function was only making use of one processor core. Maybe if I run the OCR code in separate threads then perhaps I can carry on grabbing frames at a steady rate while OCRing selected ones in the background. Ideally you'd read two or three images of the same plate as it passes to obtain a higher confidence in the read. Without threading, multiple 6 second reads would risk missing some vehicles during the time spent reading others.

Sure enough running 3 concurrent OCR threads brings processor usage up to 80-100%, and allows the grab and plate-classifier loop to cycle at <2 seconds per frame. Unfortunately the threaded code suffers from very occasional untraceable non-python 'std::out_of_range' RTEs. Reducing the OCR threads to a max of two seems to make this go away and results in an average frame rate of 40/min with a plate-read rate of 14/min. So usable in applications encountering up to about 5vehicles/min. This approach could make some applications feasible on the UP platform, and be of use for widening the range of applications viable on UPSquared.

See BaslCam_threaded.py in Code section

GitHub Repository

https://github.com/mcl-uk/ANPR_Embedded_vision

Commits

Steve Marchant
pushed c1b29c0d2c0b5d42cd0f92185dd5568847828186
Update README.md
2018-10-29 09:39:40 UTC
mcl-uk
pushed 79e3e259d5b8194c521d325f9d56c8005d982a21
Add files via upload
2018-10-28 18:26:49 UTC
mcl-uk
pushed bc886234218a3b5beba65fc7992caf4587684741
Merge pull request #1 from mcl-uk/readme-edits
2018-10-28 18:23:35 UTC
mcl-uk
pushed 9802ee95077e41092b56a1d1d90ce14cd22060f5
Update README.md
2018-10-28 18:18:32 UTC
mcl-uk
pushed c179f09e3683dce52360caa39204bf51b0f9979c
Update README.md
2018-10-28 18:17:33 UTC
mcl-uk
pushed 471b9b9d28ed43eb159f90ff794d1d89f4e00827
Initial commit
2018-10-28 18:04:00 UTC
Title Description Format
ANPR using UKRegOCR python module A python3 module to read UK number plates pdf
sample image sample jpg
sample image sample jpg
sample image sample jpg
1:1 zoom of M3XFL image zoom-in png

Comments

Ready to join the project?

You'd like to participate ... Show more