ANPR/ALPR on the UP Embedded Vision Starter kit

I've already developed a reliable algorithm to read UK number plates, it's coded in python3 and uses openCV2 to do the heavy lifting. I have a working system based on an UPSquared board linked to an IP camera, but can it be made to work on an UP with a Basler usb camera?

As a hardware engineer and part-time python programmer I was not prepared to start coding in C or C++, so for the starter kit to be of any use I needed to get pypylon working. Pypylon is Basler's python wrapper for their pylon C++ camera API. To avoid a steep C++ learning curve and have any chance of success I had to get python3, openCV and pypylon working on the UP board:

From zero to Pypylon...

Download the ubilinux iso image from the up-community web site

Note ubuntu was not available at time of writing

Download rufus image-writer software

Use rufus to wite the iso image to a usb stick in UEFI / DD mode

Boot the up-board with the usb stick

Allow the os installation to proceed

Up board powers-down after installation

Once powered-down, remove the usb stick & re-start with the little white button

Note default pwd is ubilinux, change as required

$sudo apt-get update
$sudo apt-get upgrade

Reboot as required

Install Basler's pylon s/w direct from their web site:

Note goo.gl url in UP's paper documentation is broken

Download the tar ball and un-pack it

Follow the instructions in the README & INSTALL files

I needed a reboot at this stage

Now plug in the camera

Run Basler's PylonViewApp (find it in /opt/pylon5/bin)

Camera should appear under the USB devices list

Double click to connect to it

Grab a frame to test the camera

Use continuous grab mode to set-up the lens's focus

Close the application once test is complete

$sudo apt-get install pip3 # get python3's pip package installer
$sudo pip3 install opencv-python # also installs numpy

At github.com/basler/pypylon...

From the releases page download pypylon wheel for py3.5 on x86-64 linux

In my case filename was pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl

Also, from samples download or copy opencv.py

$sudo pip3 install pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl
$sudo apt-get install idle3
$idle3  # test the pypylon install

file->open opencv.py

F5 to run

The camera's live image appears on-screen, success!

Testing ANPR

OK, perfect, I have an openCV python3 development platform with a live image source from the Basler dart usb camera. Now to test the UKRegOCR number-plate identification & ocr code - this was developed earlier for an UPSquared board, but how well will it work on the lower-performance UP board?

Starting with the opencv.py code from github I added some lines to make it easier to work on as I like to edit on my Windows PC. Next I downloaded the UKRegOCR package from: http://www.marvellconsultants.co.uk/ANPR/UKRegOCR.zip and unpacked the files to the same directory. I simply added an import for the UKRegOCR module to opencv.py and a call to the lookForPlate() function on each frame grabbed from the camera. This function uses a well-trained Haar Cascade classifier to spot and locate number-plate-like objects in an image. Doing this slowed the frame-rate down to about 1 per second - much slower than on an UPSquared. Calling the ReadPlate() function to do the actual OCR process (on just the portion of the frame that has a plate in it) as expected slowed things down further to around 5-6 seconds per frame, about three times slower than an UPSquared but still faster than a RPi3. Oh well, that will maybe limit the range of applications that the UP platform could service but there's always the UPSquared for those jobs - the UP's advantage is its low cost.

See BaslCam1.py in Code section

After checking processor usage during the operation of the above code it became clear that the OCR function was only making use of one processor core. Maybe if I run the OCR code in separate threads then perhaps I can carry on grabbing frames at a steady rate while OCRing selected ones in the background. Ideally you'd read two or three images of the same plate as it passes to obtain a higher confidence in the read. Without threading, multiple 6 second reads would risk missing some vehicles during the time spent reading others.

Sure enough running 3 concurrent OCR threads brings processor usage up to 80-100%, and allows the grab and plate-classifier loop to cycle at <2 seconds per frame. Unfortunately the threaded code suffers from very occasional untraceable non-python 'std::out_of_range' RTEs. Reducing the OCR threads to a max of two seems to make this go away and results in an average frame rate of 40/min with a plate-read rate of 14/min. So usable in applications encountering up to about 5vehicles/min. This approach could make some applications feasible on the UP platform, and be of use for widening the range of applications viable on UPSquared.

See BaslCam_threaded.py in Code section