While there's nothing new about Automatic Number Plate Recognition it has long been restricted to those applications where cost is no object. What is new is doing it reliably (if somewhat less quickly) on low-cost hardware, there are many many more applications for this tech if it can be delivered at a low enough cost. The Up Embedded Vision Starter Kit could be the ideal platform for such a system.
I've already developed a reliable algorithm to read UK number plates, it's coded in python3 and uses openCV2 to do the heavy lifting. I have a working system based on an UPSquared board linked to an IP camera, but can it be made to work on an UP with a Basler usb camera?
As a hardware engineer and part-time python programmer I was not prepared to start coding in C or C++, so for the starter kit to be of any use I needed to get pypylon working. Pypylon is Basler's python wrapper for their pylon C++ camera API. To avoid a steep C++ learning curve and have any chance of success I had to get python3, openCV and pypylon working on the UP board:
From zero to Pypylon...
Download the ubilinux iso image from the up-community web site
Note ubuntu was not available at time of writing
Download rufus image-writer software
Use rufus to wite the iso image to a usb stick in UEFI / DD mode
Boot the up-board with the usb stick
Allow the os installation to proceed
Up board powers-down after installation
Once powered-down, remove the usb stick & re-start with the little white button
Note default pwd is ubilinux, change as required
$sudo apt-get update $sudo apt-get upgrade
Reboot as required
Install Basler's pylon s/w direct from their web site:
Note goo.gl url in UP's paper documentation is broken
Download the tar ball and un-pack it
Follow the instructions in the README & INSTALL files
I needed a reboot at this stage
Now plug in the camera
Run Basler's PylonViewApp (find it in /opt/pylon5/bin)
Camera should appear under the USB devices list
Double click to connect to it
Grab a frame to test the camera
Use continuous grab mode to set-up the lens's focus
Close the application once test is complete
$sudo apt-get install pip3 # get python3's pip package installer $sudo pip3 install opencv-python # also installs numpy
From the releases page download pypylon wheel for py3.5 on x86-64 linux
In my case filename was
Also, from samples download or copy opencv.py
$sudo pip3 install pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl $sudo apt-get install idle3 $idle3 # test the pypylon install
F5 to run
The camera's live image appears on-screen, success!
OK, perfect, I have an openCV python3 development platform with a live image source from the Basler dart usb camera. Now to test the UKRegOCR number-plate identification & ocr code - this was developed earlier for an UPSquared board, but how well will it work on the lower-performance UP board?
Starting with the opencv.py code from github I added some lines to make it easier to work on as I like to edit on my Windows PC. Next I downloaded the UKRegOCR package from: http://www.marvellconsultants.co.uk/ANPR/UKRegOCR.zip and unpacked the files to the same directory. I simply added an import for the UKRegOCR module to opencv.py and a call to the lookForPlate() function on each frame grabbed from the camera. This function uses a well-trained Haar Cascade classifier to spot and locate number-plate-like objects in an image. Doing this slowed the frame-rate down to about 1 per second - much slower than on an UPSquared. Calling the ReadPlate() function to do the actual OCR process (on just the portion of the frame that has a plate in it) as expected slowed things down further to around 5-6 seconds per frame, about three times slower than an UPSquared but still faster than a RPi3. Oh well, that will maybe limit the range of applications that the UP platform could service but there's always the UPSquared for those jobs - the UP's advantage is its low cost.
See BaslCam1.py in Code section
After checking processor usage during the operation of the above code it became clear that the OCR function was only making use of one processor core. Maybe if I run the OCR code in separate threads then perhaps I can carry on grabbing frames at a steady rate while OCRing selected ones in the background. Ideally you'd read two or three images of the same plate as it passes to obtain a higher confidence in the read. Without threading, multiple 6 second reads would risk missing some vehicles during the time spent reading others.
Sure enough running 3 concurrent OCR threads brings processor usage up to 80-100%, and allows the grab and plate-classifier loop to cycle at <2 seconds per frame. Unfortunately the threaded code suffers from very occasional untraceable non-python
'std::out_of_range' RTEs. Reducing the
OCR threads to a max of two seems to make this go away and results in an average frame rate of 40/min with a plate-read rate of 14/min. So usable in applications encountering up to about 5vehicles/min. This approach could make some applications feasible on the UP platform, and be of use for widening the range of applications viable on UPSquared.
See BaslCam_threaded.py in Code section
|Steve Marchant||pushed c1b29c0d2c0b5d42cd0f92185dd5568847828186||2018-10-29 09:39:40 UTC|
|mcl-uk||pushed 79e3e259d5b8194c521d325f9d56c8005d982a21||2018-10-28 18:26:49 UTC|
|mcl-uk||pushed bc886234218a3b5beba65fc7992caf4587684741||2018-10-28 18:23:35 UTC|
|mcl-uk||pushed 9802ee95077e41092b56a1d1d90ce14cd22060f5||2018-10-28 18:18:32 UTC|
|mcl-uk||pushed c179f09e3683dce52360caa39204bf51b0f9979c||2018-10-28 18:17:33 UTC|
|mcl-uk||pushed 471b9b9d28ed43eb159f90ff794d1d89f4e00827||2018-10-28 18:04:00 UTC|