While there's nothing new about Automatic Number Plate Recognition it has long been restricted to those applications where cost is no object. What is new is doing it reliably (if somewhat less quickly) on low-cost hardware, there are many many more applications for this tech if it can be delivered at a low enough cost. The Up Embedded Vision Starter Kit could be the ideal platform for such a system.
I've already developed a reliable algorithm to read UK number plates, it's coded in python3 and uses openCV2 to do the heavy lifting. I have a working system based on an UPSquared board linked to an IP camera, but can it be made to work on an UP with a Basler usb camera?
As a hardware engineer and part-time python programmer I was not prepared to start coding in C or C++, so for the starter kit to be of any use I needed to get pypylon working. Pypylon is Basler's python wrapper for their pylon C++ camera API. To avoid a steep C++ learning curve and have any chance of success I had to get python3, openCV and pypylon working on the UP board:
From zero to Pypylon...
Download the ubilinux iso image from the up-community web site
Note ubuntu was not available at time of writing
Download rufus image-writer software
Use rufus to wite the iso image to a usb stick in UEFI / DD mode
Boot the up-board with the usb stick
Allow the os installation to proceed
Up board powers-down after installation
Once powered-down, remove the usb stick & re-start with the little white button
Note default pwd is ubilinux, change as required
$sudo apt-get update
$sudo apt-get upgrade
Reboot as required
Install Basler's pylon s/w direct from their web site:
Note goo.gl url in UP's paper documentation is broken
Download the tar ball and un-pack it
tar -xzf pylon-5.1.0.12682-linux-x86_64.tar.gz
cd pylon-5.1.0.12682-x86_64
Follow the instructions in the README & INSTALL files
sudo tar -C /opt -xzf pylonSDK-5.1.0.12682-linux-x86_64.tar.gz
./setup-usb.sh
Follow the instructions in the README & INSTALL files
I needed a reboot at this stage
Now plug in the camera
Run Basler's PylonViewerApp (find it in /opt/pylon5/bin)
Camera should appear under the USB devices list
Double click to connect to it
Grab a frame to test the camera
Use continuous grab mode to set-up the lens's focus
Close the application once test is complete
$sudo apt-get install python3-pip # get python3's pip package installer
$sudo pip3 install opencv-python # also installs numpy
At github.com/basler/pypylon...
From the releases page download pypylon wheel for py3.5 on x86-64 linux
In my case filename was pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl
Also, from samples download or copy opencv.py
$sudo pip3 install pypylon-1.3.1-cp35-cp35m-linux_x86_64.whl
$sudo apt-get install idle3
$idle3 # test the pypylon install
file->open opencv.py
F5 to run
The camera's live image appears on-screen, success!
Testing ANPR
OK, perfect, I have an openCV python3 development platform with a live image source from the Basler dart usb camera. Now to test the UKRegOCR number-plate identification & ocr code - this was developed earlier for an UPSquared board, but how well will it work on the lower-performance UP board?
Starting with the opencv.py code from github I added some lines to make it easier to work on as I like to edit on my Windows PC. Next I downloaded the UKRegOCR package from: http://www.marvellconsultants.co.uk/ANPR/UKRegOCR.zip and unpacked the files to the same directory. I simply added an import for the UKRegOCR module to opencv.py and a call to the lookForPlate() function on each frame grabbed from the camera. This function uses a well-trained Haar Cascade classifier to spot and locate number-plate-like objects in an image. Doing this slowed the frame-rate down to about 1 per second - much slower than on an UPSquared. Calling the ReadPlate() function to do the actual OCR process (on just the portion of the frame that has a plate in it) as expected slowed things down further to around 5-6 seconds per frame, about three times slower than an UPSquared but still faster than a RPi3. Oh well, that will maybe limit the range of applications that the UP platform could service but there's always the UPSquared for those jobs - the UP's advantage is its low cost.
See BaslCam1.py in Code section
After checking processor usage during the operation of the above code it became clear that the OCR function was only making use of one processor core. Maybe if I run the OCR code in separate threads then perhaps I can carry on grabbing frames at a steady rate while OCRing selected ones in the background. Ideally you'd read two or three images of the same plate as it passes to obtain a higher confidence in the read. Without threading, multiple 6 second reads would risk missing some vehicles during the time spent reading others.
Sure enough running 3 concurrent OCR threads brings processor usage up to 80-100%, and allows the grab and plate-classifier loop to cycle at <2 seconds per frame. Unfortunately the threaded code suffers from very occasional untraceable non-python 'std::out_of_range'
RTEs. Reducing the
OCR threads to a max of two seems to make this go away and results in an average frame rate of 40/min with a plate-read rate of 14/min. So usable in applications encountering up to about 5vehicles/min. This approach could make some applications feasible on the UP platform, and be of use for widening the range of applications viable on UPSquared. Running the same script on an UPSquared Pentium N4200 board the average frame rate was 36/min but
with a plate-read rate of 29/min.
See BaslCam_threaded.py in Code section
Back on the original UP board, I wasted much time trying to nail down the RTE without success. If anyone can help I'd be very grateful.
AI
So next I wanted to try a Movidius AI compute-stick with the UP board to see how well Intel's secutity barrier camera demo works on UK sample images. However this requires changing the os from UbiLinux to Ubuntu 16.04: Follow the steps at: https://wiki.up-community.org/Ubuntu I removed a load of bloat (games/libre-office/etc) after the first ubuntu boot. Allow a good couple of hours and note the Intel graphics driver install has been obsoleted. Now install pip3, pylon, open-cv, pypylon, idle, etc as above. Once that's all working again go to developer.movidius.com/start and download openVINO via the link - I selected the latest 2018.4.420 full-package option then followed these [laborious] instructions:
software.intel.com/en-us/articles/OpenVINO-install-Linux
BUT added -d MYRIAD to the ./demo-squeezenet-download-convert-run.sh command line. I got 'C++ internal compiler error's at the above Image Classification demo step. It seems C++ needs at least 6GB of ram storage (so great error message guys!), I set up a 6G swapfile and re-started the demo script, which took hours to complete, but ran without error.
Now to try the security-barrier-camera-demo - this sample code uses three neural nets: a vehicle recogniser, a vehicle attributes recogniser (for identifying where the number-plate is) and finally a number-plate reader. It works well on Intel's one sample image (car_1.bmp) but failed, to varying degrees, on every single one of my 18 sample images! See SAMPLES.zip in FILES. The MCL reader accurately reads 16 of those 18. So much for AI!
So, in summary, the UP (or UPSquared) is a great little platform for ANPR and by all means check out AI techniques (if you've the spare time) but if you want to reliably read number plates in the real world, for now, do it the old-fashioned way.
Further work
For optimal results in day and night it's necessary to operate in infra red, with an illuminator. To save power in the illuminator it should be tied to the exposure time of the camera which should itself employ a global-shutter type sensor. I've now started work on a Raspberry pi IR camera as proof of concept and initial results are promising, despite the sub-optimal rolling shutter. Next step will be to investigate a more specialised IR-optimised Basler camera with global shutter.
Steve Marchant | pushed c1b29c0d2c0b5d42cd0f92185dd5568847828186 | 2018-10-29 09:39:40 UTC | |
mcl-uk | pushed 79e3e259d5b8194c521d325f9d56c8005d982a21 | 2018-10-28 18:26:49 UTC | |
mcl-uk | pushed bc886234218a3b5beba65fc7992caf4587684741 | 2018-10-28 18:23:35 UTC | |
mcl-uk | pushed 9802ee95077e41092b56a1d1d90ce14cd22060f5 | 2018-10-28 18:18:32 UTC | |
mcl-uk | pushed c179f09e3683dce52360caa39204bf51b0f9979c | 2018-10-28 18:17:33 UTC | |
mcl-uk | pushed 471b9b9d28ed43eb159f90ff794d1d89f4e00827 | 2018-10-28 18:04:00 UTC |
Title | Description | Format |
---|---|---|
ANPR using UKRegOCR python module | A python3 module to read UK number plates | |
sample image | sample | jpg |
sample image | sample | jpg |
sample image | sample | jpg |
1:1 zoom of M3XFL image | zoom-in | png |
18 full HD sample images | Full HD sample vehicle images | zip |
Want to comment this ...
Show more