This project verifies the hypothesis whether it is possible to create an Automatic Optical Inspection system of printed circuit-boards from common resources, using freely available software frameworks. The work emphasizes the overall simplicity and affordability of the resulting solution.
Optical Inspection of printed circuit boards is not a new technology. It started in 1940s with traditional optical comparators. The control was performed by a human worker by basically comparing two magnified images of tested boards. The testing device was purely mechanical and the whole system depended on the workers experience and eye-sight. During the years the systems were improved and with incoming automation began to be more and more complex, assuring a higher level of inspection and wide range of fault detection. Currently, the Automated Optical Inspection (AOI) system is generally a very expensive and sophisticated machine focused on large-scale production.
The main benefit of this work should be to verify possibility to create a functional, affordable and accessible automated optical inspection system which could be beneficial either for developers or even enthusiasts and their home projects. Since most of available AOI systems are mainly focused on a large-sized serial production, the goal of this work is to use the exactly opposite philosophy and try to find the solution for a small-sized companies or private individuals.
The inspection system should by focused on the assembled board inspection i.e. the inspection of the soldered parts onboard. Nowadays, the SMD types of electronic packages are mostly used. so the inspection is focused on control of these types of packages. The inspected parameters are should be:
The main concept of the inspection is based on obtaining a detailed color image of the scene (board). Components are then segmented from the image by using a color thresholding in multiple color spaces (BGR, HSV, Lab) and combination with Watershed segmentation algorithm. Defects are extracted by comparing the obtained data with a defectless reference "Golden Board" which has to be created in advance.
The project is not presented as a final or a compact solution. The problematics was separated into several standalone blocks for a certain image processing operations which can be separately developed and consequently tested. Each block represents a certain important image processing operation necessary for the inspection itself. Inputs and outputs of the blocks is/are mostly image/s The blocks are:
We must not forget for a necessary color library required for a color thresholding and processLib.py library for inspection itself.
The image capturing device was designed as simple as possible. As it might not be obvious, the image capturing device is a major part of the whole inspection system. Even the best and the most sophisticated inspection control software cannot provide a good results without a suffitient input data. The exact documentation is not provided in this project. Nevertheless, the main concept is visualized in picture below and consists of these parts:
Concept and photos of the final device:
The source code is based on the Pylon tutorial for obtaining a video stream using a Basler camera. Follow the instruction in the code. For more information about the instalation of OpenCV library and further description the reader can be reffered to Pylon+OpenCV tutorial.
To enhance contrast of the scene, the white balance is suggested for initial pre-processing. The simple process of rescaling the image histogram was based on Simple White Balance by David Young-Chan Kay. The process should be performed on a board captured on a white background. A component with a black colored package (resistor, IO) should be present on the board since the histogram is stretched between the lightest and the darkest pixel found.
Camera calibration is a basic process in most of the image processing tasks. For more the reader can follow an OpenCV tutorial.This process is separated into three steps:
In first step a suffitient number of calibration pattern images has to be captured. For this operation the BaslerCapture.cpp application can be used. The calibration pattern is mostly a reference chessboard with known dimensions.
By running getCalibrationParams.py the calibration parameters are estimated. Follow the instructions in the code.
In the last step the image can be calibrated by undistort.py with obtained calibration parameters. Follow the instruction in the code. Calibration has to be performed before cropping of the image!
The board itself is segmented from the image. The board should be captured on a white background. Selected board crop is then rotated so the longer side of the outer rectangle is horizontal. Thanks to this operation the board does not have to be strictly placed in the same position/rotation in the capturing scene, but only small difference in position and rotation is recommended.
Before using the inspection itself the color library has to be estimated. The color library contains a set of thresholds for each electronic part (a unique color). The color threshold is saved in three color spaces (BGR, HSV, CIEL*a*b*). New color should be placed in a following form:
new_part_color = color((B_min, G_min, R_min], [B_max, G_max, R_max]), (V_min, V_max), ([a_min, b_min], [a_max, b_max]))
Values of suitable threshold has to be obtained experimentally. Very usefull tool for threshold values estimation can be for example ImageJ or MATLAB Image processing toolbox. Some values of basic packages are provided, but it is suggested to adjust color thresholds based on particulary used ones.
Provides an interactive process for creating the reference Golden Board. Reference image is loaded and then the user provides a necessary information. The process description follows:
By running this script the inspection itself is executed. The tested image and
the golden board has to be proved by selecting the paths in the code. Follow
the instructions the code.
The inspection is performed part by part. At first the color threshold in three
color spaces is performed and combined together. The most significant color
estimated the part's type.
In the next step the output of threshold operations is posted as an initial seed into the watershed algorithm. This algorithm segments the part's package from the board. From the segmented area the rotation/position is estimated.
Information about the inspection is presented via the console during the process. Detected faults are stored in the fault list. Study of the code is recommended.
|autoWhiteBalance.py||Python function performing a white balance of an image.||py|
|getCalibrationParams.py||Python function for estimation calibration parameters.||py|
|undistort.py||Python function for image calibration based on previously obtained cal. parameters||py|
|cropping.py||Python function for a board cropping from an image.||py|
|colorLib.py||Python file containing a color library for color thresholding||py|
|processLib.py||Python file containing necessary support functions for the inspection||py|
|maskCreate.py||Python function providing an interactive process for the Golden Board creation||py|
|main_process.py||Python function providing the board inspection process||py|