Automatic Optical Inspection of Printed Circuit Boards

Description

Optical Inspection of printed circuit boards is not a new technology. It started in 1940s with traditional optical comparators. The control was performed by a human worker by basically comparing two magnified images of tested boards. The testing device was purely mechanical and the whole system depended on the workers experience and eye-sight. During the years the systems were improved and with incoming automation began to be more and more  complex, assuring a higher level of inspection and wide range of fault detection. Currently, the Automated Optical Inspection (AOI) system is generally a very expensive and sophisticated machine focused on large-scale production.

The main benefit of this work should be to verify possibility to create a functional, affordable and accessible automated optical inspection system which could be beneficial either for developers or even enthusiasts and their home projects. Since most of available AOI systems are mainly focused on a large-sized serial production, the goal of this work is to use the exactly opposite philosophy and try to find the solution for a small-sized companies or private individuals.

Goals

The inspection system should by focused on the assembled board inspection i.e. the inspection of the soldered parts onboard. Nowadays, the SMD types of electronic packages are mostly used. so the inspection is focused on control of these types of packages. The inspected parameters are should be:

• Correct type of the package
• Correct position/rotation of the package
• Missing component detection

Methodology

Software (inspection)

The main concept of the inspection is based on obtaining a detailed color image of the scene (board). Components are then segmented from the image by using a color thresholding in multiple color spaces (BGR, HSV, Lab) and combination with Watershed segmentation algorithm. Defects are extracted by comparing the obtained data with a defectless reference "Golden Board" which has to be created in advance.

The project is not presented as a final or a compact solution. The problematics was separated into several standalone blocks for a certain image processing operations which can be separately developed and consequently tested. Each block represents a certain important image processing operation necessary for the inspection itself. Inputs and outputs of the blocks is/are mostly image/s The blocks are:

• Image capturing (BaslerCapture.cpp)
• White balance (autoWhiteBalance.py)
• Camera distortion compensation (getCalibrationParams.py , undistort.py)
• Board selection (cropping.py)
• "Golden Board" creation (maskCreate.py)
• Inspection process (main_process.py)

We must not forget for a necessary color library required for a color thresholding and processLib.py library for inspection itself. 

• Color threshold library (colorLib.py)
• Inspection function library (processLib.py)

Hardware (image capturing device)

The image capturing device was designed as simple as possible. As it might not be obvious, the image capturing device is a major part of the whole inspection system. Even the best and the most sophisticated inspection control software cannot provide a good results without a suffitient input data. The exact documentation is not provided in this project. Nevertheless, the main concept is visualized in picture below and consists of these parts:

• Horizontally placed board (scene), white background.
• Vertically placed camera above the scene (stacionary placed).
• Bi-directional lighting with dispersion layer.
• Surrounded with a white-box to provide shadowless environment.

Concept and photos of the final device:

Software description/functionality

Image capturing (BaslerCapture.cpp)

The source code is based on the Pylon tutorial for obtaining a video stream using a Basler camera. Follow the instruction in the code. For more information about the instalation of OpenCV library and further description the reader can be reffered to Pylon+OpenCV tutorial. 

White balance (autoWhiteBalance.py)

To enhance contrast of the scene, the white balance is suggested for initial pre-processing. The simple process of rescaling the image histogram was based on Simple White Balance by David Young-Chan Kay. The process should be performed on a board captured on a white background. A component with a black colored package (resistor, IO) should be present on the board since the histogram is stretched between the lightest and the darkest pixel found.

Camera distortion compensation (getCalibrationParams.py, undistort.py)

Camera calibration is a basic process in most of the image processing tasks. For more the reader can follow an OpenCV tutorial.This process is separated into three steps:

• Capturing suffitient number of calibration images.
• Estimating the calibration parameters.
• Removing the optical distortion on a processed image.

In first step a suffitient number of calibration pattern images has to be captured. For this operation the BaslerCapture.cpp application can be used. The calibration pattern is mostly a reference chessboard with known dimensions.

By running getCalibrationParams.py the calibration parameters are estimated. Follow the instructions in the code.

In the last step the image can be calibrated by undistort.py with obtained calibration parameters. Follow the instruction in the code. Calibration has to be performed before cropping of the image!

Board cropping (cropping.py)

The board itself is segmented from the image. The board should be captured on a white background. Selected board crop is then rotated so the longer side of the outer rectangle is horizontal. Thanks to this operation the board does not have to be strictly placed in the same position/rotation in the capturing scene, but only small difference in position and rotation is recommended.

Color library (colorLib.py)

Before using the inspection itself the color library has to be estimated. The color library contains a set of thresholds for each electronic part (a unique color). The color threshold is saved in three color spaces (BGR, HSV, CIEL*a*b*). New color should be placed in a following form:

new_part_color = color((B_min, G_min, R_min], [B_max, G_max, R_max]), (V_min, V_max), ([a_min, b_min], [a_max, b_max]))

Values of suitable threshold has to be obtained experimentally. Very usefull tool for threshold values estimation can be for example ImageJ or MATLAB Image processing toolbox. Some values of basic packages are provided, but it is suggested to adjust color thresholds based on particulary used ones.

Golden Board creation (maskCreate.py)

Provides an interactive process for creating the reference Golden Board. Reference image is loaded and then the user provides a necessary information. The process description follows:

1. By four double clicks on the outer corners of the pad the user determines a correct position/rotation of the part (a red rectangle is created).
2. By dwo diagonal double clicks the user determines an area around the part where the inspection process will perform (a green rectangle is created).
3. User inserts the correct part type and polarity into the command line.
4. The user is asked if continue to set a new part or finish the process.
5. In the end of the process the user is asked about the name of the golden board and the board is saved after confirmation.

Inspection process (main_process.py)

By running this script the inspection itself is executed. The tested image and

the golden board has to be proved by selecting the paths in the code. Follow

the instructions the code. 

The inspection is performed part by part. At first the color threshold in three

color spaces is performed and combined together. The most significant color

estimated the part's type.

In the next step the output of threshold operations is posted as an initial seed into the watershed algorithm. This algorithm segments the part's package from the board. From the segmented area the rotation/position is estimated.

Information about the inspection is presented via the console during the process. Detected faults are stored in the fault list. Study of the code is recommended.