Image Stitching Improvements on the APEXVIEW™ APX100 Benchtop Fluorescence Microscope

Introduction

The APEXVIEW™ APX100 benchtop fluorescence microscope is used to observe cells and tissue specimens. For tissue specimens in particular, the overall structure of the sample may be searched for regions of interest that are wider than the microscope’s field of view.

Stitching together multiple images of the field of view to obtain a wide-area, high-resolution image is an important function for observing tissue specimens. The following three points are essential for image stitching.

1. Stitching accuracy

The stitched image is constructed by moving the motorized XY stage for each field of view.

However, since some misalignment occurs on the XY stage, if the stitching position is determined only by the XY coordinates of each field of view, the stitched image will shift for each field of view when stitched together. Therefore, image processing is used to determine the stitching position by searching within the image. It is necessary to avoid any deviation during the image stitching, as it will affect the subsequent image analysis.

2. Stitched image quality

In the periphery of the field of view, optical factors affect the image quality in various ways. It is necessary to avoid the degradation of image quality when obtaining a stitched image.

3. Throughput

Stitching images requires accuracy and image quality, so it takes time to capture the images of each field of view required to create a stitched image. It takes even longer to process the stitched images after image acquisition, which hinders the experimental workflow. The acquisition of stitched images requires both image quality and throughput to obtain experimental data efficiently.
 

Image Stitching Improvements on the APX100 Microscope

Capturing stitched images is one of the key applications of the APX100 microscope.

For this reason, we have improved the quality of the stitched images by upgrading the software. At the same time, we have further improved the experimental efficiency using the APX100 microscope.

cellSens™ APEX software, the control software of the APX100 microscope, offers three key improvements to the image stitching function in version 4.3.1.
 

1. Improved stitching accuracy

Improvement of the stitching algorithm

The stitching of adjacent images in the field of view is performed by image processing. Until the previous version, the images were stitched immediately after each single field of view—a method known as sequential stitching.

In the sequential stitching method, the stitched area contains only a small amount of information when the stitched part of the adjacent field of view images is only a background area without a sample. This causes misalignment in the stitching.

In cellSens APEX software version 4.3.1, we have improved the alignment. The stitching algorithm now retains images of a certain range of fields of view until the optimal position is obtained, then finds the optimal position within the retained range (Figure 1).

Figure 1. The previous stitching algorithm (left figure) and the updated stitching algorithm in cellSens APEX software version 4.3.1 (right figure). Fixed frame: positioned frame. Buffered frame: held frame before positioning. Aligned frame: frame for optimal positioning.
 

In the sequential stitching method, only frames that have already been positioned can be used for the alignment. In the stitching method that holds an image of a certain range of fields of view, the frame to be aligned can be positioned from the frame already positioned and the frame within the retained range.

With this updated method, the algorithm introduced in software version 4.3.1 now makes it possible to acquire a stitched image without deviation—even in the stitched area that includes the background (Figure 2).
 

Figure 2. Comparison of the stitched image taken with the updated stitching algorithm in cellSens APEX version 4.3.1 (left image) and the stitched image taken without the algorithm (right image). In the stitched image using version 4.3.1, the structures do not appear to overlap.
 

The amount of time to obtain stitched images remains the same in the 4.3.1 version.
 

Correction of optical phenomena that affect the stitching accuracy

Optical factors such as magnification error can affect the stitching accuracy. cellSens APEX software has a function to compensate for this optical phenomenon.

Magnification correction function

Objective lenses have a slight magnification error, so correcting this error enables more accurate stitching.

With the APX100 microscope, the total magnification of the entire microscope, including the objective lens, can be corrected by using the magnification correction function introduced in cellSens APEX software version 4.2. This makes it possible to generate highly accurate stitched images (Figure 3).

Figure 3. Comparison of images taken with the magnification correction enabled (left image) and disabled (right image) in cellSens APEX software.

Since the magnification correction process is performed in real time each time an image is taken, the acquisition time does not change with or without correction.
 

2. Improved image quality of the stitching

Optical factors such as distortion can affect the image quality of the stitching. cellSens APEX software has a function to compensate for this phenomenon.

Distortion correction function

When images are formed through an objective lens on a system microscope, a phenomenon called distortion occurs in the field of view (Figure 4).

Figure 4. Schematic diagram of the ideal image (dotted line) and the image (solid line) when the objective lens is attached. The amount of deviation increases toward the edge of the field of view.
 

If image stitching is performed while distortion is present in the field of view, structures will appear twice at the four corners of the stitching position. This double appearance of structures at the corners increases the possibility that the image quality will degrade.

To avoid distortion and obtain a high-quality stitched image, one method is to use only the center of the image. However, this approach will increase the number of images taken and reduce throughput (Figure 5).

Figure 5. The field of view (FOV) needs to be reduced to avoid the effects of distortion (a). The number of fields of view required for stitching differs depending on the difference in the FOV (b).
 

In cellSens APEX version 4.3.1, a function has been added to compensate for this distortion (Figure 6). In this function, the amount of distortion is measured for each objective lens using a dedicated calibration slide. Next, when acquiring a stitched image, the distortion is corrected based on the amount of distortion measured for each image in one field of view.

Figure 6. Comparison of distortion correction enabled (left image) and distortion correction disabled (right image).
 

3. Workflow improvement

The APX100 microscope automatically performs the entire process from image acquisition to stitching as a series of workflows, making it possible to efficiently acquire stitched images.

The APX100 microscope also features a smart sample navigator that shortens the experimental workflow. The smart sample navigator can automatically detect samples from the capture of macro images in 10 seconds. The Convert Sample Area to Scan Area function added in cellSens APEX software version 4.2 lets the user set the detected sample area as the stitching acquisition area (Figure 7).

This greatly shortens the workflow from pre-observation preparation, to setting the stitching position, to image acquisition.

Figure 7. In cellSens APEX software, the Convert Sample Area to Scan Area function creates a rectangular scan area from a sample area.
 

Conclusion

The APEXVIEW APX100 benchtop fluorescence microscope now offers enhanced image quality and accuracy of stitched images, as well as streamlined experimental efficiency from image setup to capture. In addition to improving the stitching algorithm of the software, our engineers took the correction of optical factors into account so that the best performance can be achieved.

In summary, the image stitching enhancements include:

• Improved accuracy in the stitched area, including the background frame
• Accuracy improved by correcting magnification error
• Improved image quality by correcting distortion
• Shortened workflow to acquire wide-area, high-resolution stitched images
   

Author

Takuma Kimura
R&D, Software Development, Evident