Development of Automatic 3 D Blood Vessel Search and Automatic Blood Sampling System by Using Hybrid Stereo-Autofocus Method

We developed an accurate three-dimensional blood vessel search (3D BVS) system and an automatic blood sampling system. They were implemented into a point-of-care system designed for medical care, installed in a portable self-monitoring blood glucose (SMBG) device. The system solves problems of human error caused by complicated manual operations of conventional SMBG devices. We evaluated its accuracy of blood-vessel position detection. The 3D BVS system uses near-infrared (NIR) light imaging and the stereo and autofocus hybrid method to determine blood vessel locations accurately in three dimensions. We evaluated the accuracy of our 3D BVS system using a phantom of human skin, blood vessels, and blood. Additionally, we established an automatic blood sampling system for SMBG and assessed its performance in relation to punctures, blood suction, transport, and discharge on an enzyme sensor. The 3D BVS and automatic blood sampling system are adequate for use in a portable SMBG device.


Introduction
Recently, the rapidly increasing number of diabetic patients has [1] become a major social problem throughout the world.Diabetic patients must independently measure blood glucose concentrations using a self-monitoring blood glucose (SMBG) device and self-inject insulin several times each day [2,3].Diabetes patients and candidates might constitute up to one-fifth of the world population [1].Therefore, SMBG devices are expected to play an increasingly important role in daily medical care.The commercial SMBG devices of various kinds that are on the market today can be categorized as either noninvasive or minimally invasive.Noninvasive devices adopt infrared spectroscopy [4], FT-IR-ATR method [5], photoacoustic spectroscopy [6], and so on.However, these methods present severe problems of low accuracy and unreliability.In contrast, minimally invasive devices [7,8] can measure blood glucose concentrations with high accuracy [9], but puncture with a needle or knife edge is painful.In addition, such devices frequently induce human error because of their complicated manual operations during puncture, treatment of small blood samples, self-measurement, and so on.Therefore, many improvements are necessary for SMBG devices.Several groups have developed minimally invasive SMBG devices equipped with a blood vessel search system to reduce human error.For example, Yamakoshi et al. developed a device to detect the positions of blood vessels in the arm using reflected near-infrared (NIR) light [10], but the blood vessel depth could not be determined accurately.Another example is a bio-medical-chip developed by Horiike et al., which can detect the position of a blood vessel in three dimensions using electric signals [11].However, their chip showed low accuracy of blood vessel location detection, and the device entailed operations that were time-consuming and painful for patients.Although Kandani et al. developed a threedimensional blood vessel search (3D BVS) system using the stereo method [12], the value of the standard deviation of measured positions was higher for deeper blood vessel locations.In this study, we set our goal as integration of a 3D BVS system and an automatic blood sampling system.To accomplish that goal, we combined the autofocus method with the original stereo method to establish a hybrid method.We set the allowed value of 100 μm for the targeted blood vessel, which had diameter of 500-1000 μm.An automatic blood sampling system for collection and transport of the blood for blood glucose measurements was connected with 3D BVS to produce the portable medical care system.For patients with diabetes, this system is expected to improve quality of life to a considerable degree.As described in this paper, we first discuss the hybrid stereo-autofocus method for 3D BVS systems by introducing images processed with NIR light.Then we examine the accuracy of the 3D BVS system.Secondly, to evaluate our newly developed automatic blood sampling system for blood glucose measurement, we verify its capabilities of (1) accurate puncture, (2) stable blood transport, and (3) blood discharge on a glucose enzyme sensor.In light of those results, we assess the performance of this integrated portable medical care system.

Hybrid Stereo-Autofocus Method and
Blood Vessel Visualization 2.1.Autofocus Method and Stereo Method.We used a hybrid stereo-autofocus method to detect the position of a blood vessel located between the first and second joints of the finger using transmitting NIR light.Our previous studies, which were done using a stereo method, did not determine the blood vessel depths accurately [12].The measurement error was high for blood vessels located deep in the finger.Therefore, we adopted an autofocus method to obtain a very clear image of blood vessels using the template search method in the region of blood vessel [13].This method improved the accuracy considerably.As presented in Figure 1(a), we took blood vessel images using different camera positions in relation to the finger based on the autofocus method.D means the length between the initial set position and the fixed base, and S the slide length as indicated in Figure 1(a).Fixed base means the upper surface of the glass plate, which is located at the position D s and on which the finger was set.We evaluated the blood vessel image sharpness based on the template-matching algorithm and selected the clearest image at the camera position D s as shown in Figure 1(a) [13].In the auto focus procedure, we fix the incline angle θ as indicated in Figure 1(b).
Next, as shown in Figure 1(b), we measured blood vessel positions using the stereo method [12].In Figure 1(b), L stands for the distance between the two CMOS cameras, f represents the focal length, D b signifies the vertical distance between the center of the blood vessel and the line drawn between two center points of the lenses, and D s denotes the vertical distance between the reference surface of the fixed base and the line drawn between respective center points of the lenses.In Figure 1(b), ΔD is the depth of the blood vessel center from the reference surface of fixed base. Figure 1(b) shows θ 1 and θ 2 as the angles of the light axes from the center of the blood vessel against the horizontal line between the respective centers of the two lenses.As shown in Figure 1(c), the center of the blood vessel can be detected from the luminance distribution of the blood vessel images obtained using the two CMOS cameras.Furthermore, θ 1 and θ 2 are calculable using trigonometric functions.The size of image plane pixel, 5.55 μm, was used to evaluate the blood vessel center position with the same length ratio as the pixel.Therefore, we obtain the deviation lengths B 1 and B 2 as indicated in Figure 1(c), which can be calculated as a number of deviated pixels times by pixel size 5.55 μm.Finally, D b is expressed as here, Length D s is already determined by using the autofocus method as shown in Figure 1(a).Therefore, the blood vessel depth (ΔD) can be calculated as (2)

3D BVS System and Blood Vessel Visualization.
We incorporated the hybrid stereo-autofocus components into a 3D BVS system, which included four NIR LED lights (peak wavelength of 870 nm, KED871M51A; Kyosemi Corp.) as the light sources, two CMOS cameras (PPV404C; Asahi Denshi Inc.), and two image-processing units (PP-DEB-007; Asahi Electronics Laboratory Co., Ltd.).The NIR LEDs were set behind the finger to irradiate the finger with NIR light.We used two CMOS cameras to take images using the NIR transmitted light.Details of the CMOS cameras were the following: 5.32 mm focal length, 8 × 8 × 5.55 mm size, and 5.55 × 5.55 μm pixel size.Images were of 8-bit digital grayscale values ranging from black (0) to white (255) for one pixel.The 640 × 480 pixel images were stored as JPEG image files.The images were processed by an image processing unit (AT401; Asahi Denshi Inc.) to calculate the blood vessel depth using the autofocus and stereo algorithms.Figure 2 presents a photograph of our 3D BVS system.The respective centers of CMOS cameras and LEDs are set on the same   ).The size of our 3D BVS device became 40 × 30 × 10 mm.Therefore, our system had size and features that allowed its installation into our new portable SMBG device.We used the transmitted NIR light to visualize blood vessels located in the human finger [14][15][16].We selected small veins as our measurement targets and employed NIR light with wavelengths of 650-950 nm.
The NIR light was transmitted through the finger and was recorded in images by the cameras in the positions presented in Figures 1(b) and 2. Figure 3 shows images obtained using  the visible light and NIR light of the wavelengths shown in the figures.According to the steepest gradient search of the luminance distribution, especially in the close area to the blood vessel, finally we chose the wavelength of 870 nm [11,13].Figure 4 presents blood vessel images and the luminance distribution along line A-A as indicated in Figure 4(b).We concluded that our imaging system can provide a clearer image of the blood vessel to determine its location in three dimensions.
Next we discuss the image processing associated with blood vessel detection.We adopted template matching to determine the image sharpness.The templates were produced on the assumption that the luminance distributions of blood vessels are convex-downward quadratic curves, as shown by the red curve in Figure 5, where the luminance distribution is portrayed as a blue line.Figures 6(a)-6(d) show results of template matching.Figure 6(a) presents the distribution of residual error obtained by template matching calculation.It shows two local maximum points, which correspond to the blood vessel position.Figure 6(b) portrays the distribution of the derivative of residual error.Figures 6(c) and 6(d) show the point at which the derivative value becomes zero, which is the center of the blood vessel.Moreover, we assumed the inclination of the diagram of derivative values to be the rate of luminance change.Therefore, we confirmed that the blood vessel can be detected clearly as the rate of luminance change.The sharpness can be evaluated using the angle of inclination of derivatives, as shown in Figures 6(c) and 6(d).Results show that we can select a clear image of a blood vessel taken by the camera located at distance D s shown in Figure 1(a).

Evaluation of Blood Vessel Position Detection
We evaluated the accuracy of our 3D BVS system by carrying out experiments using finger phantoms containing artificial blood vessels.The photograph and the finger phantom size are depicted in Figure 7. Small veins of a human finger, with diameters of 0.5-1.0mm, exist at depths of 0.5-2.0mm from the surface.Therefore, we used the diameter of 0.75 mm for artificial blood vessels.We generated finger phantoms of seven types with artificial blood vessels embedded at different depths of 0.5-2.0mm with 0.25 mm intervals.The finger phantoms were made of the polyacetal resin (TPS-POM; Toyo Plastic Seiko Co., Ltd.), which had optical International Journal of Optics    characteristics similar to those of human skin [17].We included colored water in the artificial blood vessels.We first increased the input power of NIR light to reduce the influence of scattering and thereby obtain a clear image.The scattering of NIR light reduces both the intensity of the transmitted light and the contrast between the artificial blood vessel and the tissue.We adopted four levels of input power of the NIR light: 10, 30, 50, and 60 mW.Figures 8(a) and 8(b) show photographs of blood vessels and luminance distributions of finger phantom images with the four input powers.Through comparison with the slope and concentration of the luminance distribution, we finally selected 50 mW to overcome the scattering and provide the clearest image of artificial blood vessels.
We measured the depths of the artificial blood vessels using our 3D BVS system, which is based on the hybrid stereo-autofocus method.To avoid the influence of light from the environment, we covered the whole system with a light-proof cloth.First, we analyzed the images and determined the vertical distance between the reference surface of the fixed base and the line drawn between respective center points of the lenses as D s , presented in Figure 1(a), using autofocus methods.Secondly, we measured the position of the artificial blood vessel based on the stereo method.The rows of pixels on the center lines represented as A-A and B-B , as portrayed in Figure 9(a), were used to determine the center and the blood vessel depth.Pixels located on the line (X, Y) from value (0, 240)-(640, 240) provide luminance according to the eight-byte digital number, where X-Y coordinates were defined as portrayed in Figure 9(a).Figure 9(b) shows that a rapid decrease in brightness is observed at the area of the blood vessel.Therefore, we are able to determine the artificial blood vessel position.The center of the artificial blood vessel position of camera 1 was set to the 327th pixel and that of camera 2 to the 316th pixel.According to Figures 1(b) and 1(c), the deviations between the center position of blood vessel and the fixed base plate in the image planes of CMOS camera 1 and 2 were 7 pixels and 4 pixels, respectively.Because the pixel size was 5.55 μm, the actual deviation lengths were, respectively, 64.8 μm in camera 1 and 93.6 μm in camera 2. Next, the angles of light axes, θ 1 and θ 2 , as indicated in Figure 1(b), were obtained using these deviation length B 1 and B 2 and focal length f as shown in Figure 1(c).We then calculated the blood vessel depth ΔD using (1) and (2). Figure 10 shows stereo images of the artificial blood vessels embedded at the indicated depths, where the left image was taken by camera 1 and the right image by camera 2, ( International Journal of Optics Next, to assess the position detection accuracy of this 3D BVS detection device, we strove to eliminate the influence of the refraction of NIR light in the visualization procedure.Because the refractive index of polyacetal differs from that of air, NIR light is refracted at the interface and the difference occurs between the blood vessel position of the image and that of the actual value.Therefore, we corrected the result of the measurements based on Snell's law using the refractive index of the polyacetal as 1.48.Figure 11 shows the trajectory of NIR light from the finger phantom to the camera.Figure 11 shows that the measured depth of the blood vessel becomes smaller than the actual depth because of the refraction of NIR light.Formulas for the incidence angle, refraction angle, and the blood vessel depth are shown in (3).Using these values, we corrected the measured depth obtained using ( 1) and (2).
Figure 12 shows the relation between the corrected depth and the actual one, which we named "true" depth.We calculated the standard deviations in seven cases of "true" depths.In each "true" depth, we measured more than ten times and calculated the standard deviation by using the differences between the "true" depth and measured depths.It shows that the deeper the depth, the larger the standard deviation.The maximum standard deviation was 110 μm, and the average value of standard deviations was 63 μm.The average of all deviations was 29.8 μm, which means that average value of differences between measured and true depths in seven cases.Good correlation (R 2 0.946) was found between the corrected depths and the actual depths.These errors were less than the allowance value of 150 μm for the targeted blood vessel with diameter greater than 500 μm.

Automatic Blood Sampling System
We fabricated an automatic blood sampling system that had been designed to be built in a portable SMBG device for use in conjunction with the 3D BVS system.In addition to two small stepping motors, we used a glucose enzyme sensor made by Horiba, Ltd. for accurate and reliable blood glucose measurement purposes [18], which was expected to perform accurate and quick blood collection.Figure 13 shows a prototype of the newly developed automatic blood sampling system, with size of 47 × 30 × 20 mm.Regarding performance evaluation, we used an artificial finger made of silicone rubber (RTV2 VP7550; Asahi Chemical Ind.Co. Ltd.), which had similar elastic properties to those of human skin.The artificial blood vessel diameter was set as 1.0 mm; the blood vessel was embedded to 1.0 mm depth in the artificial finger.We used a 50% glycerin solution, which had similar viscosity to that of human blood, as artificial blood.
International Journal of Optics This artificial blood was kept in the artificial blood vessel.We examined the capabilities for puncture, blood extraction, transport, and discharge on the enzyme sensor.The glucose enzyme sensor required at least 5.0 μL of blood for blood glucose measurements.Figures 14(a)-14(e) show four stages of blood sampling procedures, such as puncture, extraction, transport, discharge, and measurement.We analyzed blood sampling and measurement capabilities to provide 5.0 μL of blood on the enzyme sensor.Figure 15 presents the relation between the artificial blood volume on the enzyme sensor and the piston speed for blood transport and discharge.As the piston speed decreases, the amount of artificial blood increases.We concluded that an adequate piston speed to provide over 5.0 μL of blood for measurement was 1.0 mm/s.

Conclusions
We developed a 3D BVS system and an automatic blood sampling system to fabricate a portable SMBG device.We adopted the hybrid stereo-autofocus method to detect the position of blood vessels in three dimensions.We also examined the performance of our automatic blood sampling system, which features puncture, blood extraction, transport, and discharge for an enzyme sensor to measure the blood glucose.Results of the evaluation of our 3D BVS and automatic blood sampling devices are summarized as follows.(1) Accuracy evaluation of 3D BVS system was confirmed.The maximum standard deviation was 111 μm and the average of the standard deviation was 63 μm when detecting 0.5 mmdiameter blood vessels embedded at depths of 0.5-2.5 mm with 0.25 mm intervals.Results show very good correlation with actual measurements: R 2 0.946.(2) Blood sampling performance evaluation was done with the piston speed for blood suction and discharge selected as 1.0 mm/s to provide 0.5 μL artificial blood on the glucose enzyme sensor.That amount was sufficient for measuring the blood glucose concentration.Our integrated system, combined with 3D BVS and automatic blood sampling systems, can be used effectively as a portable medical care system.

2 Figure 1 :
Figure 1: Schematic diagram of the autofocus and stereo hybrid methods embedded in a 3D BVS system.

Figure 3 :
Figure 3: Blood vessel photos in cases of NIR lights, wavelengths 625-950 nm, and visible light.

Figure 4 :
Figure 4: Image of the blood vessel in the human finger.The NIR light wavelength is 870 nm.

Figure 5 :
Figure 5: Luminance distribution of the image including the blood vessel and the template to search the blood vessels.

Figure 6 :
Figure 6: Residual errors calculated by template matching algorithm and its derivatives distributions to determine the sharpness.

Figure 7 :
Figure 7: Finger phantom image and its dimensions.
(a) Photographs of artificial blood vessel with the described light powers Luminance distributions of artificial blood vessel for the described input light powers

Figure 8 :
Figure 8: of light power on the distributions.
blood vessels (b) Luminance distributions on lines A-A and B-B

Figure 9 :
Figure 9: Stereo images and luminance distributions obtained using two cameras, wavelength of 870 nm.

Figure 10 :
Figure 10: Stereo images of artificial blood vessels embedded in the finger phantom at the indicated depth.

Figure 9 (
Figure 9(a) shows stereo images of the artificial blood vessels.The rows of pixels on the center lines represented as A-A and B-B , as portrayed in Figure9(a), were used to determine the center and the blood vessel depth.Pixels located on the line (X, Y) from value (0, 240)-(640, 240) provide luminance according to the eight-byte digital number, where X-Y coordinates were defined as portrayed in Figure9(a).Figure9(b) shows that a rapid decrease in brightness is observed at the area of the blood vessel.Therefore, we are able to determine the artificial blood vessel position.The center of the artificial blood vessel position of camera 1 was set to the 327th pixel and that of camera 2 to the 316th pixel.According to Figures1(b) and 1(c), the deviations between the center position of blood vessel and the fixed base plate in the image planes of CMOS camera 1 and 2 were 7 pixels and 4 pixels, respectively.Because the pixel size was 5.55 μm,

Figure 11 :Figure 12 :
Figure 11: Refraction of light in the skin tissue and depth correction.

Figure 13 :
Figure13: Automatic blood sampling system equipped with artificial finger and blood vessel.
(a) Initial position (b) Puncture and blood extraction (c) Blood transport (d) Discharge of artificial blood (e) Enzyme sensor to measure the blood glucose

Figure 14 :
Figure 14: Photos of automatic blood sampling procedure and the enzyme sensor used to measure the blood glucose.

Figure 15 :
Figure 15: Relation between the volume of artificial blood on the sensor and the piston travel speed (±SD).