Robots rely on sensors to provide them with information about their surroundings. However, highquality sensors can be extremely expensive and costprohibitive. Thus many robotic systems must make due with lowerquality sensors. Here we demonstrate via a case study how modeling a sensor can improve its efficacy when employed within a Bayesian inferential framework. As a test bed we employ a robotic arm that is designed to autonomously take its own measurements using an inexpensive LEGO light sensor to estimate the position and radius of a white circle on a black field. The light sensor integrates the light arriving from a spatially distributed region within its field of view weighted by its spatial sensitivity function (SSF). We demonstrate that by incorporating an accurate model of the light sensor SSF into the likelihood function of a Bayesian inference engine, an autonomous system can make improved inferences about its surroundings. The method presented here is data based, fairly general, and made with plugandplay in mind so that it could be implemented in similar problems.
Robots rely on sensors to provide them with information about their surroundings. However, highquality sensors can be costprohibitive and often one must make due with lower quality sensors. In this paper we present a case study which demonstrates how employing an accurate model of a sensor within a Bayesian inferential framework can improve the quality of inferences made from the data produced by that sensor. In fact, the quality of the sensor can be quite poor, but if it is known precisely how it is poor, this information can be used to improve the results of inferences made from the sensor data.
To accomplish this we rely on a Bayesian inferential framework where a machine learning system considers a set of hypotheses about its surroundings and identifies more probable hypotheses given incoming sensor data. Such inferences rely on a likelihood function, which quantifies the probability that a hypothesized situation could have given rise to the data. The likelihood is often considered to represent the noise model, and this inherently includes a model of how the sensor is expected to behave when presented with a given stimulus. By incorporating an accurate model of the sensor, the inferences made by the system are improved.
As a test bed we employ an autonomous robotic arm developed in the Knuth Cyberphysics Laboratory at the University at Albany (SUNY). The robot is designed to perform studies in autonomous experimental design [
There are two aspects to this work. First is the characterization of the light sensor and second is the incorporation of the light sensor model into the likelihood function of the robot’s machine learning system in a demonstration of improved efficacy. In Section
In this section we begin by discussing various aspects of the robotic test bed followed by a discussion of the techniques used to characterize the light sensor.
The robotic arm is designed to perform studies in autonomous experimental design [
A photograph showing the robotic arm along with the circle it is programmed to characterize. The robotic arm is constructed using the LEGO NXT Mindstorms System. It employs one motor to allow it to rotate about a vertical axis indicated by the black line in the top center of the image and two motors to extend and lower the arm about the joints located at the positions indicated by the short arrows. The LEGO light sensor, also shown in the inset, is attached to the end of the arm as indicated by the long arrow.
The LEGO light sensor (LEGO Part 9844) consists of a photodiodeLED pair. The white circle is the photo diode, and the red circle is the illuminating LED. Note that they are separated by a narrow plastic ridge, which prevents the LED from shining directly into the photo diode. This ridge, along with the plastic lenses and the presence of the illuminating LED, affects the spatial sensitivity of the sensor. When activated, the light sensor flashes for a brief instant and measures the intensity of the reflected light. The photo diode and its support circuitry are connected to the sensor port of the LEGO Brick (LEGO Part 9841), which runs on a 32 bit ARM7 ATMEL microcontroller. The measured intensities are converted by internal software running on the ATMEL microcontroller to a scale of 1 to 100, which we refer to as
The robotic arm is designed to deploy the light sensor to take measurements of the surface albedo at locations within a playing field of dimensions approximately 131 × 65 LEGO distance units (1048 mm × 520 mm), within an automated experimental design paradigm [
In this experiment, the robotic arm is instructed that there is a white circle of unknown radius arbitrarily placed on the black field. Such an instruction is encoded by providing the robot with a model of the surface albedo consisting of three parameters: the center location
Again, it is important to note that the robot does not scan the surface to solve the problem nor does it try to find three points along the edge of the circle. Instead, it employs a general system that works for any expected shape or set of shapes that autonomously and intelligently determines optimal measurement locations based both on what it knows and on what it does not know. The number of measurements needed to characterize all three circle parameters within the desired accuracy is a measure of the efficiency of the experimental procedure.
The machine learning system employs a Bayesian Inference Engine to make inferences about the circle parameters given the recorded light intensities, as well as an Inquiry Engine designed to use the uncertainties in the circle parameter estimates to autonomously select measurement locations that promise to provide the maximum amount of relevant information about the problem.
The core of the Bayesian Inference Engine is centered around the computation of the posterior probability
The likelihood term,
The robot not only makes inferences from data, but also designs its own experiments by autonomously deciding where to take subsequent measurements [
This figure illustrates the robot’s machine learning system’s view of the playing field using the naïve light sensor model. The axes label playing field coordinates in LEGO distance units. The previously obtained measurement locations used to obtain light sensor data are indicated by the black and white squares indicating the relative intensity with respect to the naïve light sensor model. The next selected measurement location is indicated by the green square. The blue circles represent the 50 hypothesized circles sampled from the posterior probability. The shaded background represents the entropy map, such that brighter areas indicate the measurement locations that promise to provide greater information about the circle to be characterized. Note that the low entropy area bounded by the white squares indicates that this region is probably inside the white circle and that measurements made here will not be as informative as measurements made elsewhere. The dark jagged edges at the bottom of the colored high entropy regions reflect the boundary between the playing field and the region that is outside of the robotic arm’s reach.
The efficacy of the sensor model will be quantified by the average number of measurements the robot needs to make to estimate the circle parameters within a given precision.
In this section we discuss two models of a light sensor and indicate precisely how they are integrated into the likelihood function used by both the Bayesian Inference and Inquiry Engines.
A naïve light sensor model would predict that if the sensor was centered on a black region (surface albedo of zero), the sensor would return a small number on average, and if it were centered on a white region (surface albedo of unity), it would return a large number on average. Of course, there are expected to be noise variations from numerous sources, such as uneven lighting of the surface, minor variations in albedo, and noise in the sensor itself. So one might expect that there is some expected squared deviation from the two mean sensor values for the white and black cases. For this reason, we model the expected sensor response with a Gaussian distribution with mean
A more accurate likelihood can be developed by taking into account the fact that the photo diode performs a weighted integral of the light arriving from a spatially distributed region within its field of view, the weights being described by the spatial sensitivity function (SSF) of the sensor. Since the SSF of the light sensor could be arbitrarily complex with many local peaks, but is expected to decrease to zero far from the sensor, we characterize it using a mixture of Gaussians (MoG) model, which we describe in this section.
The sensor’s response situated a fixed distance above the point
The response of the sensor is described by
We employ a mixture of Gaussians (MOG) as a parameterized model to describe the SSF in the sensor’s frame coordinates
We assign a Student
In this section we describe the collection of the light sensor readings in the laboratory that were used to estimate the SSF of the light sensor. The SSF is a function not only of the properties of the photo diode, but also of the illuminating LED and the height above the surface. For this reason, measurements were made at a height of 14 mm above a surface with a known albedo in a darkened room to avoid complications due to ambient light and to eliminate the possibility of shadows cast by the sensor or other equipment [
The surface, which we refer to as the blackandwhite boundary pattern, consisted of two regions: a black region on the left and a white region on the right separated by a sharp linear boundary as shown in Figure
(a) This figure illustrates the laboratory surface, referred to as the blackandwhite boundary pattern, with known albedo, which consisted of two regions: a black region on the left and a white region on the right separated by a sharp linear boundary. (b) This illustrates the four sensor orientations used to collect data for the estimation of the SSF along with the symbols used to indicate the measured values plotted in Figure
The blackandwhite boundary pattern does not provide sufficient information to uniquely infer the SSF, since the sensor may have a response that is symmetric about a line oriented at 45° with respect to the linear boundary. For this reason, we employed four additional albedo patterns consisting of black regions with one white quadrant as illustrated in Figure
Four additional symmetrybreaking albedo patterns were employed. In all cases, the sensor was placed in the
In this section we describe the application of Bayesian methods to estimate the SSF MoG model parameters. Keep in mind that in this paper we are considering two distinct inference problems: the robot’s inferences about a circle and our inferences about the light sensor SSF. Both of these problems rely on making predictions about measured intensities using a light sensor. For this reason many of these equations will not only look similar to what we have presented previously, but also depend on the same functions mapping modeled albedo fields to predicted sensor responses, which are collectively represented using the generic symbol
The posterior probability for the SSF MoG model parameters, collectively referred to as
All five sets of data
We assign uniform priors so that this is essentially a maximum likelihood calculation with the posterior being proportional to the likelihood. We assign a Student
The joint likelihood for the five data sets is found by taking the product of the likelihoods for each data set, since we expect that the standard deviations that were marginalized over to get the Student
We employed nested sampling [
In this section we present the SSF MoG light sensor model estimated from the laboratory data and evaluate its efficacy by demonstrating a significant improvement of the performance in the autonomous robotic platform.
The light sensor data collected using the blackandwhite boundary pattern are illustrated in Figure
This figure illustrates the intensity measurements,
The nested sampling algorithm produced the mean SSF fields for each of the MoG models tested, as well as the corresponding log evidence. Table
A comparison of the tested MoG SSF models and their respective log evidence (in units of
MoG model order  Log evidence  Number of parameters 

1 Gaussian 

6 
2 Gaussian 

12 
3 Gaussian 

18 
4 Gaussian 

24 
This figure illustrates the SSF obtained from the four MoG models along with their corresponding logevidence values.
A comparison of the observed data (red) with predictions (black) made by the SSF field estimated using the single twodimensional Gaussian MoG model.
In the next section, we demonstrate that explicit knowledge about how the light sensor integrates light arriving from within its fieldofview improves the inferences one can make from its output.
The mean SSF field obtained using a single twodimensional Gaussian model (Figure
The three panels comprising Figure
(a) These three panels illustrate the robot’s machine learning system’s view of the playing field using the naïve light sensor model as the system progresses through the first three measurements. The previously obtained measurement locations used to obtain light sensor data are indicated by the blackandwhite squares indicating the relative intensity with respect to the naïve light sensor model. The next selected measurement location is indicated by the green square. The blue circles represent the 50 hypothesized circles sampled from the posterior probability. The shaded background represents the entropy map, which indicates the measurement locations that promise to provide maximal information about the circle to be characterized. Note that the low entropy area surrounding the white square indicates that the region is probably inside the white circle (not shown) and that measurements made there will not be as informative as measurements made elsewhere. The entropy map in Figure
In contrast, the three panels comprising Figure
This additional information can be quantified by observing how many measurements the robot is required to take to obtain estimates of the circle parameters within the same precision in the cases of each light sensor model. Our experiments revealed that on average it takes 19 ± 1.8 measurements using the naïve light sensor model compared to an average of 11.6 ± 3.9 measurements for the more accurate SSF light sensor model.
The quality of the inferences one makes from a sensor depend, not only on the quality of the data returned by the sensor, but also on the information one possesses about the sensor’s performance. In this paper we have demonstrated via a case study, how more precisely modeling a sensor’s performance can improve the inferences one can make from its data. In this case, we demonstrated that one can achieve about 18% reduction in the number of measurements needed by a robot to make the same inferences by more precisely modeling its light sensor.
This paper demonstrates how a machine learning system that employs Bayesian inference (and inquiry) relies on the likelihood function of the data given the hypothesized model parameters. Rather than simply representing a noise model, the likelihood function quantifies the probability that a hypothesized situation could have given rise to the recorded data. By incorporating more information about the sensors (or equipment) used to record the data, one naturally is incorporating this information into the posterior probability, which results in one’s inferences.
This is made even more apparent by a careful study of the experimental design problem that this particular robotic system is designed to explore. For example, it is easy to show that by using the naïve light sensor model, the entropy distribution for a proposed measurement location depends solely on the number of sampled circles for which the location is in the interior of the circle and the number of sampled circles for which the location is exterior to the circle. Given that we represented the posterior probability by sampling 50 circles, the maximum entropy occurs when the proposed measurement location is inside 25 circles (and outside 25 circles). As the robot’s parameter estimates converge, one can show that the system is simply performing a binary search by asking “yes” or “no” questions, which implies that each measurement results in one bit of information. However, in the case where the robot employs an SSF model of the light sensor, the question the robot is essentially asking is more detailed: “to what degree does the circle overlap the light sensor’s SSF?” The answer to such a question tends to provide more information, which significantly improves system performance. One can estimate the information gain achieved by employing the SSF model. Consider that the naive model reveals that estimating the circle’s position and radius with a precision of 4 mm given the prior information about the circle and the playing field requires 25 bits of information. The experiment using the SSF model requires on average 11.6 measurements, which implies that on average each measurement obtained using the SSF model provides about 25/11.6 = 2.15 bits of information. One must keep in mind, however, that this is due to the fact that the SSF model is being used not only to infer the circle parameters from the data, but also to select the measurement locations.
Because the method presented here is based on a very general inferential framework, these methods can easily be applied to other types of sensors and equipment in a wide variety of situations. If one has designed a robotic machine learning system to rely on likelihood functions, then sensor models can be incorporated in more or less a plugandplay fashion. This not only promises to improve the quality of robotic systems forced to rely on lower quality sensors, but it also opens the possibility for calibration on the fly by updating sensor models as data are continually collected.
The authors have no conflict of interests to report.
This research was supported in part by the University at Albany Faculty Research Awards Program (Knuth) and the University at Albany Benevolent Research Grant Award (Malakar). The authors would like to thank Scott Frasso and Rotem Guttman for their assistance in interfacing MatLab to the LEGO Brick. They would also like to thank Phil Erner and A. J. Mesiti for their preliminary efforts on the robotic arm experiments and sensor characterization.