Sometimes, Noise Helps

Mike Perkins

I’ve been working on a fun problem lately that involves estimating a scalar parameter from a set of repeated observations. It turns out that in certain circumstances, the presence of noise in the system can actually make the estimate more accurate, which is a little counterintuitive and also kind of cool.

In my case, I have a number of observations (pixels), and I am trying to analyze a large number of them to estimate the parameter of interest. The image capture system I’m using quantizes the incident light into 8 bits per pixel—that is, to an integer between 0 and 255. However, I have reason to believe that the underlying analog sensor is fundamentally more accurate than the quantizer’s step size, and I would like to recover the most accurate possible value for the parameter.

The diagram below abstracts the problem:

Quantizer System

Here alpha is the parameter I want to estimate, the Yi values are noise, Q represents the quantizer, and the Xi values are the observations available to my estimator. Can it be that my estimator will perform better in the presence of noise than in its absence?

Let’s assume that the quantizer is uniform and that analog values are rounded off to the nearest integer. The diagram below helps to visualize Q’s operation.

Fig-2

We’ll call the integer values, indicated by the blue vertical lines, the reconstruction levels. These are the outputs of the quantizer. We’ll call the values corresponding to the red lines the decision levels. The quantizer operates by determining between which two decision levels a value lies, and then outputting the reconstruction level between the same two decision levels. For example, if the value to be quantized is 126.4, the quantizer determines that the value lies between 125.5 and 126.5 and outputs 126. (Not to digress too far into quantization theory, but if the marginal distribution of the random process being quantized isn’t uniform, a uniform spacing of the decision levels is not optimal in a mean-square sense).

Now, how about my problem of estimating alpha? Let’s assume alpha is a constant 126.4. If there is no noise, then every Xi value will be identical and equal to 126. There is no reason to estimate alpha as anything other than 126. In the absence of noise, the quantizer limits my ability to estimate alpha. However, if noise is added to each observation, then nearby integers will also be output by the quantizer: 124, 125, 127, 128 etc. It’s reasonable to ask if the mean of the Xi observations will yield a better estimate.

To approach this analytically we need to make some assumptions about the noise. We’ll make life easy and assume the Yi values are independent identically distributed zero mean normally distributed random variables. In this case it is easy to write a script for computing the expected value of the quantizer’s output: E[X]. For those interested in the computation, the accompanying python script provides the details. In practice, we would observe as many of the Xi values as we can, average them together, and use that average as our estimate. The average should converge to E[X] as more and more observations are made.

The graph below, where alpha is set to 123.4, shows how noise helps. Clearly, as the standard deviation of the noise increases, the estimate becomes better. Fast. With a standard deviation as small as 0.5 the estimate is essentially perfect.

Accuracy

Fortunately for me, there is noise in my image capture system, and I have thousands and thousands of pixels over which to average, so my estimate can converge to E[X].

Cardinal Peak
Learn more about our Audio & Video capabilities.

Dive deeper into our IoT portfolio

Take a look at the clients we have helped.

We’re always looking for top talent, check out our current openings. 

Contact Us

Please fill out the contact form below and our engineering services team will be in touch soon.

We rely on Cardinal Peak for their ability to bolster our patent licensing efforts with in-depth technical guidance. They have deep expertise and they’re easy to work with.
Diego deGarrido Sr. Manager, LSI
Cardinal Peak has a strong technology portfolio that has complemented our own expertise well. They are communicative, drive toward results quickly, and understand the appropriate level of documentation it takes to effectively convey their work. In…
Jason Damori Director of Engineering, Biamp Systems
We asked Cardinal Peak to take ownership for an important subsystem, and they completed a very high quality deliverable on time.
Matt Cowan Chief Scientific Officer, RealD
Cardinal Peak’s personnel worked side-by-side with our own engineers and engineers from other companies on several of our key projects. The Cardinal Peak staff has consistently provided a level of professionalism and technical expertise that we…
Sherisse Hawkins VP Software Development, Time Warner Cable
Cardinal Peak was a natural choice for us. They were able to develop a high-quality product, based in part on open source, and in part on intellectual property they had already developed, all for a very effective price.
Bruce Webber VP Engineering, VBrick
We completely trust Cardinal Peak to advise us on technology strategy, as well as to implement it. They are a dependable partner that ultimately makes us more competitive in the marketplace.
Brian Brown President and CEO, Decatur Electronics
The Cardinal Peak team started quickly and delivered high-quality results, and they worked really well with our own engineering team.
Charles Corbalis VP Engineering, RGB Networks
We found Cardinal Peak’s team to be very knowledgeable about embedded video delivery systems. Their ability to deliver working solutions on time—combined with excellent project management skills—helped bring success not only to the product…
Ralph Schmitt VP, Product Marketing and Engineering, Kustom Signals
Cardinal Peak has provided deep technical insights, and they’ve allowed us to complete some really hard projects quickly. We are big fans of their team.
Scott Garlington VP Engineering, xG Technology
We’ve used Cardinal Peak on several projects. They have a very capable engineering team. They’re a great resource.
Greg Read Senior Program Manager, Symmetricom
Cardinal Peak has proven to be a trusted and flexible partner who has helped Harmonic to deliver reliably on our commitments to our own customers. The team at Cardinal Peak was responsive to our needs and delivered high quality results.
Alex Derecho VP Professional Services, Harmonic
Yonder Music was an excellent collaboration with Cardinal Peak. Combining our experience with the music industry and target music market, with Cardinal Peak’s technical expertise, the product has made the mobile experience of Yonder as powerful as…
Adam Kidron founder and CEO, Yonder Music
The Cardinal Peak team played an invaluable role in helping us get our first Internet of Things product to market quickly. They were up to speed in no time and provided all of the technical expertise we lacked. They interfaced seamlessly with our i…
Kevin Leadford Vice President of Innovation, Acuity Brands Lighting
We asked Cardinal Peak to help us address a number of open items related to programming our systems in production. Their engineers have a wealth of experience in IoT and embedded fields, and they helped us quickly and diligently. I’d definitely…
Ryan Margoles Founder and CTO, notion