Using Lossy Video Compression in the Courtroom

Howdy Pierce

I’m at the DSI conference in Las Vegas today, presenting a primer for law enforcement investigators on how video compression works and trying to answer the question of why “lossy” compression should be considered reliable for use in courtrooms. (My slides are available here, and I welcome comments on them.) I think I was invited to speak because of our CaseCracker product, which is used to record custodial interrogations, although what I’m discussing is only slightly related.

The lack of trust in digital media compression in a forensic setting is primarily a PR issue for the media compression industry, if such an industry can be said to exist. We use terms like “lossy compression” and “predicted blocks”—terms that have relatively precise technical meaning. But these terms also have a slightly different meaning to laymen, and that everyday meaning isn’t exactly reassuring if you’re a judge relying on testimony compressed using a lossy compression algorithm. So it’s important for lawyers and investigators working in the criminal justice system to understand how image compression works.

The technical meaning of “lossy compression” is that the process of encoding followed by the process of decoding doesn’t output the exact same file as the source file you started out with:

When we say the output file isn’t the same as the source file, what we mean is that a byte-for-byte comparison of the two files will fail—not that a guy protesting his innocence will be turned into a different guy admitting his guilt. In fact, with a well-implemented codec, the mathematical lossiness shouldn’t be subjectively noticeable at all. Intuitively, everyone knows that: Nobody worries about using lossy media compression for recording videos of their kids’ birthdays or pictures of their vacations.

But still, it’s worth thinking about the question as to how to state with certainty that lossy compression algorithms should be considered reliable for courtroom use.

In preparing for this talk, I tried to think of all the ways that video compression is lossy. I came up with four independent sub-processes that each contribute to a codec’s overall lossiness:

  • Resolution reduction: Often the video resolution is reduced prior to encoding, because this can dramatically diminish the number of bits to encode. The result is that the output is fuzzier and less crisp.
  • Color sub-sampling: The human eye is not equally sensitive to luminance and chrominance changes, so chroma is normally subsampled, which typically reduces the color information in the picture by a factor of 4 and the total uncompressed size of the picture by a factor of 2. The color sub-sampling is not usually perceptible except in test patterns explicitly designed to expose it.
  • Noise reduction and other pre-filtering: Sometimes video encoders, particularly expensive ones, will filter the image prior to encoding in order to remove noise and otherwise make the image easier to compress. This might result in a softer image in certain cases, but again it normally won’t make any subjective difference in the output.
  • Quantization: This is a technical term that loosely translates to “rounding”. The basic idea is that the human eye can’t usually discern small differences in intensity. So why waste a lot of bits faithfully preserving the difference between a 66% gray block and a 69% gray block, when the viewer will perceive them as the same thing anyway? By quantizing both blocks to an average value—say, 67% gray—the encoder is able to dramatically reduce the amount of information it needs to send. (The same concept applies to high frequencies in the image.) Quantization is responsible for the majority of lossiness in video compression, but again, its use is normally not perceptible except in the lab.

I’m not a lawyer, thank heaven, but I’m pretty sure the relevant legal issue is whether a piece of video evidence accurately reproduces the event it purports to record. And so in a law enforcement setting, the ultimate answer is that someone who is trusted needs to be able to testify that a particular video clip faithfully represents what happened.

Related posts:

Categories: Howdy, Video

Cardinal Peak
Learn more about our Audio & Video capabilities.

Dive deeper into our IoT portfolio

Take a look at the clients we have helped.

We’re always looking for top talent, check out our current openings. 

Contact Us

Please fill out the contact form below and our engineering services team will be in touch soon.

We rely on Cardinal Peak for their ability to bolster our patent licensing efforts with in-depth technical guidance. They have deep expertise and they’re easy to work with.
Diego deGarrido Sr. Manager, LSI
Cardinal Peak has a strong technology portfolio that has complemented our own expertise well. They are communicative, drive toward results quickly, and understand the appropriate level of documentation it takes to effectively convey their work. In…
Jason Damori Director of Engineering, Biamp Systems
We asked Cardinal Peak to take ownership for an important subsystem, and they completed a very high quality deliverable on time.
Matt Cowan Chief Scientific Officer, RealD
Cardinal Peak’s personnel worked side-by-side with our own engineers and engineers from other companies on several of our key projects. The Cardinal Peak staff has consistently provided a level of professionalism and technical expertise that we…
Sherisse Hawkins VP Software Development, Time Warner Cable
Cardinal Peak was a natural choice for us. They were able to develop a high-quality product, based in part on open source, and in part on intellectual property they had already developed, all for a very effective price.
Bruce Webber VP Engineering, VBrick
We completely trust Cardinal Peak to advise us on technology strategy, as well as to implement it. They are a dependable partner that ultimately makes us more competitive in the marketplace.
Brian Brown President and CEO, Decatur Electronics
The Cardinal Peak team started quickly and delivered high-quality results, and they worked really well with our own engineering team.
Charles Corbalis VP Engineering, RGB Networks
We found Cardinal Peak’s team to be very knowledgeable about embedded video delivery systems. Their ability to deliver working solutions on time—combined with excellent project management skills—helped bring success not only to the product…
Ralph Schmitt VP, Product Marketing and Engineering, Kustom Signals
Cardinal Peak has provided deep technical insights, and they’ve allowed us to complete some really hard projects quickly. We are big fans of their team.
Scott Garlington VP Engineering, xG Technology
We’ve used Cardinal Peak on several projects. They have a very capable engineering team. They’re a great resource.
Greg Read Senior Program Manager, Symmetricom
Cardinal Peak has proven to be a trusted and flexible partner who has helped Harmonic to deliver reliably on our commitments to our own customers. The team at Cardinal Peak was responsive to our needs and delivered high quality results.
Alex Derecho VP Professional Services, Harmonic
Yonder Music was an excellent collaboration with Cardinal Peak. Combining our experience with the music industry and target music market, with Cardinal Peak’s technical expertise, the product has made the mobile experience of Yonder as powerful as…
Adam Kidron founder and CEO, Yonder Music
The Cardinal Peak team played an invaluable role in helping us get our first Internet of Things product to market quickly. They were up to speed in no time and provided all of the technical expertise we lacked. They interfaced seamlessly with our i…
Kevin Leadford Vice President of Innovation, Acuity Brands Lighting
We asked Cardinal Peak to help us address a number of open items related to programming our systems in production. Their engineers have a wealth of experience in IoT and embedded fields, and they helped us quickly and diligently. I’d definitely…
Ryan Margoles Founder and CTO, notion