Among the many consumer products Cardinal Peak is involved with, we have worked on multiple true wireless earbuds projects. These systems have several unique engineering challenges beyond your standard Bluetooth wireless headphones.
The first challenge is coordinating the actions of multiple independent devices (left bud, right bud, smart case and audio source) into one cohesive system. Part of the challenge here is that the reference designs from the silicon vendors are often not sufficient. Therefore, it is left to the development to build out additional components to incorporate into a robust system. With truly wireless earbuds, increasing battery size is not an option, so the second major challenge is minimizing power as needed.
This case study is particularly focused on the system issues of embedded software engineering (the state machine) and how they impact user experience and power consumption.
For more information on developing wireless earbuds with Qualcomm parts, check out our case study.
What are True Wireless Earbuds (TWE)?
True wireless Bluetooth earbuds are two separate Bluetooth earbud devices communicating wirelessly between each other and an audio source to provide convenient cord-free access for on-the-go music streaming, remote gaming and hands-free voice/video calling. Often, a smart charging case is integrated into the system as well. When included with extended features such as noise cancellation, voice assistant integration and biometric monitoring, the set is also referred to as a hearable (a hybrid of “wearable headphone”).
In 2020, true wireless earbuds were the most desired form factor for on-the-go headphone use, and adoption doubled from the previous year. Additionally, users demand the best audio fidelity without glitching or dropouts, and they want the most battery life possible between charges (9-18 hours).
The above diagram details a typical TWE system. An audio source (typically a PC or smartphone) connects to the primary earbud and transmits stereo left/right music (A2DP) and voice call data (HFP) over classic Bluetooth BR/EDR as normal for a Bluetooth headset receiver. However, the primary earbud maintains a multipoint music share connection with the secondary earbud, synchronizing the audio play over Bluetooth SPP and caching streaming data where necessary. A smart case can be involved to control charging, provide power stability during firmware updates and control power management features, such as ship mode, communicating status messages to the system over Bluetooth BLE GATT.
What are the Challenges When Designing Truly Wireless Earbuds?
Developing a TWE system is technically challenging because it involves coordinating the actions of two independent Bluetooth devices and, frequently, an integrated smart case into a unified system behavior in order to appear as a single unit to an audio source. The system must be engineered to provide consistent, predictable behavior even when one of the elements is out of communication, such as an earbud being left in its charging case. Ideally, post-manufacturing system configuration should be supported to allow a user to replace one malfunctioning component rather than the entire system.
Glitch-free audio is always the user expectation, and TWE systems are particularly challenged in this area. In addition to sharing the ISM (Industrial, Scientific, and Medical) frequency band with other products in the environment, truly wireless earbuds must overcome radio interference from their own component devices sending/receiving data in the same moments. Solving this issue requires detailed knowledge of the frequency band and the Bluetooth configuration.
In addition, since earbuds need to be as lightweight and compact as possible, battery size must be minimized and battery life maximized to provide a useful performance period between charges. The common solution is to turn off power-consuming features when the earbuds are not in active use, but the challenge is to recognize and respond to these user experience transitions accurately since one misinterpretation can result in nonperformance of the product.
Typically, the software development kit and drivers provided by Bluetooth and component device hardware manufacturers (such as Qualcomm) are sufficient for standard headphone development and usage. However, these starter solutions are often not technically advanced enough to manage the challenges detailed above, so it is left to the product engineering team to manage the increased load on communication, data caching and peer device coordination. Designing and deploying a reliable TWE system requires a digital product engineering team familiar with the details of Bluetooth streaming data management.
How Does a State Machine Streamline Wireless Earbud Engineering?
When Cardinal Peak was presented with the requirement to engineer a TWE platform based on a standard Bluetooth headset implementation, we leaned on our substantial audio experience to specify an extensible two-domain state manager to control and monitor a multi device system as it transitioned through its user and software behavior states. This manager performed on both buds in parallel, continually synchronizing over Bluetooth SPP (Serial Port Profile). This allowed for predictable and stable performance of both earbuds whether paired or operating individually.
In the diagram above, the user state of the TWE earbud is managed separately from its software state, reducing the complexity of the state interactions. Without this separation, the number of required states and interactions multiplies in order to capture all the potential combinations, increasing the chance of a bug that places the unit in an unexpected state and exploding the size of the test matrix necessary to cover it. The strategy also allows for alternate physical architectures on the same firmware platform, such as a product having no case or having a case lid open/closed condition as part of the system behavior. Here, the product not only has a case (detected by the earbud when having a charging voltage present), but it also has sensors to detect in-ear insertion. Heuristics help distinguish genuine in-ear seating from other proximity situations, such as being held in hand.
As an earbud cycles through its user states of being removed from its case to in-ear insertion, the user state machine issues the required events to update the software state machine, which has direct control over the system behavior. This platform allows for significantly extended battery life by adding context awareness of the operating environment: accurately detecting whether an earbud is active in an ear, stored in its case, idle out of its case or in transit between these states. This allows power-consuming features (such as amplification, active noise cancellation, etc.) to be turned off in moments of idle usage and restored back when detecting user intent to operate the device.
For example, battery drain from a sensor subcomponent was able to be managed by mapping the various software states to sensor user behaviors. Its accelerometer was placed into a minimum power state during hibernation by being tuned to low-frequency sampling of major movements (watching for significant changes in motion related to being picked up from a surface), placed into full power performance along with infrared (IR) and proximity sensors when awakened (watching for small changes in motion relating to in-ear placement) and then completely turned off once seated (relying on only the IR sensor to detect removal from ear).
When turning off system behaviors to save battery power, close analysis and testing of the state decisions must be performed in order guarantee a unit will return to expected functionality when the user interacts with it. In a noisy Bluetooth environment, it is possible for a communication-based event (such as an earbud signaling its peer that it has been removed from the case) to not be received. It is useful in these situations to distinguish reliable, determinate events from unreliable events and give those priority to update the system. In the example above, noticing the application of charger voltage to the earbud will reset the unit into its in-case user state, regardless of what state the unit previously thought it was in.
Further building on this platform, Cardinal Peak’s Bluetooth product experts were able to overcome radio interference between buds, allowing clear stereo music streaming and voice calling, by manipulating Bluetooth frequencies and services based on this state knowledge. For example, secondary earbud BLE advertising can destructively interfere with HFP voice call data transmission when happening simultaneously in close proximity. Having a context awareness of the current usage of the TWE system allows for secondary advertising to be turned off in instances when it is out of case and connected to its primary bud via Bluetooth.
By approaching the challenge with a “system view” rather than just a “get it working” attitude, combined with industry-standard specification, design and testing practices, Cardinal Peak’s team of experienced engineers was crucial in developing a line of successful TWE products from specification through to final product release.
References
Qualcomm / The State of Play Report 2020 / A global analysis of user behaviors and desires driving consumer audio / https://assets.qualcomm.com/the-state-of-play-2020.html
