The gaming industry has come a long way since vector graphics (think: 1983 Star Wars arcade game) and 8-bit sound. Today’s companies are innovating rapidly, delivering high-definition visuals, surround sound and realistic content while providing developers with the advanced tools to create deeper immersive experiences.
In contrast, innovation in haptics has been slow. Haptics have been used in gaming since the early gamepads - but, generic, dull buzzes did not add much to the experience. To match the giant leaps made in audio and visual, forward-thinking gaming companies need to make high-resolution, intelligent haptics a core feature of their products. As more studies confirm that haptics contribute to enhanced gaming performance, there is a new, fresh focus on high-quality tactile feedback.
Vector graphics in 1983 Atari Star Wars arcade game
The Nintendo Switch entered the market with a big splash as the first console/portable hybrid for gaming on the go. The Switch Console touchscreen and the gamepads (called “Joy-Cons”) both offer haptic feedback dubbed ‘HD Rumble’.
The Lofelt Engineering team couldn’t wait to take a look under the hood of the Switch and replace the built-in haptic actuators inside the Joy-Cons with the Lofelt L5 haptic actuators for some upgraded haptics. Lead Prototyping Engineer, Andrew, walks us through the tear-down and haptic enhancement.
Day 1: Discovery
For any type of hack, it’s important to get a deeper understanding of what you’re working with, rather than just opening things up and pulling out wires. I want to identify the internal construction and components, see how they’re laid out in relation to one another.
In the first prototype, I’m going to replace the existing Joy-Con actuator with a Lofelt L5 actuator and have the Joy-Con drive it from the original Switch audio signal. To do this, I opened up the Joy-Con to determine how to best position the L5 actuator.
Although space is always tight in small devices, after conducting measurements, the L5 actuator (20.5mm (L) x 17mm (W) x 6.2mm (H)), actually fits easily inside the Joy-Con. Next step is to determine the best position.
Day 2: Tear Down
Today, I will be taking out the existing electronics of the Joy-Con to replace it with the L5 actuator. This involves drilling holes and removing the PCBs and flexible connectors without damaging them.
I want to understand how the vibration will interact with the unit. All non-critical and non-structural plastics can be altered to fit the L5 actuator dimensions. I need to be careful not to compromise the initial design and weaken the unit structurally, which could lead to mechanical vibration and premature failure of the unit.
Day 3: Installation
Next, it’s time to position and place the L5 actuator inside the Joy-Con. I need to pay close attention to displacement and also make sure the membrane doesn't “hit” against the unit when vibrating. Before I do a more permanent fix of the L5 actuator inside the Joy-Con, I’ll first install it with double-sided tape and test that nothing rattles.
In a first quick test, the L5 actuator works out of the box and the lower resonant frequency gives a slightly different experience. Playing the game, Thumper, explosions can be felt better, but some high frequency textures and clicks are less prominent.
One of the more important findings so far, is that there may be ways to improve the haptic feedback by also hacking the Switch software and driving the L5 actuator with the Lofelt intelligent firmware. Let’s go deeper with this scenario.
Day 4: Digging Deeper
The original game titles on Switch don’t have full wideband haptic information to maximize the potential of the L5. For this purpose, Lofelt developed a sophisticated DSP that converts game audio into wideband haptics in real-time. The DSP is part of a firmware running on an STM M4 processor. Today, I will modify another Joy-Con with an L5 actuator and then also attach it to the Lofelt EVK running Lofelt’s firmware and DSP algorithms. The Lofelt EVK will replace the built-in actuator control IC and drive haptic feedback directly from the headphone output of the Switch. Because the Lofelt audio-to-haptic firmware is a stereo signal and adapts in real-time, the L5 actuator’s large bandwidth is suited perfectly and creates more nuanced, natural vibrations.
Like above, I’ll position and place the L5 actuator inside the Joy-Con and then run cables from the headphone output on the Switch to test the connections.
Next, I’ll configure the software by connecting the Lofelt EVK to a computer and use a Command Line Interface (CLI) to adjust parameters in real-time and match the signal processing to the output of the Nintendo.
The Switch is now set up to be used with the new L5 actuator and Lofelt EVK Joy-Con configurations.
Day 5: Comparison
Below is a graph comparing the frequency response between the original actuator and the L5 actuator. This frequency response results are purely each actuator attached to 30g and acceleration measured for a frequency sweep. No additional DSP, just each actuator on their own.
The lower resonant frequency of the L5 actuator allows haptic feedback that is in line with the perceptible tactile range of the skin. Mechanically, it’s far more challenging to design an actuator that produces powerful low frequencies. For human perception, low frequencies are key to creating a pleasant experience. High frequencies above 250Hz can create numbness and tingling over a longer period of time.
Looking at the frequency graph for our L5 actuator there is a range from 45 to 425 Hz (total 380 Hz wide range) where you can feel 1G or more acceleration force.
For the Nintendo actuator that range is much smaller 162 to 190 Hz and from 325 to 350 Hz (total 53 Hz wide range).
At an all-hands meeting, my colleagues conducted user testing with some of their favorite games - TumbleSeed and Thumper - and then our Experience Designer, Bianca, compared results of the original actuator vs the L5 actuator.
During longer vibrations lasting 0.5 seconds or more, the original actuator peaks in quick repetition, giving the sense of a series of discrete instances of haptic feedback rather than a single continuous one. On the other hand, the Lofelt L5 actuator can achieve this more refined sensation if compared on the same vibration track, one to one.
The real magic happens when the Lofelt L5 actuator is driven by the patented dynamic audio-to-haptic DSP in response to the game audio. The stereo audio track from Lofelt firmware maps to the player’s two hands, making gameplay more immersive and natural. Haptics by Lofelt also simulates different effects instead of simple, uniform vibrations: the thundering boom of a moving tank and the sound of footsteps on different surfaces feel very different when interpreted as haptic feedback, improving gamers’ reaction times when speed is important. With its larger bandwidth, combined with a stereo audio track and conversion updated in real-time, the Lofelt L5 actuator results are impressive.