What a ball hears?
AT&T 5G Sports Hackathon 2021 (Winner)
UX Research | UX Design | Accessibility | Market Research | Business Model | User Study
User research through talking with experts and research professors
Brainstorming through market research & understanding the business model impact through domain study
User interviews and testing with participants
2 Researchers, 2 Designers & 1 Developer.
48 hours at Butler University in October 2021.
The AT&T Hackathon was held to further the inclusion of 5G in sports challenges like Fan Engagement, IoT and Venue Safety.
The team worked on creating an inclusive solution that uses the 5G low latency and spatial audio of devices like the Beats headphones and AirPods Pro. The solution proposes the use of audio sensors inside the ball to capture and relay the game audio to the users. Location sensors within the ball will guide the spatial audio technology to simulate the game play through the left or right headphones.
User Interface flows and simulation of the audio for left and right halves of the court. As a part of the final deck, we also researched the business impact of the project.
The team won prizes for Best Accessibility, Best use of 5G and Grand Prize - Best Overall Solution.
Exploratory research into understanding feasibility of our idea for users and testing out the concept with participants.
How might we enhance the experience of a basketball game for the average NBA fan and make it inclusive for visually impaired fans
The 94Fifty Smart Basketball Sensor and Wilson X connected basketball can be used for shot perfection.
Apple AirPods Pro and Beats promise a surround sound experience.
Deep learning has been used to track player movement across the court to identify defense and offense strategies.
Samsung and IrisVision have collaborated on a headset that records video and replays a magnified version helping legally blind fans experience the match.
The visually impaired, on an average, can comprehend speech sped up to 25 syllables per second vs others who can process maximum of 10 syllables. The cerebral cortex that normally responds to vision responds to speech.
Beep Baseball - The ball and the bases emit a beeping sound acting as beacons to help the visually impaired play.
Bell Basketball - a basketball with embedded audio enables the visually impaired to play basketball by tracking the position of the ball via its sound.
.png)
We explored technologies like Refreshable Braille displays and tactile displays to make the solution more inclusive. Spatial audio was included to make the experience more immersive for all fans.
Tactile graphic displays are heavy and awkward to carry, furthermore, they are quite expensive so only a handful of blind people are able to accurately operate and read them. Using something like this for relaying a game would involve a high number of manhours dedicated to identifying and coding each unique game event into the display.
- Expert User with research experience in the field of accessibility
1. Audio sensor inside the basketball captures sounds from the gameplay.
2. Cameras around the court transmit the gameplay live which is used to track ball location across the court.
3. The data is transmitted to the cloud where Machine Learning and Image Processing is used to track ball location. Noise reduction is applied to the audio data to remove background data.
4. The resultant audio feed is transmitted to the mobile application.
The 5G network can ensure zero latency and fast transmission in the entire process for the audio so users can participate in a stadium like experience.
.png)
To simulate the experience of watching a live game with no visual cues we tested with blindfolded participants who were asked to describe what is happening in the game simply by listening to it.
Proper knowledge of the game was required to accurately understand what was happening on court
Participants were able to identify location of the ball on the court and passing of the ball from one player to another
Audience noise and player noise were counted as misleading to make conclusions about what was happening on the court
Periods when the game was suspended was confusing to the players as there was no audio cue to understand this occurrence
Read our paper titled ' Immersive Fan Engagement in Basketball Using Spatial Audio and 5g' for the results of our user study and exploratory research. The paper is purely for academic research purposes and is not published on any forum.
Click Here to read our paperThe team walked out of the hackathon extremely proud of what we had achieved in 3 days. Our main aim going into the hackathon was to create and learn which I believe we fulfilled completely.
.jpeg)
This project was a great learning experience in terms of designing for accessible products. In the beginning most of us were overthinking the accessibility aspect of the product. William Lawrence (pictured here), Accessibility Solutions Engineer at AT&T, was key in understanding how accessible products are designed and created in the real world.
.jpeg)
Overall, I believe that the team's focus on creating a valuable product rather than a winning product is what made us stand out in the end. I take this mindset with me into every project that I work on, focusing on adding value and creating solutions.
