Knockout League

User Experience Report

SAMPLE

May 21st, 2017

Vivestudios Grab studios

About this report

Between May 19th and May 21st, 2017, Fishbowl VR conducted confidential usability playtests of Knockout League with 10 members of our testing panel. The testers played from home using their own VR devices and followed the think-aloud protocol. The only instructions provided were to use the app as they normally would, and to complete a short post-test survey. All playtest sessions were recorded, and recordings show both what the testers are doing and what they are seeing in their HMD.

What follows is a summary and details of the user experience of Knockout League. The results include:

  • A Net Promoter Score, an index ranging from -100 to 100 that measures the willingness of users to recommend the app to others. This serves as a proxy for gauging overall satisfaction with the application, and how user satisfaction compares to that of similar applications.
  • An overview of technical and performance issues encountered, with each issue classified on a scale from Severity 1 (critical issue or crash) to Severity 4 (fit and finish issues).
  • High-level explanations for what testers liked the most about the app, what if anything they found frustrating, and what they would change about the app if given a "magic wand".
  • Detailed tester profiles and system specifications, as well as log files from each playtest session and the raw video of each test conducted.

We recommend having user experiences reports conducted with each major update throughout the development of an app as these tests will help you to identify problems areas more efficiently, to generate ideas for new features, and to solve design arguments and questions around product prioritization.

For questions or to order additional reports visit www.fishbowlvr.com or email founders@fishbowvr.com

About Fishbowl VR

Fishbowl VR has built a testing and user insights platform to help developers figure out what works and what doesn't in VR via a remote panel of 800+ active VR users. The company is a graduate of both Techstars and HTC's Vive X accelerator programs, and counts Schell Games, Vive Studios, High Fidelity and many other top VR studios among its client base.

User Satisfaction

Net Promoter Score: 40

50% Promoters
40% Neutral
10% Passive
Individual Responses
Score Name Comment
10 Tim S. Huge fan of Punch Out and this is the closest thing yet to seeing that in VR. Video
10 Jacob G. The gameplay is fantastic, with great opponent design and animations. Video
10 Alexander S. Great cardio workout in vr. Close to perfect tracking. It reminds me of Mike Tysons Puchout on NES. Video
10 Tom B. Replay value. I broke enough of a sweat that I could not complete the game before beating all of the fighters and definitely see myself coming back to attempt to do so. They offered a high enough challenge that my competitive spirit broke out. Video
9 Ron D. Very polished, lots of fun, good workout Video
8 Thomas P. The overall game experience is fun and makes me want to experience more of it. Video
8 Chris T. It feels polished, visually and sound wise. The fighting patterns feels a bit rigid at times, but overall, I feel like the style is done with taste and polish. It's sort of difficult, would be nice to have a difficultly option. Video
7 Robert P. Great production values and was fun to play when it has all the issues ironed out I could see this being a 9 or 10! Video
7 Chris E. I found my self more entertained by the app. I think the teaching session in the ring could be improved by visual demonstration in addition to text instruction. I think the boxing match part would be more fun if a coach was there helping or the option of having a coach. Video
6 Aaron E. Really fun but still has immersion problems Video

Technical and Performance Issues

Critical
Name Comment
Robert P. Game got stuck with no audio and video for a while. Game didn't want to start when I had the Rift OVR service running in the background. One point I lost sound and had to restart the title to get it to work. Then I changed my audio output to my speakers and then the sound started working again, BUT from the Vive instead of the speakers. Video
Tom B. The first time I launched, I got to the screen where I was given the option of higher or lower visual quality. Out of curiosity, I changed to the higher quality. The display changed to the SteamVR loading screen and would not return to the game. I could hear the ambient noise from the game as if the changes had been made but my vision never returned. Had to reboot the game to fix. Video
Major
Name Comment
Ron D. 2nd boss froze, had to cancel out of the match and restart. Video
Thomas P. In the match menu, the video that plays of the fighter (your opponent) has a massive impact on the performance. And when I then started the match, the FPS was too low to comfortably continue playing the game. I had to stop. A restart didn't fix this. Although I do suspect other processes may have had an impact on this (Oculus app auto starting for example, or Windows anti malware suddenly kicking in). Video
Minor
Name Comment
Chris T. Second fighter would freeze randomly but follow my movements. Had to exit fight to create a rematch. Video
Cosmetic
Name Comment
Jacob G. Objects in the distance had a shimmering effect to them, possibly due to trying to display hi-res textures too small on the Vive's screens. Video
Alexander S. My chaperone wasn't working at first had to restart. One time the gloves disappeared. Video
Aaron E. With a VR game, I don't think I shoudl have to be alt tabbed in to hear sound. Video

What People Liked

Name Comment
Tim S. character animation. Also really enjoyed the new training!
Robert P. Graphics were fantastic, great sense of scale and depth. Really felt like I was getting punched in the face! Reminds me of the good qualities of punch out.
Jacob G. The character animations were very fluid and suited their character design. Reaction timing was specific enough to feel skillful without being too difficult.
Ron D. Satisfaction of knockout. Minigames were good challenge
Thomas P. It went beyond my expectations by offering a pretty complete package, starting with fun training, matches, and a very cool visual style.
Alexander S. The cartoony art design. The trainer and his attitude. The characters you fight are all different, unique and fun not what you'd expect.
Chris T. It was fun and interesting.
Tom B. All of the ambient sounds, visuals, etc offered a level of immersion that really brought you into what it must feel like to be in the boxing ring. I also highly enjoyed the challenge that the fighters provided as well as the varying fighting styles and moves. I needed to completely change strategy for the second fighter from the first.
Chris E. The graphics were really good. Sounds and the coach character was entertaining. I like the mitt training and the speed bag training.
Aaron E. Success feels great!

What Frustrated People

Name Comment
Tim S. Nothing really. If I had to pick anything I would say lack of characters I sure will be frustrating when I get that far only because the game is such a blast!
Robert P. Sound issues, crashing issues, failure to close properly issues.
Jacob G. I kept getting hit because I failed to dodge, not sure if its my own mistake from not moving far enough or if there's problems with the dodge detection.
Ron D. 2nd boss is tough :) Music is repetitive.
Thomas P. The FPS drops and the fact that it made it unplayable for me.
Alexander S. Not being able to move from green spot or moving out and not realizing it.
Chris T. Difficultly and a bug that occurred with the second fighter where she wouldn't move and I had to restart.
Tom B. It may be that I had to spend more time in the training ring but I could not seem to get the blocking mechanic down. I think I understood that I had to have the gloves glowing blue and when they were purple, the block would not work but I could never seem to get them to do that during the fight. During training, it was frustrating because when I was trying to find that happy zone where they were blue, DJ would throw a bunch and then I would have to start over.
Chris E. It felt like the height of my character was off and that broke immersion. I think the instructions on how to play the game would be better if they had visual component. Just reading text, it wasn't always clear how to perform the moves.
Aaron E. Correct move seems to be to dodge real early.

Suggested Improvements

Name Comment
Tim S. More characters of course but also maybe add more to the training other than just the basics like you have now. Maybe more intensive dodging and fighting skills. Oh also, star punches giving your character extra umf!
Jacob G. Fix the visual issues with distant objects shimmering (may be hardware and not be the game's fault). Add more opponents, and more variety to each opponent's attacks so the fight is more about reaction and less about moveset memorization.
Ron D. More social aspects like trophies, unlocks, high scores. Don't think its possible but multiplayer would be amazing. I assume more bosses will come at launch
Thomas P. I would obviously fix the performance issues with the videos. I would also tease or explain better how the progression works.
Alexander S. Let me move around. Have online multiplayer, lan or local.
Chris T. Fix the bug I faced, ability to remove jagged edges on graphics, difficulty options.
Tom B. Better blocking system. I see the difficulty because if they were able to block all of the time, the fight might become way too easy. I suppose then, I better tutorial for the blocking (starting with more time to find the sweet spot).
Chris E. The height problem, it felt like the placement of the floor was off and my feet were in the floor. During a fight, it would be nice to have a coach giving me advice. It would be nice to be able to adjust sound volume in the "home" menu.
Aaron E. Allow for shorter, later dodges, or at least show me the view of how I got hit or what happened so I know what to adjust.

Tester Profiles and System Specifications