Getting Started With Playtesting
When developing a Virtual Reality application it is necessary to assess the state of the application and how it is received with real users. Playtesting is an important part of the VR application development cycle at all stages.
Common Types of Testing
Before beginning a playtest it is important to understand how playtesting is different from other common types of testing.
|Quality Assurance||Formal testing plans and focusing on software bugs.|
|Automated Testing||Testing software where someone writes tests in code to perform against an application.|
|Focus Groups||Interviewing groups of people for demographic and market reception questions.|
|Playtesting||Observing users interact with an application to reveal if it is delivering the value intended.|
Playtesting can provide benefits of other types of testing (such as finding bugs) but, is unique in it's ability to provide insight into how the user is really using an application.
Playtesting is a crucial part of making informed decisions about your VR application.
- Real human reactions.
- Identify hidden problem areas.
- Idea generation.
- Solve design arguments.
- Quickly evaluate a hypothesis.
What Stage Should an Application be Playtested?
Not playtesting enough is common feedback from post-mortems of underperforming applications. Playtesting can prevent issues from going out in release as well as refine existing ideas after they've been in the wild.
Questions Before Beginning a Playtest
Before beginning, it's helpful to consider the questions you're interested in answering.
- How do different demographics use this app differently?
- Do users understand how to use this app?
- Do users want to use this app again?
- Are users bored, confused, experiencing motion sickness?
- Does the application have hidden bugs?
- Are users discovering things that haven't been noticed?
- Which parts are the most enjoyable?
- Is part of the overall experience too long or short?
Who should playtest your application?
Common groups of users available fall into 3 categories:
|Co-Workers||Because of the closeness with the application, it's easy to be unaware of issues that would be revealed to users without the same level of internal knowledge.|
|Friends||It's easy to assist or interact in a way that may alter the experience; such as leaning over and helping through a difficult part of the interface etc.|
|Outside Users||They don't have the internal knowledge of co-workers, and are separated and less likely to be biased about their opinions of their experience.|
How Many Testers?
We recommend test sizes based on the questions you're trying to address:
|Small||3-9 testers||Precise features or question such as "How easy is it to login to my app?"|
|Medium||10-30 testers||Demographic or targeted testing (Playtest for Oculus Touch users, Gear VR users, etc)|
|Large||30+ testers||Build milestone playtest, pre-release playtest, market reception or playtesting with a larger survey component|
What Instructions Should I Provide?
The instructions or tasks should be completable within a short timeframe (under 15 min each task.) Providing simple tasks is not just important for the tester to be able to complete the test but also in providing results that are easier to analyze and reason about.
|Easy||1 task or open ended|
By defaults playtests are created with instructions for testers to record their session and feedback for 15-45 minutes.
Custom instructions may be provided under the "Playtesting Instructions" section when you create a new test.
Analyzing the Playtest Results
Once your playtest has completed, areas to pay attention to when analyzing the results:
- How long did it take the testers to complete the tasks?
- Did testers encounter a common error?
- Did testers react in a way that was unexpected?
- Was there neck or body movement, or other signs of physical strain?