To Answer this question, or better not, is to go directly to the user at street to do the test. By doing so, it will remove the big assumption if it is working with real life or not. So what are the problems with lab test? Here are some points you may consider when your run test in the lab:
- The tester is not the real users, sometimes. Some people will be show in the lap test just for sack of doing favor to someone, or just to earn some extra cash.
- People will do take an extra time to come to your place, which may stop them from coming in at first place.
When testing with a camera on and everything is recorded, do you think the users will be act same as everything is offline. Just ask yourself? Is your answer will be same when you chat freely with friend, or will think twice before you provide the answer? - TEST! Is a freaky word the people are do not like. When you told to come to test and you will be given something in return, I will start thinking if I provide a “correct” answer is that entitle me to get more rewards! And in lab setup, it is similar to seat for major exams or GMAT test!
- The users in the lap will spend the time not focusing on the test subject (the website or apps) instead they will be thinking about the device you handle them! I remember one time I was attending a demo for MS SharePoint 2013 and there was a laptop in the room to check it out live. The people start asking the sales person what is the type of the laptop and from where they can buy it!!!!
All of the above, and many others, led to prefer user testing on street, or guerilla usability test. Even better, to run the test directly with current customers at the retails store space. For example, if you are building a website for bookstore, it will be great idea to run the test in that store directly with users/customers to obtain opinions. You mat also seat there for while and observe their behavior. But still you need to go out (maybe near a competitor store) and run another set of test there with potential customer.
…on 2019 with COVID-19 pandemic, new roles and restriction have been change the way we do usability testing, in such situation, the only and more effective way is using remote usability test (using zoom, for example) there are lots of advantages, like wide range of users that can be studied and lower cost, and easy to record the session
I am sure by following this approach; you will get more accurate, yet true and valuable feedback from the users. Even by this way there is no recording, which is not a requirement, but it will make the user speaks more freely. You may trade this off with good and quick note taking. Remember the goals of the testing is to collect the users feedback and see their interactive and “feeling” while using the product.
