Best practices for Saas User Testing

Assume your product is b2b Saas with ideal users as account executives, sales managers and sales leaders. Please understand that the feedback from users is important, however sometimes it may get challenging. My approach would be to handle the end to end Saas user testing from planning, recruiting, execution, feedback collection, insights gathering and suggesting changes to the Development team in the upcoming releases, tracking metrics and iterating. 

Here are some of the best practices for Saas user testing - 

1 - First start with a clear goal. What is the next big thing/ use case the user is expecting? Prepare an open ended questionnaire. 

2 - Run the user test for internal users first. The app has to be defect free and fully tested - production grade. Hide features which are not fully baked. Otherwise the test may fail and users will lose interest in doing the tasks. One more thing - make sure to track the user transactions or movements. Google analytics, full story, or similar are recommended.   

3 - Once this is done, recruit unknown people who are in the same role as that of our Ideal User Profile. Maybe give incentive for their 30 to 45 minutes time. For us T-shirts and/ or a lunch coupon worked well. Prefer in-person tests only. 

4 - Get 3 to 4 cameras/ laptops which can record the room, environment and start google meet/ zoom; so you can see everyone's facial expressions. Make sure to get this ready and nail it well. Check and make sure that the recording has good sound/ video quality. Get consent from users to record. Keep their name anonymous. 

5 - Schedule time slots of 1 hour for each participant. Prepare the test well to make sure they align with our goals. Ask the user to do the high level task; do not give a solution which is on screen to them. The goal is to know if users can find the solution by themselves.  

6 - Once recordings are done, store those in google drive or similar for analysis. Watch the videos multiple times and see what users are getting and where they have struggled and so on. Look for patterns within multiple users. Gather insights from those patterns and convert those into high, medium, low priority tickets for the development team. Share insights with the management team as well. Recruiting, getting app ready (defect free and tracking), execution and pattern understanding are the key steps. 

Do relevant changes in the app. Remember to repeat once Saas user testing changes are done in the app for user validation. 

Do use external tools, surveys, recruitment agencies and/ or similar as needed. 

Remember these - 

Once you have the user profiles, demographic data, video recordings, analytics data and everything; analyse them towards the goals defined. Create a point based ranking system for impact of each request. Then use Effort vs Impact 4x4 metrics. Prioritise those which are high impact and low to medium effort.  

We used the MoSCoW framework as well. This really helped eliminate what we will not do in the next 6 months or so. 

One other framework which helped us was - Kano model. This really helped us in understanding delighters. What are moments where users smiled, they had a good surprise and so on. 

I have used these best practices and frameworks to increase the monthly adoption of a b2b product by more than 60%. 

Sometimes during trial Users may go silent and/ or stop engaging. These are Saas User testing sessions which need multiple visits from participants. In this case, someone should nudge the users frequently if a visit is expected to be face to face. Calls, chat messages, email etc. will also help nudge them to come back. 

Additionally - 

• Analyse each task or use case. Is the use case frequent (RFM model)? Buying a mobile phone is not a frequent use case. In this case users may have completed their goal and may not come back unless it is time for a new goal. 

• Are there other ways they can solve the same problem? How they used to do it before our product and so on. 

• Last but not least - we should avoid operational challenges - is there anything not allowing them to use it. 

If we take care of all these, we can make Saas user testing a great success. 

----

In the picture, think about and visualise how large scale events manage audience. The way the staff guides, signs are posted, notes are written and so on. 

They follow Saas user testing approach.

……. 

Swagat Irsale is Growth Advocate. He works with startups and scale ups to grow revenue and build products which enterprises love to use. 

Reach him for work and partnership opportunities.