Moderated user testing mixed with remote user testing

There is no doubt that running moderated, in-person user testing sessions is an extremely valuable way to gain a lot of insight into the mental process of people who interact with your site. When running these sessions your users are able to explain their thoughts, their frustrations and the things that make them happy.

However these moderated user testing sessions have a few downsides. Firstly, they can be very expensive in terms of cost and time and it's not always entirely clear if this cost is paid off in terms of the final quality of your user flows. Secondly, you have to be very careful that you don't end up curating very specific problems for the customer to solve. If your questions are too leading you may end up almost forcing the user to perform an action through a method that would otherwise be entirely unnatural for them. There's no doubt that a skilled user testing moderator can lead these sessions in such a way as to minimise the risk of this happening but it's still a potential shortcoming.

The perfect accompaniment to moderated user testing is unmoderated remote user testing where the user is not even aware they are being observed. This can be achieved by using tools such as session cam or inspectlet which record every single action a user takes on your site. There are plenty of other tools that provide the same service but these are two that I have used personally.

Where moderated user testing sessions can give you huge amounts of insight it's typically at a low volume because of the associated costs. Remotely recording your users gives you much less personal insight but at a much higher volume. When a user has not been set a curated problem to solve (and are not even aware they are being recorded), you are much more likely to see their natural behaviour as they attempt to achieve their goal. You are also able to spot emerging trends and persistent pain points. If you're not comfortable with the idea of recording all of your user sessions then you can limit them in such a way that you only record users who are in a particular test group or who are engaged in a flow that you are currently trying to improve or build.

Here's a quick summary of what I see as the pros and cons of each approach:

Moderated user testing	Remote user testing
✘ Expensive	✔ Cheap
✘ Low volume	✔ High volume
✘ Unnatural user behaviour	✔ Natural user behaviour
✔ High insight	✘ Low insight
✔ Wide scope of test	✘ Tests interaction only
✔ Accurate, targeted	✘ Random, untargeted

Both approaches are extremely valuable for different reasons but it is the mixture of the two that gives you the best results as they make up for each other's shortcomings.