{"id":1053,"date":"2018-04-04T19:29:17","date_gmt":"2018-04-04T17:29:17","guid":{"rendered":"https:\/\/www.thediar.com\/?p=1053"},"modified":"2019-07-09T19:21:03","modified_gmt":"2019-07-09T17:21:03","slug":"why-do-online-testing-and-what-kind-of-testing-tools-you-can-use","status":"publish","type":"post","link":"https:\/\/www.thediar.com\/en\/blog\/why-do-online-testing-and-what-kind-of-testing-tools-you-can-use\/","title":{"rendered":"Why do online testing and what kind of testing tools you can use"},"content":{"rendered":"

The digital environment provides businesses with a very convenient opportunity to quickly evaluate ideas using controlled experiments. The aim is to carry out experiments using the scientific method: observing the effect of the changes in the behaviour or opinion of the users that we assign randomly between different variations \/ combinations.<\/span><\/p>\n

Why do online testing?<\/span><\/h2>\n

Unlike other methodologies, such as post-hoc analysis or analysis of time series (quasi-experimentation), controlled experiments evaluate cause-effect relationships.<\/span><\/p>\n

The simplest experiment would consist in the random assignment of the users that visit a page between 2 variants: (A) control, with the original version and (B) alternative variation, the new version that is evaluated against the control.<\/span><\/p>\n

The evaluation uses performance metrics that are directly or indirectly related to data on user behavior and opinion. Statistical tests evaluate the probability of not finding evidence of significant differences between the data collected from the variation versus the original variation. The use of secondary metrics and user segmentation of each group can help us understand the results and refine and explore new ideas.<\/span><\/p>\n

Web teams that take into account the behavior and opinion of users are more likely to conduct successful experiments. With a data-driven approach we can report on the subjective results of our experiments and go beyond the opinion of the expert or Highest Paid Person’s Opinion (HiPPO).<\/span><\/p>\n

When a company already has a testing tool, the cost of doing one more test is very low. In this sense, putting a testing tool at your disposal will enhance the innovation capabilities of your company and optimize business results.<\/span><\/p>\n

A losing test is more interesting than an inconclusive test. Finding alternative variations of ideas that could be harmful allows you to analyze the results and refine the idea to make it work.<\/span><\/p>\n

“The best way to have a good idea, is to have many ideas.”<\/span><\/p><\/blockquote>\n

 <\/p>\n

Important concepts in testing<\/span><\/h2>\n

We use the type of testing tool that we use, we need to have clear the following concepts when we perform tests:<\/span><\/p>\n

    \n
  • Main objective:<\/strong> It is the main metric or key performance indicator with which the achievement of the objectives of the experiment will be evaluated. It should not be based on clicks but a result or an “outcome”.<\/span><\/li>\n
  • Variable:<\/strong> It is the factor that is believed to influence the outcome of the main objective.<\/span><\/li>\n
  • Variations:<\/strong> They are the different versions of the variable, including the original version.<\/span><\/li>\n
  • Experimental unit:<\/strong> Refers to the entity with which the metrics are calculated for each of the variants. On the web, a user with his userId in a browser cookie is usually the experimental unit. It is important that the user experience a consistent experience throughout the experiment depending on what variation was assigned.<\/span><\/li>\n
  • Null hypothesis (H0):<\/strong> Consider that the different variants do not produce differences in the main objective beyond random differences.<\/span><\/li>\n
  • Confidence level:<\/strong> The probability of refuting the null hypothesis when in fact it is true.<\/span><\/li>\n
  • Statistical power: The probability of detecting difference when it really exists.<\/span><\/li>\n
  • Test A\/A:<\/strong> Exercise to evaluate the statistical power of the method of random assignment between two variations equal to the original version.<\/span><\/li>\n
  • Standard deviation (Std-Dev):<\/strong> It is a measure that indicates how scattered the data are with respect to the average.<\/span><\/li>\n
  • Standard error (Std-Err):<\/strong> It is the estimate of the standard deviation, calculated from an estimate derived from a sample.<\/span><\/li>\n
  • Effect:<\/strong> It is the difference of the main objective between the variations. It is useful to give the confidence intervals for the differences between the means of the variations why it gives an idea of \u200b\u200bthe ranges of the effect.<\/span><\/li>\n<\/ul>\n

     <\/p>\n

    Types of testing tools<\/span><\/h2>\n

    The testing tools can be classified according to their method of random assignment.<\/span><\/p>\n

    Every testing tool must execute a code to randomly assign users in one variation or another. Then it should be possible to manipulate what is necessary, from the content to the proper functioning of the modules of the platform.<\/span><\/p>\n

    Each method has its advantages and disadvantages.<\/span><\/p>\n