A/B Testing in QuestionPro Surveys

A/B testing, also known as split testing, is a method of comparing two versions of a web page or email to see which one performs better. You can test anything on a page, from the text to the layout to the call-to-action button. By doing A/B testing, you can ensure that your website is constantly improving and generating more revenue for your business.

A/B testing is a valuable technique for surveys because it allows researchers to determine if there is a statistically significant difference between two groups of survey respondents. This information can help researchers make decisions about the design of their survey, the questions they ask, and the way they analyze their data

What is A/B Testing in Surveys?

There are two main types of A/B testing:

-Test A vs. B: In this type of test, you compare two versions of a web page or email to see which one performs better.

-Split Test: In a split test, you randomly divide your respondents into two groups and send each group a different version of your survey. This type of test is used to determine if there is a statistically significant difference between the two groups

When doing A/B testing, it is important to make sure that you are testing something that will have a meaningful impact on your website’s performance. For example, you should not test the color of a button or the wording on a headline. Instead, you should test variables that are likely to have an impact on your website’s conversion rate, such as the layout of a page or the text of a call-to-action.

A/B Testing can be performed in QuestionPro by randomly assigning two (or more) conditions to the respondents, without letting the respondents know what the other condition looks like. This allows respondents to provide unbiased opinions by the desire to compare and contrast options.

Setting up survey blocks

Without any survey questions, you will see the survey template blocks where you can add or merge them together.

Block randomization:

Go to your survey. Click on ‘Tools’.
Click on ‘Block Randomizer’ from the drop-down menu.

Click on ‘Add Randomizer’ to add a section for the randomizer.

Drag and drop the blocks in the section and select the number of blocks you want to randomly display to the respondents.
Click on save.

The order decided under block randomizer will be the default order of your survey.

Can I see which blocks were displayed and in which order for each respondent?

Yes. All randomization data is saved for each respondent and can be downloaded in excel/csv format.

Go to: My Surveys(Select Survey) » Analytics » Manage Data » Export

Here, switch on the toggle for Include Randomization Data

In the exported file, block order data and question order data will be included.

License

This feature is available with the following license :

Research Edition

Was the article helpful?

Yes No

How can we improve it?

Submit

1 Question

How does QuestionPro's A/B test work in this hypothetical scenario?

We have two A/B test questions in the same survey. The first A/B test includes three question patterns, each requiring 100 responses, for a total of 300 respondents. The second A/B test consists of two question patterns, each needing 100 responses, totaling 200 respondents.

Given the difference in required respondent numbers between the two A/B tests, could you clarify how respondents will be distributed across these tests?

Will 200 of the 300 respondents from the first A/B test proceed to the second A/B test, leaving 100 respondents from the first test not participating in the second? If so, only 200 respondents would be involved in the second A/B test.

I would appreciate your clarification on the respondent allocation process as they progress through the survey.

0 Answer

You may also be interested in... Block randomizer Survey Blocks Question Randomization Advanced Randomization

GET STARTED WITH YOUR FIRST SURVEY NOW

Still have questions?

Webinar

Free Training

Support

Certification