Rank Order

Introduction

Rank order survey questions are close-ended questions that allow respondents to evaluate multiple row items based on a specific factor. A ranking survey is a group of multiple-choice questions presented in a single-column format. Rank order questions are used in both pen and paper questionnaires as well as online surveys. They offer accuracy of responses and detailed insights.

Rank order scale questions are one of the most commonly used survey question types. They are used in a survey when the market researcher would like to understand the order of importance of items when there are multiple items. This ranking could be on the basis of features, needs, likes or dislikes, effectiveness, etc.

The ranking is by a simple ordinal position where one variable or option is higher than another. The ranking could also be by a relative position where one variable has a higher relative rating than another and subsequent ones.

Example of rank order scale questions

Consider a fitness tracker manufacturing company that wants to know what features are ranked most important by their respondents and what do they buy fitness tracker primarily for. Their ideal target audience is a group of people who already use a fitness tracker.

Then you can ask the below question if you have determined in the previous questions, that the respondent is currently or planning to evaluate fitness trackers within three months. Otherwise, the question is a way too hypothetical.

If you are looking to offer a new, differentiated product, the first step is to know what the “table stakes” are. If you know certain features are well-liked in current products then you need to match respondents’ expectations at a minimum. Also, once you know the features that least liked, it offers you insights on what differentiators you should seize.

While this approach is wordier, it reflects a reality: choosing between products in many product categories is often ultimately driven by one or two items, and all other features don’t matter much.

Uses of rank order scale

Rank order questions are effective in collecting feedback quickly. Respondents find it quite easy to answer them and don’t have to spend time typing their feedback as in open-ended questions.

An airline company might have many ideas to improve the passenger experience but can’t implement all of them at one go. They can create a rank order survey and request their customers to rate the ideas on a scale from “not at all” to “very” important options.

A rank order survey is used for conducting secondary research. If preliminary research has been conducted and possible areas of product enhancement have been marked out, it builds a platform to validate the findings. You could present customers with a list of possible product improvements and ask them to select what product tweaks matter the most to them within the product.

But what if they select multiple options?

How would you find out which are they most interested in?

This is where rank order scaling questions help prioritize. They ask respondents to order the options from high to low. With such type of questions, respondents cannot say that all things are equally important. They are forced to prioritize, which helps you gather insightful and actionable data.

Types of rank order survey questions

Rank order scaling question: The rank order scaling question type provides respondents a unique opportunity to rank a set of items against each other. Rank order scaling based multiple-choice questions allow a certain set of brands or products to be ranked based on a specific attribute or characteristic. Respondents can rank the options by selecting a number from the dropdown.

Constant sum question: A constant sum question allows respondents to enter numerical values for a set of variables but requires the values to add up to a pre-specified total. Each numeric entry is summed and can be displayed to the respondent. This is a great question type to use when asking financial, budget-related questions or percentage based questions."

Drag and drop question: Respondents can drag and drop the answer choices to re-order them as they choose. This question type gives complete freedom to the respondent to order the answer options to best suit their preference. These drag and drop option questions are interactive and will help the respondents choose wisely.

There are multiple ways of sorting ordinal data using rank order scaling questions. Some of the major sorting types are:

Priority sorting: Priority sorting looks for the most important and subsequent options in order of importance. This method is widely used in customer satisfaction surveys (CSAT) to capture customer insights. For example during a new product launch or a product upgrade, priority sorting is used to capture what customers perceive as the important features in a descending order.

Block sorting: Block sorting is used when there are many options that can be bucketed into distinctive blocks of quantitative data. This gives the interviewer a granular insight into exactly what a customer wants. For example if a steakhouse wants to understand what meats, cuts and rubs to have on the menu asks a respondent to sort options by bunching the above three parameters.

Score sorting: Score sorting is one of the most widely used methods of rank order scaling questions. Respondents are expected to add one unique number to each option and the options are then sorted from 1 being the most wanted feature. For example if an airline conducts a survey where respondents are required to score options like seat size and pitch, in-flight food, luggage allowance, lounge access and frequent flier program.

Data analysis of a rank order question

Once you receive the responses, you can view and analyze data to generate various reports. For each response, you can see how a respondent has ranked each option.

The survey results dashboard allows you to download rank order question data in below ways.

Excel: This report contains the rank order result analysis of all responses. It shows the number of responses for each option, the rank given to each option, the average rank of each option and the percentage share of each rank for all the options.

Trend report: Trend analysis highlights the change in data, and hence, a change in the behavior or experience of your target audience. Survey creators can compare the results with historical data and gain insights into the shift in behavior.

To generate a trend report, set the start date and end date. Optionally, you can also set the data filters, frequency and output data.

Banner table: Banner tables let you visualize your data relative to a pivot question. You can view how respondents of the rank order question answered another question without creating segments."

Explore our help file on the banner/pivot table report.

Share rank order survey
You can share ranking scale survey dashboard with your stakeholders like colleagues, clients, employees or on social media sites in below ways:

Share URL: Preview dashboard report and share the url or QR code with anyone. Post the link directly to social media sites like Facebook, Twitter and LinkedIn.

Embed URL: Embed report url in your web pages.

Explore our help file on sharing dashboard reports.

Advantages of using rank order survey questions in surveys

Advanced consumer insights: Rank order survey questions help to delve deeper into the profile of the respondent. This question type not only talks about the ‘what’ but also the ‘why’ of a research subject. There is a value tied into each individual response and this helps derive statistical analysis about a certain subject. The data from such a question also helps with making better business decisions and offer your customers what is important to them.

Higher response rate: Due to the nature of the answers, the respondent feels a heightened sense of responsibility towards the question being asked. The rank order survey question also pushes the respondent towards providing as realistic responses as possible.

Easy of use: The instructions of a rank order survey question are simple and easy to understand. The respondents are comfortable with ranking the different options as they are comfortable with the options provided and have a heightened sense of belonging.

Validating primary research: Rank order survey questions help validate any prior primary research. The options provided in such a question are brief inferences from the research. The respondent data from this question helps the researcher understand if the initial theories you set out with are true or not.

Introduction: Sets the respondent's expectations – may also be covered in the survey invite.

Survey duration: Give respondents a realistic time estimate and remind them their opinions are secure.

Screeners: Questions designed to ensure the respondent meets your criteria.

Typical screeners include demographics (age, gender, region), level of responsibility or purchase ability (B2B), product/service usage, or brand/company awareness. Screeners are useful for establishing quotas.

Body: The bulk of the survey. Questions on usage, attitudes, awareness, concept testing, competitive perceptions, etc.

Demographics: Questions used for profiling or segmenting the respondents. No need to repeat any demographics used as screeners.

Final thoughts: Close with an open-ended question asking for any additional ideas.

Redirects: Could be a simple thank you page, a redirect to your website, or back to a panel provider.

2. Put the essential questions

Now that you have your core objectives brief ready, it is time for you to convert these core points to real survey questions. It is rather easy to start deviating from the core objectives and boundaries once it starts creating the survey on QuestionPro, given the amount of freedom and choices that the platform provides. However, it is essential to stick to what you had already decided unless you know that the objectives need some alterations to make your survey better.

3. Keep it short and straightforward.

Respondents may get irritated and less likely to complete the survey if a topic is not precise and tend to bounce back and forth. Therefore, follow a logical order in placing the questions and stick to a specific topic. If the survey is too lengthy, respondents might not stay interested in filling out the entire study, and survey results might get compromised. Hence, keep it short and straightforward.

4. Ask one question at a time.

Branching questions or asking more than one question at a time can confuse respondents, and they might not be able to choose the correct option/options from the list of answer options provided to them. Therefore, it is advisable to ask one question at a time, to give clarity to respondents.

QuestionPro features over 20 advanced logic mechanisms to make a survey more systematic, and in most cases, it also helps make them shorter. Another advantage of using logic in your surveys is making them more user friendly by simplifying the answers.

5. Avoid using jargon.

For respondents to respond to the survey, avoid using jargon and too many technical terms. Using a language that is easily understood by respondents will help them understand and respond to surveys better.

Language is extremely important while designing a survey. The idea must be to make it as simple as possible for the respondent to answer your survey. The more complicated it becomes, the more likely it is that respondents will leave their surveys incomplete.

6. Open-ended questions or closed-ended questions?

It is always the tricky part, whether to ask open-ended questions or closed-ended questions. The trick here is, balance out both these options. Try and include descriptive and objective questions to balance the survey. Again question types should be carefully sorted as it defines the tone and importance of asking a question in the first place.

7. Spend time to design your survey.

A survey needs to create brand awareness with respondents. For this, the survey creator can use the brand language (brand logo and brand color), so respondents can correlate with it.

QuestionPro lets you fully own your surveys through self-branding with logos. You can also personalize your research studies for your respondents using customer variables. It is useful for capturing responses in earlier questions from your target audience and use them to create automated form fills that personalize the following questions. For example, obtaining the respondent's key demographics at the beginning of the survey allows you to set questions that use this data and personalizes the survey.

8. Analyze the responses.

Once you've sent out the survey, collect all the responses, and maintain a document or excel sheet for the collected responses. This excel sheet should have all the mentioned classifications of the survey to have all the data in place.

You need to make sure you follow this step and categorize data into Behaviour, Psychographics, and Demographics. Any researcher needs to have organized data so that analyzing, predicting outcomes, and writing reports become easy.

9. Put a summary report together.

After collecting responses and analyzing it, share your analysis with your customers using a summary report. You should have all the data gathered from surveys in a particular format by this stage. The readers should get a clear picture of your goal, i.e., what were you trying to attain from the survey? Make sure you cover questions such as do users prefer/use a particular product or service? Which product do the users prefer? Any comment?

10. Conclude your plan of action.

Prepare your final action plan based on your goal, responses put together and conclude. Mend your final plan and start executing the changes.