Website internal search queries, answers to open questions on a poll, comments on your vlog or social media feed, user recordings, customer interviews: Qualitative data is key to have a better understanding of your audience. But how can you go from an endless list of character strings to actionable insights on your audience expectations and experience?
In this article you will find the steps to follow to effectively do quantitative analysis on qualitative data:
- Get context
- Get to know your data
- Create descriptive and interpretive categories
1. Get context
Imagine this scenario: You are facing your raw qualitative data. You browse through a sample. What’s next? You need to find out how the data was generated and gather as much information about context as possible.
Even if you already know, for example, that you are looking at answers to a website survey, it is not the same to ask a question in the homepage, in the pricing page or in the checkout.
Whatever information you can get about your users will be key to putting yourself in the respondents’ shoes. We will touch on this later: empathy plays a key role in interpretation.
I recommend using some of the following questions when researching the context of your data set:
- For behavioural data. What did the user do right before the observation started? Were users aware that they were being observed?
- For declarative data. Were users incentivized to tell the truth? Do you have any reason to believe that the results are skewed? (For example, unsatisfied customers are incentivized to leave detailed product reviews, but happy customers, less so)
“Whatever information you can get about your users will be key to putting yourself in the respondents’ shoes”
2. Get to know your data
The second way to get to know your data is through metadata. Here you need to ask questions such as: What is the data format? What are the data types? What is the volume of data that you need to process? How many entries do you have per month?
Also, depending on what the source is, you’ll need to get familiar with key attributes. If you are looking at user recordings, you will want to answer questions such as: What is the average and median duration? What is the minimum and maximum duration? Are there outliers?
The metadata will help you have an initial idea of how to approach data quality, transformation and analysis. Can you process a sample manually? How big will it be? Do you need to use artificial intelligence to interpret the data or to transform it into a more manageable format? Will you need to use a programming language or will a simple spreadsheet be sufficient?
“The metadata will help you have an initial idea of how to approach data quality, transformation and analysis”
3. Create descriptive and interpretive categories
By this point you should have a pretty good feel of what your dataset is and you already have some ideas about the value it has to offer. Now we come to one of the hardest problems to solve: How do you reach quantitative metrics when every datapoint in the field is unique?
In order to have metrics you need to aggregate data. You will have to attach attributes to each row or instance of your data.
Once you have these attributes in place, you will be able to categorize your information and perform a count operation. Then you can start looking for trends and patterns.
These are some good examples of attributes you can use for open text fields:
- Type of statement: declarative / imperative / interrogative / exclamatory
- All caps: yes / no
- Is the user angry?: yes / no
- Intended task: Find information / purchase / contact support / sign up
There are two different things you will be doing when categorizing your data: describing and interpreting.
The difference between describing and interpreting
In order to attach attributes to your data you will have to either describe it or interpret it.
When you are describing, you are simply giving a detailed account of your object. In turn, when you are interpreting, you are also providing meaning. Interpretation is subjective. Even though we are about to perform quantitative analysis, since we are dealing with qualitative data, subjective interpretation will be part of the process.
When interpreting what a user said or did, you will get the best results by trying to put yourself in the user’s shoes. Empathy will be key in trying to understand the meaning behind the data.
It is important to clearly differentiate when you are aiming for an objective description and when you are going for an interpretation.
Interpreting data is an iterative process. You will create better interpretations after gaining knowledge from your previous analysis. When you come back to your data set you will want to reuse descriptions and remove the old interpretations. To do that you need to be able to tell them apart.
Let’s look at the category examples again. In what cases are we describing and where are we interpreting?
- Type of statement: Declarative / imperative / interrogative / exclamatory
- All caps: yes / no
- Is the user angry?: yes / no
- Intended task: Find information / purchase / contact support / sign up
You will notice that some fields are clearly descriptive:
- Type of statement: Declarative / imperative / interrogative / exclamatory
- All caps: yes / no
One of them is evidently an interpretation:
- Is the user angry?: yes / no
What about this one?:
- Intended task: Find information / purchase / contact support / sign up
Well, it depends. It could be descriptive if you only categorize the comments where the user states their intended task, for example if they say “I was trying to purchase a t-shirt, but couldn’t find it in my size” they are stating their intended task and you can complete your field without any subjective interpretation. If, however their comment was: “I couldn’t find the size of the t-shirt I wanted” you would have to mark that as a “null” in the descriptive “intended task” field, even if it is your interpretation that the intent of the user was to make a purchase (why else would they be looking for a specific size?). So in this case, you need to decide if the field will be descriptive or interpretive.
This is a common pitfall, and it bears repeating: when creating an attribute for your qualitative data, differentiate fields that contain descriptions from fields that contain interpretations.
“Empathy will be key in trying to understand the meaning behind the data”
Final words
In summary, you shouldn’t miss out on gathering insights from qualitative information, and quantitative analysis is one of the great ways of doing so. By following the steps outlined in this article (get context, create metadata, create descriptive and interpretive categories) you can transform data into numeric values and discover trends and patterns that will deliver a better understanding of your audience.
