Gathering qualitative data

This section provides an overview and examples of how qualitative data can add depth and insight to your wellbeing evaluation.

What is qualitative data?

Qualitative data is information which comes from observation, description or other non-numerical sources. Words are the most common form of qualitative data, which, unlike quantitative data can’t be counted. It comprises people’s opinions, stories and views expressed in their own words. Qualitative data includes what people tell you in interviews or focus groups, diaries, videos or photographs, and through other means.

Every item of qualitative data will be unique, so you won’t be able to compare them to each other directly like the quantitative data from your surveys. But you can still analyse a set of statements and narratives together to find themes and patterns to gain a deeper understanding of how your project may be affecting people’s wellbeing.

The main focus of this guide is measuring wellbeing impact through quantitative methods, so this section is an introduction to qualitative data rather than an in-depth guide.

You can find practical advice and guidance on qualitative research for beginners here. A dedicated guide on gathering qualitative data for wellbeing evaluations is available from the Centre for Thriving Places.

Uses of qualitative data in wellbeing evaluation

There are several reasons for using qualitative data in a wellbeing evaluation and qualitative methods can be employed at different points during your evaluation, including in the planning phase, and when you are collecting and interpreting wellbeing data. 

You will typically use qualitative data to:      

1. Develop a Theory of Change – understand and interpret how change happens, build outcomes chains and draw a link to wellbeing evidence

You can ask your beneficiaries to tell you in their own words about their experience, and the effects of your project or programme. You can read more about Theories of Change in the section on Planning your wellbeing evaluation in this guide.

Conducting focus groups or interviews can help you explore how wellbeing fits into your programme theory, when you might expect wellbeing changes to occur, and for whom they are likely to occur. This will help you draw a link to any wellbeing evidence to support your programme theory.    

Participants can also tell you what else is going on in their lives, which could affect how successful your activities could be in improving their wellbeing, so you can understand the barriers to your impact.

2. Identify – to draw out information on your beneficiaries’ wellbeing

For some people, topics such as loneliness or mental health may feel socially stigmatising, and their experience might be best drawn out in one-to-one interview settings or within group settings with members of the same community or background.    

3. Explain – to investigate unexpected wellbeing scores

Your survey data may return unexpected results which you’ll need to explore further by collecting rich textual data and detailed information on the experiences of your beneficiaries. This type of information can help shed light on why wellbeing hasn’t improved or why it has worsened following participation in your project or programme. It can also be helpful to explore differences and capture nuances or factors that are not observable or measurable in a survey.  

Covid-19: things to consider

Now is a good time to collect qualitative data which can tell you how and why people’s wellbeing is changing in the context of the Covid-19 pandemic. As well as carrying out interviews or focus groups over video link, you may also want to ask people to keep diaries, take images or videos that tell you about their experiences.

Qualitative methods in a mixed-method evaluation

Using qualitative methods alongside quantitative data in your evaluation will strengthen your confidence in the results and conclusions, as your wellbeing findings will have drawn from different sources of information, such as a survey and focus groups. 

Very few evaluations ask one single question, and using qualitative techniques can help you answer questions about how your activities generate changes in wellbeing. 

For example, you may want to know: 

• ‘Does my art therapy class improve wellbeing?’ in which case quantitative data will help you find an answer
• but you may also want to know ‘How does my art therapy improve wellbeing?’ which is best answered using qualitative data

Gathering data through focus groups and interviews

This is a technique for collecting data in a group setting and an interactive environment on a topic or area of interest defined by a researcher. 

Focus groups can vary in their structure, ranging from group interviewing to semi-structured approaches, where you agree on a set of core questions to ask in advance, and settings in which the facilitator’s role is almost imperceptible.   

When collecting wellbeing data through a focus group you will be talking about sensitive topics and some of your beneficiaries may be: 

• In distress 
• Experiencing feelings of shame and stigma
• Reluctant to share personal feelings and experiences 

Make sure you: 

• Have an experienced facilitator with deep knowledge of the wellbeing topics, who is discreet and as neutral as possible. Care must be taken to avoid participants thinking that they are in any way being judged.  

• Carefully select your participants and the size of the group to ensure your group is balanced.    
• Uphold the confidentiality and anonymity of your participants when you record, analyse and disseminate any data collected.  

Recording qualitative data

There are a number of ways you can capture qualitative data. You need to think about how and what to record. You might choose to audio record an interview and write up what is said word for word as a transcript. As well as making sure you know who is talking, it might also be important to note in your transcript how people are describing their experiences: are they laughing, pausing, happy or uncertain?  Are any phrases emphasised? 

Similarly to one-to-one interviews, focus groups can help you understand what else is going on in the lives of your beneficiaries. For example, you may find that wellbeing gets worse for some beneficiaries by the end of your project or programme. Qualitative methods can help you investigate the underlying causes for this and explore any unintended consequences of your activities that are not included in your Theory of Change. 

Gathering qualitative data through surveys

Qualitative data can also be collected as part of a survey through open questions. This guidance focuses on the use of closed questions; however, using open questions can benefit your evaluation by: 

1. Giving beneficiaries the opportunity to answer however they want, providing you with information on unexpected topics or effects. Make sure you’re clear on what you’re asking and why, so you can collect relevant and usable data. 

‘Since you got involved with (X), what, if anything, has changed about how you feel about yourself and your life?’

2. Providing information on why wellbeing has changed. Make sure you analyse and test your findings against your Theory of Change or established wellbeing concepts and models.

‘Why do you think things have changed? What is it about (X) that has helped this happen?’

3. Providing insight into other parts of your service or organisation. Make sure you have allocated enough time to analyse this data and have a clear audience/s in mind for dissemination. 

‘Have these changes affected anyone else (e.g. family members)? If so, who have they affected and how?

For more information on developing and using open questions in your evaluation, see our guidance leaflet here.

Analysing qualitative data 

A good place to start when analysing qualitative data is to ask questions. Here are five questions you might start with (Shank, 2002):

1. ‘What sorts of findings did you get that you expected to get?’
   
2. ‘How can you best organise these expected findings?’
   
3. ‘What sorts of findings did you get that you did not expect to get?’
   
4. ‘How can you best organise these unexpected findings?’
   
5. ‘Which three findings do you feel are the most important?’
   

One way of organising your findings is to look for patterns and/or themes, sometimes called codes.  For example, if you were analysing interview transcripts from people participating in a community choir who were asked to share their experiences, you might find themes relating to friendship or social connections, feelings of happiness and joy, a sense of belonging, confidence and participating in something meaningful or purposeful. 

Sharing your findings

When qualitative data is analysed and written up, the following information is usually included: 

• Research question and reason for the study
• Who did you collect data from? Number of beneficiaries, relevant demographics, etc.
• How were beneficiaries chosen and invited, and how many refused to participate (and why)?
• Context in which you collected your data (location, date, time, face to face/telephone)
• A list of the main questions asked
• Data analysis techniques used (Did you use coding? What were your themes?)
• Summary of main findings
• Include some direct quotes to illustrate key points
• Any strengths or limitations
• Recommendations for future research, and implications for organisations thinking about delivering similar projects in the future

In this section you learned:

• What qualitative data can help you learn in evaluations
• How to collect qualitative data through focus groups and surveys
• How to make sense of qualitative data.

You’re ready to learn how to analyse and interpret your evaluation findings.