During my PhD I took over co-chairing an informal qualitative discussion group which was set up by Marta Cecchinato called QUDDLE (pronounced Cuddle), which stands for QUalitative Data Discussion Led by Experience.
We discuss many topics in these fortnightly meetings from methods to use, common pitfalls, advice on analysis and beyond.
Together with some colleagues from this group at UCLIC, we wrote a quick blog to discuss some common qualitative research constraints. Original blog can be found here.
Whether studying or working within academia, or conducting user research within industry, at some point you will be faced with research constraints. Below we share some of the constraints that we have faced when conducting qualitative user research at UCL’s Interaction Centre.
Time Constraints
As user researchers, we’re often under pressure to produce results, constrained by time. A PhD student may be chasing a publication deadline, whilst industry professionals may be racing against the clock towards product release date. How do we manage these time constraints within user research?
I was aware of the limited time I had to complete my own doctoral research. I was engaging with a sensitive population that I knew very little about at the beginning of my studies and was worried that I’d be unable to recruit participants.
When faced with time constraints, you may have to think creatively.
- Try to avoid reducing the time you spend on data analysis as this is likely to jeopardise the quality of your findings and insights.
- Look for methods that can be conducted quicker. Could you draw from secondary sources? Or engage in focus groups, rather than interviews?
- If financial resources are not a constraint, could you outsource parts of the study? For instance, whilst transcribing your own interviews can be good for immersion, the time could be better spent on data analysis.
Financial Constraints
Research projects are often constrained with financial limits which can be challenging to manage. Research often involves costly equipment, payments to participants, and other costs such as printing for recruitment flyers and professional transcription services. Moreover, for academics and students who are submitting papers to conferences, publishing fees and travel expenses are likely to take a significant chunk from these budgets. Financial limitations can constrain the research being conducted and limit the avenues available for promoting and publishing our work.
I had a number of engaged participants who wanted to participate in my research, but I was afraid how many to recruit, as this impacted how much I had to spend on paying them and then how much would be remaining in my budget to attend conferences to present my work.
If financial constraints occur, plan ahead and don’t be shy to ask for additional funds from other sources.
- Some departments may have additional funds that only covering travel expenses, so carefully prioritise how you spend your main budget.
- Share resources, you may not need to buy a new piece of equipment if someone else in the department already has it.
- Plan your time carefully, funds may limit how much you can incentivise participants or how many participants you can recruit.
- Apply for student volunteer programs at conferences, the conference fees for student volunteers are often waived.
Technical Constraints
As a user researcher, you may develop an application that needs evaluating or develop software for analysing research data. As such, you may face technical constraints such as: database, performance, operational, maintainability, reliability and safety.
I wanted to develop a novel technology to deploy as part of a study. I conducted an early auto-ethnographic study with myself to experience and understand first-hand the different elements of the technology which may or may not work well within the study. This influenced and informed the study design, allowing me to overcome and shape the study around the technical constraints.
When faced with technical constraints:
- Consider speaking to peers from your own research lab or even online, to observe and learn how they have overcome technical constraints.
- It’s also worth keeping in mind that technical constraints can be influenced by other resources, namely time and money. In other words, most technical constraints can be overcome by leveraging one of these levers. You could spend more time, for instance, increasing your technical skills to be able to resolve constraints and develop a solution to overcome these constraints. If budget is no issue, you could spend more money on faster hardware, or outsource software development.
Cultural Constraints
Cultural constraints relate to community, country, or region-specific customs, ideals and behaviours. As a user researcher, you may be conducting studies with partners in other areas of the world. Therefore, various dimensions of a research project are affected as the deployment of the research requires interactions between people from different cultures with different cultural practices and working styles.
At the start of my project I was invisible to my overseas stakeholders and my suggested research methods were questioned. This was concerning as in order to be successful in the project I was reliant on my overseas colleagues understanding the proposed methods, and collaboratively drafting the protocol, accepting its relevance, and conducting the research.
When faced with cultural constraints:
- Acknowledge the risks of cultural clashes and perform a risk assessment that consider the potential cultural issues that may arise.
- Build rapport with colleagues early on in the project and include overseas colleagues during the process of drafting the research protocols.
- Set out the project’s values early on in the process. Partners may have different views on, for example, equality, diversity and inclusion. Discussing and documenting the research around a set of project values may help to align the different project partners and avoid issues later down the road.
- Understand early on the cultural differences around work and communication, and plan for it. Whilst it may be common practice for you to communicate via Slack or email, this may be an uncommon practice for people in different parts of the world. Special considerations should be made when developing a communication strategy across cultures and especially when involving people working in remote places.
Knowledge Constraints
However well trained we are as user researchers, there will be times when we will need the help of others — we may lack understanding of a certain theory, a particular research method, or software.
I was working on a project which required me to create a new checkout form for a shopping website: “What elements will guarantee a purchase?”, I was asking myself. I did not work with forms before and didn’t have enough time to learn the ins and outs of form design, but my co-worker was able to help me. They helped me understand the typical design patterns for form layouts and pointed me to some useful resources online.
When faced with knowledge constraints:
- Ask for help. Look out for people in your department/larger organisation who might have the skills or knowledge you need. In most cases, people will be happy to help.
- If your co-workers can’t help you, turn to academia. It’s likely that the topic you’re after has been covered in depth. A good place to start is Google Scholar.
- When the time comes, offer to help. Think about the set of skills that you possess, which might be useful to the people and projects around you.
Recruitment Constraints
Researchers often face challenges in recruiting participants for studies. Difficulties arise in identifying who the target population should be, how to recruit a sample that adequately represents this population and being able to recruit enough participants to meet the sample size/power requirements of the study. These challenges are particularly relevant when trying to engage with sensitive populations, who may be hard to reach.
My research focuses on experiences of having non-conforming identities. This is a very personal, sensitive topic, and given the lack of visibility/representation and stigma surrounding certain identities, it can be difficult to start engaging with people who may not feel comfortable discussing this. Some people are not ready to disclose their identity, and others may not have sufficient trust in the researcher to be willing to engage with them. I am in the process of considering how to build enough trust and mitigate these barriers to engagement.
When faced with recruitment constraints:
- Be flexible with the practicalities of the research, including the study methods! Would people feel more comfortable sharing their experiences in a different format to traditional in-person interviews or focus groups, such as online methods?
- Some methods have been proposed to conduct research with participants remotely — see work on asynchronous remote communities
- If you’re working with a group you have not engaged with before on a sensitive topic, you may want to engage in PPI (patient and public involvement) activities. This may involve holding workshops with relevant stakeholders (including people with ‘lived experience’ of whatever sensitive issue you are researching). PPI participants can give feedback on recruitment materials to ensure they are clear and acceptable to potential participants.
- Talk to other researchers who have worked within the same area/with hard-to-reach populations to see what resources they used to reach people and advertise projects, and how they built up trust with participants.
Ethical Constraints
You should always consider the ethical implications of running user research. Whether you are required to submit your study protocol to an ethics committee, or evaluate it locally, there are always ethical constraints with any research.
I planned to run a study using online “public domain” data to save time. The data is freely available online so I initially assumed it would be fine to use. When deliberating on the ethics of using such data, it soon became clear that ethically, there are some constraints around using this type of data in research. For instance, whilst it may be publicly available, it was not published with the intention or expectation of being used in academic research.
When faced with ethical constraints:
- Discuss the ethical implications of the research methods being used with colleagues and co-workers even before submitting for ethical review.
- Understanding the ethical constraints early will allow you to develop your study protocol around these constraints to ensure you do not get into hot water later down the road.
- Engage with your ethics team and ask for their advice — they have probably seen similar study designs and can advise you on ways to approach any problems related to ethics.
Conclusion
Constraints within research can be frustrating. Yet, managing constraints can lead to creative work arounds and innovative new ways of working. Be open and honest about the constraints you face when reporting the results of your research. After all, everyone is in the same boat, and perhaps other may be able to learn from your experiences.
Contributors: Mark Warner, Dilisha Patel, Dafne Zuleima Morgado Ramirez, Laura Lascau, Leya George and Tu Dinh Duong