Research tool kit ( draft version), Study Guides, Projects, Research of Research Methodology

Download Research tool kit ( draft version) and more Study Guides, Projects, Research Research Methodology in PDF only on Docsity!

RESEARCH

TOOLKIT

TABLE OF CONTENTS

ACKNOWLEDGEMENTS

The content of this Research Toolkit was developed from the three-day training workshops

conducted by the Data for Development (D4D) in Nepal Program that were held in May and June 2019 in Kathmandu, Nepal and covered the following topics:

1. Improving Research Impact
2. Quantitative Research Skills
3. Qualitative Research Skills

In collaboration with:

ABBREVATIONS

AIDS Acquired Immune Deficiency Syndrome AIIM Alignment, Interest, Influence Matrix ANCOVA Analysis of Covariance ANOVA Analysis of Variance CAQDAS Computer-assisted qualitative data analysis software CIRT Center For Innovation in Research and Teaching CSV Comma Separated Values D4D Data for Development in Nepal DFID Department for International Development DI Development Initiatives DV Dependent Variable ECOSOC Economic & Social Research Council ERB Ethical Review Board FGD Focus Group Discussion GAGE Gender and Adolescence: Global Evidence GESI Gender Equality and Social Inclusion GIF Graphics Interchange Format HIV Human Immunodeficiency Virus IRC Institutional Review Committee ISBN International Standard Book Number IV Independent Variable KII Key Informant Interview M&E Monitoring and Evaluation MANOVA Multivariate Analysis of Variance MS Excel Microsoft Excel MS Work Microsoft Work NHRC The Nepal Health Council ODA Open Data Assistance ODI Overseas Development Institute PDF Portable Document Format SDG Sustainable Development Goals SPSS Statistical Package for the Social Sciences UK United Kingdom

INTRODUCTION

Purpose of the toolkit

The Data for Development in Nepal (D4D) Program aims to contribute to the increased use of evidence by government agencies, civil society and private sector in their decision-making. Through the use of this toolkit D4D aims to help ensure the research has impact, it is critical that research is targeted towards influencing specific goals and audiences; that it builds upon the existing body of evidence; that it is clearly and compellingly presented and in formats that meet audience needs; that it is promoted effectively in way that supports use of the research. Well-designed strategies in these areas can support research to have broader influence and result in change. As well as building the capacity of Nepali organisations in conducting user research; finding and analysing existing available research and data; designing of research methodologies and tools; collection and processing of data; analysis of data, development of engaging knowledge products; and in ensuring uptake of research findings and use of data.

This Research Toolkit brings together the training workshops conducted by D4D with local partner organisations to grow the influence of their research by teaching them strategies to improve research uptake and technical research skills

Target audience

This Research Toolkit is suited for people who are currently running or planning research and want to learn techniques that can improve their product, impact and technical research skills. People who do not already have detailed knowledge of user research, situating research within the existing knowledge base, presentation, research uptake and/ or quantitative and/or qualitative research skills.

Structure of the toolkit

The guide is divided into three modules covering the following topics: Module 1 – Improving Research Impact Module 2 – Quantitative Research Skills Module 3 – Qualitative Research Skills

MODULE 1: IMPROVING RESEARCH IMPACT

Module Aims

The module aims to support researchers to grow the influence of their evidence by teaching them strategies to identify their desired impact; analyse the relevant existing knowledge base; conduct audience-research; present research; and grow research uptake.

Key learning objectives

By the end of the module participants will:

1. Know how to identify the desired impact of the research
2. Know how to identify users and conduct user research
3. Know how to analyse the relevant existing knowledge base
4. Know how to effectively present research for different users
5. Know how to monitor and evaluate uptake of the research

By the end of the module you will have the tools needed to design strategies that grow the impact of your research.

▪ Economic and societal impact is the demonstrable contribution that excellent social and economic research makes to society and the economy, and its benefits to individuals, organisations and/or nations^1.

According to the Economic and Social Research Council, the impact of research, be it academic, economic and social can include: ▪ Instrumental: influencing the development of policy, practice on service provision, shaping legislation, altering behaviour ▪ Conceptual: contributing to the understanding of policy issues, reframing debates ▪ Capacity building: through technical and personal skill development^2

Importantly, for research evidence to have impact, it needs to be used - by the right people, at the right time, to inform decision-making and action.

Why does research not often achieve impact? Unfortunately, research is often not used, and therefore fails to have impact. If research does not have impact then, again, what is the point of research?

According to the London School of Economics, “85% of health research is wasted, usually because it asks the wrong questions, is badly designed, not published or poorly reported”^3. What issues are preventing research evidence from being used?

Activity:

1. How much impact do you think research has in Nepal? a. Think about whether the research you have been involved with has informed decisions or action b. Think about whether there are other examples of where research in Nepal has made an impact
2. What do you think are the barriers to achieving impact with research in Nepal?

(^1) UK Research and Innovation - Pathways to Impact, https://www.ukri.org/innovation/excellence-with- impact/pathways-to-impact/ (accessed 5 September 2019) (^2) Economic and Social Research Council (2019) What is Impact, https://esrc.ukri.org/research/impact- toolkit/what-is-impact/ 3 (accessed 5 September 2019) London School of Economics (2015) Impact of Social Sciences Blog, Available at: http://eprints.lse.ac.uk/65847/1/85%20percent%20of%20Health%20Research%20is%20Wasted%20How %20to%20do%20great%20research%20get%20it%20published%20and%20improve%20health%20outco mes.pdf

a. Think about whether people are using research evidence in their decisions b. Think about whether decision-makers are able to access the research that they need c. Think about whether the research created, and research products developed are designed to meet these needs.

There are many factors which can prevent research from having an impact, regardless of the country. Some common themes include: ▪ Research evidence is often not communicated in ways that are useful to decision-makers ▪ Research evidence often does not meet the specific needs of the decision-maker ▪ Research evidence is sometimes perceived by decision-makers to lack credibility, particularly if they have not been engaged with it. ▪ There is often a large gap between researchers and research users meaning that users cannot access research and researchers cannot reach users. ▪ Political context and power relationships sometimes disincentivise the use of research evidence in decision making

Case Study In 2016 a study was conducted by Dhimal et all that looked at barriers preventing research

having impact in Nepal. The research report ‘Translation of Health Research Evidence into

Policy and Planning in Nepal: An Appraisal 2016' raised the following issues^4 :

Direct Contact and Negotiation: Lack of direct contact of researchers and policy makers thereby limiting opportunities for direct negotiation was cited by researchers as challenge in pushing research findings towards informing policy making. In some cases, geographical inaccessibility like being located outside Kathmandu Valley was found to limit the opportunity of interaction and thus utilization of research findings.

Poor Research Reading culture: Among other factors impeding the utilization of research findings were the lack of coordination between concerned stakeholders, a poor culture of reading articles among policy makers, unethical conduct of research, noncompliance of research findings with interest of donor agencies, and a failure to write appealing papers.

(^4) Dhimal, Meghnath & Pandey, Achyut & Aryal, Krishna & Subedi, Madhusudan & Karki, Khem. (2016). Translation of Health Research Evidence into Policy and Planning in Nepal: An Appraisal. Available at: http://nhrc.gov.np/wp-content/uploads/2017/06/HEALTH-POLICY-book.pdf

• Communicating research effectively
• Synthesising and repacking research for different audiences ▪ Support the Usage of Research
• building the capacity and commitment of users to access, evaluate, synthesise and use research evidence

To find out more, watch these two videos: ▪ What is Research Uptake and how is it done? https://www.youtube.com/watch?v=- 2wteUq7t3s ▪ An example of a research uptake strategy by GrOW https://www.youtube.com/watch?v=- 2wteUq7t3s

The London School of Hygiene and Tropical Medicine has summarised the evolution of research uptake as follows:

Different Types of Research Uptake Research Update is not always “up”. There can also be research sidetake and downtake^6.

Research Sidetake: Not all ideas flow ‘upwards’ and towards decision-makers. For many researchers the most immediate audience is other researchers. Ideas take time to develop and researchers often need to share them with their peers first. ‘Uptake’ therefore can very well be ‘sidetake’.

(^6) Mendizabal, Enrique (2013) Research Uptake: What is it and can it be measured? https://onthinktanks.org/articles/research-uptake-what-is-it-and-can-it-be-measured/ (accessed 5 September

Research Downtake: Not all research is aimed at decision makers or academics. So, ‘uptake’ could also be ‘downtake’. Much of the research done is directed not at high level political decision-makers but at the public (such as public health information) or practitioners (toolkits, manuals).

Activity: Reflect on practices of research uptake, sidetake or downtake in your last research project. What activities did you do that would fall into each of the three categories?

Planning for research uptake To help you improve the potential of your research to have impact, we have developed a five- stage research uptake process for you to follow.

1. Strategic area 1: Identify the desired impact of the research
2. Strategic area 2: Identify users and conduct user research
3. Strategic area 3: Analyse the relevant existing knowledge base
4. Strategic area 4: Present research effectively for different users and grow uptake of research
5. Strategic area 5: Monitor and evaluate uptake of the research

This has been developed into a template which can be found here. The following sections of this toolkit will help you navigate each of these strategic areas.

related to one another causally) for the goals to occur. These are all mapped out in an Outcomes Framework. The Outcomes Framework then provides the basis for identifying what type of activity or intervention will lead to the outcomes identified as preconditions for achieving the long-term goal. Through this approach the precise link between activities and the achievement of the long-term goals are more fully understood. This leads to better planning, in that activities are linked to a detailed understanding of how change actually happens. It also leads to better evaluation, as it is possible to measure progress towards the achievement of longer-term goals that goes beyond the identification of program outputs.”

The Center for Theory of Change website has a range of specific project examples you can explore: https://www.theoryofchange.org/what-is-theory-of-change/how-does-theory-of-change- work/example/

To develop a Theory of Change you need to follow several distinct steps. The Center for Theory of Change outlines the following stages: a) Identify your long-term goals b) Backwards map and connect your outcomes c) Complete the outcomes framework d) Identify your assumptions e) Develop your indicators f) Identify your interventions

For more information about how to conduct a Theory of Change please see the factsheet here. Following these steps will help you understand the dynamics impacting the broader change you wish to influence and the role that your research will play in that.

Creating the Policy Window If your theory of change involves informing or influencing of policy to get to your desired impact, then it is helpful to understand the “Multiple Streams Model” developed by John Kingdon in

1984. The Multiple Streams Model conceptualizes how three independent streams of thinking must come together in order to create change: ▪ the problem must be recognised (problem stream)

▪ there must be a policy solution to the problem (policy stream) ▪ there must be political will to solve the problem (politics stream)

Your research should have an impact on one or more of these streams in order for change to occur as when these streams come together you create a policy window.

Image credit: https://findings.org.uk/PHP/dl.php?f=rooms.hot

Guldbrandsson et al describe the streams as follows^7 : ▪ Problem stream: Those holding power or decision makers may not be giving equal attention to all problems. Some conditions are not even defined as problems until there is something to do about it, i.e. there is a solution (a policy) available and recognised by politicians. ▪ Policy stream: Before a problem can reach the decision agenda, decision makers must be given at least one alternative solution, worked out and ready to put in place. Politicians concerned with an array of problems will prioritize to act on the ones where the administration, the scientific community, or somebody else, could provide a constructive solution, often worked out in advance.

(^7) Guldbrandsson, Karin & Fossum, Bjöörn, An exploration of the theoretical concepts policy windows and policy entrepreneurs at the Swedish public health arena, Health Promotion International, Volume 24, Issue 4, December 2009, Pages 434–444, https://doi.org/10.1093/heapro/dap

Policy can be national, sectoral, regional (provincial), local, community-level, or international, and it can relate to plans, budgets and the setting of government priorities. Understanding how policy is made in the specific context you are working within is essential if you are to increase your research impact. These factors can influence the timing of your research outputs and who your outputs are targeted at. In Nepal policy is made as a result of:

▪ Routine Government Procedures: for example, the development of annual periodic plans and budgets by the National, Provincial and Local bodies or line ministries ▪ Public Advocacy: for example, the development of policy as a result of advocacy by rights groups or interest groups; media; research organisations and development partners.

It is critical to feed policy with research evidence at the ‘right’ time. For example, to influence the allocation of resources of development of planning for the expenditure of resources, research will need to target researchers during their planning phases. The timeline of the budgeting process in Nepal falls between April-June of every fiscal year and involves the following steps:

For your research to have impact on budget allocations it should target the decision-makers early on in their budgeting processes.

1.3 Strategic Area 2: Identifying users & conducting user research

Chapter Objectives By the end of this chapter you will:

1. Know why understanding research users and their needs is important
2. Learn about tools than help you understand research users’ needs

Understanding your research users The users of research are often identified as being really broad, covering everyone from other researchers, programme staff, donors, multi-lateral organisations to evidence brokers, the media, non-government organisations, policymakers and the general public. However, while the users of research may be varied and wide it is impossible for you to reach all these intended audiences and have an impact with one simple product (such as a results paper).

Activity: Who are all the different users who can be influenced or informed by research? Thinking about the research you have been involved with, who were your intended users, who were the actual users?

In order to truly have impact researchers need to take a flexible approach and begin to understand how you want each of these users to act and what their preferred methods of communication are. Users are a static concept; we must regularly reconsider our users of research as their needs may change or new users may emerge as we move ahead with our research.

Often the best way to start thinking about your target users and to ensure that as a research team you all understand who you are trying to influence and what you want them to do is to carry out a user mapping exercise.

Tools to help you map users and identify their needs There are a number of different tools and approaches that can be used to map users and their needs. They typically involve steps to:

▪ identify multiple audiences who might be interested in your work