[As research director at Human Impact Partners, Holly Avey spends a lot of time not just looking at our findings but thinking about how we conduct and use research. This is one in a series of blogs about the role of research in HIA.]
Last week I discussed philosophies of research, and how different people might see the same information as either an appropriate source of data or a source of bias. This week, let’s think about different approaches to answering research questions. While your philosophy influences how you think about research, the questions you ask influence how you collect and analyze your data.
A document from the National Institutes of Health (NIH), explains the difference between quantitative and qualitative approaches to research. When people have strong reactions about the pros and cons of these, I believe it stems from a difference in their underlying philosophy of research.
Quantitative research uses numeric data that can be analyzed statistically to assess relationships among variables and understand cause and effect
Qualitative research uses interviews, observations, and reviews of documents (among other methods) to understand the context and meaning of the situation
So which is right for HIA? Our personal philosophy of research will guide how we think about this initially, but the next question should be what kinds of questions do we want our research to answer?
First, what is the purpose of HIA? In 2001, the Merseyside Guidelines for Health Impact Assessment, were published for HIA practitioners in the UK. They state that the aims of HIA are:
- “to assess the potential health impacts, both positive and negative, of projects, programmes and policies
- to improve the quality of public policy decision making through recommendations to enhance predicted positive health impacts and minimise negative ones”
Based on this thinking, your overarching research questions might be: “What are the relationships between the pending decision and any potential health impacts?” “Is the pending decision likely to cause any health effects?” The quantitative approach is good for assessing relationships among variables and cause-and-effect, so you should use a quantitative approach, right? But what happens when you don’t have the quantitative data to answer those questions? Often there are times when HIAs are focused on neighborhood or local-level decisions, with significant limitations on the available quantitative data. In these cases, a combination of methods may be the best bet.
Let’s look back at that NIH document, which defines this combination of methods in this way:
Mixed methods research “involves the intentional collection of both quantitative and qualitative data and the combination of the strengths of each to answer research questions.” (p. 4-5).
One example of combining quantitative and qualitative data is a story that is often told by Aaron Wernham, of the Health Impact Project. Wernham tells about a small community where a natural resource extraction processing facility was operating. Quantitative air quality data for the area did not show any significant violations of air quality standards after the facility began operating. Asthma rates tracked by the state also didn’t show an increase. But community members consistently reported that they perceived asthma rates to be higher. During the HIA, community members offered testimony at public meetings, which was tracked by the HIA team. During the testimony, one of the community members specified that the asthma rates got worse for people when certain conditions aligned – when the facility flared gas under certain weather conditions, with the wind directed toward the village. Community members also testified that the air quality data would not be likely to detect emissions under these conditions because of the location of the air quality monitor for the area.
In this case, quantitative data was available but limited to one monitor, which provided a limited perspective on conditions for the area. Qualitative data from initial discussions with community was also limited, as it provided general perceptions without specificity. Additional qualitative data from the testimony provided the specific context that allowed the HIA team to interpret some of the quantitative data from a new perspective, and understand the discrepancies between the two types of data in other cases. The combination of these two approaches allowed the HIA team to explore a new causal pathway for the HIA to investigate potential health impacts. Thus, combining the two approaches provided the opportunity for the HIA researchers to explore a more complete and accurate picture, and identified data gaps that were limiting the ability to address community concerns. Ultimately, this contributed to a recommendation that was adopted by the decision-makers as a formal requirement for more specific air quality modeling and modeling near potentially affected communities.
How far should we go with qualitative research in HIA? Is it just used when we don’t have enough quantitative data to answer our research question, or are there other reasons to consider incorporating qualitative research into your HIA work? That’s the next research blog topic.