Tag Archives: Methods

HIA Research: When is Qualitative Research Warranted?

[As research director at Human Impact Partners, Holly Avey spends a lot of time not just looking at our findings but thinking about how we conduct and use research. This is one in a series of blogs about the role of research in HIA.]

In my research blog published back in 2013, I asked: How far should we go with qualitative research in HIA? Is it just used when we don’t have enough quantitative data to answer our research question, or are there other reasons to consider incorporating qualitative research into your HIA work?

A national evaluation of HIAs conducted by the Environmental Protection Agency states that “stakeholder and community input lend themselves to qualitative analysis”, and beyond that, qualitative analysis is warranted in HIAs in the following circumstances: “lack of available scientific research, unavailability of local data, time limitations, limited resources, etc.” (p. 39). The implication is that qualitative data is warranted as a means of stakeholder input, but from a data perspective, you might only pursue qualitative data if you don’t have and/or can’t get quantitative data.

The authors further state, “most HIAs qualitatively characterized impacts; the use of quantitative analysis was lacking.” (p. 80). This statement implies that qualitative characterization of impacts is not sufficient or appropriate when quantitative data is available and the process allows it to be obtained.

This perspective is not unique to the EPA, or to the field of HIA. As Margarete J. Sandelowski states in her editorial Justifying Qualitative Research, quantitative research is often the default modality for the health sciences and is therefore introduced first. This results in many health researchers being trained to think of the ways qualitative research is different from, less than, or deficient in comparison to quantitative research. For example, qualitative research may be described as “less mathematically precise and as producing findings that are not generalizable” when compared to quantitative research. Alternatively, one never sees a comparison that assumes the qualitative research perspective and describes quantitative research as, “less descriptively precise and attentive to context” and limited to generalizations based on objective (nomothetic) phenomena (p. 193).

Thus it is no surprise that one of the EPA’s evaluation review criteria assumed the quantitative default perspective and was originally labeled “quantification of impact” but later changed to “characterization of impact” after the full-scale review had been completed, as a means of reflecting the fact that impacts can be characterized both qualitatively and quantitatively (p 12). Although the authors were trying to accommodate the multitude of research approaches that can be used in HIA, their quantitative default perspective still resulted in the summary statement that “quantification of impacts was lacking” (p. 80). How often might we similarly challenge health researchers to say “qualitative analysis was lacking”?

There may be two underlying assumptions here. One, that quantitative research is more rigorous and defensible in comparison to qualitative research, and two, that quantitative data is more compelling to decision-makers (note how both use the quantitative default perspective). To the first point, I would reiterate what I mentioned in my last blog, which is that qualitative and quantitative research are designed to answer different research questions. They are often based on different research philosophies (see my first research blog). They can both be executed in a manner that is rigorous or a manner that is sloppy. Rigor and defensibility are not the domains of one over the other, but many health researchers who are trained with the quantitative default perspective may assume a higher level of rigor with their default approach.

To the second point, what kind of data is more compelling to decision-makers? Well, in an interesting article published in the American Journal of Public Health titled Understanding Evidence-Based Public Health, the authors argue that “there is no single, ‘best’ type of evidence .” (p. 1578). … “Studies from the communication field have shown that the combination of [both qualitative and quantitative] evidence appears to have a stronger persuasive impact than either type of evidence alone.” (p. 1577).

The authors go on to state, “Qualitative evidence can make use of the narrative form as a powerful means of influencing policy deliberations, setting priorities, and proposing policy solutions by telling persuasive stories that have an emotional hook and intuitive appeal. This often provides an anchor for statistical evidence…”(p. 1577). They suggest that quantitative evidence be incorporated within a compelling story that is created with the qualitative data to maximize the potential use of the data in the policy process. They also go on to report that “in a survey of 292 US state policymakers, respondents expressed a strong preference for short, easy-to-digest data” (p. 1577). This finding may contradict what many quantitatively-focused HIA researchers may assume, which is that the more thorough and specific the data, the better.

While quantitative research can provide powerful data to inform our predictions with numerical specificity, we do not need to sacrifice research rigor for qualitative research. Qualitative research can inform new theories about connections to health that have not yet been studied. It can provide the localized context and community-specific perspectives that can create a compelling narrative and provide relevance and meaning. Qualitative data collection analysis processes can be powerful experiences for stakeholders, when they are offered in a participatory fashion.

So, returning to my original question and the title of this blog – when is qualitative research warranted for HIAs? Hmmm. Now isn’t that a question you’d only ask if you were coming from the quantitative default perspective? We should stop dismissing qualitative research as less-than or if-needed. We need both in HIA.

Is There a Role for Epidemiology in HIA?

In graduate school, I fell in love with the field of epidemiology – the study of the distribution and causes of disease – because it can pull together disparate pieces of information into a quantifiable representation of truth. My mind works like that –– using deductive reasoning and pattern recognition to piece together seemingly unrelated mysteries that cumulatively point to a plausible answer.

But after graduation, working in public health departments made me want not just to connect the data on health outcomes to their immediate cause, but to work for change that addresses the root causes. Conducting Health Impact Assessments at Human Impact Partners has proven to be a perfect fit.

Epidemiologic methods are not completely absent in HIA. Mapping the pathways of community health allows us to test possible causes against a conceptual and visual model, and working with community members often identifies possible causes that can be quantified. That said, there is not a lot of use of epi methods to dig into those associations – and I’m OK with that.

Over time HIA has shifted from a modeling approach derived from its pedigree in Environmental Impact Assessment to a more community-based, qualitative approach that borrows more from the social sciences – addressing real-life circumstances on a human scale. This makes examining causes, even within a single individual or a small community, more legitimate, because traditional quantitative methods can’t really accommodate this notion.

So instead of crystallizing information into a single, unifying truth, HIA – at least the model promoted by HIP and like-minded practitioners – will continue to seek multiple truths, because change does not happen equitably in a one-size-fits-all approach. Life is more dynamic and unpredictable than that, and so the tools we use must be more inclusive and adaptable. Epidemiology will – and should – continue to provide enormous value. In HIA, as we start to examine much more upstream factors and how they might affect health, I’m excited to look to other disciplines to see what methods might supplement traditional public health approaches.

HIA Research: What’s the Right Approach for Your Question?

[As research director at Human Impact Partners, Holly Avey spends a lot of time not just looking at our findings but thinking about how we conduct and use research. This is one in a series of blogs about the role of research in HIA.]

Last week I discussed philosophies of research, and how different people might see the same information as either an appropriate source of data or a source of bias. This week, let’s think about different approaches to answering research questions. While your philosophy influences how you think about research, the questions you ask influence how you collect and analyze your data.

A document from the National Institutes of Health (NIH), explains the difference between quantitative and qualitative approaches to research. When people have strong reactions about the pros and cons of these, I believe it stems from a difference in their underlying philosophy of research.

Quantitative research uses numeric data that can be analyzed statistically to assess relationships among variables and understand cause and effect

Qualitative research uses interviews, observations, and reviews of documents (among other methods) to understand the context and meaning of the situation

So which is right for HIA? Our personal philosophy of research will guide how we think about this initially, but the next question should be what kinds of questions do we want our research to answer?

First, what is the purpose of HIA? In 2001, the Merseyside Guidelines for Health Impact Assessment, were published for HIA practitioners in the UK. They state that the aims of HIA are:

  • “to assess the potential health impacts, both positive and negative, of projects, programmes and policies
  • to improve the quality of public policy decision making through recommendations to enhance predicted positive health impacts and minimise negative ones”

Based on this thinking, your overarching research questions might be:  “What are the relationships between the pending decision and any potential health impacts?” “Is the pending decision likely to cause any health effects?” The quantitative approach is good for assessing relationships among variables and cause-and-effect, so you should use a quantitative approach, right? But what happens when you don’t have the quantitative data to answer those questions? Often there are times when HIAs are focused on neighborhood or local-level decisions, with significant limitations on the available quantitative data. In these cases, a combination of methods may be the best bet.

Let’s look back at that NIH document, which defines this combination of methods in this way:

Mixed methods research “involves the intentional collection of both quantitative and qualitative data and the combination of the strengths of each to answer research questions.” (p. 4-5).

One example of combining quantitative and qualitative data is a story that is often told by Aaron Wernham, of the Health Impact Project. Wernham tells about a small community where a natural resource extraction processing facility was operating. Quantitative air quality data for the area did not show any significant violations of air quality standards after the facility began operating. Asthma rates tracked by the state also didn’t show an increase. But community members consistently reported that they perceived asthma rates to be higher. During the HIA, community members offered testimony at public meetings, which was tracked by the HIA team. During the testimony, one of the community members specified that the asthma rates got worse for people when certain conditions aligned – when the facility flared gas under certain weather conditions, with the wind directed toward the village. Community members also testified that the air quality data would not be likely to detect emissions under these conditions because of the location of the air quality monitor for the area.

In this case, quantitative data was available but limited to one monitor, which provided a limited perspective on conditions for the area. Qualitative data from initial discussions with community was also limited, as it provided general perceptions without specificity. Additional qualitative data from the testimony provided the specific context that allowed the HIA team to interpret some of the quantitative data from a new perspective, and understand the discrepancies between the two types of data in other cases. The combination of these two approaches allowed the HIA team to explore a new causal pathway for the HIA to investigate potential health impacts. Thus, combining the two approaches provided the opportunity for the HIA researchers to explore a more complete and accurate picture, and identified data gaps that were limiting the ability to address community concerns. Ultimately, this contributed to a recommendation that was adopted by the decision-makers as a formal requirement for more specific air quality modeling and modeling near potentially affected communities.

How far should we go with qualitative research in HIA? Is it just used when we don’t have enough quantitative data to answer our research question, or are there other reasons to consider incorporating qualitative research into your HIA work? That’s the next research blog topic.