Which of the following terms refers to the extent to which research findings are generalizable?

The goal of scientific research is to increase our understanding of the world around us. To do this, researchers study different groups of people or populations. These populations can be as small as a few individuals from one workplace or as large as thousands of people representing a cross-section of Canadian society. The results of this research often provide insights into how work and health interact in those groups. But how do we know if a study's results can be applied to another group or population?

To answer this question, we first need to understand the concept of generalizability.

In its simplest form, generalizability can be described as making predictions based on past observations.

In other words, if something has often happened in the past, it will likely occur in the future. In studies, once researchers have collected enough data to support a hypothesis, they can develop a premise to predict the outcome in similar circumstances with a certain degree of accuracy.

Two aspects of generalizability

Generalizing to a population. Sometimes when scientists talk about generalizability, they are applying results from a study sample to the larger population from which the sample was selected. For instance, consider the question, “What percentage of the Canadian population supports the Liberal party?” In this case, it would be important for researchers to survey people who represent the population at large. Therefore they must ensure that the survey respondents include relevant groups from the larger population in the correct proportions. Examples of relevant groups could be based on race, gender or age group.

Generalizing to a theory. More broadly, the concept of generalizability deals with moving from observations to scientific theories or hypotheses. This type of generalization amounts to taking time- and place-specific observations to create a universal hypothesis or theory. For instance, in the 1940s and 1950s, British researchers Richard Doll and Bradford Hill found that 647 out of 649 lung cancer patients in London hospitals were smokers. This led to many more research studies, with increasing sample sizes, with differing groups of people, with differing amounts of smoking and so on. When the results were found to be consistent across person, time and place, the observations were generalized into a theory: “cigarette smoking causes lung cancer.”

Requirements for generalizability

For generalizability we require a study sample that represents some population of interest — but we also need to understand the contexts in which the studies are done and how those might influence the results.

Suppose you read an article about a Swedish study of a new exercise program for male workers with back pain. The study was performed on male workers from fitness centres. Researchers compared two approaches. Half of the participants got a pamphlet on exercise from their therapist, and half were put on an exercise program led by a former Olympic athlete. The study findings showed that workers in the exercise group returned to work more quickly than workers who received the pamphlet.

Assuming the study was well conducted, with a strong design and rigorous reporting, we can trust the results. But to what populations could you generalize these results?

Some factors that need to be considered include: How important is it to have an Olympian delivering the exercise program? Would the exercise program work if delivered by an unknown therapist? Would the program work if delivered by the same Olympian but in a country where he or she is not well-known? Would the results apply to employees of other workplaces that differ from fitness centres? Would women respond the same way to the exercise program?

To increase our confidence in the generalizability of the study, it would have to be repeated with the same exercise program but with different providers in different settings (either worksites or countries) and yield the same results.

Source: At Work, Issue 45, Summer 2006: Institute for Work & Health, Toronto

refers to portraying accurately the meanings attached by participants to what is being studied by the researcher. More specifically, it refers to the degree to which the qualitative researcher accurately understands research participants' viewpoints, thoughts, feelings, intentions, and experiences and portrays them in the research report. Perhaps the most important skill required for conducting qualitative research is understanding the research participants' "inner worlds" (i.e., their subjective worlds), and interpretive validity refers to the degree of accuracy in presenting these inner worlds. Accurate interpretive validity requires that the researcher get inside the heads of the participants, look through the participants' eyes, and see and feel what they see and feel. In this way, the qualitative researcher can understand things from the participants' perspectives and thus provide a valid account of these perspectives.

Perhaps the biggest limitation of action research is that it sometimes relies on weaker methods and validity strategies than other methods using a more regular scientific research.

Action researchers are reflective practitioners. One important part of this reflexivity is considering how to obtain results that are defensible and trustworthy not only to the researcher but also to insiders (the people studied) and "objective outsiders" who might have the power to make decisions about sanctioned practices. Keeping track of evidence of success and effectiveness is very important to successful action researchers.

Given these interests and concerns of action researchers, as well as the variety of ways to do action research, action researchers consider most of the types of validity evidence that other researchers do. For example, in order to make up for their inability to control variables or randomly assign people to groups, weakness minimization is often used by action researchers. Triangulating qualitative evidence against the quantitative data can help bolster the validity of inferences made from the study. There is a need for construct validity so that there is consistency of definition of groups or measures. Perhaps the one area of less concern is the generalizability aspect of external validity. Studies done for organizational improvement, for example, may be very particular to the organization examined. While generalization might be nice,the focus is on the problem at hand and not on whether the problem can be solved in general (although it is viable that a solution in one context might generalize to another in that setting). Finally, as noted above, there is often a need to focus on the sociopolitical validity of action research results. Such studies are often done in organizational settings (including schools), and with organizations come different values and needs of different stakeholders. Hence, focusing on the sociopolitical validity of the study is important.

What type of research is generalizable?

What does it mean when findings are generalizable?

What type of validity refers to how generalizable the findings are?

What name is given to the generalizability of the findings of a study?