What steps will you take to ensure the best generalization
of your findings?
The Generalization
Generalization is applied by researchers in academia. It can be defined as the extension of the results and conclusions of a research carried out on a population sample to the general population. Although the reliability of this extension is not absolute, it is statistically probable.
Since good generalization requires data on large populations, quantitative research—experimental, for example—provides the best basis for producing a broad generalization. The larger the sample population, the more the results can be generalized. For example, a comprehensive study of the role that computers play in the writing process might reveal that students who compose most of their text on a computer are statistically likely to move more chunks of text than students who do not compose on a computer.
Transferability
Transferability is applied by readers of the research. Although generalizability usually applies only to certain types of quantitative methods, transferability can apply to varying degrees to most types of research. Unlike generalizability, transferability does not imply broad statements, but rather invites readers of the research to make connections between the elements of a study and their own experience. For example, high school teachers could selectively apply the results of a study showing that heuristic writing exercises help students at the college level to their own classrooms.
Interrelationships between Generalization and Transferability
Generalizability and transferability are important elements of any research methodology, but they are not mutually exclusive. Generalization, to varying degrees, relies on the transferability of research results. It is important for researchers to understand the implications of these two aspects of research before designing a study. Researchers seeking to make a generalizable claim must carefully examine the variables involved in the study.
Among them are the population sample used and the mechanisms for formulating a causal model. Furthermore, if researchers want the results of their study to be transferable to another context, they must maintain a detailed account of the environment surrounding their research, and include a rich description of that environment in their final report. With the knowledge that the sample population was large and varied, as well as detailed information about the study itself, readers of the research can generalize and transfer the results to other situations with greater confidence.
Generalization
Generalization is not only common to research, but also to everyday life. In this section, we establish a working definition of generalization as it applies within and outside of academic research. We also define and consider three different types of generalization and some of their likely applications. Finally, we discuss some of the potential shortcomings and limitations of generalizability that researchers should consider when constructing a study that they hope will produce potentially generalizable results.
Definition
In many ways, according to Shavelson et al (1991), generalization is nothing more than making predictions based on recurring experience. If something happens frequently, we hope that it will continue to happen in the future. Researchers use the same type of reasoning when generalizing the results of their studies.
Once researchers have collected enough data to support a hypothesis, a premise can be formulated about the behavior of that data. This is what makes it generalizable to similar circumstances. However, due to its foundation in probability, this generalization cannot be considered conclusive or exhaustive.
Although generalization can occur in informal and non-academic contexts, in academic studies it usually only applies to certain research methods. Quantitative methods allow some generalization. Experimental research, for example, often produces generalizable results. However, this experimentation must be rigorous to obtain generalizable results.
Generalization Example 1
An example of generalization in everyday life is driving. Driving a car in traffic requires drivers to make assumptions about the likely outcome of certain actions. When approaching an intersection where a driver is preparing to turn left, the driver passing through the intersection assumes that the driver turning left will yield to him before turning. The driver passing through the intersection applies this assumption with caution, recognizing the possibility that the other driver may turn prematurely.
American drivers also generalize that everyone drives on the right side of the road. However, if we try to generalize this assumption to other settings, such as England, we will be making a potentially disastrous mistake. It is therefore evident that generalization is necessary to form coherent interpretations in many different situations. However, we do not expect our generalizations to work the same in all circumstances. With enough evidence we can make predictions about human behavior. At the same time we must recognize that our assumptions are based on statistical probability.
Generalization Example 2
Consider this example of generalizable research in the field of English studies. A study of students’ evaluations of composition instructors could reveal that there is a strong correlation between the grade students expect to earn in a course and whether they give their instructor high marks.
The study may find that 95% of students who expect to receive a “C” or lower in their class give their instructor a grade of “average” or lower. Therefore, there would be a high probability that prospective students who expect a “C” or less will not give their instructor high grades. However, the results would not necessarily be conclusive. Some students might buck the trend.
A second form of generalization focuses on measurements rather than treatments. For a result to be considered generalizable outside the test group, it must produce the same results with different forms of measurement. In terms of the heuristic example above, the results will be more generalizable if the same results are obtained when evaluated “with questions that have slightly different wording, or when we use a six-point scale instead of a nine-point scale” (Runkel and McGrath, 1972, p.46).
A third type of generalization concerns the subjects of the test situation. Although the results of an experiment may be internally valid, that is, applicable to the group being tested, in many situations the results cannot be generalized beyond that particular group. Researchers hoping to generalize their results to a broader population should ensure that their test group is relatively large and chosen at random. However, researchers must take into account the fact that test populations of more than 10,000 subjects do not significantly increase generalizability (Firestone,1993).
Potential limitations
No matter how carefully these three forms of generalizability are applied, there is no absolute guarantee that the results obtained in a study will occur in all situations outside the study. To determine causal relationships in a test environment, precision is of utmost importance. However, if researchers want to generalize their findings, range and variance must take precedence over precision.
Therefore, it is difficult to test accuracy and generalizability simultaneously, as focusing on one reduces the reliability of the other. One solution to this problem is to make a greater number of observations. This has a double effect: first, it increases the sample population, which increases generalizability. Second, precision can be reasonably maintained because random errors across observations will be averaged (Runkel and McGrath, 1972).