1
Basic Concepts in Research

1.1 The Scientific Method

This is a textbook about statistics. Simply defined, statistics are the mathematical tools used to analyze and interpret data gathered for scientific study. It is paramount to remember that statistical analyses and interpretations do not exist in a vacuum. They occur within the larger scientific research process. Both how to analyze and how to interpret the data are quite dependent upon the surrounding research context. While the subject of statistics can be singled out and studied in isolation (as this textbook demonstrates), it is inextricably linked to the larger scientific enterprise. As such, it is appropriate to review the basic features of “doing” science before we delve into statistics proper.

The scientific method can be conceptualized as a three‐step recursive process. Each can be summarized as follows:

Theory. Theories are an attempt to explain and organize collections of data observed about the topic (or “phenomenon”) under scrutiny by appealing to general principles and relationships that are independent of the topic itself. Take, for example, a line of research on the endurance of friendships. In theorizing why some acquaintances lead to enduring friendships while others do not, one could propose that personalities are a bit like magnets; similar ones repel one another, while dissimilar personalities are drawn together (i.e. opposites attract). Clearly, this theory appeals to the prior concepts “magnets,” “personality,” “similarity,” and “dissimilarity” in purporting to explain why certain friendships pass the test of time while others do not.

Not all theories can be considered “scientific.” For a theory to qualify as properly “scientific,” it must be testable. By testable we mean: is it potentially falsifiable? Can it be placed into jeopardy and potentially observed to be untrue? If it cannot, it still remains a theory, but it is not considered to be properly “scientific.” Using testability as a criterion, for example, the theory that each of our choices, past and future, is actually predetermined by some combination of our DNA and behavioral conditioning through our previous experiences, could hardly be considered “scientific.” While many people believe it to be true, how exactly would we go about testing it? And chiefly, how exactly could we place this theory in jeopardy and observe it to be true or untrue?

Hypothesis. In the light of any scientific theory, it should be possible to generate predictions about the data one expects to observe – this is a hypothesis. Sticking with the aforementioned magnetic theory of friendships, one hypothesis might be as follows: If we measure the personalities of incoming university students who are randomly assigned to live on a given hall in a freshman dorm, we might expect to find that students who have quite discrepant personality profiles are more likely to be friends at the end of the semester than those who had similar personality profiles.

Because it is possible to find evidence that would not support this hypothesis, we can say that this theory is “testable.” However, we cannot stop at hypothesizing. To say anything meaningful, we have to complete the research process and actually go out and do the work, set up the study, gather the participants, and carefully collect the data. This leads us to the final step.

Observation. The gathering of scientific observations is done by careful and systematic measurements of events occurring in the world by using our five senses, often with the aid of various scientific tools and instruments. In our example, we would want to measure meticulously our incoming freshman’s personalities as well as the nature of the friendships on the hall at the end of the semester. These observations, then, would be organized and interpreted. Ultimately, what is concluded would reflect back upon the theory. Observations will either support the theory, fail to support it, or, perhaps, partially support it. The circle is complete as we relate our findings to our original theoretical proposition.

In our particular example, we should not be too confident that supporting data will be found – previous research suggests we will probably be disappointed (e.g. Buss, 1985). And that is an important point – if supporting data is not found, so much the worse for the theory. We may need to think differently about why some friendships begin and endure while others do not. As would be expected, accurate theories will be supported by our observations. Supporting observations can both affirm a theory and lead to clearer and more refined articulations of that theory. More precise theories, in turn, lead to new hypotheses, and the cycle starts over again. The process is circular and recursive, with each cycle ideally spiraling toward a more accurate understanding of the topic under investigation.

The specific role of statistical analysis is found in the interpretation of our numerically represented observations. What do the numbers mean? What do they not mean? For whom do they have meaning? Furthermore, how certain are we that our conclusions are accurate? On what do we base our sense of certainty? These are often not easy determinations to make. The central purpose of this text is to dissect and explain how this part of the research process works. The remainder of this introductory chapter will lay out an overview of the research enterprise.

1.2 The Goals of the Researcher

Scientific researchers set out with earnest intention to study carefully, logically, and objectively a particular topic of interest. Depending upon what is already known about the topic, what one wants to learn about the topic, and what one realistically can learn about the topic, researchers adopt different “goals” for their projects. Often, the initial goal a researcher has when first addressing a topic of interest is that of description. Scientific description is the process of defining, identifying, classifying, categorizing, and organizing the topic of interest. Explicit delineation of the boundaries of the topic is crucial. What exactly constitutes the topic and what clearly does not constitute the topic? How many forms can it take? How frequently are these various forms found?

For example, if we were interested in studying the various ways in which people take vacations, we would first have to define what a vacation is and what it is not. Is an afternoon day trip to a community park a vacation? What about an extra day tacked onto a work‐related business trip? It is not a requirement for all researchers to agree on the same definition of what “is” and “is not” a vacation, in order for vacations to be studied. However, it is absolutely imperative that the readers know explicitly what we, the researchers, mean when we say that we are counting days spent on vacation. In other words, concepts must be operationally defined. An operational definition is a precise verbal description of the concrete measurement of that concept, as it will be used in a given research project.

Another issue would be to decide how many different ways “vacation” can take place. For example, someone might suggest that there are fundamentally two different kinds of vacations: one kind that is designed around relaxation and focuses on bodily rest and another kind that is designed around engaging in new and exciting experiences. Another researcher may come along and suggest that there is actually a third kind of vacationing – one that combines the two and incorporates both time dedicated to bodily rest and time dedicated to having new experiences (e.g. traveling the country in a motor home). Widespread agreement regarding the particulars of the concept “vacation” is not required, of course, for it to be studied. The crucial point to be made is that researchers who are dealing with a topic at this level are going to gather statistics that reflect the relative frequencies and averages pertaining to the categories of the topic under investigation, as they understand them to exist. For example, one researcher might find that only 20% of vacations are of the relaxation variety, while another researcher, using a different operational definition, might find a quite different percentage. Statistical statements, then, can only be properly interpreted once the larger research context is correctly understood. Finally, it should be noted that the statistical needs associated with meeting this initial goal of “description” are usually not too sophisticated.

Another goal of the researcher would be one of correlation (or prediction or association – these are all analogous terms). Correlation involves a description of the degree of relationship between the topic of interest and other variables. For example, in our study of vacations, we might be interested to see if there were a relationship between the age of the vacationer and the type of vacation chosen. Here, we would be measuring two variables (the “age of the vacationer” and the “type of vacation chosen”) and determining if there was a relationship between them. As a rule, it requires more sophisticated mathematical work to establish correlations. It is critical to realize that research designed to show correlations does not allow us to draw causal conclusions. For...