Cognition Research Methods

Essay 1Essay 2

Defining the Dependent Variable

In the Research Methods essay for Chapter 1, we discussed the importance of testable hypotheses—that is, hypotheses that are framed in a way that makes it clear what evidence will confirm them and what evidence will not. Sometimes, though, it’s not obvious how to phrase a hypothesis in testable terms. For example, in Chapter 12 we discuss research on creativity, and within this research investigators often present hypotheses about the factors that might foster creativity or might undermine it. Thus, one hypothesis might be: “When working on a problem, an interruption (to allow incubation) promotes creativity.” To test this hypothesis, we would have to specify what counts as an interruption (5 minutes of working on something else? an hour?). But then we’d also need some way to measure creativity; otherwise, we couldn’t tell if the interruption was beneficial or not.

For this hypothesis, creativity is the dependent variable—that is, the measurement that, according to our hypothesis, might “depend on” the thing being manipulated. The presence or absence of an interruption would be the independent variable—the factor that, according to our hypothesis, influences the dependent variable.

In many studies, it’s easy to assess the dependent variable. For example, consider this hypothesis: “Context reinstatement improves memory accuracy.” Here the dependent variable is accuracy, and this is simple to check—for example, by counting up the number of correct answers on a memory test. In this way, we would easily know whether a result confirmed the hypothesis or not. Likewise, consider this hypothesis: “Implicit memories can speed up performance on a lexical decision task.” Here the dependent variable is response time; again, it is simple to measure, allowing a straightforward test of the hypothesis.

The situation is different, though, for our hypothesis about interruptions and creativity. In this case, people might disagree about whether a particular problem solution (or poem, or painting, or argument) is creative. This will make it difficult to test our hypothesis.

Psychologists generally solve this problem by recruiting a panel of judges to assess the dependent variable. In our example, the judges would review each participant’s response and evaluate how creative it was, perhaps on a scale from 1 to 5. By using a panel of judges rather than just one, we can check directly on whether different judges have different ideas about what creativity is. More specifically, we can calculate the inter-rater reliability among the judges—the degree to which they agree with each other in their assessments. If they disagree with each other, it would appear that the assessment of creativity really is a subjective matter and cannot be a basis for testing hypotheses. In that case, scientific research on this issue may not be possible. But if the judges do agree to a reasonable extent—if the inter-rater reliability is high—then we can be confident that their assessments are neither arbitrary nor idiosyncratic.

Let’s be clear, though, that this is a measure of reliability—that is, a measure of how consistent our measurements are. As the text describes, reliability is separate from validity (i.e., whether we’ve succeeded in measuring what we intended to measure). It’s possible, for example, that all of our judges are reacting to, say, whether they find the responses humorous or not. If the judges all have similar senses of humor, they might agree with each other in this assessment (and so would have a high level of inter-rater reliability), but, even so, they would be judging humor, not creativity (and so would not offer valid assessments). On this basis, measures of inter-rater reliability are an important step toward establishing our measure—but we still need other steps (perhaps what the chapter calls a “predictive validation”) before we’re done.

Notice, in addition, that this way of proceeding doesn’t require us to start out with a precise definition of creativity. Of course, a definition would be very useful because (among other benefits) it would allow us to give the judges on our panel relatively specific instructions. Even without a definition, though, we can just ask the judges to rely on their own sense of what’s creative. This isn’t ideal; we’d prefer to get beyond this intuitive notion. But having a systematic, nonidiosyncratic consensus measurement at least allows our research to get off the ground.

In the same way, consider this hypothesis: “College education improves the quality of critical thinking.” This hypothesis—and many others as well—again involves a complex dependent variable, and might also require a panel of judges to obtain measurements we can take seriously. But by using these panels, we can measure things that seem at the outset to be unmeasurable, and in that way we appreciably broaden the range of hypotheses we can test.

Critical Questions

1. Why would defining a dependent variable be difficult in a study of creativity?
2.
fiogf49gjkf0d
What is inter-rater reliability? For what kinds of measurements would it be necessary to measure inter-rater reliability?
3.
fiogf49gjkf0d
Select a problem-solving study that was covered in your text. What were the independent variables and dependent variables in this study?

Submit to Gradebook:

First Name:
 
Last Name:
 
Your Email Address:
 
Your Professor's Email Address:
 
Section: