Analysis of Variance

Mass Communication Research
ANOVA

One-Way ANOVA

ANOVA, a short hand for analysis of variance, is used to compare three or more means, i.e. researchers use ANOVA to determine if the samples represent the same population in terms of their means. Like the t-test, the independent variable must be nominal and the dependent variable must be interval or ratio.

What is this thing called "F?"

Back in the old days before most of you were born we had to calculate statistics by hand. Statistics textbooks contain elaborate directions and mathematical formulae for doing those calculations.

The primary thing that was being calculated was called a "statistic." Once you calculated a statistic, you had to go to a table to look up the significance value associated with it. (You also needed to know the "degrees of freedom" associated with the statistic to figure out the level of significance. Statistics textbooks usually had hundreds of pages of tables and the Wimmer and Dominick book still has a few of them.)

When computer programmers began writing programs to calculate statistics so we don't have to do all that math, they decided to report the statistics and degrees of freedom. The reported these things although the computer calculates the significance so we don't have look it up any more.

So, what is F anyway? F is a statistic, a value that we used to calculate by hand so we could look up the significance level in a table. We persist in looking at such statistics and reporting them in academic research, but we really don't much need them any more. For that reason, we haven't said much about them.

In Media Research we learn about several statistics, Chi Square, t, F, and r. Notice that Chi Square is in the Greek alphabet and the others, in the Roman alphabet. F is capitalized and t and r are lower case. These conventions persist although the ideas of statistics as individual values to be calculated on the way to determining a significance level is obsolete. At least some of us think so.

We hope the clears up any confusion. We're also glad that nobody has to calculate statistics by hand. Calculating statistics used to really be tough.

ANOVA provides an f-value, much like the t-test provides a t-value. We want to know if the f-value is significant at the .05 level. SPSS provides several ways of calculation the f-value, but we will use the One-Way ANOVA, which allows the comparison when there is only one dependent and one independent variable. The set of procedures for calculating the F statistics is called ANOVA.

When an F value is significant we conclude there is a difference between at least two of the means. While ANOVA will tell us if there is significant variation among the means in a total statement, it will NOT tell us about the comparison of individual means. If we are testing a set of three means, then there are three possible combinations of pairs, any of which or all of which might be different.

In addition to determining that differences exist among the means, we may want to know which means differ. There are two types of tests for comparing means: A priori contrasts and post hoc tests. Contrasts are tests set up before running the experiment, and post hoc tests are run after the experiment has been conducted. For this reason, we will be using the LSD test, one of many POST HOC (follow-up) comparisons, to tell us if there is significance. We can also test for trends across categories.

How do I run the test?

The One-Way ANOVA procedure in SPSS produces a one-way analysis of variance for a quantitative dependent variable by a single-factor independent variable. Remember the hypothesis saying the means will be equal is called the null hypothesis. If we reject the null, then we can conclude that at least two of the means are different — this is the research or alternative hypothesis. This technique is an extension of the Independent-Sample t-test.

Usually one-way is used to "snoop" the data and see if there is any evidence of relationships between selected variables. In other words, the researcher does not have good reasons for hypothesizing that there will be differences between the means of groups.

In other words, ANOVA tests two-tailed hypothesis concerning the differences between all possible pairs of means. Notice that as the number of pairs to be tested increases much faster than the number of means: for three means, three tests; four means, six tests; five means 10 tests; six means 15 tests, etc.

The more tests we make on a data set, the more likely we are to make Type I errors, that is infer that there are significant differences that do not really exist. The ANOVA procedure "protects" against this problem by not interpreting the post hoc tests unless the over-all F is statistically significant.

In other words, DO NOT INTERPRET THE POST HOC COMPARISONS TO MEAN THAT SIGNIFICANT DIFFERENCES EXIST UNLESS THE OVERALL F TEST IS SIGNIFICANT. If there were significance, we would scroll to the Post Hoc tests, which compare each group with the others.

How do I interpret the data?

As in the t-test, we must interpret our findings by stating our conclusions in terms of the differences in means, i.e. don't stop just because you've determined significance. You've got to know what the means are to understand what the significance indicates.

When the overall F is significant, you indicate the differences in means. When the overall F is not significant you should conclude that there are no differences between any pair of means even if the post hoc comparison tests indicate that there are. When the researcher has strong hypotheses concerning differences among means (one-tailed hypotheses), he or she should use t-tests to test them — even if that means running lots of different tests.

If you don't understand something in this Web note, please e-mail Dr. Sitton.

©M. Mark Miller & Ronald W. Sitton 2009
Revised 092811 — http://www.uamont.edu/FacultyWeb/sitton/crz/mrea/anova.html