Lab | ANOVA

Mass Communication Research
ANOVA Lab

Here's our hypothesis: Class rank plays a role in students rating of their overall experience at UT.

We want to compare freshmen to sophomores, freshmen to juniors, freshmen to seniors, sophomores to juniors, sophomores to seniors and juniors to seniors. We could use a t-test on each combination, but that increases the odds of making a Type I error, that is inferring a relationship when none exists. ANOVA (analysis of variance) solved this problem by calculating all of the mean comparisons in one procedure and providing significance tests for them.

We'll use the clean300.sav SPSS data set, and the variables of class rank (var. 1) and overall evaluation of experience at UT (var. 2). It's important to note that the overall evaluation variable is a 5-point scale, which meets the requirements of the interval level of measurement.

To do the ANOVA, go to Analyze, Compare Means then One-Way ANOVA and a window will open.

Move the overall evaluation of UT experience variable (q2) into the window labeled dependent list. Move class rank (q1) into the window labeled "factor" (what SPSS calls the independent variable in ANOVA).

After you have done this, click Post Hoc. Then check LSD. This is the follow-up test we want to run to determine where differences exist. In this test, there are six possible comparisons as to where a difference exists. There could be a difference in all of the comparisons or none of them.

After checking LSD, click continue. Then select descriptive statistics under options. It's important to have this information in order to interpret the results. Press continue and then OK. This will produce output.

The first thing you see is a box of the descriptive statistics for each group. The mean for Freshmen is 2.28 (remember to round) for Sophomores, 2.31, Juniors, 2.59, and Seniors, 2.54. There seems to be a pattern here with Freshmen and Sophomores having lower scores than Juniors and Seniors. But are there any significant differences?

Usually one-way is used to "snoop" the data and see if there is any evidence of relationships between selected variables. In other words, the researcher does not have good reasons for hypothesizing that there will be differences between the means of groups.

In other words, ANOVA tests two-tailed hypothesis concerning the differences between all possible pairs of means. Notice that the number of pairs to be tested increases much faster than the number of means: for three means, three tests; four means, six tests; five means 10 tests; six means 15 tests, etc.

The more tests we make on a data set, the more likely we are to make Type I errors, that is infer that there are significant differences that do not really exist. The ANOVA procedure "protects" against this problem by not interpreting the post hoc tests unless the over-all F is statitically significant. In other words, DO NOT INTERPRET THE POST HOC COMPARISONS TO MEAN THAT SIGNIFICANT DIFFERENCES EXIST UNLESS THE OVERALL F TEST IS SIGNIFICANT.

Looking at the next box which indicates the ANOVA for rating overall experience, we see that the F statistic is 4.682, which has a significance level of 0.003, so there is at least one significant difference between groups in the data set.

Once again, you should NOT interpret the Post Hoc Comparisons to mean that significant differences exist unless the overall F test is significant. If there were significance, we would scroll to the Post Hoc tests, which compare each group with the others.

Look at the asterisks (*) in the mean difference column. These asterisks indicate significant differences at the 0.05 level. The output indicates significant differences between freshmen and juniors, freshmen and seniors, sophomores and juniors, and sophomores and seniors. None of the other comparisons offer significant differences.

How do I interpret the data?

As in the t-test, we must interpret our findings by stating our conclusions in terms of the differences in means, i.e. don't stop just because you've determined significance. Significance only gives you PERMISSION to interpret. You've got to know what the means are to understand what the significance indicates.

When the overall F is not significant you should conclude that there are no differences between any pair of means even if the post hoc comparison tests indicate that there are. When the researcher has strong hypotheses concerning differences among means (one-tailed hypotheses), he or she should use t-tests to test them — even if that means running a lot of different tests.

When the overall F is significant, you indicate the differences in means. You find the means in the descriptives box at the top of the output. In this case we'd say, "There are significant differences in rating overall experience at UT between freshmen (2.28) and juniors (2.59), freshmen (2.28) and seniors (2.54), sophomores (2.31) and juniors (2.59), and between sophomores (2.31) and seniors (2.54). This supports our hypothesis that class rank plays a role in students rating of their overall experience at UT. The results indicate that students rate their overall experience at UT more favorably the longer they've remained in school.

If you don't understand something in this Web note, please e-mail Dr. Sitton.

©M. Mark Miller & Ronald W. Sitton 2009
Revised 092811 — http://www.uamont.edu/FacultyWeb/sitton/crz/mrea/anovalab.html