This page introduces the typical application of One-Way and Factorial ANOVA and how to report the findings.
A brief introduction to the study:
A school-based intervention program was designed to prevent adolescent substance use in middle schools. Randomization was based on schools with 1,275 participants in 3 schools assigned into the treatment group and 1,088 participants in other 3 middles schools assigned into the control group. All 6 schools were located in the same school district with a majority of residents living in middle class families.
Substance use was a composite score on the frequencies of adolescent smoking, drinking and marijuana use measured on a scale from 1 to 5 indicating stages. At the end of the intervention program, students were asked how many times they used cigarettes, alcohol and marijuana in the last 30 days. The maximum possible score on Substance Use was 15.
One question in the evaluation of the effectiveness of the prevention program was who had benefited the most from the intervention program and who were resistant to the program. Race was one of the possible moderator variables examined. A Factorial ANOVA was performed to investigate whether the treatment effect was the same for Whites, Blacks and Hispanic ethnicity groups.
Results:
A 2 x 3 Factorial ANOVA was conducted to test the null hypothesis of no difference in the treatment effects among ethnicity groups including Whites (N = 1,570), Blacks (N = 592) and Hispanics (N = 203). The dependent variable was Substance Use, a composite score of the use of cigarettes, alcohol and marijuana ranging from 1 to 15. The two independent variables were Group Status (treatment and control) and Ethnicity with 3 levels. The major interest of the analysis was a significant interaction, a direct indication of different treatment effects for ethnicity groups. SPSS version 13.0 was used to perform the analyses and produce the figure.
Outliers were first examined prior to ANOVA analysis. The dependent variable was converted into standardized scores within each of the 6 groups. A univariate outlier was defined as a z-score greater than 3 or less than -3. After exclusion of the univariate outliers, the total sample size reduced from 2,365 to 2,297, with 66 outliers and 2 with missing values excluded from the analysis.
The Normality Assumption was examined using the skewness index, with the index greater an 1 or less than -1 indicating a sign of non-normality. Even after the exclusion of outlier, Substance Use was still positively skewed with the skewness index = 1.16, indicating violation of Normality assumption. A logarithmic transformation was then performed on the Substance Use, resulting in a near normal dependent variable with the skewness index = 0.250.
Levene's Test of Equality of Variance showed that the variances for groups were significantly different with F (5, 2291) = 49.30, p < 0.001, indicating that the homoscedastisity Assumption was violated. Several options are available when this assumption is violated. We used one of the adjusted test provided by SPSS, the Dunnett T3 Test for the violation of this assumption. Another reason to choose the adjusted test was that the sample sizes were very different across groups. Unequality in sample size and violation of homoscedastisity Assumption justified the use of an adjusted test.
Two main effects and one interaction effect were tested by the Factorial ANOVA. Table 1 presents the summary table for this 2 x 3 Factorial ANOVA. It can be seen from Table 1 that the main effect of Ethnicity was significant with F (2, 2291) = 192.29, p < 0.000, indicating that Substance Use was different in ethnicity groups. The main effect of Treatment was also significant with F (1, 2291) = 2358.24, showing that the intervention program had an overall effect on reducing substance use for all ethnicity groups. The interaction effect was the most relevant hypothesis of the study It was also significant with F (2, 2291) = 227.51, indicating that the treatment effect was different for ethnicity groups.
Table 1
Summary Table for Factorial ANOVA
| Source | Degrees of Freedom | Mean Square | F | p - value |
| Intercept | 1 | 546.01 | 4740.36 | 0.000 |
| Ethnicity | 2 | 22.15 | 192.29 | 0.000 |
| Treatment | 1 | 271.63 | 2358.24 | 0.000 |
| Ethnicity*Treatment | 2 | 26.21 | 227.51 | 0.000 |
| Error | 2291 | 0.15 |
Table 2 presents the observed means and standard deviations of log substance use for ethnicity groups separately by treatment groups. The Dunnett T3 Post Hoc Test based on unequal variance comparing ethnicity groups was also presented in Table 2. It can be seen from Table 2 that for each ethnicity group, the treatment group had significantly lower substance use score than the control groups, indicating the overall treatment effect. It can also be seen from Table 3 that the difference in substance use between control and treatment groups was the widest for Whites, indicating that Whites benefited the most from the treatment program relative to other ethnicity groups.
Table 2
Observed Means and Standard Deviations of Log Substance Use for Ethnicity Groups Separated by Treatment Conditions
| Ethnicity | Control | Treatment |
| Black(a) | 0.81 (0.25) (N = 211) | 0.22 (0.44)** (N = 368) |
| Hispanic(b) | 1.18 (0.19) (N = 85) | 0.20 (0.43)** (N = 113) |
| White(c) | 1.50 (0.23) (N = 761) | 0.19 (0.39)** (N = 759) |
Figure 1 shows the interaction plot for better understanding a significant interaction effect. Adjusted means were presented because of unequal sample sizes for groups. It is clear from Figure 1 that Whites in the control group had highest substance use level, but had lowest substance level in the treatment group, indicating that Whites benefited the most from the intervention program. Further analysis comparing only Hispanics and Blacks was conducted. Results showed that the interaction term was also significant with F (1, 772) = 38.99, p < 0.001, showing that the Hispanics benefited more from the intervention program than Blacks.
Figure 1
Estimated Means of Log Substance Use for Ethnicity Groups Separated by Treatment and Control Conditions
|
|
To download the source data, please click here.