Chapter 23 – Multi-factorial ANOVA designs

Exercises

Exercise 23.1

Calculating two way unrelated ANOVA on a new data set

The data set used to calculate the example of a two-way unrelated ANOVA in this chapter is provided below and is named two way unrelated (book).sav. An Excel file with the same name is also provided.

The data set provided below (two-way unrelated ex) is one of fictitious data from a research project on leadership styles. Each participant has an LPC score, which stands for ‘least preferred co-worker’. People with high scores on this variable are able to get along with and accept relatively uncritically even those workers whom they least prefer to interact with. Such people make good leaders when situations at work are difficult (they are ‘people oriented’). By contrast low LPC people make good task leaders and are particularly effective when working conditions are good but tend to do poorly as leaders when conditions are a little difficult.

The variables in the file are sitfav with levels of highly favourable and moderately favourable (work conditions) and lpclead with levels of high and low being the categories of high and low LPC scorers. Hence in these results we would expect to find an interaction between situation favourability and LPC leadership category. High LPC people should do well in moderately favourable conditions whereas low LPC people should do well in highly favourable conditions. How well the leaders do is the dependent variable and is measured on a scale of 1-10 as assessed by a panel of independent raters. Let’s see what the spoof data says. Conduct a two-way unrelated ANOVA analysis, including relevant means and standard deviations, and checking for homogeneity of variance and for effect sizes and power for each test.

The answers I got are revealed when you select the button below.

Show answer

The main effect for LPC leadership is not significant (overall one type of leader did no better than the other), F(1,20) = 0.220, p = .644. The main effect for situation was also not significant (leadership performances overall were similar for highly and moderately favourable conditions), F(1,20) = 0.220, p = .644). However, there was a significant interaction between situation and leadership type. In highly favourable conditions, High LPC leaders (M = 5.33, SD = 1.03) scored lower than low LPC leaders (M = 6.5, SD = 1.64), whereas in moderately favourable conditions they scored higher (M = 7.0, SD = 1.41) than low LPC leaders (M = 5.33, SD = 1.03), F(1,20) = 7.049, p = .015. Levene’s test for homogeneity of variance was not significant so homogeneity was assumed. Partial eta-squared for the interaction was .261 with power estimated at .714.

Exercise 23.2

Interpreting an SPSS output for a two-way unrelated analysis.

Here is part of the SPSS output data for a quasi-experiment in which participants were grouped according to their attitude towards students. This is the ‘attitude group’ variable in the display below. Each group was exposed to just one of several sets of information about a fictitious person including their position on reintroducing government grants to students. Participants were later asked to rate the person on several characteristics including ‘liking’. It can be assumed for instance that participants who were pro students would show a higher liking for someone who wanted to introduce grants than someone who didn’t. Study the print out and try to answer the questions below.

Levene’s test of equality of error variancesa

Dependent Variable: liking

F	df1	df2	Sig.
2.757	5	41	.031

Tests of between-subjects effects

Dependent Variable: liking

Source	Type III sum of squares	df	Mean square	F	Sig.
Corrected Model	114.601a	5	22.920	7.947	.000
Intercept	1880.558	1	1880.558	652.033	.000
Information	3.670	2	1.835	.636	.534
Attitudegroup	15.953	1	15.953	5.531	.024
Information * attitudegroup	93.557	2	46.778	16.219	.000
Error	118.250	41	2.884
Total	2135.000	47
Corrected total	232.851	46

1. R Squared = .492 (Adjusted R Squared = .430)

---

Weblinks

Multi-factorial ANOVA designs

Laerd statistics pages for two-way unrelated ANOVA.

Two-way ANOVA Output and Interpretation in SPSS Statistics – Including Simple Main Effects | Laerd Statistics

The Hyperstat pages for between subjects factorial designs:

HyperStat Online: Factorial Between-Subjects ANOVA (davidmlane.com)

Two sites on the argument about interpreting main effects when there is a significant interaction effect:

How does the interpretation of main effects in a Two-Way ANOVA change depending on whether the interaction effect is significant? – Cross Validated (stackexchange.com)

Actually, you can interpret some main effects in the presence of an interaction – The Analysis Factor

---

Further Information

In Chapter 21 of the book we calculated a two-way ANOVA on the data that are provided here in Exercise 1. The book tells you that you can obtain effect size and power using SPSS or G*Power, but that the by-hand calculations are rather complex. For the sake of completeness though I will give the detail here.

Remember we are dealing with an experiment where participants consume either strong coffee, decaffeinated coffee or nothing. These conditions are referred to as caffeine, decaff and none. Two groups of participants are tested, those who have just had five hours’ sleep and those who have been awake for a full 24 hours. We therefore have a 2 (sleep) x 3 (caffeine) design. The results Table 21.2 from the book is:

Skill scores

* Overlong decimal figures are used here in order that our figures come close to those given by SPSS. With sensible rounding, our ANOVA results would be more different from the SPSS result than they are.
Table 21.2: Driving skill scores by caffeine and sleep conditions

Main effects

The general rules for calculating effect size and power for main effects in a two-way design are as follows, where we will refer to one factor with the general term A and the other with the term B.

For factor A, $${\mathrm{\Phi }}_A^{\prime }=\sqrt{\frac{\sum {\alpha }_i{}^2}{a{\sigma }_e{}^2}}$$ = where αi is the difference between the grand mean and each mean for factor A ignoring any difference across factor B. Ideally, these means would be the true population means if known but in calculating power after an experiment we use the sample means and assume these are good estimates of the population means. For our caffeine/sleep study, then, the grand mean is 4.48 and the α values would be the differences between 4.48 and each value, that is, 5.13 (caffeine), 4.25 (decaff) and 4.06 (none). Each of these differences is squared, the results added and this result for the top of the equation is divided by a, the number of levels of A (three in the caffeine case) multiplied by the mean square for error (MSE) from the ANOVA calculation.

Calculating for the caffeine effect we get:

$${\mathrm{\Phi }}_A^{\prime }=\sqrt{\frac{(4.48-5.125{)}^2+(4.48-4.25{)}^2+(4.48-4.0625{)}^2}{3\times 2.074}}$$

= 0.321

Our effect size, also referred to (by Cohen, 1988) as f, is 0.321. If we want to consult tables then we now need $${\mathrm{\Phi }}^{\prime }{}_{\mathrm{A}}\sqrt{n}$$ which = in general terms but with a factorial ANOVA we substitute n’ for n where , $$\frac{d{f}_{\text{error }}}{d{f}_{\text{effect }}+1}$$ . Our df are broken down like this:

Total df = N – 1 = 47
Main effect (caffeine) df = 2
Main effect (sleep) df = 1
Interaction df = 2 x 1 = 2
Error df = 47 – 2 – 1 – 2 = 42

Hence

$$n^{\prime }=\frac{42}{2+1}+1=15$$

and = 0.321√15 = 1.24

We go to appendix Table 13 with Φ = 1.24, df1 = 2 and dfe = 42 and α = 0.05. With a bit of extrapolation, we find that power is around 0.48 (don’t forget that power = 1 – β). This is close enough to SPSS and G*Power which both agree on 0.471.

As explained in the book SPSS provides effect size and power if you select this before your analysis and in G*Power select F tests, ANOVA: Fixed effects, special, main effects and interactions and Post hoc: Compute achieved power – given α, sample size and effect size. The values to enter are 0.321 for effect size, .05 for α err probability, 48 for total sample size, 2 for numerator df and 3 for number of groups.

Interaction effects

For the interaction things are a bit tricky. The formula is

$${\mathrm{\Phi }}_{\mathrm{AB}}^{\prime }=\sqrt{\frac{\sum \left({\alpha }_i{\beta }_j^2\right)}{ab{\sigma }_e^2}}$$

The calculation of is carried out for each individual cell of the data table. The calculation is:

$${\bar{x}}_{a1b1}-{\bar{x}}_{a1}-{\bar{x}}_{b1}+{\bar{x}}_g$$where$${\bar{x}}_{a1b1}$$is the cell mean (5 hours/caffeine if working from the top left in table 1 above),$${\bar{x}}_{a1}$$is the mean of the caffeine condition which that cell is in (caffeine and 5.125 in this case),$${\bar{x}}_{b1}$$is the mean for the sleep condition of that cell (5 hours and 4.7917) and finally$${\bar{x}}_g$$is the grand mean. Each of these six values (one for each of the cells in the table) is squared and the results are added together. This sum is divided by n’ times the MSE where n’ is calculated as above.

Put perhaps more simply for each cell you subtract from the cell mean the mean of the row it is in and the mean of the column it is in and then add the grand mean. Square the result and divide by n’ times the MSE. Let’s do this now. The calculation may look horrific but if you stick to the rule just stated you should be able to follow each step:

1. top line of the fraction inside the square root sign is:
(6.25 – 5.125 – 4.7917 + 4.48)2 + (4 – 5.125 – 4.1667 + 4.48)2 + (4.125 – 4.25 – 4.7917 + 4.48)2 +
(4.375 – 4.25 – 4.1667 + 4.48)2 + (4 – 4.0625 – 4.7917 + 4.48)2 + (4.125 – 4.0625 – 4.1667 + 4.48)2

Which comes to 1.983!

Dividing this by$$ab{\sigma }_e^2$$we get: 1.983/(2 x 3 x 2.074) = 0.159

The square root of 0.159 is 0.399. This is our value for Φ’ (or f)

We need to calculate n’ and we use

$$n^{\prime }=\frac{d{f}_{error}}{d{f}_{effect}+1}=1$$

as above but here is for the interaction and hence dfcaffeine x dfsleep which is 2 x 1 = 2. Hence n’ will again be 15.

Φ then is Φ’ x √n’ = 0.399 x √15 = 1.54 Going to appendix Table 13 with Φ = 1.54, df for the interaction being 2, dferror = 42 and α = 0.05 we need (again) to do some extrapolation between the table values but I get a table value of around 0.35 and, remembering that power = 1 – β this gives power at 0.65 which is pretty close to the SPSS and G*Power values of 0.662. To use G*Power proceed exactly as given above but for the number of groups enter 6.