Chapter 9

Multivariate analysis

Exploring differences among three or more variables

In most studies in the social sciences we collect information on more than just two variables. Although it would be possible and more simple to examine the relationships between these variables just two at a time, there are serious disadvantages to restricting oneself to this approach, as we shall see. It is preferable initially to explore these data with multivariate rather than bivariate tests. The reasons for looking at three or more variables vary according to the aims and design of a study. Consequently, we shall begin by outlining four design features which only involve three variables at a time. Obviously these features may include more than three variables and the features themselves can be combined to form more complicated designs, but we shall discuss them largely as if they were separate designs. However, as has been done before, we shall use one set of data to illustrate their analysis, all of which can be carried out with a general statistical model called multivariate analysis of variance and covariance (MANOVA and MANCOVA). This model requires the Advanced Statistics option. Although the details of the model are difficult to understand and to convey simply (and so will not be attempted here), its basic principles are similar to those of other parametric tests we have previously discussed such as the t test, one-way analysis of variance and simple regression.

MULTIVARIATE DESIGNS

Factorial design

We are often interested in the effect of two variables on a third, particularly if we believe that the two variables may influence one another. To take a purely hypothetical case, we may expect the gender of the patient to interact with the kind of treatment they are given for feeling depressed. Women may respond more positively to psychotherapy where they have an opportunity to talk about their feelings while men may react more favourably to being treated with an antidepressant drug. In this case, we are anticipating that the kind of treatment will interact with gender in alleviating depression. An interaction is when the effect of one variable Figure 9.1 An example of an interaction between two variables

is not the same under all the conditions of the other variable. It is often more readily understood when it is depicted in the form of a graph as in Figure 9.1. However, whether these effects are statistically significant can only be determined by testing them and not just by visually inspecting them. The vertical axis shows the amount of improvement in depression that has taken place after treatment, while the horizontal one can represent either of the other two variables. In this case it reflects the kind of treatment received. The effect of the third variable, gender, is depicted by two different kinds of points and lines in the graph itself. Men are indicated by a cross and a continuous line while women are signified by a small circle and a broken line. An interaction is indicated when the two lines representing the third variable are not parallel. Consequently, a variety of interaction effects can exist, three of which are shown in Figure 9.2 as hypothetical possibilities. In Figure 9.2a, men show less improvement with psychotherapy than with drugs while women derive greater benefit from psychotherapy than from the drug treatment. In Figure 9.2b, men improve little with either treatment, while women, once again, benefit considerably more from psychotherapy than from drugs. Finally, in Figure 9.2c, both men and women improve more with psychotherapy than with drugs, but the improvement is much greater for women than it is for men. The absence of an interaction can be seen by the lines representing the third

Figure 9.2 Examples of other interactions

Figure 9.3 Examples of no interactions

variable remaining more or less parallel to one another, as is the case in the three examples in Figure 9.3. In Figure 9.3a, both men and women show a similar degree of improvement with both treatments. In Figure 9.3b, women improve more than men under both conditions while both treatments are equally effective. In Figure 9.3c, women show greater benefit than men with both treatments, and psychotherapy is better than drugs.

The results of treatment and gender on their own are known as main effects. In these situations, the influence of the other variable is disregarded. If, for example, we want to examine the effect of gender, we would only look at improvement for men and women, ignoring treatment. If we are interested in the effect of the kind of treatment, we would simply compare the outcome of patients receiving psychotherapy with those being given drugs, paying no heed to gender.

The variables which are used to form the comparison groups are termed factors. The number of groups which constitute a factor are referred to as the levels of that factor. Since gender consists of two groups, it is called a two-level factor. The two kinds of treatment also create a two-level factor. If a third treatment had been included such as a control group of patients receiving neither drugs nor psychotherapy, we would have a three-level factor. Studies which investigate the effects of two or more factors are known as factorial designs. A study comparing two levels of gender and two levels of treatment is described as a 2 × 2 factorial design. If three rather than two levels of treatment had been compared, it would be a 2 × 3 factorial design. A study which only looks at one factor is called a one-way or single factor design.

The factors in these designs may be ones that are manipulated, such as differing dosages of drugs, different teaching methods, or varying levels of induced anxiety. Where they have been manipulated and where participants have been randomly assigned to different levels, the factors may also be referred to as independent variables since they are more likely to be unrelated to, or independent of, other features of the experimental situation such as the personality of the participants. Variables which are used to assess the effect of these independent variables are known as dependent variables since the effect on them is thought to depend on the level of the variable which has been manipulated. Thus, for example, the improve- ment in the depression experienced by patients (that is, the dependent variable) is believed to be partly the result of the treatment they have received (that is, the independent variable). Factors can also be variables which have not been manipulated, such as gender, age, ethnic origin, and social class. Because they cannot be separated from the individual who has them, they are sometimes referred to as subject variables. A study which investigated the effect of such subject variables would also be called a factorial design.

One of the main advantages of factorial designs, other than the study of interaction effects, is that they generally provide a more sensitive or powerful statistical test of the effect of the factors than designs which investigate just one factor at a time. To understand why this is the case, it is necessary to describe how a one-way and a two-way (that is, a factorial) analysis of variance differ. In one-way analysis of variance, the variance in the means of the groups (or levels) is compared with the variance within them combined for all the groups:

The between-groups variance is calculated by comparing the group mean with the overall or grand mean, while the within-groups variance is worked out by comparing the individual scores in the group with its mean. If the group means differ, then their variance should be greater than the average of those within them. This situation is illustrated in Figure 9.4 where the means of the three groups (M 1, M2 and M3) are quite widely separated, causing a greater spread of between-groups variance (VB) while the variance within the groups (V1, V2 and V3) is considerably less when combined (VW).

Figure 9.4 Schematic representation of a significant one-way effect

Now the variance within the groups is normally thought of as error since this is the only way in which we can estimate it, while the between-groups variance is considered to consist of this error plus the effect of the factor which is being investigated. While some of the within-groups variance may be due to error, such as that of measurement and of procedure, the rest of it will be due to factors which we have not controlled, such as gender, age and motivation. In other words, the within-groups variance will contain error as well as variance due to other factors, and so will be larger than if it just contained error variance. Consequently, it will provide an overestimate of error. In a two-factor design, on the other hand, the variance due to the other factor can be removed from this overestimate of the error variance, thereby giving a more accurate calculation of it. If, for example, we had just compared the effectiveness of the drug treatment with psychotherapy in reducing depression, then some of the within-groups variance would have been due to gender but treated as error, and may have obscured any differential effect due to treatment.

Covariate design

Another way of reducing error variance is by removing the influence of a non-categorical variable (that is, one which is not nominal) which we believe to be biasing the results. This is particularly useful in designs where participants are not randomly assigned to factors, such as in the Job Survey study, or where random assignment did not result in the groups being equal in terms of some other important variable, such as how depressed patients were before being treated. A covariate is a variable which is linearly related to the one we are most directly interested in, usually called the dependent or criterion variable.

We shall give two examples of the way in which the effect of covariates may be controlled. Suppose, for instance, we wanted to find out the relationship between job satisfaction and the two factors of gender and ethnic group in the Job Survey data and we knew that job satisfaction was positively correlated with income, so that people who were earning more were also more satisfied with their jobs. It is possible that both gender and ethnic group will also be related to income. Women may earn less than men and non-white workers may earn less than their white counterparts. If so, then the relationship of these two factors to job satisfaction is likely to be biased by their association with income. To control for this, we shall remove the influence of income by covarying it out. In this case, income is the covariate. If income were not correlated with job satisfaction, then there would be no need to do this. Consequently, it is only advisable to control a covariate when it has been found to be related to the dependent variable.

In true experimental designs, we try to control the effect of variables other than the independent ones by randomly assigning participants to different treatments or conditions. However, when the number of participants allocated to treatments is small (say, about ten or less), there is a stronger possibility that there will be chance differences between them. If, for example, we are interested in comparing the effects of drugs with psychotherapy in treating depression, it is important that the patients in the two conditions should be similar in terms of how depressed they are before treatment begins (that is, at pre-test). If the patients receiving the drug treatment were found at pre-test to be more depressed than those having psychotherapy despite random assignment, then it is possible that because they are more depressed to begin with, they will show less improvement than the psychotherapy patients. If pre-test depression is positively correlated with depression at the end of treatment (that is, at post-test), then the effect of these initial differences can be removed statistically by covarying them out. The covariate in this example would be the pre-test depression scores.

Three points need to be made about the selection of covariates. First, as mentioned before, they should only be variables which are related to the dependent variable. Variables which are unrelated to it do not require to be covaried out. Second, if two covariates are strongly correlated with one another (say 0.8 or above), it is only necessary to remove one of them since the other one seems to be measuring the same variable(s). And third, with small numbers of participants only a few covariates at most should be used, since the more covariates there are in such situations, the less powerful the statistical test becomes.

Multiple measures design

In many designs we may be interested in examining differences in more than one dependent or criterion measure. For example, in the Job Survey study, we may want to know how differences in gender and ethnic group are related to job autonomy and routine as well as satisfaction. In the depression study, we may wish to assess the effect of treatment in more than one way. How depressed the patients themselves feel may be one measure. Another may be how depressed they appear to be to someone who knows them well, such as a close friend or informant. One of the advantages of using multiple measures is to find out how restricted or widespread a particular effect may be. In studying the effectiveness of treatments for depression, for instance, we would have more confidence in the results if the effects were picked up by a number of similar measures rather than just one. Another advantage is that although groups may not differ on individual measures, they may do so when a number of related individual measures are examined jointly. Thus, for example, psychotherapy may not be significantly more effective than the drug treatment when outcome is assessed by either the patients themselves or by their close friends, but it may be significantly better when these two measures are analysed together.

Mixed between–within design

The multiple-measures design needs to be distinguished from the repeated-measures design which we encountered at the end of Chapter 7. A multiple-measures design has two or more dependent or criterion variables such as two separate measures of depression. A repeated-measures design, on the other hand, consists of one or more factors being investigated on the same group of participants. Measuring job satisfaction or depression at two or more points in time would be an example of such a factor. Another would be evaluating the effectiveness of drugs and psychotherapy on the same patients by giving them both treatments. If we were to do this, we would have to make sure that half the patients were randomly assigned to receiving psychotherapy first and the drug treatment second, while the other patients would be given the two treatments in the reverse order. It is necessary to counterbalance the sequence of the two conditions to control for order effects. It would also be advisable to check that the sequence in which the treatments were administered did not affect the results. The order effect would constitute a between-subjects factor since any one participant would only receive one of the two orders. In other words, this design would become a mixed one which included both a between-subjects factor (order) and a within-subjects one (treatment). One of the advantages of this design is that it restricts the amount of variance due to individuals, since the same treatments are compared on the same participants.

Another example of a mixed between–within design is where we assess the dependent variable before as well as after the treatment, as in the study on depression comparing the effectiveness of psychotherapy with drugs. This design has two advantages. The first is that the pre-test enables us to determine whether the groups were similar in terms of the dependent variable before the treatment began. The second is that it allows us to determine if there has been any change in the dependent variable before and after the treatment has been given. In other words, this design enables us to discern whether any improvement has taken place as a result of the treatment and whether this improvement is greater for one group than the other.

Combined design

As was mentioned earlier, the four design features can be combined in various ways. Thus, for instance, we can have two independent factors (gender and treatment for depression), one covariate (age), two dependent measures (assessment of depression by patient and informant), and one repeated measure (pre- and post-test). These components will form the basis of the following illustration, which will be referred to as the Depression Project. The data for it are shown in Table 9.1.

There are three treatments: a no treatment control condition (coded 1 and with eight participants); a psychotherapy treatment (coded 2 and with ten participants); and a drug treatment (coded 3 and with twelve participants). Females are coded as 1 and males as 2. A high score on depression indicates a greater degree of it. The patient’s assessment of their depression before and after treatment is referred to as patpre and patpost respectively, while the assessment provided by an informant before and after treatment is known as infpre and infpost. We shall now turn to methods of analysing the results of this kind of study using MANOVA or MANCOVA.

Table 9.1 The Depression Project data

Id

Treat

Gender

Age

Patpre

Infpre

Patpost

Infpost

01

1

1

27

25

27

20

19

02

1

2

30

29

26

25

27

03

1

1

33

26

25

23

26

04

1

2

36

31

33

24

26

05

1

1

41

33

30

29

28

06

1

2

44

28

30

23

26

07

1

1

47

34

30

30

31

08

1

2

51

35

37

29

28

09

2

1

25

21

24

9

15

10

2

2

27

20

21

9

12

11

2

1

30

23

20

10

8

12

2

2

31

22

28

14

18

13

2

1

33

25

22

15

17

14

2

2

34

26

23

17

16

15

2

1

35

24

26

9

13

16

2

2

37

27

25

18

20

17

2

1

38

25

21

11

8

18

2

2

42

29

30

19

21

19

3

1

30

34

37

23

25

20

3

2

33

31

27

15

13

21

3

1

36

32

35

20

21

22

3

2

37

33

35

20

18

23

3

1

39

40

38

33

35

24

3

2

41

34

31

18

19

25

3

1

42

34

36

23

27

26

3

2

44

37

31

14

11

27

3

1

45

36

38

24

25

28

3

2

47

38

35

25

27

29

3

1

48

37

39

29

28

30

3

2

50

39

37

23

24

MULTIVARIATE ANALYSIS

The example we have given is the more common one in which there are unequal numbers of cases on one or more of the factors. Although it is possible to equalise them by randomly omitting two participants from the psychotherapy treatment and four from the drug one, this would be a waste of valuable data and so is not recommended.

There are four main ways of analysing the results of factorial designs with SPSS. The first method, referred to as Type I in SPSS and previously known as the hierarchical or sequential approach, allows the investigator to determine the order of the effects. If one factor is thought to precede another, then it can be placed first. This approach should be used in non-experimental designs where the factors can be ordered in some sequential manner. If, for example, we are interested in the effect of ethnic group and income on job satisfaction, then ethnic

group would be entered first since income cannot determine the ethnic group to which we belong.

Type II, previously known as classic experimental or least squares approach, can be used with balanced or unbalanced designs where there are some cases in all cells. Tabachnick and Fidell (1996) recommend that this method should be used for non-experimental designs where there are unequal numbers of cases in cells and where cells having larger numbers of cases are thought to be more important.

Type III, previously known as the regression, unweighted means or unique approach, is used where there are an equal number of cases in each cell or where the design is balanced in that the cell frequencies are proportional in terms of their marginal distributions. Tabachnick and Fidell (1996) recommend that this method should also be used for true experimental designs where there are unequal numbers of cases in cells due to random drop-out and where all cells are considered to be equally important.

Type IV is applied with balanced or unbalanced designs where there are no cases in some cells.

All four methods produce the same result for analysis of variance designs when there are equal numbers of participants in cells. For an unbalanced two-way analysis of variance design they only differ in the way they handle main effects so the results for the interaction effects and the error are the same. Where the main effects are not independent, the component sums of squares do not add up to the total sum of squares.

In this chapter, we have followed the recommendation of Tabachnick and Fidell (1996) in using the Type II method for true experimental designs.

Factorial design

To determine the effect of treatment, gender and their interaction on post-test depression as seen by the patient, the following procedure is needed:

➔Analyze ➔General Linear Model ➔Univariate… [opens Univariate dialog box shown in Box 9.1]

➔patpost ➔► button beside Dependent Variable: [puts patpost in this box]

➔gender ➔► button beside Fixed Factor[s]: [puts gender in this box] ➔treat

➔► button beside Fixed Factor[s]: [puts treat in this box]

➔Options... [opens Univariate: Options subdialog box shown in Box 9.2]

➔Descriptive statistics ➔Homogeneity tests ➔Continue [closes Univariate: Options subdialog box]

➔Plots… [opens Univariate: Profile Plots subdialog box shown in Box 9.3]

➔gender ➔► button beside Horizontal Axis: [puts gender in this box] ➔treat

➔► button beside Separate Lines: [puts treat in this box] ➔Add [puts gender*treat in box under Plots:] ➔Continue [closes Univariate: Profile Plots subdialog box]

➔OK

Page 206

Box 9.1 Univariate dialog box

Box 9.2 Univariate: Options subdialog box

Box 9.3 Univariate: Profile Plots subdialog box

The means for the three treatments for women and men are shown in Table 9.2 and Figure 9.5.

Levene’s test for homogeneity of variance is displayed in Table 9.3. It is not significant, which means there are no significant differences between the variances of the groups, an assumption on which this test is based. If the variances had been grossly unequal, then it may have been possible to reduce this through transforming the data by taking the log or square root of the dependent variable. This can easily be done with the Compute procedure on the Transform menu which displays the Compute Variable dialog box shown in Box 3.8. For example, the natural or Naperian log (base e) of patpost is produced by selecting the LN (numexpr) Function and inserting patpost as the numerical expression. The square root of patpost is created by selecting the SQRT(numexpr) Function. It is necessary to check that the transformations have produced the desired effect.

The tests of significance for determining the Type III sum of squares for each effect are shown in Table 9.4. These indicate that there is a significant effect for the treatment factor ( p < 0.0005) and a significant interaction effect for treatment and gender (p = 0.016). If we plot this interaction, we can see that depression after the drug treatment is higher for women than men, while after psychotherapy it is higher for men than women. The mean square of an effect is its sum of squares divided by its degrees of freedom. Thus, for example, the mean square of the treatment effect is 767.942 divided by 2 which is 383.971. The F value for an effect is its mean square divided by that of the within-cells and residual term. So for the treatment effect this would be 383.971 divided by 16.163 which is 23.756. Multivariate analysis: exploring differences 207

Table 9.2 Means of post-test depression ( patpost) in the three treatments for men and women (Depression Project)

Descriptive Statistics

Dependent Variable: PATPOST

GENDER

TREAT

Mean

Std. Deviation

N

1

1

25.50

4.80

4

2

10.80

2.49

5

3

25.33

4.76

6

Total

20.53

8.10

15

2

1

25.25

2.63

4

2

15.40

4.04

5

3

19.17

4.36

6

Total

19.53

5.33

15

Total

1

25.38

3.58

8

2

13.10

3.98

10

3

22.25

5.41

12

Total

20.03

6.75

30

Figure 9.5 Post-test patient depression of men and women

Table 9.3 Homogeneity tests output (Depression Project)

Levene’s Test of Equality of Error Varianceas

Dependent Variable: PATPOST

F

df1

df2

Sig.

1.213

5

24

.333

Tests the null hypothesis that the error variance of the dependent variable is equal across groups.

a. Design: Intercept+GENDER+TREAT+GENDER *TREAT

Table 9.4 Tests of significance for main and interaction effects in an unrelated factorial design (Depression Project)

Tests of Between-Subjects Effects

Dependent Variable: PATPOST

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected Model

935.050a

5

187.010

11.570

.000

Intercept

11959.543

1

11959.543

739.924

.000

GENDER

2.676

1

2.676

.166

.688

TREAT

767.942

2

383.971

23.756

.000

GENDER * TREAT

159.608

2

79.804

4.937

.016

Error

387.917

24

16.163

Total

13363.000

30

Corrected Total

1322.967

29

a. R Squared = .707 (Adjusted R Squared = .646)

Having found that there is an overall significant difference in depression for the three treatments, we need to determine where this difference lies. One way of doing this is to test for differences between two treatments at a time. If we had not anticipated certain differences between treatments, we would apply post hoc tests such as Scheffé to determine their statistical significance, whereas if we had predicted them we would use unrelated t tests (see Chapter 7).

Covariate design

If the patients’ pre-test depression scores differ for gender, treatment or their interaction and if the pre-test scores are related to the post-test ones, then the results of the previous test will be biased by this. To determine if there are such differences, we need to run a factorial analysis on the patients’ pre-test depression scores. If we do this, we find that there is a significant effect for treatments (see the output in Table 9.5), which means that the pre-test depression scores differ between treatments.

Covariate analysis is based on the same assumptions as the previous factorial analysis plus three additional ones. First, there must be a linear relationship between the dependent variable and the covariate. If there is no such relationship, then there is no need to conduct a covariate analysis. This assumption can be tested by plotting a scatter diagram (see Chapter 8) to see if the relationship appears non-linear. If the correlation is statistically significant, then it is appropriate to carry out a covariate analysis. The statistical procedure Univariate… also provides information on this (see pp. 212–13). If the relationship is non-linear, it may be possible to transform it so that it becomes linear using a logarithmic transformation of one variable. The procedure for effecting such a transformation with SPSS has been described on page 207.

The second assumption is that the slope of the regression lines is the same in each group or cell. If they are the same, this implies that there is no interaction between the independent variable and the covariate and that the average within-cell regression can be used to adjust the scores of the dependent variable. This information is also provided by Univariate…. If this condition is not met, then the Johnson–Neyman technique should be considered. This method is not available

Table 9.5 Tests of significance for effects on pre-test depression (Depression Project)

Tests of Between-Subjects Effects

Dependent Variable: PATPRE

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected Model

693.283a

5

138.387

13.657

.000

Intercept

26119.676

1

26119.676

2521.779

.000

GENDER

4.227

1

4.227

.408

.529

TREAT

686.475

2

343.237

33.139

.000

GENDER * TREAT

3.475

2

1.737

.168

.847

Error

248.583

24

10.358

Total

28424.000

30

Corrected Total

941.867

29

a. R Squared = .736 (Adjusted R Squared = .681)

on SPSS but a description of it may be found elsewhere (Huitema, 1980).

The third assumption is that the covariate should be measured without error. For some variables such as gender and age, this assumption can usually be justified. For others, however, such as measures of depression, this needs to be checked. This can be done by computing the alpha reliability coefficient for multi-item variables (such as job satisfaction) or test–retest correlations where this information is available. A coefficient of 0.8 or above is usually taken as indicating a reliable measure. This assumption is more important in non- than in true-experimental designs, where its violation may lead to either Type I or II errors. In true-experimental designs, the violation of this assumption only leads to loss of power. As there are no agreed or simple procedures for adjusting covariates for unreliability, these will not be discussed.

The following procedure is necessary to test whether the regression lines are the same in each of the cells for the analysis of covariance in which the effect of treatment on the patients’ post-test depression scores, controlling for their pre-test ones, is examined:

➔Analyze ➔General Linear Model ➔Univariate… [opens Univariate dialog box shown in Box 9.1]

➔patpost ➔► button beside Dependent Variable: [puts patpost in this box]

➔treat ➔► button beside Fixed Factor[s]: [puts treat in this box] ➔patpre

➔button beside Covariate[s]: [puts patpre in this box]

➔Model… [opens Univariate: Model subdialog box shown in Box 9.4]

Box 9.4 Univariate: Model subdialog box

Table 9.6 Analysis of covariance table showing test of homogeneity of slope of regression line within cells (Depression Project)

Tests of Between-Subjects Effects

Dependent Variable: PATPOST

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected Model

1071.498a

5

214.300

20.453

.000

Intercept

43.642

1

43.642

4.165

.052

TREAT

16.905

2

8.453

.807

.458

PATPRE

299.436

1

299.436

28.578

.000

TREAT * PATPRE

4.907

2

2.454

.234

.793

Error

251.468

24

10.478

Total

13363.000

30

Corrected Total

1322.967

29

a. R Squared = .810 (Adjusted R Squared = .770)

➔Custom ➔treat ➔► button [puts treat under Model:] ➔patpre ➔► button [puts patpre under Model:] ➔Interaction ➔treat ➔patpre ➔► button [puts treat*patpre under Model:] ➔Continue [closes Univariate: Model subdialog box]

➔OK

The output for this procedure is presented in Table 9.6. The interaction between the independent variable of treatment and the covariate of patpre is not significant since p is 0.793. This means that the slope of the regression line in each of the cells is similar and therefore the second assumption is met.

Consequently we can proceed with the main analysis of covariance using the following procedure:

➔Analyze ➔General Linear Model ➔Univariate… [opens Univariate dialog box shown in Box 9.1]

➔patpost ➔► button beside Dependent Variable: [puts patpost in this box]

➔treat ➔► button beside Fixed Factor[s]: [puts treat in this box] ➔patpre ➔► button beside Covariate[s]: [puts patpre in this box]

➔Model… [opens Univariate: Model subdialog box shown in Box 9.4]

➔Full factorial ➔Continue [closes Univariate: Model subdialog box]

➔Options... [opens Univariate: Options subdialog box shown in Box 9.2]

➔treat ➔► button [puts treat under Display Means for:] ➔Continue [closes Univariate: Options subdialog box]

➔OK

The analysis of covariance table in Table 9.7 shows that the relationship

Table 9.7 Analysis of covariance table (Depression Project)

Tests of Between-Subjects Effects

Dependent Variable: PATPOST

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected Model

1066.591a

3

355.530

36.056

.000

Intercept

41.426

1

41.426

4.201

.051

PATPRE

298.650

1

298.650

30.287

.000

TREAT

339.161

2

169.580

17.198

.000

Error

256.375

26

9.861

Total

13363.000

30

Corrected Total

1322.967

29

a. R Squared = .806 (Adjusted R Squared = .784)

between the covariate (patpre) and the dependent variable (patpost) is significant. Consequently, it is appropriate to proceed with the interpretation of the covariate analysis. This table also shows there is a significant treatment effect when pre-treatment depression is covaried out.

An inspection of the adjusted means for the three treatments presented in Table 9.8 with the observed means in Table 9.2 shows that controlling for pre-treatment depression has little effect on the mean for the control group, which remains at about 25. However, it makes a considerable difference to the means of the two treatment conditions, reversing their order so that patients who received psychotherapy report themselves as being more depressed than those given the drug treatment. The means have been adjusted using the weighted rather than the unweighted covariate grand mean (Cramer, 1998). The Bryant–Paulson post hoc test for determining whether this difference is significant is described in Stevens (1996).

Table 9.8 Adjusted means of post-test depression in the three treatments (Depression Project)

TREAT

Dependent Variable: PATPOST

TREAT

Mean

Std. Error

95% Confidence Interval

Lower Bound

Upper Bound

1

25.528a

1.111

23.245

27.811

2

19.660a

1.551

16.471

22.849

3

16.681a

1.359

13.888

19.474

a. Evaluated at covariates appeared in the model: PATPRE = 30.27

Multiple measures design

So far, we have analysed only one of the two dependent measures, the patient’s self-report of depression. Analysing the two dependent measures together has certain advantages. First, it reduces the probability of making Type I errors (deciding there is a difference when there is none) when making a number of comparisons. The probability of making this error is usually set at 0.05 when comparing two groups on one dependent variable. If we made two such independent comparisons, then the p level would increase to about 0.10. Since the comparisons are not independent, this probability is higher. Second, analysing the two dependent measures together provides us with a more sensitive measure of the effects of the independent variables.

The following procedure is necessary to test the effect of treatment on both the patient’s and the informant’s post-test assessment of the patient’s depression:

➔Analyze ➔General Linear Model ➔ Multivariate… [opens Multivariate dialog box shown in Box 9.5]

➔patpost ➔► button beside Dependent Variables: [puts patpost in this box]

➔infpost ➔► button beside Dependent Variables: [puts infpost in this box]

Box 9.5 Multivariate dialog box

Box 9.6 Multivariate: Options subdialog box

➔treat ➔► button beside Fixed Factor[s]: [puts treat in this box]

➔Options… [opens Multivariate: Options subdialog box shown in Box 9.6]

➔Descriptive statistics ➔Residual SSCP matrix [gives Bartlett’s test of sphericity] ➔Homogeneity tests [gives Box’s M and Levene’s test]

➔Continue [closes Multivariate: Options subdialog box]

➔OK

The means and standard deviations for patpost and infpost for the three treatment conditions are shown in Table 9.9. The results for Box’s M and Levene’s test are presented in Tables 9.10 and 9.11 respectively. Box’s M test determines whether the covariance matrices of the two dependent variables are similar while Levene’s test assesses whether their variances are similar. For this example, both tests are not significant, which means that the covariance matrices and the variances do not differ significantly across the three conditions.

Table 9.12 shows the output for Bartlett’s test of sphericity which assesses whether the dependent measures are correlated. If the test is significant, as it is here, it means the two dependent measures are related. In this situation, it is more

Table 9.9 Means and standard deviations of post-test depression scores for patients ( patpost) and informants (infpost) for the three treatments (Depression Project)

Descriptive Statistics

TREAT

Mean

Std. Deviation

N

PATPOST

1

25.38

3.58

8

2

13.10

3.98

10

3

22.25

5.41

12

Total

20.03

6.75

30

INFPOST

1

26.38

3.42

8

2

14.80

4.54

10

3

22.75

6.73

12

Total

21.07

6.99

30

Table 9.10 Box’s M test (Depression Project)

Box’s Test of Equality of Covariance Matriceas

Box’s M

7.985

F

1.182

df1

6

df2

9324.416

Sig.

.313

Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.

a. Design: Intercept+TREAT

Table 9.11 Levene’s test (Depression Project)

Levene’s Test of Equality of Error Varianceas

F

df1

df2

Sig.

PATPOST

.456

2

27

.638

INFPOST

2.540

2

27

.098

Tests the null hypothesis that the error variance of the dependent variable is equal across groups.

a. Design: Intercept+TREAT

Table 9.12 Bartlett’s test of sphericity (Depression Project)

Bartlett’s Test of Sphericity a

Likelihood Ratio

.000

Approx. Chi-Square

44.656

df

2

Sig.

.000

Tests the null hypothesis that the residual covariance matrix is proportional to an identity matrix.

a. Design: Intercept+TREAT

Table 9.13 Multivariate tests of significance for the treatment effect (Depression Project)

Multivariate Testsc

Effect

Value

F

Hypothesis df

Error df

Sig.

Intercept

Pillai’s Trace

.956

280.195a

2.000

26.000

.000

Wilks’ Lambda

.044

280.195a

2.000

26.000

.000

Hotelling’s Trace

21.553

280.195a

2.000

26.000

.000

Roy’s Largest Root

21.553

280.195a

2.000

26.000

.000

TREAT

Pillai’s Trace

.618

6.032

4.000

54.000

.000

Wilks’ Lambda

.393

7.724a

4.000

52.000

.000

Hotelling’s Trace

1.513

9.456

4.000

50.000

.000

Roy’s Largest Root

1.494

20.169b

2.000

27.000

.000

a. Exact statistic

b. The statistic is an upper bound on F that yields a lower bound on the significance level.

c. Design: Intercept+TREAT

appropriate to use the multivariate test of significance to determine whether there are significant differences between the treatments. The result of this test is presented in Table 9.13 and shows the treatment effect to be significant when the two measures are taken together.

The univariate F tests for the treatment effect, which are presented in Table 9.14, show that the treatments differ on both the dependent measures when they are analysed separately. To determine which treatments differ significantly from one another, it would be necessary to carry out a series of unrelated t tests or post hoc tests as discussed previously.

Table 9.14 Univariate tests of significance for the two dependent measures (Depression Project)

Tests of Between-Subjects Effects

Source

Dependent Variable

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected Model

PATPOST

767.942a

2

383.971

18.679

.000

INFPOST

652.142b

2

326.071

11.497

.000

Intercept

PATPOST

11959.543

1

11959.543

581.789

.000

INFPOST

13253.207

1

13253.207

467.317

.000

TREAT

PATPOST

767.942

2

383.971

18.679

.000

INFPOST

652.142

2

326.071

11.497

.000

Error

PATPOST

555.025

27

20.556

INFPOST

765.725

27

28.360

Total

PATPOST

13363.000

30

INFPOST

14732.000

30

Corrected Total

PATPOST

1322.967

29

INFPOST

1417.867

29

a. R Squared = .580 (Adjusted R Squared = .549)

b. R Squared = .460 (Adjusted R Squared = .420)

Mixed between–within design

The procedure for determining if there is a significant difference between the three conditions in improvement in depression as assessed by the patient before (patpre) and after (patpost) treatment is:

➔Analyze ➔General Linear Model ➔Repeated Measures… [opens Repeated Measures Define Factor[s] dialog box shown in Box 9.7]

➔highlight factor1 and type time in Within-Subject Factor Name: box

➔Number of Levels: box and type 2 ➔Add ➔Define [opens Repeated Measures subdialog box shown in Box 9.8]

➔patpre ➔► button beside Within-Subjects Variables [time]: [puts patpre in this box] ➔patpost ➔► button beside Within-Subjects Variables [time]: [puts patpost in this box] ➔treat ➔► button beside Between-Subjects Factor[s]: [puts treat in this box]

➔Options... [opens Repeated Measures: Options subdialog box shown in Box 9.9]

➔Descriptive statistics ➔Continue [closes Repeated Measures: Options subdialog box]

➔Plots… [opens Repeated Measures: Profile Plots subdialog box shown in Box 9.10]

➔treat ➔► button beside Horizontal Axis: [puts treat in this box] ➔time ➔►

Box 9.7 Repeated Measures Define Factor[s] dialog box

Box 9.8 Repeated Measures subdialog box

button beside Separate Lines: [puts time in this box] ➔Add [puts time*treat in box under Plots] ➔Continue [closes Repeated Measures: Profile Plots subdialog box]

➔OK

Box 9.9 Repeated Measures: Options subdialog box

Box 9.10 Repeated Measures: Profile Plots subdialog box

Table 9.15 Test of significance for interaction between time and treatment (Depression Project)

Tests of Within-Subjects Contrasts

Measure: MEASURE_1

Source

TIME

Type III Sum of Squares

df

Mean Square

F

Sig.

TIME

Linear

1365.352

1

1365.352

285.697

.000

TIME * TREAT

Linear

175.650

2

87.825

18.377

.000

Error(TIME)

Linear

129.033

27

4.779

There is a significant effect for the interaction between treatment and time (that is, the change between the pre- and the post-treatment scores), as indicated in the output in Table 9.15.

If we look at the means of the patients’ pre-test and post-test depression scores in Table 9.16 and Figure 9.6, we can see that the amount of improvement shown by the three groups of patients is not the same. Least improvement has occurred in the group receiving no treatment (30.13 − 25.38 = 4.75), while patients being administered the drug treatment exhibit the most improvement (35.42 − 22.25 = 13.17).

Statistical differences in the amount of improvement shown in the three treatments could be further examined using One-Way ANOVA where the dependent variable is the computed difference between pre- and post-test patient depression.

Table 9.16 Means and standard deviations of patient pre-test ( patpre) and post-test (patpost) depression in the three treatments (Depression Project)

Descriptive Statistics

TREAT

Mean

Std. Deviation

N

PATPRE

1

30.13

3.72

8

2

24.20

2.78

10

3

35.42

2.84

12

Total

30.27

5.70

30

PATPOST

1

25.38

3.58

8

2

13.10

3.98

10

3

22.25

5.41

12

Total

20.03

6.75

30

Figure 9.6 Pre- and post-test patient depression for the three treatments (Depression Project)

Combined design

As pointed out earlier, it is possible to combine some of the above analyses. To show how this can be done, we shall look at the effect of two between-subjects factors (treatment and gender) and one within-subjects factor (pre- to post-test or time) on two dependent variables (depression as assessed by the patient and an informant), covarying out the effects of age which we think might be related to the pre- and post-test measures. The following procedure would be used to carry this out:

➔Analyze ➔General Linear Model ➔Repeated Measures… [opens Repeated Measures Define Factor[s] extended dialog box shown in Box 9.7]

➔highlight factor1 and type time in Within-Subject Factor Name: box

➔Number of Levels: box and type 2 ➔Add ➔Measure>> [opens more of Repeated Measures Define Factor[s] dialog box shown in Box 9.11] ➔type pat in Measure Name: box ➔Add ➔type inf in Measure Name: box ➔Add

➔Define [opens Repeated Measures subdialog box shown in Box 9.12]

➔patpre ➔► button beside Within-Subjects Variables [time]: [puts patpre in this box] ➔patpost ➔► button beside Within-Subjects Variables [time]: [puts patpost in this box] ➔infpre ➔► button beside Within-Subjects Variables [time]: [puts infpre in this box] ➔infpost ➔► button beside Within-Subjects

Box 9.11 Repeated Measures Define Factor[s] extended dialog box

Box 9.12 Repeated Measures subdialog box (combined design analysis)

Variables [time]: [puts infpost in this box] ➔treat ➔► button beside Between-Subjects Factor[s]: [puts treat in this box] ➔gender ➔► button beside Between-Subjects Factor[s]: [puts gender in this box] ➔age ➔► button beside Covariate(s): [puts age in this box]

➔Options… [opens Repeated Measures: Options subdialog box shown in Box 9.9]

➔Descriptive statistics ➔Transformation matrix ➔Continue [closes Repeated Measures: Options subdialog box]

➔OK

In conducting a multivariate analysis of covariance, it is necessary to check that the covariate (age) is significantly correlated with the two dependent variables, which it is as the output in Table 9.17 shows.

The output for the multivariate tests is reproduced in Table 9.18. This shows a significant effect for the time by treatment by gender interaction effect.

The univariate tests in Table 9.19 demonstrate the interaction effect to be significant (p < 0.0005) for the patient measure only (pat). It is not significant (p = 0.176) for the informant measure (inf). Pat refers to the transformed score which is the difference between the patient’s pre- and post-test measures as can be seen in Table 9.20. Inf represents the difference between the informant’s pre- and post-test scores. To interpret these results, it would be necessary to compute the mean pre- and post-treatment patient depression scores, adjusted for age, for men and women in the three treatments. Additional analyses would have to be conducted to test these interpretations, as described previously.

Table 9.17 Relationship between the covariate of age and the two transformed variables (Depression Project)

Tests of Between-Subjects Effects

Transformed Variable: Average

Source

Measure

Type III Sum of Squares

df

Mean Square

F

Sig.

Intercept

PAT

200.217

1

200.217

16.471

.000

INF

322.787

1

322.787

19.914

.000

AGE

PAT

298.580

1

298.580

24.562

.000

INF

192.043

1

192.043

11.848

.002

TREAT

PAT

549.908

2

274.954

22.619

.000

INF

531.276

2

265.638

16.389

.000

GENDER

PAT

9.063

1

9.063

.746

.397

INF

8.461

1

8.461

.522

.477

TREAT * GENDER

PAT

100.978

2

50.489

4.153

.029

INF

324.974

2

162.487

10.025

.001

Error

PAT

279.587

23

12.156

INF

372.799

23

16.209

Table 9.18 Multivariate tests for the interaction between time, treatment and gender (Depression Project)

Multivariatec,d

Within Subjects Effect

Value

F

Hypothesis df

Error df

Sig.

TIME

Pillai’s Trace

.364

6.305a

2.000

22.000

.007

Wilks’ Lambda

.636

6.305a

2.000

22.000

.007

Hotelling’s Trace

.573

6.305a

2.000

22.000

.007

Roy’s Largest Root

.573

6.305a

2.000

22.000

.007

TIME * AGE

Pillai’s Trace

.009

.097a

2.000

22.000

.908

Wilks’ Lambda

.991

.097a

2.000

22.000

.908

Hotelling’s Trace

.009

.097a

2.000

22.000

.908

Roy’s Largest Root

.009

.097a

2.000

22.000

.908

TIME * TREAT

Pillai’s Trace

.781

7.364

4.000

46.000

.000

Wilks’ Lambda

.241

11.423a

4.000

44.000

.000

Hotelling’s Trace

3.066

16.099

4.000

42.000

.000

Roy’s Largest Root

3.037

34.927b

2.000

23.000

.000

TIME * GENDER

Pillai’s Trace

.098

1.196a

2.000

22.000

.321

Wilks’ Lambda

.902

1.196a

2.000

22.000

.321

Hotelling’s Trace

.109

1.196a

2.000

22.000

.321

Roy’s Largest Root

.109

1.196a

2.000

22.000

.321

TIME * TREAT *

Pillai’s Trace

.576

4.653

4.000

46.000

.003

GENDER

Wilks’ Lambda

.439

5.606a

4.000

44.000

.001

Hotelling’s Trace

1.245

6.537

4.000

42.000

.000

Roy’s Largest Root

1.217

13.998b

2.000

23.000

.000

a. Exact statistic

b. The statistic is an upper bound on F that yields a lower bound on the significance level.

c.

Design: Intercept+AGE+TREAT+GENDER+TREAT * GENDER Within Subjects Design: TIME

d. Tests are based on averaged variables.

Table 9.19 Univariate tests for the interaction effect between time, treatment and gender (Depression Project)

Tests of Within-Subjects Contrasts

Source

Measure

TIME

Type III Sum of Squares

df

Mean Square

F

Sig.

TIME

PAT

Linear

32.706

1

32.706

12.916

.002

INF

Linear

31.259

1

31.259

4.244

.051

TIME * AGE

PAT

Linear

9.094E-02

1

9.094E-02

.036

.851

INF

Linear

.218

1

.218

.030

.865

TIME * TREAT

PAT

Linear

174.857

2

87.428

34.526

.000

INF

Linear

185.773

2

92.886

12.610

.000

TIME * GENDER

PAT

Linear

6.274

1

6.274

2.478

.129

INF

Linear

6.758

1

6.758

.917

.348

TIME * TREAT *

PAT

Linear

60.289

2

30.144

11.904

.000

GENDER

INF

Linear

27.641

2

13.820

1.876

.176

Error(TIME)

PAT

Linear

58.242

23

2.532

INF

Linear

169.423

23

7.366

Table 9.20 Transformed variables (Depression Project)

Average

Transformed Variable: AVERAGE

Dependent Variable

Measure

PAT

INF

PATPRE

.707

.000

PATPOST

.707

.000

INFPRE

.000

.707

INFPOST

.000

.707

EXERCISES

1

What are the two main advantages in studying the effects of two independent variables rather than one?

2

What is meant when two variables are said to interact?

3

How would you determine whether there was a significant interaction between two independent variables?

4

A colleague is interested in the relationship between alcohol, anxiety and gender on performance. Participants are randomly assigned to receiving one of four increasing dosages of alcohol. In addition, they are divided into three groups of low, moderate and high anxiety. Which is the dependent variable?

5

How many factors are there in this design?

6

How many levels of anxiety are there?

7

How would you describe this design?

8

If there are unequal numbers of participants in each group and if the variable names for alcohol, anxiety, gender and performance are alcohol, anxiety, gender and perform respectively, what is the appropriate SPSS procedure for examining the effect of the first three variables on performance?

9

You are interested in examining the effect of three different methods of teaching on learning to read. Although participants have been randomly assigned to the three methods, you think that differences in intelligence may obscure any effects. How would you try to control statistically for the effects of intelligence?

10

What is the appropriate SPSS procedure for examining the effect of three teaching methods on learning to read, covarying out the effect of intelligence when the names for these three variables are methods, read and intell respectively?

11

You are studying what effect physical attractiveness has on judgements of intelligence, likeability, honesty and self-confidence. Participants are shown a photograph of either an attractive or an unattractive person and are asked to judge the extent to which this person has these four characteristics. How would you describe the design of this study?

12

If the names of the five variables in this study are attract, intell, likeable, honesty and confid respectively, what is the appropriate SPSS procedure you would use to analyse the results of this study?

Đăng ký:
Đăng Nhận xét (Atom)

## Không có nhận xét nào:

## Đăng nhận xét