Illustration (and the use of HLM) Chapter 4

Illustration (and the use of HLM) Chapter 4

The Illustration Data Now we cover the example. In doing so we does the use of the software HLM. In addition, we will discuss typical questions that can be answered. 2 LTRC HLM Workshop

The Illustration Data The data set consists of 7,185 students who are nested in schools (there are 160 schools). Variables that we will use: MATHACH (Math Achievement at student level). CSES (group centered SES at student level). MEANSES (Average School SES, school level). l) SECTOR (Public 0 versus Catholic 1, school level). 3 LTRC HLM Workshop

Student Label School Label Level-1 Variables Level-2 Variables 4 LTRC HLM Workshop

The Questions 1. How much do U.S. high schools vary in their mean math achievement? 2. Do schools with high MEANSES also have high math achievement? 3. Is the strength of association between student CSES and math achievement similar across schools? Or is CSES a better predictor of student math achievement in some schools than others? 4. How do public and Catholic schools compare in terms of mean math achievement and in terms of the strength of the SES-math achievement relationship, after we control for MEAN SES? 5 LTRC HLM Workshop

1 st Question 1. How much do U.S. high schools vary in their mean math achievement? Notice that this question only addresses the variance of the observations, We will use a random effects ANOVA, 6 LTRC HLM Workshop

1 st Question We assume that u 0j has a normal distribution with mean 0 and variance τ 00 School j s mean MATHACH β γ + 0 = γ + u 0 j 00 0 j MATHACH β + ij = 0 j r ij Here all we want to know is the amount of variability of the mean math achievement across the schools 7 LTRC HLM Workshop

HLM The program does allow one to keep two separate files in the case of a 2 level analysis, SO in our case we could have a file for each examinee and a file for school, You will need to give the ID variable (school) in both files, Here we have it as an SPSS file and so we will just use it. 8 LTRC HLM Workshop

HLM To begin using HLM open the program and click on file This will pop-up a question about what format of dt data you are using At this point we want to start a new analysis so click on Make a new MDM file 9 LTRC HLM Workshop

HLM This brings up an option window. We only discuss how to do a two level analysis, so click on HLM2 10 LTRC HLM Workshop

HLM Pick the SPSS file that you are using 11 LTRC HLM Workshop

HLM Pick variables 12 LTRC HLM Workshop

HLM Pick the SPSS file that you are using 13 LTRC HLM Workshop

HLM Pick variables 14 LTRC HLM Workshop

HLM Then save the mdmt file, a control file Add name of data file Finally, click on Make MDM file, completes your set up. Examine basic statistics file Click Done 15 LTRC HLM Workshop

HLM That will get you to this part where you define your model. 16 LTRC HLM Workshop

HLM Output The output for HLM is given as a text file (.txt). This file can be opened from HLM. Here I will just provide some of the main parts 17 LTRC HLM Workshop

Summary Information This gives us the random components 18 LTRC HLM Workshop

Fixed Effects Fixed effect for our model (intercept,,γγ 00 ) 19 LTRC HLM Workshop

Random Effects Same random components as before only now we have SE and significance tests 20 LTRC HLM Workshop

Estimates Think back to the original question. 1. How much do U.S. high schools vary in their mean math achievement? We have a parameter that measured that for us, which was τ 00 =8.61. Also, the average e achievement ement across all school averages (i.e., the grand mean). γ 00 =12.64. 21 LTRC HLM Workshop

Range of School Averages Given the mean and its variance I can even compute a confidence interval to describe a range that includes 95% of all schools average math achievement. ˆ γ ± 1.96( ˆ τ ) 00 00 1 2 1 12.64 ± 1.96(8.61) 2 = (6.89,18.39) 22 LTRC HLM Workshop

CI for Grand Mean We can also put a CI around the our estimate of the grand mean using the same equation. Only now we use the standard error of our estimate for γ 00. 1 ± 2 12.64 ± 1.96(.24) = (12.17,13.11) 17 13 11) 23 LTRC HLM Workshop

Interclass Correlation We can also compute the intraclass correlation. ˆ τ 8.61 ˆ τ 00 + σ 861 8.61+ 3915 39.15 00 ρ = = = 2 0.18 24 LTRC HLM Workshop

Reliability Lastly we can compute the reliability of our estimate of School j s mean. ˆ λ reliability ( Y ) ˆ00 λ j =. j = ( 2 ˆ τ ) 00 + ( σ / n j ) τ 25 LTRC HLM Workshop

Reliability If we compute this value for all schools and average it we will get a summary of reliability which is the same that is given in the HLM output. 26 LTRC HLM Workshop

2 nd Question 1. Do schools with high MEANSES also have high math achievement? Focus is how a level-2 variable effects math achievement (average math achievement). Means as outcomes model. 27 LTRC HLM Workshop

2 nd Question MATHACH β + ij = 0 j r ij β 0 j = γ00 + + γ01meanses u0 j 28 LTRC HLM Workshop

HLM 29 LTRC HLM Workshop

Model Summary Information 30 LTRC HLM Workshop

Results (Fixed and Random) The estimate of our effect for MEANSES, γ 01 Hypothesis test Here are our variance This tells us that t there are still components differences across schools and that we 31 LTRC HLM Workshop do need the random effect

Additional Results There are other things we can compute. For example, we can compute the expected range of school means adjusted for MEANSES. 1 1 γˆ 1.96 ˆ 2 12.65 1.96 2.64 2 00 ± τ00 = ± = ( ) ( ) ( 9.47,15.83) 32 LTRC HLM Workshop

Additional Results We can also compute the proportion of variance that can be explained by MEANSES using. τˆ 00 00 = = ( ANOVA) τˆ ( MEANSES) τˆ00 ( ANOVA) 8.61 2.64 8.61 0.69 33 LTRC HLM Workshop

Additional Results Lastly we can still compute the intraclass correlation as before. Now it is conditional on MEANSES. A measure of dependence within school given that we are now accounting for MEANSES. τˆ 2.64 ρ = 00 = =.06 τˆ +σ 2 2.64 39.16 00 + 34 LTRC HLM Workshop

3 rd Question 1. Is the strength of association between student CSES and math achievement similar across schools? Or is CSES a better predictor of a students math achievement in some schools than others? Now the focus is on the effect of a level-1 variable on the dependent variable. Also how this effect varies across schools. 35 LTRC HLM Workshop

3 rd Question That means that we are interested in: MATHACH = β ( 0 + β1 CSES ) + r β β = γ + u 0 j 00 = γ + u j j i ij oj 1j 10 1j 36 LTRC HLM Workshop

HLM 37 LTRC HLM Workshop

HLM Output Notice now that we have two level-2 random effects we can estimate each variance and their covariance, this is their covariance matrix 38 LTRC HLM Workshop

HLM Output 39 LTRC HLM Workshop

Average Regression Line Can determine the average regression line across school based on the level-2 fixed effects. β β = γ + u 0j 00 = γ + u oj 1 j 10 1 j So the average line is: MATHACH = 12.65 + 2.19( CSES) 40 LTRC HLM Workshop

Variability of Line Can determine variability of the regression lines across schools. Based on variance components (or random effects) τ 00 and τ 11 12.64 ± 1.96(8.68) = 6.87,18.41 1 1 2 ( ) 2.19 1.96(0.68) 68) 0.57,3.81 ± 2 = ( ) 41 LTRC HLM Workshop

Relationships of Coefficients Can interpret the relationship between average math achievement with in a school and the relationship between CSES and MATHACH. Looking at τ 01 42 LTRC HLM Workshop

Additional Results Lastly, we could look at the proportion of error, at level 1, that can be explained by CSES. Because we are at level-1 we use σ 2 σ 2 ( ) 2 ANOVA σ ( CSES ) 2 σ ( ANOVA) = 39.15 36.70 39.15 = 0.063 Remember that the school MEANSES accounted for 60% 43 LTRC HLM Workshop

4 th Question 4. How do public and Catholic schools compare in terms of mean math achievement and in terms of the strength of the SES-math achievement relationship, after we control for MEAN SES? 44 LTRC HLM Workshop

4 th Question Do MEANSES and SECTOR significantly predict the intercept. Do MEANSES and SECTOR significantly predict the slope. How much variation in the intercept and slope is explained by MEANSES and SECTOR. 45 LTRC HLM Workshop

HLM 46 LTRC HLM Workshop

HLM Output 47 LTRC HLM Workshop

Fixed Effects 48 LTRC HLM Workshop

Random Effects 49 LTRC HLM Workshop

Summary The purpose this morning was to get you familiar with terminology, notation, and the types of tests and conclusions we can make with HLM. Are there any questions? Lunch Time 50 LTRC HLM Workshop