Resources devoted to arts education
This is part of an extended look at the NYC School Arts Survey:
Part One: Arts Education Liaisons
Part Two: Arts Education Supervisors
The survey had several questions regarding which resources schools are devoting to arts education, whether the administrator thought they were sufficient and which non-Department of Education sources of support were available to them.
Schools were asked how many rooms they have dedicated to arts education. Rooms were divided into two categories: “rooms designed and used solely for the arts” and “multi-purpose or general education classroooms used for arts education”. Five arts disciplines were considered: dance, music, theater, visual arts, and media arts (a category which includes film and photography). Note that the term media arts refers to film programs.
Number of rooms is certainly not a direct metric for a school’s commitment to the arts. I hypothesize that it may prove useful for assessing the resources that are made available for arts education more broadly. Of course, there may be confounding factors, such as school size and borough.
All disciplines show the same trend; many schools have zero to two rooms for a given discipline and a few schools have more than twenty rooms. Having twenty-some rooms might be bordering on unrealistic, but the above plots depict both categories of rooms, those designed solely for the arts and multi-purpose. Will we get more reasonable numbers by examining the room counts separately?
Looking at the number of rooms designed solely for the arts, the maximum number of rooms is a more reasonable thirteen. For a school with several hundred students and a dedicated arts program I can picture that. If we look at multi-purpose rooms, we see that this class contains the bulk of the rooms. Not many schools have dozens of multi-purpose rooms for arts disciplines, but again, in the context of a large school, it seems like a possible number for some schools to have.
I would be curious to discover the overlap for multi-purpose rooms among arts disciplines. In other words, is a school with twenty multi-purpose rooms reporting some of those rooms as in use for multiple arts discipline? I do not see any way to completely correct for that possibility, but it would be interesting to control for school size to try to get to the bottom of it. If this does not work, then we could consider only the number of rooms designed solely for art, but this could penalize small schools unduly.
##
## Call:
## lm(formula = perc_34_all_2018_ela ~ ., data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -38.706 -15.286 -3.482 13.757 57.410
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 37.172301 1.540811 24.125 < 2e-16 ***
## rm_dance 0.198147 0.203517 0.974 0.331
## rm_music -0.019724 0.170177 -0.116 0.908
## rm_thtr -0.046773 0.195201 -0.240 0.811
## rm_visart 0.186354 0.137639 1.354 0.176
## rm_media -0.141977 0.162441 -0.874 0.382
## total_enrollment_2017 0.014001 0.002242 6.245 7.33e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 19.62 on 706 degrees of freedom
## (552 observations deleted due to missingness)
## Multiple R-squared: 0.06461, Adjusted R-squared: 0.05666
## F-statistic: 8.127 on 6 and 706 DF, p-value: 1.642e-08##
## Call:
## lm(formula = perc_34_all_2018_math ~ ., data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -44.704 -17.891 -4.972 15.429 60.377
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 30.379442 1.751426 17.346 < 2e-16 ***
## rm_dance 0.187916 0.231335 0.812 0.4169
## rm_music 0.142808 0.193439 0.738 0.4606
## rm_thtr 0.048594 0.221883 0.219 0.8267
## rm_visart 0.258630 0.156453 1.653 0.0988 .
## rm_media -0.206618 0.184645 -1.119 0.2635
## total_enrollment_2017 0.018216 0.002549 7.147 2.21e-12 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.3 on 706 degrees of freedom
## (552 observations deleted due to missingness)
## Multiple R-squared: 0.09644, Adjusted R-squared: 0.08876
## F-statistic: 12.56 on 6 and 706 DF, p-value: 1.747e-13Conditioning on the effect of school size, through total enrollment, there is no statistically significant effect for the total number of rooms for arts education on student academic performance.
##
## Call:
## lm(formula = perc_34_all_2018_ela ~ ., data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -39.233 -15.704 -3.011 13.495 54.952
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 36.98124 1.56537 23.625 < 2e-16 ***
## Q8_R1_C1 -1.92913 1.36332 -1.415 0.157452
## Q8_R2_C1 3.30383 0.91647 3.605 0.000332 ***
## Q8_R3_C1 0.92597 1.04167 0.889 0.374312
## Q8_R4_C1 0.16888 1.18807 0.142 0.886998
## Q8_R5_C1 -1.84882 1.00594 -1.838 0.066449 .
## total_enrollment_2017 0.01312 0.00221 5.938 4.3e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 19.51 on 797 degrees of freedom
## (461 observations deleted due to missingness)
## Multiple R-squared: 0.08419, Adjusted R-squared: 0.0773
## F-statistic: 12.21 on 6 and 797 DF, p-value: 3.603e-13##
## Call:
## lm(formula = perc_34_all_2018_math ~ ., data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -39.170 -17.415 -4.067 14.767 58.385
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 31.887968 1.772957 17.986 < 2e-16 ***
## Q8_R1_C1 -3.390478 1.544105 -2.196 0.0284 *
## Q8_R2_C1 2.964480 1.037997 2.856 0.0044 **
## Q8_R3_C1 0.795531 1.179807 0.674 0.5003
## Q8_R4_C1 -0.890543 1.345613 -0.662 0.5083
## Q8_R5_C1 -2.400571 1.139337 -2.107 0.0354 *
## total_enrollment_2017 0.019294 0.002503 7.707 3.82e-14 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.1 on 797 degrees of freedom
## (461 observations deleted due to missingness)
## Multiple R-squared: 0.104, Adjusted R-squared: 0.09722
## F-statistic: 15.41 on 6 and 797 DF, p-value: < 2.2e-16Controlling for school size, through total student enrollment, we can see a statistically significant (p-value less than 0.001) positive coefficient for the effect of additional rooms designed and used solely for music, on ELA state test scores. If we consider math scores, rooms solely dedicated to music still have a positive effect, but at a 0.0044 p-value. No other arts disciplines have a statistically significant effect on ELA scores, but dance and media arts rooms have statistically significant (p-value less than 0.05) negative effects on math scores.
When we consider multi-purpose rooms, no arts discipline has a statistically significant effect on academic performance.
I am curious about the reason behind why music rooms appear to be associated with higher test scores, and why dance and media arts are associated with lower scores. To investigate we can try to control for different features and divide the performance metric by grade.
## [1] "perc_34_5_2018_ela"
##
## Call:
## lm(formula = f.ela, data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -37.072 -16.524 -3.894 15.154 65.915
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 30.393260 2.172697 13.989 < 2e-16 ***
## total_enrollment_2017 0.010224 0.002857 3.579 0.000375 ***
## Q8_R1_C1 -1.070094 1.677094 -0.638 0.523694
## Q8_R2_C1 3.047105 1.061100 2.872 0.004239 **
## Q8_R3_C1 -0.076777 1.189206 -0.065 0.948546
## Q8_R4_C1 -0.120480 1.501049 -0.080 0.936057
## Q8_R5_C1 -0.440556 1.291528 -0.341 0.733148
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 20.13 on 556 degrees of freedom
## Multiple R-squared: 0.04519, Adjusted R-squared: 0.03489
## F-statistic: 4.386 on 6 and 556 DF, p-value: 0.000242
##
## [1] "perc_34_6_2018_ela"
##
## Call:
## lm(formula = f.ela, data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -41.030 -16.090 -3.834 12.920 55.424
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 32.107149 2.400189 13.377 < 2e-16 ***
## total_enrollment_2017 0.019201 0.003511 5.469 9.17e-08 ***
## Q8_R1_C1 -0.386109 2.053572 -0.188 0.85098
## Q8_R2_C1 2.890346 1.386475 2.085 0.03789 *
## Q8_R3_C1 3.378799 2.188257 1.544 0.12357
## Q8_R4_C1 1.694171 1.881632 0.900 0.36860
## Q8_R5_C1 -4.765419 1.542141 -3.090 0.00218 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 20.26 on 319 degrees of freedom
## Multiple R-squared: 0.1908, Adjusted R-squared: 0.1756
## F-statistic: 12.53 on 6 and 319 DF, p-value: 1.08e-12## [1] "perc_34_5_2018_math"
##
## Call:
## lm(formula = f.math, data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -42.241 -19.010 -3.535 16.810 65.205
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 30.370138 2.450852 12.392 < 2e-16 ***
## total_enrollment_2017 0.015948 0.003225 4.944 1.01e-06 ***
## Q8_R1_C1 -1.021941 1.892332 -0.540 0.589384
## Q8_R2_C1 3.959590 1.195636 3.312 0.000988 ***
## Q8_R3_C1 -0.436070 1.340461 -0.325 0.745066
## Q8_R4_C1 -1.058457 1.691447 -0.626 0.531723
## Q8_R5_C1 -0.904146 1.456523 -0.621 0.535015
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.68 on 555 degrees of freedom
## Multiple R-squared: 0.07056, Adjusted R-squared: 0.06051
## F-statistic: 7.023 on 6 and 555 DF, p-value: 3.232e-07
##
## [1] "perc_34_6_2018_math"
##
## Call:
## lm(formula = f.math, data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -48.190 -16.714 -4.699 10.401 67.053
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 18.231937 2.592891 7.032 1.25e-11 ***
## total_enrollment_2017 0.021830 0.003808 5.733 2.29e-08 ***
## Q8_R1_C1 -0.022742 2.227755 -0.010 0.99186
## Q8_R2_C1 3.955813 1.502217 2.633 0.00887 **
## Q8_R3_C1 2.114419 2.368073 0.893 0.37259
## Q8_R4_C1 2.738886 2.038702 1.343 0.18008
## Q8_R5_C1 -4.635148 1.673390 -2.770 0.00594 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 21.97 on 319 degrees of freedom
## Multiple R-squared: 0.218, Adjusted R-squared: 0.2033
## F-statistic: 14.82 on 6 and 319 DF, p-value: 5.957e-15If we break down academic performance by grade, then the results are not as straigtforward. For sixth-, seventh- and eighth-graders’ ELA performance, media arts rooms have a more statistically significant negative coefficient than the positive coefficient for music rooms. All grades show a positive coefficient for music rooms, with a p-value no more than 0.01. There is a large jump in media arts between fifth- and sixth-grades, which is the demarcation between elementary and middle schools. A jump of this nature could have an underlying reason, beyond increased susceptibility to media arts programs at the expense of academic performance beginning in middle school. Perhaps not many elementary schools have media arts programs at all, or they are more commonly found in schools that are otherwise performing at an atypical level (higher or lower). Both of these can be checked.
Math scores show a similar trend to ELA scores, with most grades showing a statistically significant positive coefficient for music rooms. Among middle schools, media arts rooms have a statistically significant negative coefficient. The key difference with ELA scores is that there is not a statistically significant positive coefficient for music rooms on eighth-grade math scores.
I do not see a significant difference in the distribution of art rooms between elementary and middle schools. Music and visual arts rooms are the most common. Let’s look at the performance of schools with media arts programs.
We can match schools on having media arts rooms while controlling for number of students, number of students in each grade, and perhaps other demographic and academic features. Then we can examine the effect of media arts rooms on academic performance.
##
## Call:
## matchit(formula = f, data = temp, method = "nearest", distance = "logit")
##
## Summary of balance for all data:
## Means Treated Means Control SD Control Mean Diff
## distance 0.4353 0.4066 0.0652 0.0287
## perc_34_all_2018_ela 46.6335 46.3189 20.4054 0.3147
## total_enrollment_2017 655.2789 582.9615 325.9929 72.3174
## grd_1_2017 61.4113 66.4686 56.7962 -5.0573
## grd_2_2017 62.2197 67.0081 57.3009 -4.7884
## grd_3_2017 64.3408 69.3570 60.2137 -5.0162
## grd_4_2017 64.3634 68.6998 62.2163 -4.3364
## grd_5_2017 64.7155 68.8499 63.3577 -4.1344
## grd_6_2017 76.4901 42.7688 82.3900 33.7214
## grd_7_2017 77.1099 42.4604 83.9690 34.6494
## grd_8_2017 77.1690 42.1339 84.4076 35.0351
## eQQ Med eQQ Mean eQQ Max
## distance 0.0101 0.0291 0.192
## perc_34_all_2018_ela 1.1000 1.3214 11.100
## total_enrollment_2017 59.0000 74.1915 238.000
## grd_1_2017 4.0000 5.0338 53.000
## grd_2_2017 3.0000 4.8113 48.000
## grd_3_2017 3.0000 4.9746 49.000
## grd_4_2017 3.0000 5.2761 40.000
## grd_5_2017 5.0000 5.0423 42.000
## grd_6_2017 0.0000 34.1662 253.000
## grd_7_2017 0.0000 35.1944 281.000
## grd_8_2017 0.0000 35.6197 254.000
##
##
## Summary of balance for matched data:
## Means Treated Means Control SD Control Mean Diff
## distance 0.4353 0.4178 0.0721 0.0175
## perc_34_all_2018_ela 46.6335 45.0045 19.9245 1.6290
## total_enrollment_2017 655.2789 616.0394 337.3379 39.2394
## grd_1_2017 61.4113 65.1183 58.3054 -3.7070
## grd_2_2017 62.2197 65.9211 59.5015 -3.7014
## grd_3_2017 64.3408 67.9239 62.3438 -3.5831
## grd_4_2017 64.3634 68.7352 64.3851 -4.3718
## grd_5_2017 64.7155 69.4873 65.0190 -4.7718
## grd_6_2017 76.4901 53.9972 92.9389 22.4930
## grd_7_2017 77.1099 54.0197 95.1002 23.0901
## grd_8_2017 77.1690 53.8000 95.7602 23.3690
## eQQ Med eQQ Mean eQQ Max
## distance 0.0008 0.0177 0.1721
## perc_34_all_2018_ela 1.5000 2.1676 11.1000
## total_enrollment_2017 31.0000 41.0648 171.0000
## grd_1_2017 3.0000 3.8085 53.0000
## grd_2_2017 2.0000 3.9324 48.0000
## grd_3_2017 2.0000 4.1183 60.0000
## grd_4_2017 3.0000 5.2338 59.0000
## grd_5_2017 4.0000 5.3296 42.0000
## grd_6_2017 0.0000 22.6901 223.0000
## grd_7_2017 0.0000 23.3718 253.0000
## grd_8_2017 0.0000 23.8366 219.0000
##
## Percent Balance Improvement:
## Mean Diff. eQQ Med eQQ Mean eQQ Max
## distance 38.9856 91.9854 39.1348 10.3680
## perc_34_all_2018_ela -417.7110 -36.3636 -64.0375 0.0000
## total_enrollment_2017 45.7400 47.4576 44.6503 28.1513
## grd_1_2017 26.6991 25.0000 24.3425 0.0000
## grd_2_2017 22.7004 33.3333 18.2670 0.0000
## grd_3_2017 28.5688 33.3333 17.2140 -22.4490
## grd_4_2017 -0.8167 0.0000 0.8009 -47.5000
## grd_5_2017 -15.4176 20.0000 -5.6983 0.0000
## grd_6_2017 33.2976 0.0000 33.5889 11.8577
## grd_7_2017 33.3607 0.0000 33.5921 9.9644
## grd_8_2017 33.2984 0.0000 33.0803 13.7795
##
## Sample sizes:
## Control Treated
## All 493 355
## Matched 355 355
## Unmatched 138 0
## Discarded 0 0
Propensity score matching improves balance for most features.
##
## Welch Two Sample t-test
##
## data: perc_34_all_2018_ela by rm_ded_media
## t = -1.0661, df = 706.75, p-value = 0.2867
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -4.629010 1.370982
## sample estimates:
## mean in group 0 mean in group 1
## 45.00451 46.63352##
## Call:
## lm(formula = perc_34_all_2018_ela ~ rm_ded_media, data = m.data1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -45.005 -16.587 -3.434 15.588 53.895
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 45.005 1.080 41.653 <2e-16 ***
## rm_ded_media 1.629 1.528 1.066 0.287
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 20.36 on 708 degrees of freedom
## Multiple R-squared: 0.001603, Adjusted R-squared: 0.0001926
## F-statistic: 1.137 on 1 and 708 DF, p-value: 0.2867##
## Call:
## lm(formula = perc_34_all_2018_ela ~ ., data = .)
##
## Residuals:
## Min 1Q Median 3Q Max
## -125.283 -6.967 -0.691 6.486 46.678
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.781e+02 2.010e+01 33.738 < 2e-16 ***
## rm_ded_media 1.388e+00 9.212e-01 1.507 0.132
## total_enrollment_2017 2.824e-02 4.520e-03 6.248 7.23e-10 ***
## grd_1_2017 -4.954e-01 4.753e-02 -10.423 < 2e-16 ***
## grd_2_2017 1.219e+00 6.042e-02 20.167 < 2e-16 ***
## grd_3_2017 -1.194e+00 5.417e-02 -22.045 < 2e-16 ***
## grd_4_2017 4.809e-01 4.566e-02 10.533 < 2e-16 ***
## grd_5_2017 5.800e-01 4.021e-02 14.424 < 2e-16 ***
## grd_6_2017 2.189e-01 2.560e-02 8.549 < 2e-16 ***
## grd_7_2017 -7.749e-04 3.879e-02 -0.020 0.984
## grd_8_2017 1.275e+00 5.267e-02 24.206 < 2e-16 ***
## pscore -1.846e+03 5.745e+01 -32.124 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 12.2 on 698 degrees of freedom
## Multiple R-squared: 0.6466, Adjusted R-squared: 0.6411
## F-statistic: 116.1 on 11 and 698 DF, p-value: < 2.2e-16Matching suggests that rooms dedicated to media arts are associated with schools with higher ELA scores. The result that led to this investigation was that rooms dedicated to media arts are associated with lower state test scores among middle-schoolers. In light of this analysis I would not draw any firm conclusions, but would be interested to find some instrumental variable to use in lieu of an experiment.
NYC Schools Arts Survey Data
The 2017-2018 Arts Survey Data has data about arts teachers, budgets, partnerships with cultural organizations and parental involvement in NYC public schools.
In an effort to gain greater context for this data, we can examine it in conjuction with publicly available ELA and Math state test results and demographic data.
My goals were to understand the state of arts programs in NYC schools, what variables affect the resources of arts programs, and whether arts programs have an effect on the academic performance of students.