Arts Supervisors

New York City public schools were asked whether their arts supervisor was employed full- or part-time. If their supervisor was full-time, schools clarified whether they were solely working on arts programs or had other responsibilities.
In small schools, or under-resourced ones, faculty may be expected to wear many hats. In the simplest cases, physical education teachers lead gym and health classes. More extreme examples can have teachers with certification in, for example, English teaching classes in math or science. I do not assert that an arts supervisor must be employed full-time to run an effective program, but I am curious as to what features predict their employment status.

There are many full-time supervisors with duties other than the arts. I would be curious to discover the share of their responsibilities that are considered “other”. It is possible that some administrators consider teaching an arts class or doing clerical work to be “other”. Relative to the number of supervisors working full-time, there are few part-timers.

Only two schools did not respond to this question about their arts supervisor’s status. In the previous part, I looked at the question of whether a school had a designated arts liaison. 72 schools responded that they did not, while 296 schools do not have any arts supervisor. I wonder whether the presence of an arts supervisor has more or less influence on student academic performance, relative to arts liaisons.

To begin exploring academic performance, I will use the percentage of students to perform at or above standards for English Language Arts (ELA) and math.

I came into this analysis with some suspicion that arts programs might be more beneficial for ELA performance than for math, if they were to have any effect. In the plot above there is a positive relationship between ELA and math scores, regardless of art supervisor status.

## 
## Call:
## lm(formula = Math_score ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -32.578  -4.957   0.679   5.235  32.163 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -6.45658    0.72046  -8.962   <2e-16 ***
## ELA_score    1.06415    0.01424  74.718   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 8.686 on 893 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.8621, Adjusted R-squared:  0.8619 
## F-statistic:  5583 on 1 and 893 DF,  p-value: < 2.2e-16

Output 1

On inspection the relation between the two scores seems identical between supervisor statuses. In fact, fitting a linear regression to each yields coefficients close to the uncontrolled coefficient for math scores on ELA scores, 1.06415 (Output 1). \(Math\_score = \beta_0 + \beta_1 * ELA\_score\)

## 
## Call:
## lm(formula = Math_score ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -32.854  -4.955   0.520   5.357  31.868 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -20.80595    8.66873  -2.400   0.0166 *  
## Q3_1         10.11455    8.84006   1.144   0.2529    
## Q3_2         14.62550    8.66724   1.687   0.0919 .  
## Q3_3         13.37116    8.73705   1.530   0.1263    
## Q3_4         14.28427    8.68146   1.645   0.1002    
## ELA_score     1.06450    0.01421  74.922   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 8.657 on 889 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.8636, Adjusted R-squared:  0.8628 
## F-statistic:  1126 on 5 and 889 DF,  p-value: < 2.2e-16

Output 2

We can control for supervisor status with dummy variables by altering the regression like so, \(Math\_score = \beta_0 + \beta_1 * ELA\_score + \beta_2 * Q3\_1 + \beta_3 * Q3\_2 + \beta_4 * Q3\_3 + \beta_5 * Q3\_4\), where \(Q3\_i\) corresponds to the \(i^{th}\) member of the list c('Full-time, solely for arts', 'Full-time, with other duties', 'Part-time', 'None') (Output 2).

##                  2.5 %    97.5 %
## (Intercept) -37.819514 -3.792379
## Q3_1         -7.235276 27.464384
## Q3_2         -2.385138 31.636142
## Q3_3         -3.776499 30.518809
## Q3_4         -2.754271 31.322808
## ELA_score     1.036615  1.092385

Output 3

This model specification does not result in a compellingly significant coefficient for the effect of a particular supervisor status. Any status is estimated to have a positive coefficient, but a 95% confidence interval (Output 3) does not exclude the possibility that any of status coefficients could be zero.

delta.beta1 <- coefficients(lm.fit.null.summ)[2,1] - coefficients(lm.fit.summ)[6,1]
delta.beta1

## [1] -0.0003517423

se.delta.beta1 <- sqrt(coefficients(lm.fit.null.summ)[2,2]^2 + coefficients(lm.fit.summ)[6,2]^2)
se.delta.beta1

## [1] 0.02011739

Output 4

On inspection, the regression coefficients for ELA_score are close between model specifications A (no controls) and B (controlling for supervisor status). The difference in estimated coefficients \(\beta_1\) is -0.0003 (Output 4). To find the variability of the difference in regression coefficients, we use the formula \(Var(A-B) = Var(A) + Var(B) - 2*Cov(A,B)\). Assuming covariance between the two estimates is zero, we arrive at 0.0201 as the standard error of the difference.
Now I can assert that there is no difference in the relationsip between ELA and math scores from supervisor statuses.

This mosaic plot illustrates that the proportion of schools without an arts program supervisor is greater for schools without an arts liaison that for those with. This lends support to the idea that those schools are lacking the resources to fully staff their arts programs, as they have not filled two key positions. It is of course possible that arts liaisons and supervisors are not necessary to effective programs, and schools without either are running just fine. I would like to assess the quality of the arts programs themselves as a function their liaison and supervisor statuses, perhaps through some sort of measure of funding or arts resources.

The next questions concern certifications that arts supervisors may have, either in an arts discipline or in administration.

Few supervisors are certified in the arts, the majority are administrators. Could this have an impact on the efficacy of an arts program?

Linear regressions do not yield a statistically significant result for the effect of either or both supervisor certification on student academic performance. The model specifications I tried out were of the form \(score = \beta_0 + \beta_1 * arts\_cert + \beta_2 * admin\_cert + \beta_3 * both\_cert\).

## 
## Call:
## lm(formula = perc_34_all_2018_ela ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -32.521  -9.023  -1.700   7.244  54.364 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           93.980634   1.850377  50.790  < 2e-16 ***
## arts_cert              1.164997   1.758245   0.663 0.507765    
## admin_cert             1.167456   0.942800   1.238 0.215937    
## both_cert              1.002057   2.934071   0.342 0.732789    
## total_enrollment_2017  0.004566   0.001363   3.351 0.000839 ***
## perc_black_2017       -0.119051   0.018022  -6.606  6.8e-11 ***
## perc_pov_2017         -0.642529   0.020084 -31.993  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.5 on 888 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.6268, Adjusted R-squared:  0.6243 
## F-statistic: 248.6 on 6 and 888 DF,  p-value: < 2.2e-16

##                              2.5 %       97.5 %
## (Intercept)           90.349011532 97.612257433
## arts_cert             -2.285803835  4.615797128
## admin_cert            -0.682920159  3.017833144
## both_cert             -4.756465157  6.760580155
## total_enrollment_2017  0.001891955  0.007240283
## perc_black_2017       -0.154420668 -0.083680879
## perc_pov_2017         -0.681945909 -0.603112479

## 
## Call:
## lm(formula = perc_34_all_2018_math ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -41.852 -10.756  -2.115   8.057  63.109 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           91.616504   2.239070  40.917  < 2e-16 ***
## arts_cert             -1.728792   2.127584  -0.813    0.417    
## admin_cert             0.942806   1.140846   0.826    0.409    
## both_cert              3.165637   3.550406   0.892    0.373    
## total_enrollment_2017  0.006623   0.001649   4.017  6.4e-05 ***
## perc_black_2017       -0.219876   0.021807 -10.083  < 2e-16 ***
## perc_pov_2017         -0.634167   0.024302 -26.095  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 15.13 on 888 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.584,  Adjusted R-squared:  0.5812 
## F-statistic: 207.8 on 6 and 888 DF,  p-value: < 2.2e-16

##                              2.5 %       97.5 %
## (Intercept)           87.222017963 96.010989772
## arts_cert             -5.904471567  2.446887747
## admin_cert            -1.296263104  3.181874756
## both_cert             -3.802527434 10.133801854
## total_enrollment_2017  0.003386955  0.009858758
## perc_black_2017       -0.262675858 -0.177076386
## perc_pov_2017         -0.681863289 -0.586470017

Output 5

After controlling for school size and percentage of black and impoverished students, there is no statistically significant effect for certifications on academic performance. A 95% confidence interval of the coefficient for supervisor certification in administration includes zero for ELA and math scores, so we cannot claim that there is a statistically significant nonzero effect on academic performance. The confidence interval for arts or both certifications are even wider.

## 
## Call:
## lm(formula = perc_4_all_2018_ela ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -29.574  -5.502  -1.417   3.625  57.556 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           48.868172   1.405134  34.778  < 2e-16 ***
## arts_cert              0.862021   1.335171   0.646    0.519    
## admin_cert             0.543582   0.715941   0.759    0.448    
## both_cert              3.240655   2.228066   1.454    0.146    
## total_enrollment_2017  0.005410   0.001035   5.229 2.13e-07 ***
## perc_black_2017       -0.054557   0.013685  -3.987 7.25e-05 ***
## perc_pov_2017         -0.453225   0.015251 -29.718  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 9.494 on 888 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.5861, Adjusted R-squared:  0.5833 
## F-statistic: 209.5 on 6 and 888 DF,  p-value: < 2.2e-16

##                              2.5 %       97.5 %
## (Intercept)           46.110400736 51.625943358
## arts_cert             -1.758437940  3.482479797
## admin_cert            -0.861551453  1.948715865
## both_cert             -1.132235198  7.613545292
## total_enrollment_2017  0.003379317  0.007440715
## perc_black_2017       -0.081416450 -0.027698274
## perc_pov_2017         -0.483157590 -0.423293288

## 
## Call:
## lm(formula = perc_4_all_2018_math ~ ., data = .)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -35.383  -7.853  -2.417   5.387  58.260 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           59.773655   1.820197  32.839  < 2e-16 ***
## arts_cert             -1.894760   1.729567  -1.096 0.273590    
## admin_cert             0.277532   0.927423   0.299 0.764818    
## both_cert              5.689689   2.886215   1.971 0.048996 *  
## total_enrollment_2017  0.005234   0.001340   3.905 0.000101 ***
## perc_black_2017       -0.154016   0.017728  -8.688  < 2e-16 ***
## perc_pov_2017         -0.508582   0.019756 -25.743  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.3 on 888 degrees of freedom
##   (370 observations deleted due to missingness)
## Multiple R-squared:  0.5648, Adjusted R-squared:  0.5619 
## F-statistic: 192.1 on 6 and 888 DF,  p-value: < 2.2e-16

##                              2.5 %       97.5 %
## (Intercept)           56.201265512 63.346043902
## arts_cert             -5.289275597  1.499756483
## admin_cert            -1.542663701  2.097728293
## both_cert              0.025091309 11.354287317
## total_enrollment_2017  0.002603756  0.007864849
## perc_black_2017       -0.188808711 -0.119222727
## perc_pov_2017         -0.547356080 -0.469808466

Output 6

If we drill down further, looking at only the percentage of students to receive a 4 (the highest grade), then there is a statistically significant coefficient for the effect of both certifications on math scores, with a p-value of 0.048996 (Output 6). In this case, a 95% confidence interval just excludes zero, being [0.025, 11.354].
Having been able to arrive at this more tenuous result I feel a renewed belief that arts programs do have an effect on academic performance and that some proof is lying in the data somewhere. On the other hand I am suspicious as to how I manufactured this result by narrowing my focus until I hacked my way to an accceptable p-value.