The Assessment of A Grade School Environmental Education Measure

ARN Research Note on Measuring the Effectiveness of Environmental Education

Pamela M. M. Jull, PhD & Aaron Ignac

Applied Research Northwest. LLC

Project Description

In Winter of 2004-05, Applied Research Northwest (ARN) developed and piloted a test designed to estimate knowledge gains for sixth graders participating in the week-long solid waste curriculum. The program was run by RE Sources on behalf of Whatcom County Public works and the Coordinated Prevention Grants.

The development of the Assessment of Grade School Environmental Education Measure (AGSEEM) was a comprehensive process consisting of test item construction; pilot testing; psychometric analysis, using traditional and probabilistic approaches; instrument refinement; and finally, full administration. Throughout the process, content and statistical experts were included to ensure the utility of the final instrument.

Methods and Findings

In all, 162 students completed pre and post-program tests of their knowledge. A t-test comparing pre- (M = 47.4, SD = 11.6) and post-test (M = 53.6, SD = 12.4) scores indicates a significant increase in scores after the environmental education course was completed (F = 24.3, p < 0.001).

What did students learn?

Displacement scores, which indicate a shift in student ability to correctly respond to specific test questions, indicated that certain concepts became much easier as a result of the course. They include:

n Making the best choices for reducing waste (e.g. buying less packaging)

n Understanding of composting processes and practice

n Understanding the value of recycling and using recycled and recyclable products.

Displacement scores also indicated that some of the initially easy concepts became so easy as a result of the RE Sources course that they were less able to distinguish student ability. In other words, these concepts were for the most part, well understood by students before the course and well understood by students at the end of the course.

n Disposal of toxic waste

n Recycling behaviors

n Conserving natural resources

Prior Program Experience Effects

Because many of the students may have encountered RE Sources solid waste education program efforts in earlier grades through less intensive coursework (typically 30-50 minute sessions), students’ exposure to the program was believed to impact their scores positively.

Familiarity with and knowledge related to the RE Sources program were correlated with the pre-program scores of the students (r=.265, p<.001, N=161), suggesting that the short courses kids encounter in the Whatcom County school system may improve their knowledge of solid waste issues. However, since the program is not randomly assigned to classrooms, there is likely some selectivity bias for teachers who are interested in kids learning about environmental impacts of human behavior. These teachers may have developed additional curricula to which the RE Sources programs contribute.

Familiarity with other RE Sources programs did not impact the change in scores suggesting significant value is added by participating in the course, regardless of prior knowledge.

Did kids’ behavior change?

Kids reported similar levels of recycling behavior at both home and school after the program as before. It should be noted that the goal of the program and its design is not to change behavior, merely to increase their understanding of the issues.

The program instructors were interested in whether the program influenced students’ attitudes toward solid waste issues, particularly in terms of their sense of responsibility and efficacy about making a difference. To that end, two items were included in the test and scores shifted significantly on one in the expected direction.

Discussion

Overall the tests suggest RE Sources is conducting an effective program for educating students about solid waste issues. After five, one-hour sessions in the course of a week, sixth grade students at seven schools in Whatcom County showed significantly stronger understanding of solid waste issues.

Future administrations of the test would benefit from random assignment of the program to schools and school classrooms, as well as testing of matched classes of non-program students.

The findings show that the program had a significantly positive impact on student knowledge pertaining to several aspects of waste reduction: making the best decisions for reducing waste, optimizing the positive effects of composting, and recyclable products and processes. The diverse and comprehensive nature of these concepts suggests that the program effectively improves general and overall knowledge of solid waste issues.

Meanwhile, student knowledge on other concepts tended not to change dramatically from pre- to post-test assessments: plastic recycling, the recycling loop, and what packaging is recyclable. Program designers may want to focus their efforts on improving the curriculum around the content and activities that relate to these items to boost change in scores in the future.

Finally, it looks as if the program is having some effect on students’ attitudes, particularly towards making them aware of the need for paying attention to waste issues.