When the Bay State Reading Institute won an Investing in Innovation Grant in 2010, it wasn’t surprising that its leaders turned to SchoolWorks to serve as evaluators for the federal grant program. SchoolWorks had been working with BSRI for four years, conducting formative reviews of its elementary literacy model that helped guide the program’s evolution into a multi-pronged approach that includes data analysis, research-based pedagogy and differentiated instruction, all supported by extensive coaching for teachers and principals. But more importantly, the idea of applying for the i3 grant came out of conversations between BSRI and SchoolWorks leaders, who recognized that the organization’s programs had the potential to scale as a comprehensive turnaround model for high-need elementary schools.
When BSRI became one of only 49 organizations in the country to be awarded the i3 grant, SchoolWorks was named the formal program evaluator. To take on the complex task of preparing a formal evaluation, SchoolWorks was charged with developing a rigorous research model that would also frame BSRI’s work and provide a more rigorous blueprint for implementation. “Our program was built on the extensive capability of our coaches and relied heavily on their work,” says Barbara Gardner, the organization’s executive director. “We had a strong sense of what our model was, but needed to formalize it not only for the i3 grant, but also to guide implementation as we continue to grow.”
As BSRI prepared to roll out its literacy-based transformation model to more than a dozen high-need schools in Massachusetts as part of the i3 grant, SchoolWorks consultants conducted focus group meetings and interviews with key stakeholders to identify the core elements of the program. These were codified in a theory of action, which both framed the i3 program evaluation and helped BSRI better communicate its approach to school reform.
SchoolWorks used the theory of action to develop a logic model and implementation matrix that defined key measurables within the evaluation. SchoolWorks then developed an outcome study, working with experts in educational statistics to conduct a time -interrupted series analysis of state test scores. Currently in its second year, the annual evaluation process incorporates more than 30 site visits, teacher surveys, and test score analysis.
Based on its work developing the BSRI study, SchoolWorks was invited to present at the 2012 Project Directors Meeting for i3 directors and evaluators. Throughout the evaluation’s design and implementation, SchoolWorks has approached the work through its core value of “serving clients best by learning with them.”
“We came as learners, and our experience with BSRI has expanded our knowledge of literacy practices,” says Robin Coyne Hull, SchoolWorks’ project manager. “Ultimately, what was once a linear model came together as a multi-dimensional model of BSRI’s work as a result of our careful listening and learning.”
Along with satisfying the requirements of the federal grant, SchoolWorks’ work on the logic model has provided key ways to assess different components of BSRI’s work against the metric that matters most—student outcomes. These key metrics, along with the documentation of the organization’s model and its implementation, will guide the organization going forward, according to Gardner. “In many ways, SchoolWorks helped us build a more formal organizational structure that will benefit us far beyond the life of the i3 grant,” she says.