The Louisiana Department of Education (DOE) invested a substantial amount of energy and money to win grant funding from the 21st Century Community Learning Centers for administering after school programs among its schools and their nonprofit partners.
Like many school departments across the country, the Louisiana DOE needed to demonstrate performance aligned with federal requirements, and it hoped to go even further: to produce outcomes evidence of sufficient quality to meet the “gold standard” of scientific, third-party review. But the scale of the initiative, which began in the early 2000s with 20 school districts and 30 nonprofit agencies serving approximately 20,000 students, overwhelmed manual and disparate data collection obligations. Worse, participating districts were funded within staggered annual funding rounds or “cohorts”; to produce acceptable reports for federal evaluators, the state had to aggregate data by their correct cohorts, a monumental challenge made more complex by the sheer volume of the participating parties.
Solution: Sophisticated but simple platform collects higher-quality inputs, automates reporting outputs
In 2002, Louisiana turned to Cityspan, a cloud-based data management platform that expands and improves the state’s ability to manage data. The Cityspan platform included a number of important building blocks that improved the quality of the data collected, managed and reported:
Precise site-level reporting
While the state provides the data collection tools, each site gathers its data and gets reports on its own. Simple, intuitive interfaces prove vital in an environment with high personnel turnover and limited access to technology resources. Even with minimal or no training, field staff can input accurate data that conforms to the terminology required by federal regulators and other evaluators.
Flexible outcomes tracking
Under federal grant requirements for 21st Century Centers, youth outcomes can be measured through either teacher surveys, student grades or standardized state assessments. Cityspan has the flexibility to aggregate data in any of the three outcomes categories the state and school districts choose. Regardless of the categories chosen, the outputs remain compliant with federal rules.
Conformance with federal performance reports
For busy program staff, assembling data manually for federal reports can be a confusing process prone to unintentional error. Cityspan incorporated federal regulatory requirements into its platform’s algorithms, producing outputs that exactly correspond to each of the eight different sections of the annual performance reports demanded by evaluators.
Results: First scientifically rigorous proof of programs’ impact on academic performance
In 2003, the Louisiana DOE’s after school programs had to do more than meet federal performance requirements. Its outcomes reports were submitted to a rigorous analysis, conducted and reviewed by expert social scientists, that pit program outcomes data against a comparable control set.
Thanks to the depth, precision and reach of the data collected and organized through the Cityspan platform, Louisiana proved the efficacy of its programs through a 2004-2005 evaluation, funded by the U.S. Department of Education, titled, Academic Outcomes in Louisiana’s 21st Century Community Learning Centers. After reviewing 16 programs involving 5,375 youth, the evaluators concluded that the 21st Century Learning Centers out-of-school activities made a statistically meaningful improvement in academic performance for youth who participated in the programs for thirty days or more.
Benefits at a glance:
- Facilitated data collection at 229 individual sites
- Aggregated data to conform with the 21st CCLC Annual Performance Reports
- Supported annual renewals among multiple program cohorts
- Informed the first scientifically rigorous proof of programs’ contribution to academic progress
- External Study – A Profile of the Evaluation of 21st Century Community Learning Centers – Louisiana
