District Data Team Toolkit
Helping districts establish, grow, and
maintain a culture of inquiry and data use.
MODULES
Introduction
Getting Ready
Inquiry
Information
Knowledge
Action
Results
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
District Data Team Toolkit, Study notes of Data Communication Systems and Computer Networks
The District Data Team Toolkit is a resource designed to help districts establish, grow, and maintain a culture of inquiry and data use. The toolkit provides a structure that takes data use from asking the right questions to getting results. It is an iterative process in which the district uses data to target and support continuous improvement. how the toolkit can help districts and schools gather and use data systemically for their own learning and to inform decisions about curriculum, instruction, resource allocation, and other vital functions at district and school levels.
Typology: Study notes
2021/2022
1 / 313
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download District Data Team Toolkit and more Study notes Data Communication Systems and Computer Networks in PDF only on Docsity!
District Data Team Toolkit
Helping districts establish, grow, and
maintain a culture of inquiry and data use.
MODULES
Introduction
Getting Ready
Inquiry
Information
Knowledge
Action
Results
District Data Team Toolkit—Version 1.0 Module 0: Introduction
INTRODUCTION
Welcome to the District Data Team Toolkit. This Toolkit is designed to help a district establish, grow, and maintain a culture of inquiry and data use that can inform decisions that impact teaching and learning, and ultimately improve the achievement of all students. This short introduction will help you understand and navigate the tools and resources available to support this work.
WHAT IS THE TOOLKIT?
Ensuring the effective use of inquiry and data district-wide involves many tasks and types of expertise. For that reason, this Toolkit is designed to assist in the establishment of a District Data Team—a cadre of staff who is collectively responsible for the technical, organizational, and substantive aspects of data use. These Team members must be data literate and able to lead the collaborative inquiry process with both district and school staff. Members of the District Data Team work with district staff and school-level data teams to:
Craft questions about accountability, equity, and continuous improvement Coordinate the collection, analysis, and dissemination of data displays that are necessary to address these essential questions
Build action plans Monitor progress of improvement initiatives
These activities can help build the capacity of a District Data Team to engage in inquiry and use data to inform district-level decisions. Over time, the Team can engage the entire staff in using multiple data sources to continuously improve teaching and learning throughout the district. Districts that engage with the Toolkit should plan for a multi-year commitment to increase and embed a capacity for effective data use.
The Toolkit is designed around a theory of action, the Data-Driven Inquiry and Action Cycle (see diagram) on the next page, which provides a foundation for effective data use. The Cycle provides a structure that takes data use from asking the right questions to getting results. It is an iterative process in which the district uses data to target and support continuous improvement. A disciplined application of this kind of data-driven approach can build a district and school environment that is focused on continuous improvement grounded in evidence. This Cycle is also the basis for the ESE Education Data Warehouse trainings, which further provide excellent tools to access and analyze data. But analyzing data alone will not result in continuous improvement. Concrete actions that are grounded in evidence and continually monitored through the collection and analysis of multiple forms of data are critical to achieve improved results.
Massachusetts Department of Elementary and Secondary Education 1
District Data Team Toolkit—Version 1.0 Module 0: Introduction
WHAT’S IN THE TOOLKIT?
The Toolkit has seven modules – this Introduction, and six modules aligned to the Data-Driven Inquiry and Action Cycle. Each module contains objectives, detailed information, tools, and resources to help implement the work.
1. Getting Ready module provides guidance on forming a District Data Team and taking steps to build a solid foundation for building a culture of inquiry and systemic data use. The module addresses the need for systems to inventory, collect, and disseminate the data. It also has information to promote assessment literacy and help manage change. 2. Inquiry module launches a process of inquiry by identifying high-level questions of importance to the district, generating clarifying questions to focus the inquiry, and identifying data needed to answer them. The module includes guidance for effective data displays and data overviews. 3. Information module guides the process of analyzing the data identified in Inquiry, first making factual observations about the data and then generating inferences about what the data mean. 4. Knowledge module helps place information (analyzed quality data) into the context of research and practice to accurately define the problem and identify possible solutions. 5. Action module provides frameworks for putting new knowledge to work by developing a logic model and articulating clear measures that will guide and focus the work, and then by creating an action plan (if necessary). If the information and knowledge gained from quality data analysis are not acted on, data collection efforts are wasted and improvement won’t occur! 6. Results module shares methods for monitoring the work, evaluating the impact, making mid-course corrections if necessary, and communicating the outcomes with stakeholders.
HOW SHOULD THE TOOLKIT BE USED?
The Toolkit is designed for district-level staff, to promote the skills and knowledge necessary to build their capacity to effectively use inquiry and data to inform district-level decisions. As a District Data Team gains comfort with the tools, resources, and processes in the Toolkit, it can plan ways to share them with school-and teacher-level data teams.
Each of the modules provides specific tools and activities to implement the steps of the inquiry process. Some tools are best used electronically. It is important to understand, however, that superimposing a process does not necessarily yield a positive result. A district must be mindful of doing what it can to embed a culture of inquiry and data use that goes beyond technical compliance with processes.
Massachusetts Department of Elementary and Secondary Education 3
District Data Team Toolkit—Version 1.0 Module 0: Introduction
HOW DO I NAVIGATE THE TOOLKIT?
No matter the entry point taken, the Toolkit is designed to support ready access to tools and resources as a District Data Team needs them, using a numbering system to make finding the right tool quick and efficient. 3.2.1T
Tool or Resource Module Activity (^) Number When an activity has multiple tools or resources associated with it, they are numbered sequentially and designated with either a T (tool) or an R (resource).
Tools are templates, protocols, organizers, or other items that the Team will work with to build its knowledge and expertise.
Resources provide further reading, documentation, or guidance to help the Team use the tools or expand its understanding.
Activity 0. 1 Overview of Toolkit Contents
These documents summarize elements of the Toolkit in order to assist a district in engaging with ones that will be most useful to its work.
(0. 1 .1R: Objectives for All Modules) (0. 1. 2 R: Tools and Resources for All Modules)
WHERE SHOULD OUR DISTRICT BEGIN? Districts should begin by engaging with the District Data Team Self-Assessment , at the end of this Introduction module, which will help determine the district’s strengths and areas of need in regards to data use and a culture of inquiry. The Self-Assessment aligns with each of the modules and therefore can help the district identify where to begin engaging with the Toolkit – which modules, tools, and resources would most help the district move forward with its work.
If in doubt, a district might gain the most value from starting in Module 1: Getting Ready and working through the Toolkit sequentially, committing to a multi-year process of building robust use of data at the district level. If the District Data Team has been in existence for several years and needs to work on refining processes and policies that support data use, it may find it useful to go directly to certain tools in the Toolkit.
Activity 0. 2 District Data Team Self-Assessment
This self-assessment can help a district determine its strengths and needs, and how best to use this Toolkit to support inquiry and data use.
(0.2.1T: District Data Team Self-Assessment)
4 Massachusetts Department of Elementary and Secondary Education
OOBBJJEECCTTIIVV^ EESS^ FFOORR^ AALLLL^ MMOODDUULLEESS^00 ..^11 ..^11 RR
Purpose (^) To provide an overview of the objectives for all modules in the District Data Team Toolkit. Description In conjunction with the Introduction and Self- Assessment , this summary can help a District Data Team identify the parts of the Toolkit that might be most useful to its work. Time N/A
Related Documents 0–Introduction 0.1.2R: Tools and Resources for All Modules
Module 1 (Getting Ready) will help a district: Set the vision for data use across the district Build a culture of inquiry to promote systemic data use Establish a District Data Team to drive this work Build data literacy Understand types of data that inform inquiry Establish systems and policies to inventory, collect, and disseminate data Manage the change process
Module 2 (Inquiry) will help a District Data Team use the above roles and vision to: Formulate questions to drive an inquiry process Create and present effective data displays and data overviews Identify the data needed to answer the questions
Module 3 (Information) will help a District Data Team use the above questions and data to: Collect and organize data relevant to the inquiry process Distinguish between observations and inferences Make inferences from multiple sources of data
Module 4 (Knowledge) will help a District Data Team use the inferences generated above to: Clearly articulate a problem statement Identify and explore root causes of the problem Cross-reference solutions with research and local knowledge Begin to capture information on the district’s improvement efforts
Module 5 (Action) will help a District Data Team use the knowledge generated above to: Craft a logic model or theory of action to guide subsequent action and evaluation Articulate meaningful measures of implementation and change Develop action plans, if necessary, to implement new strategies or to implement existing strategies more effectively
Module 6 (Results) will help a District Data Team use the action plan generated above to: Decide what to evaluate Develop an evaluation plan Analyze evaluation data Identify and develop a communication strategy Continue the process of inquiry
0.1.1R: Objectives for All Modules—Version 1.0 1/
TTOOOOLLSS^ AANNDD^ RREESSOOUURRCCEESS^ FFOORR^ AALLLL^ MMOODDUULLEESS^00 ..^11 ..^22 RR
Purpose To provide an overview of the tools and resources available in all modules of the District Data Team Toolkit. Description In conjunction with the Introduction and Self- Assessment , this list of tools and resources can help a District Data Team identify the parts of the Toolkit that might be most useful to its work. Time N/A
Related Documents 0 – Introduction
- 1 .1R: Objectives for All Modules
Module 0: Introduction
- 1 .1R: Objectives for All Modules
- 2 R: Tools and Resources for All Modules
- 2 .1T: District Data Team Self-Assessment
Module 1: Getting Ready (Organize for Success)
1.1.1T: Functions of a District Data Team 1.2.1T: Barriers to Effective Data Use 1.3.1T: Vision for Data Use 1.4.1T: Norm Setting Protocol 1.4.2T: Data Team Meeting Agenda 1.4.3T: Data Team Meeting Minutes 1.5.1T: Data Inventory Template 1.5.2T: Data Inventory Template: SIMS and EPIMS Data
1.5.3R: ESE Data Resources 1.6.1T: Data Collection Self-Assessment 1.7.1T: Data Dissemination Schedule Template 1.7.2R: Data Dissemination Schedule Example 1.7.3R: ESE Policies for Data Access 1.8.1T: Data Literacy Training Catalog 1.8.2R: Assessment Glossary 1.9.1T: Managing Change and Understanding Concerns Protocol
Module 2: Inquiry (Ask the Right Questions)
2.1.1T: Question Formulation Protocol 2.4.2R: Data Display Rubric 2.2.1T: Inventory of District and School Initiatives 2.4.3R: Types of Data Displays 2.3.1T: Data Overview Checklist 2.4.4R: More Data Display Resources 2.3.2R: Data Overview Example 2.5.1T: Data Overview Brainstorming Protocol 2.4.1T: Building Data Displays Protocol 2.5.2T: Focusing Question Investigation Template
Module 3: Information (Collect and Analyze Data)
3.1.1T: Data Collection Planning Tool 3.2.1T: Practice Making Valid Inferences 3.3.1T: Data Analysis Protocol
Also revisit 2.4.1T: Building Data Displays Protocol
- 2 R: Data Display Rubric 2.4.3R: Types of Data Displays 2.4.4R: More Data Display Resources
0.1.2R: Tools and Resources for All Modules—Version 1.0 1/
Module 4: Knowledge (Make Meaning of Data)
4.1.1T: Writing a Problem Statement 4.2.1T: Why, Why, Why? 4.2.2T: 20 Reasons 4.2.3T: Fishbone Analysis 4.2.4T: Graphic Representation
Module 5: Action (Apply Meaning)
4.2.5T: Dimensions Bulls-Eye 4.3.1T: Problem Investigation Plan 4.3.2R: Educational Research Websites 4.4.1T: Problem Catalogue Template
5.1.1T: Logic Model Template 5.2.2R: Elements of a Well-Written Measure 5.1.2T: Logic Model Checklist 5. 3 .1T: Action Plan Template 5.2.1T: Crafting Meaningful Measures Checklist 5. 3 .2T: Action Plan Checklist
Module 6: Results (Look for Improvement)
6.1.1T: Evaluation Plan Template 6. 2 .2T: Building Data Walls
- 2 .1T: Communication Organizer 6. 2. 3 T: Evaluation Report Template
For more information on this and other district support resources, or to share feedback on these tools, visit http://www.doe.mass.edu/sda/ucd/ or email [email protected].
0.1.2R: Tools and Resources for All Modules—Version 1.0 2/
District Data Team Self-Assessment
Background: This tool is designed to give a District Data Team an indication of its strengths and challenges in a variety of areas related to promoting a district-wide culture of inquiry and data use. The self-assessment is comprised of six short surveys which are aligned to each of the six modules in the Toolkit and to the six steps in the Data-Driven Inquiry and Action Cycle. Each survey has a number of selected-response questions grouped by Data Team practice. The possible responses are described in the rubric below.
1. Print each page of this self-assessment (including this page so that the rubric is readily available to anyone taking the survey) and provide a full copy to each member of the group. 2. Individually complete the survey, assigning a rating from the rubric below to each indicator. 3. As a group, discuss each page of the survey and agree on a rating for each indicator. It is not necessarily best to average the individual scores to get this final rating. If responses among individuals vary widely, engaging in a discussion about which rating best represents the level of practice can help the Team begin the hard work of developing a common understanding of the work. 4. Enter the final rating for each indicator into the spreadsheet version of this survey. 5. Print out the Graphs page (or use a projector to display it on the wall), and as a group talk through the discussion questions for each graphical display.
Rubric for Assessing Each Practice:
0 No Knowledge Respondent/Team has no knowledge about this indicator and cannot provide a judgment about its existence in the district.
1 No Evidence (^) There is no evidence that this indicator is in place within the district.
2 Emerging Evidence
There is some evidence of this indicator in the district, but the evidence indicates that the practice is far from standard procedure and has clear room for improvement in both quality and frequency.
3 Adequate Evidence
This indicator has clear evidence of existence in the district and is consistently practiced in many places. There is room for improvement in either quality or frequency.
4 Exemplary Evidence
This indicator is evident in a variety of ways throughout the district. The practice described is clearly a part of the district culture and the way people operate within the district.
0.2.1T: District Data Team Self-Assessment 2
Interpreting the Data: When the Team completes the six surveys, the data will be displayed in a series of charts with questions to guide your review. There are two types of displays.
- A radar chart showing your Team's perceptions of strengths across each of the six modules.
- A horizontal bar chart for each step displaying the strengths within the step by practice.
A radar chart, also known as a spider chart or a star chart because of its appearance, plots the values of each category along a separate axis that starts in the center of the chart and ends on the outer ring. In the example below, each step of the Data-Driven Inquiry and Action Cycle is plotted on an axis. This makes it possible to compare your survey results across the steps. When your surveys are complete, your results will be displayed in a ring plotted on the chart. A district performing consistently across all of the steps will be displayed in a near circle. A district performing higher in some steps than others will be displayed in a more free form shape.
0.2.1T: District Data Team Self-Assessment 3
0
0 0
0.
0 0 0 0 0 0
0.
0
0
0
0.
0
0
0.
0
0
0.
Getting Ready (^) Incomplete Rating Scale: 0 = No Knowledge; 1 = No Evidence; 2 = Emerging Evidence; 3 = Adequate Evidence; 4 = Exemplary Evidence
Vision for Data Use (^) Rating: The district has a vision for data use that aligns with and furthers the wider district mission and vision.
The district’s vision for data use is widely understood and accepted by all stakeholders.
The vision is supported by district policies and published expectations that support the use of inquiry and data for instructional, school, and district improvement. Practice Average:
District Data Team Rating: The district has an established District Data Team or has designated another team to fulfill those functions. The District Data Team has a data champion, a data manager, and additional members with the range of skills and perspectives needed to address the functions of the Team. The District Data Team has a clear sense of its purpose and role in furthering the district's vision for data use. The District Data Team has a written plan that outlines the membership, roles, and responsibilities of the Team, and this is publically communicated. The District Data Team uses effective team practices (starting and ending on time, appointing a moderator, following an agenda, appointing a note-taker, and clearly communicating regular meeting times). The District Data Team accomplishes its tasks effectively (action items are clearly noted with the people responsible and timelines for completion; Team communicates with all stakeholders). Practice Average:
Assessment Literacy (^) Rating: District Data Team members are fully fluent in and have a shared understanding of contemporary, standards-based assessment terms and concepts, e.g., summative assessments, formative assessments, performance levels. District Data Team members know what assessments are in use in the district and understand each assessment's purpose. Members of the District Data Team actively support the development of assessment literacy in district colleagues, school data teams, and others through professional development offerings, coaching, and modeling. Practice Average:
Data Technology(ies) Rating: Resources, e.g., user manuals, job aids, IT support, for data technology(ies) exist and are easily accessible to district and school users. The district has technology(ies) for collecting and combining data from multiple sources, e.g., demographics, grades, assessment data, attendance. Practice Average:
Data Inventory Rating: The district has a comprehensive inventory of demographic, assessment, and other data available to inform improvement plans and instructional practices. The data inventory is published and available to all, along with a regularly updated schedule to communicate how and when each type of data becomes available. Practice Average:
0.2.1T: District Data Team Self-Assessment 5
0
0
0 0
0.
0 0
0.
0
0.
0.
Data Collection Rating: The District Data Team solicits feedback from stakeholders to identify opportunities for improvement in methods of data collection, management, and reporting. The District Data Team regularly uses established methods of communicating and coordinating with stakeholders at all levels regarding data work. Protocols or procedures are regularly used to ensure accuracy of data.
The District Data Team collects and reviews data from non-electronic sources, such as Learning Walkthroughs, Common Planning Time (CPT) Self-Assessment, and interviews with principals and other staff. Practice Average:
Data Access Rating: The district has clear policies for data access that conform to state and federal requirements.
The District Data Team has knowledge of what data reports are disseminated to which stakeholders, when, why, and what actions are taken as a result. Practice Average:
Change Management Rating: District Data Team members are able to identify and address potential resistance to change before problems occur. Practice Average:
Getting Ready Composite Average:
0.2.1T: District Data Team Self-Assessment 6