Download Study Materials for Program Evaluation - Spring 2008 | ITEC 8435 and more Exams History of Education in PDF only on Docsity!
ITEC 8435
Program Evaluation
Spring 2008
Professor: Dr. Randal D Carlson Department: Leadership, Technology, & Human Development Office: 3101 Education Building Telephone: 912-681- Office Hours M-F: 10-11, Other times by appointment Fax: 912-486- Email: [email protected] WWW: http://coe.georgiasouthern.edu/lthd/ Text s: Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation Alternative approaches and practical guidelines (3rd^ ed.). New York: Longman. (W,S,&F) Guiding Principles Success includes the creation of an environment where teachers bring students from diverse groups to high levels of learning. Success requires that graduates: use data on student performance to effect necessary instructional change. reduce the effect of environmental and institutional barriers that impede student academic success. implement instructional changes designed to close the achievement gap between students from high and low income groups and from minority and majority groups. work effectively with students, teachers, parents, and community. Course Subject Matter Content: This course has been designed as a follow-on course to ITEC 7430, Instructional Design. In it we will examine in some depth two of the steps in the Instructional Design model that have very broad application in schools and other learning situations -- Evaluation, both Formative and Summative. This is intended to be a hands-on class that reviews existing knowledge in systematic design processes, deepens candidate understanding of “back end” analysis, and helps the candidate to apply these processes in context. Candidates who have not taken a first course in Instructional Design may draw on their previous knowledge of curriculum design, assessment, and evaluation. Course Objectives:
- Describe an educational system.
- Describe fundamental Evaluation Models.
- Conduct evaluations that are designed to enhance instructional effectiveness.
- Design, investigate, analyze, and report on the effectiveness and efficiency of programs using a systematic research methodology.
Standards AECT Standards addressed in this course:
1. Design. Candidates demonstrate the knowledge, skills, and dispositions to design conditions for learning by applying principles of instructional systems design, message design, instructional strategies, and learner characteristics. 4. Management. Candidates demonstrate knowledge, skills, and dispositions to plan, organize, coordinate, and supervise instructional technology by applying principles of project, resource, delivery system, and information management. 5. Evaluation. Candidates demonstrate knowledge, skills, and dispositions to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning. Conceptual Framework COE Commitments addressed in this course: 1. Commitment to the knowledge and dispositions of the profession 2. Commitment to diversity 3. Commitment to technology 4. Commitment to the practice of continuous reflection and assessment Administrative Comments I first taught this course via the internet in 2001. The delivery mode has changed somewhat since that first time and it will again this semester. But, the fundamental processes remain stable & the book has not changed since I taught it in Spring 2006. You will see some of the “old” material in this delivery, notably the “Impaticized” Powerpoints that accompany most of the learning modules. These learning aids are an expansion of the material in the book, so if you are the type that reads & applies the material readily to your personal context, then these learning aids will be less valuable to you than for someone who may need to see what I think are the important points in the lesson. The ‘plain” PowerPoints are available for those who simply need an outline of what I believe to be the most the important material. The first week, we will decide if the class or some subset of the class wants to have periodic meetings via the Vista chat capability. Vista (WebCT): The Vista course management system will be used. Candidates will have access to all course materials (except the textbook), assignments, quizzes, and supplemental information here. Candidates must login to Vista and complete the required information. Most of you are familiar with Vista, as it is used at Georgia Southern. For those who aren’t, you will access it through MyGeorgiaSouthern. We will use the Live Classroom conferencing capability contained in Vista. This will be new to most of you & may require some computer setup. To audioconference, you must have a microphone & to videoconference, you need to obtain & install a web camera. See http://academics.georgiasouthern.edu/etc/emTech/liveclassroom-requirements.php for requirements and setup. If you are having trouble with Vista, contact the Help Desk at the Emerging Technology Center: Email: [email protected] Phone: (912) 486- IM: AIM – GSUWebCT Yahoo!M: gsuwebct
Graded Course Activities & Specifications You will be able to show the breadth and depth of your knowledge of the course materials in four different ways. Due dates for all work except the weekly quizzes are bolded on your schedule. Weekly quizzes are generally due on the 7th^ day following the assigned reading (the day of the next assigned reading). Each quiz will say when it must be completed. Read the requirements for each graded activity. If you have questions, please ask. If not, I will assume that you have read and understand instructions. NOTE: A component of every graded activity is spelling and grammar. Every document should be prepared using a word processor. Graded Components
- Weekly Quizzes 20%
- Class Participation 20%
- Evaluation Project 60% The Weekly Quizzes are primarily multiple choice, but may include some short essay questions, as well. They will be administered through Vista. They will be available the week before the assigned reading. Each week I will post a discussion question on the Vista Discussion Board. The Class Participation of each candidate will be graded at the end of the semester. It will be cumulative and include both the number of postings that you make and the quality of your thoughts. I will select three weeks from the course of the semester and assess your participation for those weeks. For example, I may assess everyone’s participation for Week 2, Week 7 and Week 9. I am not going to tell you in advance if it’s an assessment week. See the Online Class Participation Rubric later in the syllabus. The Evaluation Project will be to design and accomplish an evaluation of a program at an actual location (your school or another appropriate activity). The objective of this project is to have you demonstrate that you can perform the important activities involved in the model that is outlined in Fitzpatrick, Sanders, & Worthen. There is no need to accomplish an extensive project to complete this assignment. The important factor is that you demonstrate that you understand the concepts and can carry out a simple project properly. You may have to make assumptions about your project as you develop your solution. Document those assumptions as you make them and include them as a part of your project. This is designed to be a group project. You will work in groups of two. Candidates are encouraged to evaluate one of the programs at their school connected with AYP and to use the most recent data available, however other projects are acceptable for candidates that do not work in K-12 public schools. Use the most recent data available in your analysis. The course grade will be based on the following scale: 91 - 100 = A 81 - 90 = B 70 - 80 = C 60 - 69 = D 0 - 59 = F
The instructor's scale represents the following standards set forth in the GSU Catalog: The "A" grade may be interpreted to mean that the instructor recognized the candidate's exceptional mastery of the course material. The grade of "B" indicates that the candidate has demonstrated an above average mastery of the course material. The "C" grade is the instructor's certification that the candidate has demonstrated minimal mastery of the course material. The candidate is graded "D" or "F" when his/her grasp of the course is unacceptable. Timely Submission of Required Work: Promptness is expected in this course. Graded work submitted by 2359 (11:59 pm) the day it is due is counted as being on time. I will not accept late work. Exceptions will require documentation of the excuse. I will be more likely to approve an exception if asked before the work is done, however I expect exceptions to be rare. Academic Integrity This is of the highest importance in the academic community and at Georgia Southern University. I expect you to conduct yourself with complete honesty and integrity throughout the course of this class. I will accept nothing less. The Student Conduct Code defines academic misconduct. Confirmed cases of academic misconduct will be dealt with according to the guidelines set forth in the Student Conduct Code and related directives. Candidate Notification Policy All Georgia Southern University candidates are provided with an email address within 24 hours of candidate registration. Email is the official means of communication between the University and the candidate. It is the candidate's responsibility to check his or her email each school day for administrative messages. There is no justifiable excuse for failure to respond to a University communication or failure to act on a University communication in a timely manner. Disability Accommodations : Georgia Southern University is committed to providing an equal educational opportunity to qualified candidates with disabilities. The Student Disability Resource Center (SDRC) is the primary source of services for these candidates. Candidates with an array of disabilities are eligible for services; however, documentation standards exist for all conditions. For further information contact the SDRC at 871- 1566.
Online Class Participation Rubric
I will select three weeks from the course of the semester and assess your participation for those weeks. For example, I may assess everyone’s participation for Week 2, Week 7 and Week 9. I am not going to tell you in advance if it’s an assessment week. On the other hand I’m not going to do the assessment five minutes after the week officially ends. DO NOT GO BACK and try to improve the quality of posts for previous weeks!!! Rating Scale Minimum points possible: 12 Maximum points possible: 36 35 points and above earns a rating of 100 32 and above earns a rating of 90 28 and above earns a rating of 80 25 and above earns a rating of 70 24 points and below earns a rating of 0 Adapted from two rubrics available online at : http://community.flexiblelearning.net.au/TeachingTrainingLearners/content/article_6959.htm
Rating Rating Period
Category Drifting (1) Moving in the Right Direction (2) Valuable Performance (3) Week 1 Week 2 Week 3 Promptness and initiative Does not respond to posts; rarely participates freely Responds to some posts; shows limited initiative in advancing the discussion Regularly responds to posts; posts advance the discussion to new areas Integration of readings and resources No reference to readings or other resources Some mention of readings and other relevant resources Readings and resources clearly read and used in constructing posts Expression Numerous errors make post difficult to understand Occasional errors in usage Posts are clearly constructed and presented with few errors Synthesis and Reflection No evidence of synthesis or reflection Post includes some reflection but little synthesis of the resources used Posts are thoughtful and reflect synthesis of a wide range of materials; reflection on the topic is evident Your Score
Assignment: Evaluation Project NCATE Assessment 7 (Non-certification Masters) Course: ITEC 8435 The Evaluation Project is to design and accomplish an evaluation of a program at an actual location (a public school or another appropriate activity). The objective of this project is to have candidates demonstrate that they can perform the important activities involved in the model that is outlined in the text. There is no need to accomplish an extensive project to complete this assignment. The important factor is that the candidates demonstrate that they understand the concepts and can carry out a simple project properly. Candidates may have to make assumptions about the project as they develop the solution. Candidates should document assumptions as they are made. This is designed to be a group project. Candidates work in groups of two. Candidates are encouraged to evaluate one of the programs at their school connected with AYP and to use the most recent data available, however other projects are acceptable for candidates that do not work in K-12 public schools. Components of the assignment:
- Statement of generally what is to be investigated.
- Evaluability assessment.
- Identification of the evaluation questions, criteria, and standards.
- Completed evaluation plan. An appropriate submission would be a paragraph or an outline addressing the appropriate planning items.
- Finished report. NCATE/AECT Standards Addressed: 1.1.5.a Utilize a variety of assessment measures to determine the adequacy of learning and instruction. 1.1.5.b Demonstrate the use of formative and summative evaluation within practice and contextualized field experiences. 2.0.5 Apply appropriate evaluation strategies and techniques for assessing effectiveness of instructional and professional products. 4.1.1 Apply project management techniques in various learning and training contexts. 5.1.1 Identify and apply problem analysis skills in appropriate educational communications and instructional technology (ECIT) contexts (e.g., conduct needs assessments, identify and define problems, identify constraints, identify resources, define learner characteristics, define goals and objectives in instructional systems design, media development and utilization, program management, and evaluation). 5.2.1 Develop and apply criterion-referenced measures in a variety of ECIT contexts. 5.3.1 Develop and apply formative and summative evaluation strategies in a variety of ECIT contexts. Conceptual Framework Outcomes: Commitment to the Knowledge and Dispositions of the Profession. Candidates use theory to establish the boundaries within which actual evaluations are conducted. Commitment to Diversity. Program evaluation requires that various constituencies or stakeholders be considered when developing the evaluation question, design, and plan. Evaluators are themselves chosen because of a particular characteristic of the stakeholders or themselves. Candidates must be sensitive to these needs, so that the can formulate and implement an appropriate evaluation plan.
Evaluation Project Rubric
ITEC 8435
Element Unacceptable Acceptable Target Your Score/Commen ts Rating 1 2 3 Statement of generally what is to be investigated. 5.3. (x1) Statement is inappropriate, not relevant, or poorly focused. Questions are either not present, not appropriate, or show little thought about the context. Statement proposed may lead to a useful evaluation. Description is very general in their nature. Questions are generic. Statement promises to lead to a valuable evaluation. Questions are specific and focused. Evaluability assessment. 5.1. (x2) Assessment addresses inappropriate areas. Incomplete. Assessment topics addressed but incomplete. Thorough assessment should lead to appropriate evaluation. Identification of the evaluation questions, criteria, and standards. 2.0.5, 5.2. (x2) Questions/criteria/ standards do not elicit thoughtful responses or are biased. Questions/criteria/ standards should elicit desired information Questions/criteria/ standards exhaustive & complete. Completed evaluation plan. An appropriate submission would be a paragraph or an outline addressing the appropriate planning items. 4.1. No plan or plan very sketchy. Significant persons/groups excluded. No plan , but list of questions/persons to contact should elicit desired information. Plan reflects well thought-out approach to investigating and collecting a variety of pertinent responses.
(x3)
