clark

Back **Online Program Evaluation – Tom Clark** //This session is intended to increase awareness about the value and benefits of program evaluation and to strengthen its use in K-12 online learning programs. Program administrators and evaluators will share internal and external evaluation practices found to be effective in a variety of unique online learning environments, both in terms of helping improve quality and effectiveness and in meeting accountability needs. We will also hear from efforts underway by the NACOL Research Committee and the federal Office of Innovation and Improvement (OII) to raise awareness and disseminate effective practices. Participants will be asked to contribute their ideas on successful practices and lessons learned, and ways to strengthen evaluation practice.

After an overview and brief presentations on individual evaluation projects and on the multi-evaluation synthesis for a new guidebook developed by WestEd for OII, session participants will join presenters for round table discussion of successful practices and lessons learned. After a break, representatives of the NACOL Research Committee will present about their efforts to develop an Issue Brief that addresses one key issue, quality and effectiveness in K-12 Online Learning. Presenters and participants will then engage in round table discussion about next steps in strengthening evaluation practice, both at the program and policymaker levels.//
 * 1) //Overview//
 * 2) //Individual evaluations presentations, OII Innovations in Education guidebook//
 * 3) //Discussion: Successful practices and lessons learned//
 * 4) //NACOL research committee Issue Brief: quality/effectiveness in K-12 online teaching//
 * 5) //Discussion: Next steps for strengthening evaluation practices//

//Participants should take away from this session a better understanding of the value and benefits of evaluation, and how it might apply to their unique circumstances. Program administrators and staff will have renewed knowledge of proven methods and potential issues to keep in mind when undertaking an evaluation, while policymakers and funders will gain knowledge useful in framing evaluations that support improvement and reform efforts.//

PowerPoint Presentation -

Add your own notes for this session here by simply clicking the "Edit this Page" icon above, type your notes, and then click "Save".


 * Resources**


 * [|**Evaluating Online Learning: Challenges and Strategies for Success**] [| **from the US Department of Education**]
 * This publication features seven evaluations of online learning programs or resources. The evaluations represent variety both in method of evaluation and in the program or resource that was examined.

Group D - Feedback Tools
 * Structured Discussions**

From [|Connections]- Star Tracker system Online Lesson Analysis Built into LMS, lesson has 5 parts, at bottom of screen is a 5-star feedback tool with comment field Terrible > Great Teachers and Curriculum developers analyze

Pop-up Surveys with about 3 questions, course specific type questions

Communications Analysis Tool - [|Advanced Academics]

For Satisfaction Surveys get the Book - [|The Ultimate Questions]

D. User Feedback Tools-Embedding into program “Star Tracker” Star rating for Connections Classes 6-7 years Goal: Immediate lesson feedback Program’s curricular designers and teachers can access lesson data Data is reviewed Lesson by lesson within a course, at a grade level, in a curricular area For Elementary, what do parents think? Patterns: For example, is there a correlation between popularity of classes and effectiveness> The designers look at specific lesson and change them based on feedback and outcomes No way to know when the rating was entered, before or after student completion? ? pops up if rating timing is questionable. Ie, A student on lesson 10 rates lesson 25 Embedded in LMS. Young children work with “learning coach”-most heavily invested para-professionals- but not certified para-professionals They sign a learning coach agreement. Training is online and teachers available to learning coaches. 5 star rating: 1-Terrible, 5 – great, and text field for comments Is it the graphics, the difficulty of the lesson, or what? No orientation to describe expectations of ratings, but they feel that students are honest, for the most part. The rating is new, so they do not know if it is the same students/learning coaches submitting all evaluations, or a broad base of students rating the classes. High School class ratings are more course/curricular driven, Elementary are more affectively based. Learning coach can also rate the whole school. Learning coaches interact much more at the K-8 level that the HS level. They are considering student and learning coach ratings, also. Idaho tracks communication between teachers and students Pennsylvania 21st Century tracks time to complete courses and variance of time to completed The Ultimate Question Measures loyalty nor satisfaction End of year surveys, sometimes there is a low response rate despite incentives Pop-up surveys in courses-3 questions Challenges? How to differentiate between teacher issues and curriculum issues? Annual Parent satisfaction surveys measure satisfaction with teacher.