These are a few examples of the questions that can be answered through program evaluation. So often, however, we avoid evaluation for fear that we might find problems and/or because we believe that we don't have the capability to organize and implement a good evaluation. Many times we avoid activities that require facility with numbers, believing that evaluation requires complex statistical procedures. It's not as scary as you may think; you can do a lot with just a bit of planning and 5th grade math!
If you realize the importance of program evaluation, would like to conduct one, and lack confidence about what to do, resources are available to help you. The National Technical Support Center (NTSC) and America's Career Resource Network (ACRN) have prepared and posted numerous free downloadable resources to help you conduct an evaluation and use the results to improve programs. These resources include a description of the evaluation process, activities you can use to better understand the process, a PowerPoint training program for self development or for your use to train others, and additional web resources you can access to learn more about a specific evaluation area.
The evaluation process provided on this site is organized into four major categories of Planning, Development, Implementation, and Feedback. Within these four categories there are nine specific steps that, if followed, will guide you through a sequence of activities that will better ensure a valuable outcome. The nine steps are:
In a downloadable evaluation guide, you are given examples of both broad and narrow evaluation questions. You will receive advice on where you can find data that are already available to you without a special effort on your part, how to locate and evaluate commercial assessment instruments, and how to craft your own data collection instruments to include surveys, interviews, focus groups, and observational checklists. Considerations for using paper- or web-based data collection instruments are also discussed.
The resources offer a framework for a data collection action plan for organizing your evaluation. Following the action plan assures that you have determined what information is required to answer your evaluation questions, who will obtain the data, and how the data will be collected and analyzed. Examples for analyzing the data and presenting the results in graphic form are shown.
Once you look at the results and identify the findings, the resources on the website help you think through the dissemination process to include who should receive the evaluation results. Suggested target audiences for the evaluation findings include those clients or students that participated in the program, school personnel, program sponsors, colleagues, and others. Ideas on how to disseminate the evaluation findings through presentations, newsletters, websites, radio and TV interviews, blogs, and podcasts are offered for your consideration.
The final step in the evaluation process is to help you use the findings to identify the positive aspects and shortcomings of your program. Positive results should be celebrated. Aspects of the program that fall short of your goals and expectations give you opportunities to make adjustments in your program or intervention that will strengthen your offerings and serve your clientele better.
You can find links to several other web resources so that you can extend your knowledge and capabilities as you plan and implement your evaluation. The web resources are organized in categories as research reports, assessment resources, evaluation design, ethical guidelines, and data collection.
Recently, a Web cast has been released to help you understand the difference between research and evaluation, why one should conduct an evaluation, how to design and implement an evaluation, and how evaluations can be used to promote career development. A paper more fully outlining the details and concepts in evaluation process can be found at http://www.home.earthlink.net/~sagesolutions . Go to free downloads to access the document called Program Evaluation Model 9-step Process by Janet E. Wall.
Don't just guess that your efforts are worth it. The various resources provided will enable you to plan and implement a useful evaluation of your program or intervention.
Dr. Janet E. Wall is President of Sage Solutions, a small company that offers consulting services in the areas of career development, career and educational assessment, program evaluation, and technology. She is a certified Career Development Facilitator Instructor and author of six books related to career development or assessment, including What Do I Like to Do?: 101 Activities to Identify Interests and Plan Careers and Measuring Up: Assessment Issues for Teachers, Counselors and Administrators, both published by ProEd Inc., and Jobseekers Online Goldmine: A Step-by Step Guidebook to Government and No Cost Web Tools, published by JIST. She is a past president of the Association for Assessment in Counseling and Education and is an officer in the Maryland Career Development Association and the Association for Counselors and Educators in Government. Reach her at sagesolutions@earthlink.net.