Using Questionnaires for Program Evaluation: Collecting High Quality Information

MLA Course
Listing Archived: Tuesday, January 9, 2018

Primary contact information...
NN/LM Evaluation Office, UW Health Sciences Library
PO Box 767671
Roswell GA , 30076
United States
Cynthia Olney is the primary contact.
Phone: 678-682-3864
Resource URL: http://nnlm.gov/training/questionnaires
Region: All

Description: This course outlines the basics of conducting surveys for program evaluation. Participants will learn how to write survey items using typical question formats while avoiding common errors; distribute questionnaires to maximize response rate; determine sample size using online calculators; learn strategies for avoiding and exploring response bias in survey data.

Experience Level: Beginning
CE Contact Hours: 4
Professional Competencies: Curriculum Design and Instruction, Research + Analysis and Interpretation
Subject: Assessment/Evaluation
Course Type: Face to Face, RML Offerings, Chapter Offerings

Educational Objective: Participants will be able to (a) determine when surveys are an effective method for program evaluation (b) structure questionnaire items to avoid common item-writing errors (c) use online sample-size calculators to determine optimal sample sizes (d) follow effective practices of survey distribution to maximize response rates (e) identify response bias is survey data

Agenda:

• Introduction to topic (30 minutes: Lecture and instructor/participant introductions)
• Planning a questionnaire: Presentation on planning a questionnaire for use in program evaluation, followed by a small-group exercise during which participants develop evaluation questions and determine whether a survey is the best approach to answering questions (60 minutes; lectures and individual exercise followed by group sharing)
• Item-writing: This section starts with an exercise in which participants work independently to identify problematic questionnaire items.  A presentation follows about typical item formats and avoiding common errors when writing survey items. The section concludes with large-group discussion about the exercise started at the beginning of this section. (75 minutes: Individual exercise, large group discussion, short lecture. A 15-minute break is incorporated into this section.)
• Presentation on sample size calculators: Participants learn when and how to use online sample size calculators to determine sample size for a survey. They also will learn about various sampling methods used in surveys. (30 minutes, demonstration and lecture)
• Presentation on response rate and survey distribution: Participants learn the importance of maximizing response rate and the methods that have been shown to improve response rates. They also learn how to explore response bias in their survey data. Brief discussion of various technologies, such as mobile device software or online polls, will be included. (45 minutes; lecture)
• Participants ask questions and complete evaluation forms. (15 minutes_

Need for This Course: Survey methods are one of the most commonly used approaches to program evaluation. The NN/LM OERC has presented a number of webinars throughout the NN/LM regions that cover topics that are incorporated into this workshop. In Fall 2011, a representative of the Louisiana Library Association who participated in two of the one-hour webinars presented in the South Central Region requested that we develop this six-hour workshop for their annual conference. The topics and this workshop have received very positive ratings, so we know it is a topic of interest and need among librarians.

The instructional methods used include Lecture, Demonstration, Slides, Discussion, Simulation, and Problem-based.

Participant Materials: Information handouts,worksheets for independent and small group exercises, and Powerpoint slides

Facility Requirements: The workshop requires a computer, data projector and screen for projecting Powerpoint slides. It is best held with a classroom-style set-up with chairs that can be arranged for both lecture and small group discussion. It should NOT be scheduled in a computer lab.

Additional Evaluation: The OERC includes a short open-ended evaluation form asking participants which concepts they intend to put into practice and what questions they still have about the topic that weren't covered. It includes two quantitative questions asking them if their confidence in using evaluation methods has increased and the likelihood that they will use the workshop information in the future.