! !
!
!
!"#$%&!"%''()!*$+&,+-#'!./&01'2#&2!
34#'1#2+/&!56+,2+2+/107!*$/-/0#'!! regarding
!
89%!:%2%$;+&+&<!=&02$1,2+/&#'!*1$-/0%0! 5:=*7!8$#+&+&<!*$/<$#;! submitted to
6#$!>%02!?#@/$#2/$(!A/$!3B1,#2+/&#'!C%0%#$,9!#&B! :%4%'/-;%&2!
Ed Tech 505 July 16, 2010 !
!
!
! Introduction This document has been created and submitted on behalf of Bear Mountain Consulting, in response to the Request for Proposals (RFP), recently issued, by Far West Laboratory (FWL) for Educational Research and Development. The document is a proposal to provide the evaluation of a newly released training program called Determining Instructional Purposes (DIP). It will be conducted to provide information and recommendations 1) regarding distribution of DIP learning units, and 2) regarding purchasing and using DIP learning units in the future to administrators and grad students. Proposed Program Description FWL developed The DIP training package with a series of three units to train school administrators and graduate students. The training is intended to provide the professionals with planning and administering skills in effectively delivering educational programs. The DIP training program (print form) is comprised of the following: NAME COST Coordinatorâ&#x20AC;&#x2122;s Handbook $4.50 each Training Units $8.95 each or (Self-contained, Step-by-Step) (155 â&#x20AC;&#x201C; 259 pages) $24.95 for set of 3 (Use individually or in combinations.) 1. Setting Goals 2. Analyzing Problems 3. Deriving objectives. The Coordinatorâ&#x20AC;&#x2122;s Handbook provides details on the procedures including directions for the coordination of the three units. Each of the three units has 4-6 smaller modules within it, providing a limited number of instructional objectives regarding training. Typically each module has reading material on the skills taught, and includes practice activities related to the skills, provided either as individual or small group activities with trainee role-play and feedback. The estimated training time ranges from 10-18 hours for each unit, with unit 2 requiring the longest training time. A stipulation is that the training be both guided and facilitated by a program coordinator. It is necessary for the successful program coordinator to have worked through the training units, before commencing the facilitation. It is not necessary for the program coordinator to have any outside training or prior knowledge. Method of Evaluation Goal To provide feedback on whether or not FWL would benefit, from the capital it invests, to create and market the DIP training program package. Objectives: 1. Effectiveness: - To measure and identify module effectiveness. 2. Quality: - To assess module quality. 3. Implementation: - To measure implementation of program materials by school districts. !
3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! L!
!
!
! Audience & Evaluation Model The FWL stakeholder committee is comprised of the following members: board of directors, public shareholders, administration, and staff. All of the stakeholders will continue to be included within the evaluation process. (To ensure that the needs of everyone involved are addressed.) The decision-based evaluation model has been selected as the most effective design model to use for this particular evaluation. Dr. Driver and stakeholders from the FWL committee (as mentioned above), along with the Bear Mountain Consulting team have selected this model, based on the desired outcomes of the evaluation. Information Required , Sources & Recommendations Looking at each of the three objectives individually: effectiveness, quality, and implementation, the tables (below) detail the evaluation plan. Effectiveness EVALUATION Do the modules provide effective training? QUESTION RECOMMENDED IN THE TO MEASURING METHOD FORM OF DETERMINE Pre tests & - Entry level -The outcomes of Quantitative Post tests
Qualitative
Video or audio recorded interviews (before the pre tests) &
Video or audio recorded interviews (after all the tests)
!
knowledge/skills - Gained knowledge/skills from taking DIP training. - The attitude of clients/participants before training.
each unit
- The attitude of clients/participants after training.
- The attitude of clients/participants after training.
-The outcomes of each unit - The attitude of clients/participants before training.
DETAILS -100 % of clients/participants tested. -100 % of clients/ participants tested. - 100 % of clients/participants formally interviewed. - 10% of administrators, coordinators, and grad students randomly sampled. - 100 % of clients/participants formally interviewed. - 10% of administrators, coordinators, and grad students randomly sampled.
3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! M!
!
!
!
The evaluation team of Bear Mountain Consulting will create, conduct, and administer the measurements. They will be conducted via computer. The evaluation team will compare and contrast the results of the pre and post test data. This will provide the team with data indicating the knowledge/skills gained by the client participants in taking each of the training units. The results will be provided to the FWL stakeholders. All interviews will be recorded either face-to-face or via teleconferencing. The recordings will be saved, and provided to FWL if requested, to ensure quality of evaluation is upheld and maintained. The interviews will be conducted either face-to-face or via teleconference. The information once collected, will provide the evaluation team with an attitude assessment, before and after training, based on the trainings effectiveness. Quality EVALUATION QUESTION RECOMMENDED METHOD Qualitative
Do the modules provide quality training? IN THE FORM OF - On- site observations - On-site Survey (short Likert- 5 point) - Recorded teleconferencing interviews (video or audio) (short)
MEASURING/ DETERMINING
DETAILS
-The quality of the training procedures. -The attitude at the end of the unit. -The impact of the training. -The quality of the training procedures. -The attitude at the end of the unit. -The impact of the training.
- 1 evaluator attends min of 1 training session. -Random sample. 10% of all including participants, admin. & program coordinators. - Participant/clients
To determine the quality of the DIP program, the evaluation team of Bear Mountain Consulting will create, conduct, and administer the quality measurements. The evaluatorâ&#x20AC;&#x2122;s will have read the Coordinatorâ&#x20AC;&#x2122;s Handbook, particularly focusing on the procedures, previous to attending the on-site observation training session. This will ensure that he/she is qualified to conduct the evaluation. All of the qualitative quality data will be collected, cataloged and graphed. This information will also be presented (and recorded interviews saved) to the stakeholders of FWL.
!
3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! N!
!
!
!
Implementation EVALUATION QUESTION RECOMMENDED METHOD Qualitative
Quantitative
Are the modules ready to be disseminated? IN THE FORM OF
MEASURING/ DETERMINING
DETAILS
- Pre learning computersurvey (Likert 5 point scale) - Post learning computersurvey (Likert 5 point scale) - Interviews - Observations video recorded.
- Perceptions and attitudes of what may be gained from this program. - The impact the program had on the participants.
-100% of individuals directly involved with the program.
-Checklist (Looks at whether obj. were met)
-As above. -How to properly implement the program. -Findings
-100% of individuals directly involved with the program - 100% of indiv. directly involved with the program -Evaluator will observe a min of 1unit. -Evaluator will tabulate all findings for each unit within the training session.
The evaluation team will create, conduct, and administer the implementation measurements. The results of the surveys will indicate how willing people are to participate in the program, as well as what the impact of the program was. This will then provide information in the decision making process of whether or not to proceed with the DIP program, or choosing a better alternative, and how to implement the program in the future. The collected quantitative data will provide information as to usability of the program as well as the confidence levels in using the coordinator handbook. This checklist will provide feedback to the team as to the proper implementation of material. The evaluators will video record the observation sessions to ensure validity. They will also make all of the implementation data available to the FWL stakeholders. BY Task Schedule 1. Meeting - stakeholders + evaluators, discuss & change. BMC 2. Present everything in plan for review. BMC 3. Provide feedback on #2. FWL 4. Revisions of #2 - Final copy submitted. BMC 5. Delivery of evaluation tools to participants. BMC 6. Evaluators have attended min. 1 training session. BMC 7. Evaluators observe unit in real time. BMC 8. Administration of surveys, assessments, and interviews. BMC 9. Collect data. BMC 10. Summarized data submitted to shareholders. BMC 11. Feedback from initial findings of complied data. FWL 12. Completed/ Final report to FWL. BMC Note: FWL = Far West Laboratory, BMC = Bear Mountain Consulting !
COMPLETED BY Start: Sept 7, 2010 Oct 4, 2010 Oct 18, 2010 Oct 29, 2010 Nov 19, 2010 Dec 3, 2010 Dec 17, 2010 Dec 17, 2110 Jan 31, 2011 March 31, 2011 April 30, 2011 End: May 31, 2011
3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! O!
!
!
! Project Personnel Bear Mountain Consulting provides the services of a highly skilled team of experts (http://edtech2.boisestate.edu/kellyk/505/our_team_of _experts.docx) with 30+ years of experience, and a rich history of diversified success in the field of program evaluation. The Bear Mountain Consulting team provides full service program evaluation, rigorous analysis of existing qualitative data, and training in qualitative methods and analysis. Each member of the team is passionate about his/her profession, focused on innovation, and dedicated to providing high value solutions to his/her customers. All associates hold either a doctorate or masters degree, and have a minimum of five years experience relevant in the project area to which they may be assigned. In addition, Bear Mountain Consulting can facilitate the assistance of native speakers of French, Spanish, Hindi and Thai. To learn more about our company’s philosophy, services, our commitment to you, the credential of each of our team members, and what our clients have to say about us, check out our web page: http://www.bearmountainconsulting.ca. Fee s
(Computers will be used to cut down on costs and save time.)
TOPICS Personnel Salaries Karen Kelly, Principal Consultant Alejandro Velasco, Associate Habiba Chopras, Associate Mano Srisai, Associate Secretarial Staff TOTAL PERSONNEL Travel • 2- round trip airfares • Car rental and fuel • Food and Lodging TOTAL TRAVEL AND / DIEM Supplies • Materials & Supplies • Printing TOTAL SUPPLIES Communication • Telephone & Videoconferencing • Courier Services TOTAL COMMUNICATION ESTIMATED TOTAL FEES
DETAILS
FEE CHARGED
50 days at $300/day 30 days at $200/day 30 days at $200/day 50 days at $150/day 25 days at $100/day
$ 15,000.00 $ 6,000.00 $ 6,000.00 $ 7,500.00 $ 2,500.00 $ 37,000.00
Wildlife – Vancouver
$ 600.00 $ 500.00 $ 600.00 $ 1700.00 $ $ $
500.00 450.00 950.00
$
800.00
$ $
120.00 920.00
$ 40, 570.00
Thank you! Our team looks forward to working with you!
!
3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! P!