CAEP Team Data Meeting Agenda via Microsoft Teams
February 18, 2022
8:30 AM
Objectives:
● Institute a formalized framework to establish and monitor inter-rater reliability (IRR) of EPP- created key assessment instruments as a measure of fidelity of implementation (FOI)
● Apply the framework to collaboratively conduct the first iteration of the IRR process
● Discuss recommended changes to EPP- created key assessment instruments or directions to improve the fidelity of implementation
I. Greetings and Opening Comments
II. Overview (Whole Group)
A Review Objectives
B Review today’s IRR process
1. Review directions and rubric (breakout rooms)
2. Review submission (individuals)
3. Evaluate submissions using the rubric (individuals)
4. Discuss outcomes (breakout rooms)
5. Calculate IRR score (breakout rooms)
6. Record IRR score (breakout rooms)
7. Discuss problems and conclusions (breakout rooms)
8. Make and record recommendations (breakout rooms)
C. Q&A
D Assign breakout rooms and room captains
1 Room 1
a) Johnson (Captain) b) Russell c) Mayo-Theus d) Matthews e) Smith-Harris f) Winzer
2. Room 2 a) Jones (Captain)
b) Burkes
c) Hill
d) Dorsey
e) Monroe
f) Holt
3. Room 3
a) Jackson (Captain)
b) Proctor
c) Ensley (Captain)
d) Roebuck
e) Davis 4 Room 4
a) Foster (Co-Captain)
b) Newman (Co-Captain)
c) Noflin
d) Williams
e) Tolliver
III. Complete 1st IRR Iteration (Breakout Rooms)
A. Access your team’s folder using the links below
1 Room 1
2 Room 2
3 Room 3
4 Room 4
IV. Review recommendations (Whole Group)
V. Set dates for program meetings to discuss recommendations and make decisions about assessment instruments.
VI. Closing comments
A. Consultant- Dr. Brewer
B Dr Ensley
C. Dr. Thomas
Agenda for Group IRR Process in Breakout Rooms
1. Review directions and rubric (Captain)
2. Review submissions (individuals)
3. Evaluate submissions using the rubric (individuals)
4. Discuss outcomes (group)
a) Fill out outcomes table on percentage of agreement worksheet. (Captain)
5. Calculate IRR score using the IRR calculator here (Captain)
6. Record outcomes on percentage of agreement worksheet (Captain)
7. Discuss problems and conclusions (group)
8. Make and record recommendations on percentage of agreement worksheet (Captain)
CAEP Team Data Meeting Agenda via Microsoft Teams
February 18, 2022
8:30 AM
Theobjectiveofthismeetingwastoreviewdatausingtherubricswecreatedtoensurethe efficacyofitsmeasurementsasitrelatestoCAEPstandards.Wewerebrokeninto4groupsto reviewthefollowing:
1 Review directions and rubric (breakout rooms)
2 Review submission (individuals)
3 Evaluate submissions using the rubric (individuals)
4 Discuss outcomes (breakout rooms)
5 Calculate IRR score (breakout rooms)
6 Record IRR score (breakout rooms)
7 Discuss problems and conclusions (breakout rooms)
8 Make and record recommendations (breakout rooms)
Afterreviewthefollowingchangesweresuggested:
Group1.Thereisaneedtohavetherubricanddirectionstogetherandanarticulationof commonvocabularythroughout
Group2. Communicationisanissue.Languageneedstobemorespecificwithoutany equivocations.
Group3. LessonplanstemplateshouldbeconsistentforeveryclasswithDanielsonastheguide Group4.Languageshouldbechangedbyeliminatingwordslike(some/most, effective/outstanding.
Allgroupparticipantsscoredverysimilarlyinagreementofeachcomponent.