Mustang Assessment 101 #5: Assessment Storytelling

Workshop #5: Assessment Storytelling

Mustang Assessment

What’s happening today?

 Review: Signature inquiry whats; assessment hows

 Assessment Storytelling:

 Audiences

 Formats

 Content

 Timing

 Sharing, Using and Closing the Loop

Review

Signature inquiry basics

Signature inquiries (previously, “signature assessments”) are:

 Focused assessment projects

 Distinct from other types of less formal (e.g., “quick-check”) assessments

 “Deep dive” inquiries into one critical problem of practice over the course of the academic year

Signature inquiry

Anthology Planning report on outcomes

Review Signature inquiry basics

4A-i: Departmental Goal Number.

4A-i: Departmental Goal (Full Text). ABCD!

4B-i: Measure(s): Description. Data and methods used to assess (e.g., student survey)

4B-ii: Measure(s): Direct or Indirect?

 Direct = Observable (e.g., knowledge test, minutes of wait time)

 Indirect = Participant reported (e.g., sentiment survey, focus group, interview)

4B-iii: Measure(s): Target(s). e.g., increase from 70% to 85% positive sense of belonging

4C-i: Findings: Target Status. Met / Partially Met / Not Met / No data collected

4C-ii: Findings: Detail. e.g., increased to 82%

 When possible, disaggregate by demographics, location

4C-iii: Findings: Interpretation

4D-i: Action Plan: Steps. Now we know; what will we do?

4D-ii: Action Plan: Primary Stakeholders. Who’s involved in the Action Plan?

4D-iii: Action Plan: Products. What are we doing to implement the Action Plan?

4E-i: Status Update (from last year). Fully implemented / In progress / Not started.

4E-ii: Status Update (from last year): Detailed.

Review Assessment basics

How do we undertake an assessment project, anyway?

DECIDE

1. Develop a purpose statement

2. Identify stakeholders

3. Examine existing data

PLAN

4. Determine what data to collect

5. Decide from where/whom data should be collected

6. Determine time frame for project

7. Develop data collection tools

ACT

8. Collect data

9. Analyze data

USE

10. Share results

11. "Close the loop" by using assessment data

Review Assessment basics

A B C

udience ehavior ondition egree

A. To whom does the outcome pertain?

B. What should the audience know/be able to do?

C. Under what conditions will learning occur?

D. How much will be accomplished (i.e., what level is “success”)? This is the target you aim to meet.

Review Assessment basics

A B C

udience ehavior ondition egree

Student Affairs Division staff who participate in Mustang Assessment 101 will demonstrate increased awareness of various forms of assessment by utilizing a different assessment approach from last year (25% of departments completing a signature inquiry).

Assessment Storytelling Audiences

Audienc e Format Content Timing

Assessment Storytelling

Audience

Division/ University

Your department

 Formal annual reporting

 Informal sharing

Other stakeholders

 Departmental meeting

 Participants

 Students  Faculty

 Merit review  “Closing the loop” to shape practice

Staff

Community partners

The field

Fellow practitioners  Academics

Assessment Storytelling Format

 Email to participants

 One-pager for students

 Weekly 1-on-1 with supervisor

 Presentation to internal stakeholders (e.g., department meeting)

 Email to external partners

 Annual report submission (general)

 Impact Day poster

 Impact writeup

 Connections to University strategic plan

 Conference presentation

 Journal article

 Presentation to external stakeholders

Assessment Storytelling Content

Regardless of format, most assessment stories contain:

 Key takeaways. Focus on findings that are high-priority, surprising and/or interesting

 Methods. Brief explanation of how findings were obtained (who, when, where, how)

 Data visualization(s) and/or examples.

 To describe sample (i.e., data/participants included in assessment)

 To illuminate/enhance key takeaways

 If qualitative data, to give voice to participants

Include quotes from participants!

 So what? Why do we care, what did we learn, and what will we do?

Assessment Storytelling

Timing

The timing of sharing your assessment story is largely dictated by your audience and your format.

Also keep in mind:

 Timeliness.

 One-time or recurring.

 Connections to future data collection.

Assessment Storytelling

Sharing, Using and Closing the Loop

 Sharing assessment stories serves multiple purposes, including…

 Using assessment stories helps to…

 Closing the Loop uses findings and then re-assesses to ensure that an anticipated impact did, indeed, occur!