Assessment in Action Issue 1 by Rutgers University

Assessment in A Publication of the Rutgers University Division of Student Affairs

Action

From Napkin to Action: A look at Rutgers MAAP Summer 2012 Issue 1

Editorial Staff Scott Beesley* Andrew Campbell Patrick Love Patricia Rivas* Abby Stern* Design Scott Beesley Patricia Rivas Photography Scott Beesley Andrew Campbell Patricia Rivas Abby Stern Special Contributor: Bridget Turner Kelly Associate Professor, Educational Leadership Loyola University of Chicago Want to get in volved? We are looking for articles questions or comments. Please contact: Student Affairs Marketing, Media, & Communications 115 College Ave New Brunswick, NJ 08901 Phone: (732) 932-7949 Email: aia@rci.rutgers.edu *Students in the Rutgers College Student Affairs Master’s Program

page

Welcome to

Assessment in Action

You are holding in your hands (or reading online) the first

issue of our new magazine: Assessment in Action. We have some specific goals for this publication. They include: •Encouraging conversation around various assessment topics; •Encouraging the continuing development of a culture of assessment and a culture of evidence in the division; •Providing resources related to assessment practices; •Introducing and explaining the Mission Assessment and Alignment Planning (MAAP) program; and •Highlighting and learning from the variety of effective assessment practices from throughout the division. I would add to this list—having some fun. As some of our writers point out, assessment tends to have a bad rap—it is dull, onerous, an unnecessary addition to our work, and something people avoid unless instructed to do it. Within these pages we intend to challenge these beliefs. We argue that assessment can be fun, energizing, and even inspirational! You can let us know if we succeed. Of course, we welcome feedback (what kind of assessment magazine would we be if we didn’t?!) and hope you will participate in an ongoing conversation about assessment. Our intention is to publish Assessment in Action (AiA) five times each year – once in the summer and twice each semester. We welcome the active participation of anyone in the division wishing to assist in the production of this new resource. Thank you and happy assessing! Patrick Love Associate Vice President for Student Affairs

Assessment in Action

in This Issue

Never Stop Assessing: An Interview with Gavin Henning By Catrina Gallo

page

Weight Loss & Assessment? By Anne Newman

page

Michael’s Musings By Michael Miragliotta

page

5 Tips for Running Focus Groups By Bridget Turner-Kelly

page

Undergrad in Action By Ian Ragsdale

page

The Dreaded “A” Word By Sarah Grun

page

Cover Story “MAAPING” the Future of Undergradute Education at Rutgers University page By Ashley Nickelsen 08 Brent Ruben and Susan Lawrence (cover photo) are co-chairs of the MAAP Committee, a university wide assessment project that started on the back of a napkin.

Summer 2012

page

Never Stop Assessing: Gavin An Interview With Henning By Catrina Gallo

had the pleasure of interviewing Dr. Gavin Henning about his assessment experiences. Gavin is a national expert on assessment who recently left the position of Senior Research Analyst at Dartmouth College to become Associate Professor of Higher Education at New England College. Gavin is founder and chair of Student Affairs Assessment Leaders and he also chaired the ACPA Commission on Assessment and Evaluation. After speaking with Gavin, it was evident that despite his background and experience he has had his fair share of both successful and failed

wished to gain an understanding of the college experience for students of color. Many factors contributed to the initiative falling short. One of the main things Gavin came to realize was that the topic he chose was too broad. The “college experience” does not refer to one specific thing; it has a vast definition. He subsequently realized that he should have spent more time narrowing the topic on particular aspects of the college experience for students of color. Basically, what we might think is one issue can often be several issues rolled into one. We need to peel back those layers and get to the root issue of concern.

use the resources available to us, such as experienced professionals in other units or faculty with skills we can use in our assessment work. 3. Avoid a mixed-methods approach Gavin had chosen a mixedmethods approach for this initiative, combining a survey with focus groups. “Poor decision,” he says now! He came to realize that an online survey was not the most effective way to reach students of color. Furthermore, a survey was not the best approach for assessing something that turned out to be much more of a qualitative issue.

1. Be sure to narrow and define the focus of assessment 2. Use all available resources 3. Avoid a mixed-methods approach 4. Learn and use additional assessment techniques 5. Regularly review assessment action plans 6. Understand the potential impact of you as a person and you in your position 7. Recognize that assessment takes place within a political environment 8. Never stop assessing assessment experiences, and from the stories he shared, and especially the stories of failure, there were a number of lessons for anyone attempting to practice effective assessment. This was a wide-ranging interview on a number of topics, but the focus of this article is on the lessons he has learned through experience that he wishes to share with student affairs professional. 1. Be sure to narrow and define the focus of assessment Gavin shared a story with me about a failed assessment. In this assessment, he and a colleague page

2. Use all available resources Gavin explained that the Commission for Assessment and Evaluation was supporting him and his colleague during this failed initiative, but he did not utilize the commission as much as he should have. Had he involved the commission more, he and his colleague would not have encountered as many obstacles and they may have made different decisions that could have led to a more positive and effective outcome for the project. When we conduct assessments on campus we need to

Assessment in Action

Students were more willing to talk about their experiences in focus groups that had other students of color than respond to an anonymous survey written by people they did not know. 4. Learn and use additional assessment techniques When I asked Gavin for his best piece of advice for Student Affairs professionals doing assessment, he simply stated: “Learn more techniques, because a survey is a blunt instrument.” A survey can often be tedious for those requested to take it. Also, people who are

asked to take the survey often have no commitment to the overall focus of the assessment effort. More creative or interactive methods, such as individual interviews, focus groups, and participant observation will often lead to better and more accurate data. Gavin believes that we need to shift away from the surveyfirst mindset. 5. Regularly review assessment action plans Gavin asserted that it is critical to examine and assess our action plans on an ongoing basis. He explained that in his failed assessment example, he and his colleague should have been monitoring and following up with the students they were trying to reach with their survey, but this was another one of their shortcomings. Oftentimes, speed bumps can arise during what was thought to be a foolproof plan. Gavin now knows

he will make changes to his plan as needed. 6. Understand the potential impact of you as a person and you in your position Another mistake from his experience that Gavin highlighted was that both he and his colleague were White, heterosexual individuals, and having a homogenous group interpreting assessment responses can potentially hinder a holistic and accurate view of results. 7. Recognize that assessment takes place within a political environment When Gavin reported the results of this assessment, the recommendations that he made were actually the opposite of the recommendations that another unit had made to the president on the same topic. Of course, it is completely acceptable to present

contradicting results if that is what the project yielded. Gavin, however, was completely unaware of what the other group had originally recommended and did not address this discrepancy. He vowed to never let that happen again! 8. Never stop assessing One of the most important things that Gavin Henning pointed out to me is that assessment should never stop. We need to get in the habit of constantly improving our practices, and learning how to perform effective assessment. Look for stories from my interview with Gavin in future issues of Assessment in Action. Catrina Gallo is a 2012 graduate of the Rutgers Ed.M. College Student Affairs Program. She is currently a Residence Life Educator at Rutgers University.

Weight Loss & Assessment? By Anne Newman

he new school year is almost upon us. This is the perfect time to think about assessment plans for the upcoming year. Assessment plans can provide your office with a wealth of information and can make planning more intentional. While assessment is worthwhile, I know some student affairs professionals dread the thought of putting together a thoughtful and useful assessment plan. Assessment dredges up thoughts of numbers, complicated surveys, analysis, and judgment. Assessment also means extra work and the fear of finding out that what you are doing is not effective. Student affairs professionals struggle to find the proper motivation to begin working on an assessment

plan. Many of us search for a better understanding of the importance of assessment in order to get started on an assessment plan. I happen to love assessment. I can admit this in public. Iâ&#x20AC;&#x2122;m one of those fortunate people that have an affinity for numbers. I like figuring out what makes things tick and how I can make things better. I like coming up with questions and using data to answer those questions. I also love using assessment to budget and plan for my office. Assessment helps us bring about change, and we all know change is necessary for student affairs programs to thrive and survive. I love watching my office change and develop based on the information we collect every year. There are a lot of things in which I do not excel and do not come easy for

me. Take weight loss for instance. It is time consuming to follow a plan, to keep track of progress, to write down everything you eat and to exercise on a consistent basis. Because it was important to me to lose weight and get healthy I started a weight loss plan last October and have been able to lose 55 pounds so far. However, I have been challenged during every step of my journey and know I have a long way to go before I meet my long term goals. So why talk about weight loss in an assessment article? It makes a lot of sense to anyone who has struggled to lose weight. I canâ&#x20AC;&#x2122;t tell you the anxiety I feel at the beginning of a diet. Losing weight also requires a great deal of persistence and motivation. In addition, successful weight loss also requires an

Summer 2012

page

understanding of the science behind successful weight loss. In their book, Assessment in Student Affairs: A Guide for Practitioners, Lee Upcraft and John Schuh provide nine principles of good practice for student affairs assessment. As I was rereading these assessment guidelines, I was struck by some of the similarities between good practice for student affairs assessment and the principles of successful weight loss. Upcraft and Schuh point out that assessment works best when a program lists clear and explicitly stated goals. Outcomes for your assessment plan need to be both clear and measurable. When I set out to create measurable goals for our office

to weight goals, I also set fitness goals for myself. I decided to set a goal to complete a marathon by the time I turn 45. Each year before the marathon, I have a shorter distance to complete. I have completed a 5K race and I’m now training for an 8K this fall. These fitness goals are understandable and attainable. This keeps me motivated to continue working toward my goal. Upcraft and Schuh also note that assessment works best when it is ongoing. A lot of times we look at assessment as an “all or nothing” proposition. Those successfully using assessment know that assessment is cyclical. Assessment occurs, and then changes are made, which are

on. Numbers are important. You have to have objective information about your programs in order to know what you’re doing well and what you need to improve. How can you plan for the future if you don’t know where you stand now? It’s the same for weight loss. As much as I can’t stand my scale, I know that in order to be successful at losing weight, I have to weigh myself on a regular basis. I also needed to weigh in and take measurements at the start to accurately measure progress. I am better able to know where I’ve been and where I am going because I have accurate data about myself and my progress.

Those successful at weight loss must set measurable goals. Having a measurable outcome makes the goal seem more attainable. I understood that saying, “We want to be the best student conduct program in the country” was not good enough. How would anyone know what that means? What makes a good student conduct program? Realistic outcomes related to conduct include “reducing recidivism rate by 10 percent” or “increase student knowledge of the Code of Student Conduct by 15 percent.” Assessment is much more tangible and understandable to staff members when there is something concrete we can measure. It’s the same with any weight loss program. Those successful at weight loss must set measurable goals. Those that are successful don’t just say, “I want to be skinny” or “I want to be healthy.” They instead say, “I want to lose 30 pounds”, or “I want a BMI under 22”, or “I want my blood pressure to be 120/80.” Having a measurable outcome makes the goal seem more attainable. When I started my weight loss plan last October I tried to come up with measureable short term and long term goals. In addition page

then assessed again. This is how an organization keeps track of how it is doing and can respond when changes need to be made. It is the same with weight loss. Individuals who rely on fad diets or quick fix schemes often find themselves frustrated when they can’t lose weight. Those that do lose weight on fad diets are often frustrated when they regain the weight. It certainly is tempting to purchase “infomercial” products advertised on Saturday morning that promise success with little change or effort. Those successful at weight loss know that weight loss is a journey and it takes changes in one’s lifestyle to maintain weight loss. The weight loss program I follow has you reassess how much you eat every time your weight drops below a certain level. This helps me reassess my plan in order to restart weight loss and maintain progress. By constantly monitoring progress and altering my weight loss plan when needed, I was able to successfully lose weight and keep the weight from coming back

Assessment in Action

Weight loss and assessment have a lot in common. Both are dreaded by individuals who are daunted by the perception of the enormity of the task. Both require a great deal of motivation and commitment in order to produce successful results. Both can also provide rewards if done correctly. I can attest to the rewards of both weight loss and assessment. I know I am personally and professionally better by completing both to the best of my ability. The start of every new school year brings with it the opportunity to create new “school” year resolutions. I hope one of your resolutions for the upcoming year includes incorporating assessment into your programs and services. While it might not be easy, the rewards will be worth it. If you happen to see me eating something fattening, please remind me of my goals. Anne Newman is the Director of Student Conduct at Rutgers University.

Michael’s Musings By Michael Miragliotta Assessment. Many readers have probably stopped reading after my first sentence. Assessment can be a daunting task and conjures images of surveys, complicated statistics, and hours of work. As the chair of the Division’s Assessment Committee, I am here to provide assurance that anyone in this Division can do assessment everyday without needing any extra tools other than what she/he already has. How does Dining Services know how to best staff its dining halls? How does Student Life know if a program is successful as it is happening? How do you know if the presentation you are giving is engaging the audience? All of these questions can be answered with the term “spotlight assessment.” Spotlight assessment is the act of assessing your work every day and making adjustments with the observations you collect. Dining records the amount of swipes in every dining hall and uses this data to create staffing patterns that ensure the dining halls run smoothly. This is a form of spotlight assessment that uses easily collected data to make decisions. Student Life staff can assess a program as it is happening by monitoring the audience. They can count the number of audience members, the number of audience members engaged in the program, and the number of positive comments they hear as audience members are leaving the program. All of these are examples of spotlight assessments helping Student Life gauge the relative success of a program. I use spotlight assessment when engaging in a presentation. As I speak to my audience I am using spotlight assessments to gauge the level of attention, engagement, and comprehension from my audience. An audience full of people on their phones, yawning, or not making eye contact demonstrates to me that I

am not engaging the audience. Audience members who are making eye contact, nodding, smiling and laughing (when appropriate) shows me that I am engaging the audience. Spotlight assessments allow me to make adjustments to my presentation, review content that may not have been presented clearly, or change my presentation style on the fly to engage my audience. Spotlight assessment is used in many other aspects of our work every day. It should be used as a first step to assessing our work. If, for example, Student Veteran Services notices dozens of students starting to ask about financial aid opportunities, that office can implement various initiatives to proactively get financial information out to veterans. This spotlight assessment could lead to a more complicated assessment of the types of financial questions students are asking or a program evaluation of a newly created financial aid program geared toward veterans. Spotlight assessments are the first step in using the information that is at our fingertips to make Rutgers a better place for our students. As we begin this new academic year, I hope that you can begin to look at the copious amounts of information we all have at our fingertips. Use spotlight assessments to begin to understand how your services are affecting our students. Start small and ease into assessing your work every day. The Assessment Committee will be providing tools and help as we begin to create a culture of evidence based decisions. Good luck, and remember that assessment is not a scary word! Each issue of Assessment in Action will feature Michael’s Musings. In this column, Michael will address some of the common misperceptions about assessment, reader questions, and interesting assessment strategies. Michael Miragliotta has worked at Rutgers for three years in various roles in Housing and Residence Life. He completed his B.A. in Psychology and Sociology at The College of New Jersey and his M.A. in Psychology at New York University. Michael structured both of his degrees to focus on research methodology and statistics. He was an adjunct professor of research methodology in the Psychology Department at The College of New Jersey for two years. His passion lies in using appropriate assessment methodology to help understand how to do our jobs better for the benefit of our students.

Summer 2012

page

06 07

By Ashley Nickelsen

wenty-seven months ago in the Busch Campus Center dining area, the Mission Alignment Assessment and Planning (MAAP) system came to life as a simple sketch on the back of a Gerlandaâ&#x20AC;&#x2122;s napkin. Brent Ruben, Executive Director of the University Center for Organizational Development and Leadership, and Susan Lawrence, Dean for Educational Initiatives and the Core Curriculum, serve as the co-chairs of the MAAP Task Force. With a team of more than 10 representatives from throughout Rutgers University, the MAAP Task Force brainstormed, discussed, and discerned ways to assess undergraduate education at Rutgers University. After sharing hundreds of ideas and conducting months of research, the MAAP Task Force transformed the once rudimentary sketch on a napkin into a massive visual matrix ready for use.

Meet MAAP MAAP provides a framework to meet both external and internal assessment needs. The visual matrix consists of seven mission-critical goals that were developed based on university strategic planning documents and the Rutgers University mission. These goals for undergraduate education include: 1. Student Recruitment, 2. Rutgers Support and Pride, 3. Personal and Professional Development, 4. Academic Degree Goals, 5. Progress to Degree, 6. Post-Graduation Success, and 7. Operational Support for Faculty and Staff. Each department and unit strives to align their programming services with these mission-critical goals as well as the more specific subgoals within each category [see side bar]. The matrix page

Assessment in Action

serves as a visual aid that demonstrates how each unitâ&#x20AC;&#x2122;s goals align with the overarching university goals. Overall, the success of MAAP relies on the assessment of these goals, the use of these data to measure university attainment in meeting these mission-critical goals, and the subsequent planning to improve programs and services to better align with goals. A Collaborative Perspective On June 9, 2012, I had the pleasure of interviewing Susan Lawrence, co-chair of the MAAP Task Force. During the interview, I learned about MAAP as a collaborative assessment model that helps individual units and departments critically think about their

Susan compared the simplicity of MAAP to purpose within the university and their goals creating the dinner menu for a summer barbecue. to contribute to the undergraduate educational “MAAP is fundamentally a fairly simple concept; experience. Susan emphasized the importance assigning goals to columns and unit services to rows. of assessing the undergraduate experience by [see the image on page 8]. It’s about finding the comparing MAAP to a matryoshka, also known as match; much like you would run a potluck dinner at a Russian nesting doll. A Russian nesting doll has a barbecue. Who’s bringing the salads, burgers, and been traditionally used to tell stories, and each doll desserts? Does fruit salad fit into the salad or dessert contains increased levels of detail and decoration. category?” The fruit salad could possibly contribute Every doll has its purpose and place within the to both the dessert and salad storytelling process, and categories depending on the within the MAAP construct, type of barbecue, but the each doll represents a potluck system helps people specific unit or department avoid overlap and assess at Rutgers. Every unit and the purpose of each dish. department contributes a Similarly, MAAP provides piece of their unique story the framework for units based on their specific and departments to better goals. Susan explained understand what they are that MAAP is a process as bringing to the “university much as a result, “MAAP is table.” There are a number not only about the process of outstanding programs, of getting everybody initiatives, and efforts on board, but it’s also Susan compares MAAP to a matryoshka. occurring within each unit and about having a cohesive, Every doll has its purpose and place within department at Rutgers; MAAP coherent, and tellable the storytelling process, and within the MAAP provides a framework for story.” Susan hopes that construct, each doll represents a specific unit or these units and departments MAAP will change peoples’ department at Rutgers. to tell their stories better, to way of thinking, and that coordinate their goals to align with the university’s more faculty and staff will begin to critically think overarching mission, and to collect assessment data to about “mapping out” the goals for their individual determine unit and institutional effectiveness. department. Susan is particularly excited by the The Future of MAAP synergistic approach within the university as a result As Rutgers changes, MAAP will continue to of MAAP; both staff and faculty have been working evolve. With a few departments already in the in a collaborative and cohesive process to tell their process of utilizing MAAP to evaluate their goals, individual stories. Each part contributes to the whole, the Task Force hopes for MAAP to become the “new and people have begun to see the impact they make normal” across the university. The institutional goals on the student experience. Susan emphasized that in may change, but the basic framework will persist, order to understand our specific roles and how each because MAAP provides a comprehensive visual of our roles fit within the largest nesting doll, or in story to guide university strategic planning and to this case the greater university community, each unit tell our story to the public. Because the concept is so is encouraged to develop their own goals, specifically simple, MAAP can be adapted to other institutions aligned to the undergraduate mission goals.

Summer 2012

page

and to a variety of programs. Susan is excited for the future of MAAP because her passion for assessment is driven by her belief in the university mission to help students through the most important transformative

that story-telling process, but MAAP continuously urges staff and faculty to reflect on and assess the purpose of their stories as well as their story-telling techniques.

MAAP is not only about the process of getting everybody on board, but it’s also about having a cohesive, coherent, and tellable story. -Susan Lawrence years of their life. Susan explains, “We need to be able to tell our story about the value of the undergraduate learning experience we provide at a university that has deep historic roots and a university that is committed to meeting the challenges of education for the 21st century.” MAAP not only facilitates

More Information

Ashley Nickelsen is a 2012 graduate of the Rutgers Ed.M. College Student Affairs Program. She is currently a Residence Hall Coordinator at the University of Delaware.

A little lost? Here is some MAAP information to help you along the way.

The members of MAAP are

Brent Ruben, Professor II (Distinguished Professor) of Communication; Executive Director, University Center for Organizational Development and Leadership Susan Lawrence, Director of Core Curriculum, Office of the Vice President for Undergraduate Education; SAS Dean for Educational Initiatives and the Core Curriculum; Associate Professor of Political Science Barbara Bender, Associate Dean, Graduate School-NB Richard DeLisi, Dean, Graduate School of Education; Professor of Educational Psychology Gary Gigliotti, Associate Vice President, Teaching and Assessment Research, Executive Director of the Center for Teaching Advancement and Assessment Research, and Professor of Economics Carol Goldin, Associate Dean for Assessment, School of Pharmacy Robert Heffernan, Director of Institutional Research & Planning Kate Immordino, Director of Organizational Research and Assessment at the University Center for Organizational Development and Leadership Patrick Love, Associate Vice President for Student Affairs Don Smith, Vice President, Information Technology and Chief Information Officer, Associate Professor of Computer Science

page

Assessment in Action

In addition to Student Affairs, MAAP has also been doing pilots with:

University Office of Undergraduate Education Division of Instructional Support Division of Academic Engagement The Libraries Administration and Public Safety Information Technology Core Curriculum The School of Arts and Sciences Academic Services Honors Program

Online Resources

http://bit.ly/N59OiF - MAAP website

Special Contributor

5 Tips for Running Focus Groups

By Bridget Turner-Kelly

ou’ve seen it on television, in political campaigns, and other media advertising—focus groups are the way to gauge what consumers want. Focus groups are a relatively recent technique used in higher education to understand consumers of the college environment. Their strength lies in the knowledge gained through interactions. Many of us have not known what we wanted to say until someone else expressed their thought. Gretchen Rossman and Sharon Rallis argue that “People often need to listen to others’ opinions and understandings to clarify their own.” In assessments where you need to know what people really think and why they think that way in order to improve practice, focus groups can provide rich, efficient data that teases out people’s attitudes and beliefs as they develop through group interaction and exchange.

Here are my top 5 tips for facilitators on Running Focus Groups:

1. Be sure to include people in the focus group who have unique experiences with the topic being assessed. In other words, you need to include the students, faculty, staff, or parents that are the most informed about whatever it is you want to know. 2. Establish a safe environment where people can trust group members, examine their own ideas and feelings, and openly and honestly express their responses. Your challenge is to carefully handle issues that people are reluctant to discuss in public and make sure you avoid forcing people to disclose embarrassing or sensitive information about themselves or the university. 3. Use a structured interview guide with 4-7 questions for an hour to an hour-and-a-half session with around 6-10 people. Allow for many follow-up and group-generated questions so most of the discussion occurs among the participants rather than between facilitator and participants. 4. Focus on and return to the central assessment questions throughout the focus group so you can be sure to obtain the data you need. Gently steer the group back from areas unrelated to the study while making sure everyone has a chance to contribute to the conversation. 5. Conduct focus groups in such a way that people can publically disagree with each other and discuss their views and you can view how committed people are to their views. Encourage people to defend or provide further explanation for their beliefs so you can interpret the meaning behind the words. Below I provide a real university example to tease out these tips. HBU and PWI Campus Contexts Case: Professor Sharon Fries-Britt and I sought to examine the influence of a historically Black university (HBU) and a predominantly White institution (PWI) on successful Black

students. Utilizing tip #1 (include people who are most likely to know what you need to know), we believed the students themselves were the most knowledgeable about their experience. We conducted four focus groups, two at the HBU campus and two at the PWI campus, to obtain data about academic

and social programs as well as experiences that facilitated or blocked their success. Utilizing tip #3 (use an interview guide for the 6-10 people in focus group), we had approximately 6 students in each focus group and asked them each structured questions from an interview guide, such as:

Summer 2012

page

(a) What challenges have you encountered to your academic and social life on campus? (b) Has a topic ever come up in class that you felt you couldn’t express your opinion on? (c) How have you connected socially to the campus? And, what do you do when you need academic help? One theme evident during the focus groups conducted at the HBU was the close connection and sense of family-like atmosphere that the students exuded in their interactions with each other. Individual interviews may have communicated the degree of closeness the Black students felt with each other at the HBU. However, the focus group setting allowed the facilitators to witness the looks of pride, smiles, high-fives, and genuineness exhibited as the students responded to comments other participants made. Thus, utilizing tip #5 (find the meaning people make of their words), the group interaction allowed the facilitators to interpret the feelings and attitudes expressed in the actual stories of support and involvement Black students shared. The findings of strong support and high involvement at the HBU were in stark contrast to the findings of dispersal of energy at the PWI. Our use of tip #4 (focus on assessment questions to get needed data) allowed us to keep coming back to our central question of what facilitated or blocked student success at the PWI. The students at the PWI described their energy being taken away from their studies in the classroom and into efforts to combat isolation resulting from the lack of racial and ethnic diversity. They reported feeling as if most of the activities on the PWI campus were primarily geared for White students. We worked hard to adhere

page

to tip #2 (establish a comfortable environment) by using strategies to create a safe and open environment in the focus group and saw the fruits of our labor in how comfortable the group was trusting each other with their experiences and feelings. During the PWI focus group interviews, knowing glances of shared frustration were exchanged. Heads nodded in agreement about different coping strategies used by students to defray the effects of racist comments or situations they encountered at the PWI. As advertised, the focus group method provided rich data about the phenomenon under study and did so in an efficient way. The strength of the focus group interaction showed during the HBU focus group interviews where we observed a distinct “uplifting” atmosphere and a contrasting “coping” atmosphere during the PWI focus group interviews. The focus groups helped us more fully

Assessment in Action

understand the varying experiences students described on each campus. As difficult it is for educators to use assessment to guide practice, focus groups provide a convenient way to gather thick, descriptive data necessary to answer questions about complex or not easily-accessed information about the college environment. We would do well in higher education to utilize focus groups to ask the right people what we want to know to guide better practice. Bridget Turner Kelly is the Associate Professor and Program Co-Director of the Educational Leadership program at Loyola University. Dr. Kelly obtained her Ph.D. from the University of Maryland, College Park in Social Foundations of Education. She is an expert in assessment, and she has numerous publications on the topic of assessment and focus groups.

Summer 2012

page

Undergrad

in Action By Ian Ragsdale

s a student worker for Housing and Residence Life during the summer break, Luis Fernandez ‘15 helped the marketing team research how other schools market and communicate to their student residents. This initiative plays into division-wide goals of self-assessment and bringing innovation and clarity to branding, marketing, and discovering new technologies and processes that connect with students. For Luis, a first-generation college student who just finished his first year at Rutgers, the experience was far more than a few paychecks over the summer. It was also a deep immersion into the world of higher education and prepared him for his new role as Advocacy Director for the Residence Hall Association (RHA). “At the end of this experience, I’ll have something I can present to Housing and Residence Life, but also something that I can take back to RHA to improve the organization’s communications,” Luis says. Luis’ primary task was immense: research every Association of American Universities (AAU) institution to find out how other housing operations or residence life groups use technology to market and communicate to students. Much of the research was conducted online—he visited countless websites, social media profiles, and YouTube channels—but Luis also corresponded directly with residence life professionals at other AAU schools. These e-mail surveys and phone interviews gave Luis a deeper understanding of how other departments approach communications. The results of his research had immediate impact. A multimedia

campaign surrounding new student move in was completely rethought after Luis surveyed the successes and failures of other schools’ move in videos. Luis’ input also supported the marketing team’s creation of an exhaustive FAQ document for the department’s website and forthcoming mobile site. “We had completed an AAU residence life communications survey previously, but having a student conduct the research and focus explicitly on new technologies allowed us to get a lot of new value,” says Ian Ragsdale, Student Communications Coordinator for Housing and Residence Life. “His research allowed us to create a benchmarking document and generated a long idea list of things that Luis, as a student, thinks would be good to implement at Rutgers.” The immersive nature of the project has given Luis a rich perspective and allowed him to make a number of specific recommendations based on his assessment. These include: making the presence of professional and graduate student Residence Life staff better known and understood; more multimedia to relieve first year student anxiety; more material on sustainability in the residence halls; and contests that can support communications while also engaging students. Ultimately, Luis hopes that his work for Housing and Residence Life and his newfound knowledge can help bridge the gap between students and professionals in Housing and Residence Life. “I want to help students get more involved in life on campus,” he says. “Learning how to communicate better will allow me to do that.” The department’s marketing team is

Summer 2012

page

also counting on Luis’ work to inform promotional item intended to and guide projects as another school generate buzz about key return. year approaches. Summer ends for Luis with a “It has been really valuable to have study abroad experience in Peru, but a student be able to come to us and say he claims that he has already done a how students would like us to market lot of traveling this summer—albeit to them,” says Michael Miragliotta, virtually. Assistant Director of Marketing, “It’s like I took a national tour Assessment, and University Relations of colleges,” he says. “It has for Housing and Residence Life. been fun seeing cool things that “No matter how hard we work to other colleges have to offer—like understand our students, nothing beats mountaintop residence halls or their direct input and guidance on our dorms on the beach—but at the end work.” of the day it has also allowed me to Recent innovations, such as offering better appreciate how distinct and quiet floors and incentivizing important my undergraduate key return with foldable experience at Rutgers has Luis Fernandez ‘15 helped the marketing water bottles, were the direct team research how other schools market and been. The size of Rutgers, the results of student feedback. students, the location, all these communicate to their student residents. Luis and other members have given me the opportunities of RHA are also regularly asked to review new I was looking for. student communications initiatives, including an “This work has made me see that Rutgers is really advertisement in the Rutgers University Visitor’s the right place for me.” Guide, postcards sent to first year and transfer students, and contests that enhance social media Ian Ragsdale is the Student Communications engagement. Coordinator for Rutgers Housing & Residence Life. A central feature of the new process was a

The Dreaded A word A

By Sarah Grun

wise and innovative man once said, “I’m always thinking of what’s wrong with the thing and how it can be improved.” Much like Walt Disney, we, as higher education professionals, should also be thinking of ways to improve our work. One way we can intentionally improve our work is through assessment. According to one assessment guru, John Schuh, assessment should be an “essential element of student affairs practice.” Ideally, assessment demonstrates how we, as Student Affairs professionals, contribute to the lives, learning, growth, and development of our students. But assessment is often considered the page

dreaded “A” word. It is a process that some of us in student affairs avoid. Recently, one Rutgers assessment advocate, Krista Kohlmann, shared with me her take on assessment, why it is important to the work she does, and the value of starting small. Krista is a Program Coordinator within Student Life-Student Involvement. She has incorporated learning outcomes and assessment of those outcomes into her role advising the Student Volunteer Council at Rutgers, a 15 member student group that plans community service events for the Rutgers Community. As part of advising the students, she wanted to create learning outcomes from

Assessment in Action

scratch to specifically gear training activities to improve their learning over the course of the academic year. To do this, she spent a few weeks over the summer reading literature on service learning, including Council for the Advancement of Standards in Higher Education (CAS) for Service Learning, Community Service Attitudes Scale, and various other sources. She then defined her own “areas of competence” based on the research, each of which had 3-5 learning objectives, and she created a pre-test to give to the students prior to their August training. Throughout the year, Krista made an intentional effort to include the areas of competence and the learning

objectives in her on-going training sessions, and administered a post-test at the end of the year. After analyzing the results and compiling a report for the department, Krista has already started to use the data to inform her work in planning next year’s training for her students. When asked about assessment Krista said, “Assessment is not something to do once and then put aside for the rest of the year. This is challenging for people who have multiple responsibilities and do not have the time and energy…that is why I specifically started on a small scale to keep track of the learning of my students so that I could be

of her student affairs background, nor is it part of her job description, Krista finds it to be “extremely informative and guides the work that [she does] to make it more meaningful.” Innovation, intentional change, and improvements take time and effort. They require us to assess our current practices and look forward to better programs and initiatives for our students. However, it is not enough to say that we do assessments or evaluations. Doing them is really just one step in the assessment process. To be effective, we must learn from assessment, integrate it into everything we do,

intentional and doing assessment are essential. We, like Walt Disney and Krista Kohlmann, should embrace assessment and in doing so become better higher education professionals and people. Sarah Grun is a 2012 graduate of the Rutgers Ed.M. College Student Affairs Program.

Assessment is not something to do once and then put aside for the rest of the year. -Krista Kohlmann (Pictured Right)

thorough and intentional about their learning.” Krista also said that the assessment process keeps her on her toes, keeps her feeling invigorated, and keeps her constantly working harder at her job. Her work with the Student Volunteer Council is one small effort within the division of Student Affairs to assess and validate the work we are doing to improve the services and programs we offer here at Rutgers. While assessment can appear to be complex, time consuming, labor intensive, and/or overwhelming, it is possible to start small. For example, a unit may begin with an assessment to measure participation, satisfaction, cost effectiveness, and/or the needs of current students. For units to claim that they contribute positively to students’ lives, there must be data to support those claims. But, you don’t need to have a formal background in assessment to get started. Just ask Krista! While assessment is not part

and make it continuous. It must become a mindset and we must remember to enact intentional changes. Much of Walt Disney’s success came from being a creative innovator who was not afraid to dream and improve upon his own work. He also continuously assessed his products and parks saying, “It’s something that will never be finished. Something that I can keep developing…and adding to.” Disney understood that there is always room for improvement, but that being Rutgers University does not discriminate on the basis of race, color, national origin, sex, sexual orientation, gender identity or expression, disability, age or any other category covered by law in its programs, activities, or employment matters. The following people have been designated to handle inquiries regarding the non-discrimination policies: Judy Ryan, Title IX Coordinator for Students & ADA/Section 504 Compliance Officer Office of the Vice President for Student Affairs 83 Somerset Street, Suite 101, CAC p. 848-932-8576 ryan@oldqueens.rutgers.edu

Jayne M. Grandes, Director Office of Employment Equity, University Human Resources 57 US Highway 1, ASB II, Cook Campus p. 848-932-3980 grandes@rutgers.edu

For further information on notice of non-discrimination, visit http://wdcrobcolp01.ed.gov/CFAPPS/ OCR/contactus.cfm for the address and phone number of the Office for Civil Rights that serves your area, or call 1-800-421-3481.

Summer 2012

page page

All assessment is a perpetual work in progress. -Linda Suske

Without deviation from the norm, progress is not possible. - Frank Zappa Self-assessment is the first step to all assessment. -Unknown

USA Today has come out with a new survey. Apparently, three out of four people make up 75% of the population. - David Letterman

Experience is a hard teacher because she gives the test first, the lesson afterwards. -Vernon Law