The State of First-Year Program Assessment: Recent Evidence from the 2017 NSFYE

Resource Spotlight: The State of First-Year Program Assessment: Recent Evidence From the 2017 NSFYE

Dallin George Young Assistant Director for Research, Grants, and Assessment National Resource Center for The First-Year Experience and Students in Transition

University of South Carolina

2017 National Survey on the First-Year Experience: Creating and Coordinating Structures to Support Student Success by Dallin George Young (Ed.), Columbia, SC: University of South Carolina. Available July 2019.

The first-year experience (FYE) has been put forward as a philosophy and movement for improving first-year student transitions for most of the past four decades (Hankin & Gardner, 1996; Upcraft & Gardner, 1989) and represents a comprehensive, coordinated, and wide-reaching effort designed to support student success (Hankin & Gardner, 1996; Upcraft & Gardner, 1989; Young & Keup, 2019). Although the philosophy behind the FYE concept requires a broad approach to new-student success, research shows that it is not enough for colleges and universities to simply increase the array of educational offerings aimed at these students. Indeed, the FYE is not necessarily improved by the number of programs, but by the level of coordination and integration across them. Because firstyear student success is not easily localized or specific to one functional area on campus (Young & Keup, 2019), FYE efforts must include a cohesive, comprehensive, and campuswide mix of curricular and cocurricular initiatives (Greenfield, Keup, & Gardner, 2013; Hankin & Gardner, 1996; Upcraft & Gardner, 1989).

Assessment is also critical to the success of any FYE enterprise. As Kuh (2010) noted, only through thoughtful implementation and continual evaluation will high-impact practices, such as those that comprise a FYE, realize their full potential. Intentional designdepends on information that allows for targeting of first-year efforts as well as ongoing improvement.

This article focuses on assessment of first-year initiatives, and specifically on providing insight gathered through the 2017 administration of the National Survey on the First- Year Experience (NSFYE). A brief description of the survey precedes a presentation of data on the frequency of assessment and the formats of assessment practices of selected first-year initiatives.

2017 National Survey on the First-Year Experience

The 2017 NSFYE sought to gather information on overall institutional attention to the first year, as well as common first-year programs including academic advising, orientation, common readings, early-alert programs, first-year seminars, learning communities, and residential programs. The questionnaire sought information common to each of these seven first-year initiatives, including students served, perceived value, and assessment. An additional set of questions about assessment was asked of institutions that indicated offering first-year seminars, first-year advising, and pre-term (new-student) orientation. The 2017 NSFYE asked more in-depth questions about these initiatives because of their longevity in the scholarly practice discussion and their greater degree of professionalization, as evidenced by the presence of professional organizations (i.e., the National Resource Center, NODA, NACADA) representing these activities. In addition to asking whether these first-year initiatives were assessed, the survey asked respondents to indicate the formats used to carry out any assessment activity. Findings related to general questions on assessment and those on assessment of specific first-year initiatives follow.

Assessment of First-Year Programs by Frequency

Figure 1 shows the frequency that institutions offering certain first-year programs reported assessing those programs within the past four academic years. First-year seminars (62.7%) and pre-term orientation (54.1%) were the only programs in which more than half of respondents with these programs reported recent assessment. However, it is notable that when I don’t know responses were removed, that group includes first-year residential programs and learning communities as well. The least frequently reported assessed first-year initiatives were common-reading (27.5%) and early-alert programs (31.1%).

There were some comparative differences in reported levels of assessment by institutional characteristic. For example, two-year institutions more frequently reported assessing early alert, while four-year schools were more likely to report assessing common readings and first-year seminars. Additionally, public institutions more frequently reported assessing first-year academic advising, first-year seminars, learning communities, orientation, and residential programs.

Figure 1. Percentage of institutions reporting assessment of selected first-year programs since Fall 2013. CR = common reading (n = 182); EA = early-alert systems (n = 383); FYA = first-year academic advising (n = 387); FYS = first-year seminars (n = 366); LC = learning communities (n = 227); OR = pre-term orientation (n = 375); RES = residential programs or initiatives (n = 240).

Assessment of First-Year Programs by Format

Figure 2 displays the frequency that colleges and universities in the sample reported using specific formats to assess first-year seminars, pre-term orientation, and first-year academic advising. Individually, the most common formats for institutions that had assessed first-year seminars were course evaluations (81.1%), analysis of institutional data (72.2%), and direct assessment of learning outcomes (63.4%). For orientation, respondents most frequently indicated the use of a survey instrument (73.1%) by a large margin, more than doubling the number of schools that reported using analysis of institutional data (36.3%) and direct assessment of learning outcomes (36.3%). Finally, nearly two thirds of colleges and universities reported analysis of institutional data (68.5%) and use of a survey instrument (66.4%) to assess first-year advising efforts, followed by nearly a third of respondents who indicated carrying out a program review (32.9%).

Figure 2. Percentage of institutions reporting type of assessment conducted. FYS = first-year seminars (n = 227); OR = pre-term orientation (n = 201); FYA = first-year academic advising (n = 149).

Discussion

These results point to at least two noteworthy patterns. First, there were substantial comparative differences in the frequency that respondents indicated the use of formats of assessment by first-year program. Institutions that recently assessed their first-year seminars reported using five of the nine types of assessment referred to in the survey at greater frequency than the other two. This, combined with the results reported in Figure 1, suggests that at the institutions represented in the sample, first-year seminars are not only being assessed more frequently, but also using a greater variety of formats.

Secondly, there are formats that, while used with varying frequency relative to one another, are among the most common for the three first-year programs on which the NSFYE gathered data. Analysis of institutional data was among the top three assessment formats for all three programs. In addition, direct assessment of learning outcomes and use of survey instruments were in the top three responses of at least two of these programs. Under the collaborative ideal of the FYE philosophy, these represent potential opportunities to engage in coordinated assessment practices. Even though an individual program might have its own priorities and salient questions to consider, similar formats allow offices to thoughtfully discuss how the data collected could be fed into a similar stream for cross-functional assessment of broader institutional goals for the first year.

Assessment of FYE programs is an important piece toward ensuring the effectiveness of these offerings individually and collectively. We repeat the sentiments of Kuh (2010), who highlighted first-year experiences as high-impact educational practices: “Only when they are implemented well and continually evaluated ... will we realize their considerable potential” (p. xiii).

References

Greenfield, G. M., Keup, J. R., & Gardner, J. N. (2013). Developing and sustaining successful first-year programs: A guide for practitioners. San Francisco, CA: Jossey-Bass.

Hankin, J. N., & Gardner, J. N. (1996). The freshman year experience: A philosophy for higher education in the new millennium. In J. N. Hankin (Ed.), The community college: Opportunity and access for America’s firstyear students (Monograph No. 19, pp. 1-10). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.

Kuh, G. D. (2010). High-impact practices: Retrospective and prospective. In J. E. Brownell & L. E. Swaner, Five high-impact practices: Research on learning outcomes, completion, and quality (pp. v-xiii). Washington, DC: Association of American Colleges and Universities.

Upcraft, M. L., & Gardner, J. N. (Eds.). (1989). The freshman year experience: Helping students survive and succeed in college. San Francisco, CA: Jossey-Bass.

Young, D. G., & Keup, J. R. (2019). Council for the Advancement of Standards (CAS) first-year experience cross-functional team framework. Fort Collins, CO: Council for the Advancement of Standards in Higher Education.

Contact

Dallin George Young youngdal@mailbox.sc.edu