A Digital Solution Toward Student Success Prepared by Anthony Moore June 30, 2018
TABLE OF CONTENTS
2
Problem Statement..............................................................................................1 Background .........................................................................................................2 Research ...............................................................................................................6 Predictive Analysis .......................................................................................8 Student Input and Research....................................................................... 10 Changing the Narrative.............................................................................. 12 Colors and Iconography.................................................................................. 14 Personas............................................................................................................. 16 Low Fidelity Wireframes................................................................................. 20 High Fidelity Wireframes................................................................................ 26 Technical Specifications................................................................................... 37 Interaction Flows......................................................................................... 38 Design........................................................................................................... 40 Backend........................................................................................................ 40 Frontend....................................................................................................... 41 Deliverables....................................................................................................... 42 Resources........................................................................................................... 50 Footnotes...................................................................................................... 50 Further Reading........................................................................................... 51
PROBLEM STATEMENT
4
In August of 2012, Lindsey Wilson College moved to a hosted solution in order to help the institution communicate with at-risk students and help them to be successful. The system was met with mixed emotions from faculty and never really had full buy in. The institution signed an agreement to use the software at a discounted rate. However, in 2017 the institution was informed that the cost of the system would increase by more than double by July 1, 2018. Given the fact that the system did not have clear buy in from faculty and the increase in cost, the institution moved to develop an in-house solution.
1
BACKGROUND
2
Before the purchase of the hosted solution, Starfish, the institution used an in-house solution that I had developed in 1999 called the Early Warning System (EWS). EWS facilitated communication between faculty and advisors when instructors identified the students that were struggling in class. These warnings were purely reactionary in nature and depended on the intervention of the instructor to alert coaches and advisors that the student was not performing well. Starfish functioned much the same way by sending out email notifications to the student’s success network (advisors, coaches and etc.) encouraging them to get involved with the student. EWS was used by the college until August of 2012 as the needs of the institution had outgrown its design and the college looked for a better solution to help with student retention and academic success.
Background
3
With EWS, we were able to cater to the needs of the college without the need of “excessive” options that is typically included in a design that services a larger user base. Because of this, faculty and staff largely enjoyed EWS and its ease of use which more or less fed into the displeasure with Starfish as its interface was – clunky. In October of 2017, after the decision was made to ditch Starfish in favor of developing our own product, I decided to assemble a design team. This design team was eventually made up of faculty, staff and coaches. The purpose of this design team was twofold.
4
Background
First we wanted to make sure that faculty would be on board with this change as opposed to having a system handed to them without any input. At this stage (as well as subsequent requests for feedback) everyone had the opportunity to get in on the action and influence the design. Secondly, I needed a sounding board. With so many individuals contributing ideas I would need some way to quickly vet any design decisions that was being kicked around. This group would be privy to all aspects of the design process and could weigh in on the ideas that they felt were important to the project. The design team would provide timely feedback on • System features • Visual design • Usability testing • Content strategy To ensure everyone was able to take part in our discussions we utilized • Google Groups – to provide forum based discussions • Google Meet – virtual options for every on-site meeting
Background
5
RESEARCH
6
As I began to research how other institutions were using Early Warning systems it became clear to me that having a EWS system was not the silver bullet towards student retention that institutions might want it to be. This information need to be disclosed up front to the design team so that we would be clear on our expectations.
In the article “How to Help Students Achieve�, George Kuh identified six key areas that institutions needed to be address in order to help their students. [1] 1. Teach first-year students as early as possible how to use college resources. 2. Make the classroom the locus of community. 3. Develop networks and early-warning systems to support students when they need help. 4. Connect every student in a meaningful way with some activity or positive role model. 5. If a program or practice works, make it widely available. 6. Remove obstacles to student engagement and success What remained was the single impression that this new system needed to be very effective in the role it played regarding student success. Research
7
Predictive Analysis In both EWS and Starfish, identifying at-risk students had been predicated upon the fact that faculty would identify students that were doing poorly such as not attending classes, failing quizzes or exams. Many times it was too late to help the student because the institution quite simply didn’t recognize the problem early enough. Identifying at-risk students in this way had long been a standard in higher education. However in recent years institutions have been turning to big data and learning analytics in order to help identify at risk students. [2] Most systems rely primarily on student grade information and login frequency; developers typically display these data in relation to relative class performance. However, we are now at a point in time where it is possible to investigate which additional data points can further explain the variation in student course outcomes and thereby recognize both successful and unsuccessful individual-specific behaviours to provide more personalized feedback when designing interventions. [3] The notion of developing a predictive system that could identify potential learning issues before the first class became appealing to me. My team and I quickly identified the different data sources, or
8
Research
performance indicators that we had in place that we could use to help create a predictive product. • • • • • • •
•
•
Blackboard Attendance Data or EWS attendance tracking? Use of Blackboard? - # of LMS logins Blackboard Grades Incident report violators – in house system used to track behavioral issues on campus SLO Data – measurement of student growth in certain metrics Cell Group data – perfomance indicator that uses ACT and HS GPA to predict 1st year retention Student Engagement – are students going to campus activities Tutoring Center Usage – does the student use tutoring services, if so, how often? % of attempted hours to completed hours – measurement of financial jeopardy
With this data in place we could actually look at student success in each class offered at the institution and see what, if any, factors contributed to success. For instance, a student that meets a particular demographic may do well in the Humanities but struggle in Mathematics. If we were able to use big data to determine this ahead of time then the institution could Research
9
become more proactive with student providing critical resources before the problem ever presented itself in the classroom. This idea was pitched to the design team and even though the evidence was clearly strong to develop such a feature, the team felt that it would create too much bias on the part of an instructor. Their main concern is that some faculty would see it as an opportunity not to engage a student since the system had identified them as a potential at-risk student. At the time the decision was made I saw this as a setback in the design; however, that all changed when we actually began to talk to students about Starfish.
Student Research and Input I asked one of my team members to go out and talk to students about how they felt about Starfish. Did they know what it was? How did the system make them feel? Was it effective? Eight students were interviewed and the results were telling. Two of the students admitted that they would not be in college if it were not for Starfish. Others said that it was stressful. Of course learning this only made us want to know more. So I developed a survey and administered it to our student body.
“
Starfish makes me feel worse about myself.
10
Research
�
Indifferent
9%
Positive
35%
Negative
56%
Students respond to how Starfish made them feel abut themsleves
“
”
It gave me anxiety and depression
What we learned from the data is that far more students experienced a negative impact from being “Starfished” than a positive impact. Of the students that had been Starfished, 56% felt that it was a negative thing. This was alarming. How could something that was meant to help our students being sending such a negative message?
Research
11
Changing the Narrative The story needed to change. True, if a student failed a test then they failed the test, but could we engage the student differently so that they could sense our concern better? As it stood a lot of students saw Starfish as a “tattletale” system where instructors would report failures to coaches or others without first engaging the student. We wanted to create a different narrative than what was being told with our early warning system. We needed to re-brand the software. Then name needed to change, but not just the name, we needed to change our language too. Instead of faculty raising ”Flags”, they now raised “Concerns”. Instead of advisors and coaches being confronted with “Issues”, they were now presented with “Opportunities”; opportunities to engage their student. This subtle change in our narrative proved to be critical in how the system was received.
12
Research
I pitched several ideas for the name of the system to the design team and solicited ideas of their own. In the end the name selected for the system was Engaged for Success or E4S. This name encapsulated everything we wanted the narrative of this system to become. We wanted our students to sense that we were engaging them to help ensure their success. In addition, we realized that we needed to change the way that our message to our students was delivered. [4] [5] Students complained about faculty members sending out a flag that simply stated “Johnny missed class on Wednesday” or “Sally failed her last exam.” Students already knew they didn’t attend class or that they didn’t perform that well on the test. Quite simply - we needed to be better. To help with this I solicited the assistance of students to write templates that we could incorporate into E4S that would set the tone for the instructor on how they would interact with their student. The goal as to eliminate or reduce the tension that students were feeling under Starfish while recognizing that there indeed was a problem. Examples of these templates will be discussed further in the design section of this document.
Research
13
COLORS AND ICONOGRAPY
14
To complete the personality of the application, I wanted to finalize our story with a new logo and iconography. If we were going to change the culture in which we interacted with our students then everything about this application needed to send a clear message.
The logo itself was designed to highlight the student supported by those in her support network. The icons would impress on faculty that they were no longer raising a flag about a student but that rather openly participating in a genuine concern.
Colors and Iconography
15
PERSONAS
16
Personas
17
18
Personas
Personas
19
LOW FIDELITY WIREFRAMES
20
Notes: • The landing screen for the application will show recent activity for the user’s students. • “My Students” UI provides faculty a way to quickly see her students. • User’s are provided a way to switch between semesters. (This was removed in a later iteration.) • The use of the word “Flags” was prior to the discussion with the Design Team regarding rebranding the application.
Low Fidelity Wireframes
21
Notes: • Student Menu UI for direct access to student information, search feature to help locate students and UI to allow user to select particular classes.
22
Low Fidelity Wireframes
Notes: • Student Detail with menu options at the bottom to select different student information. This presentation would be simplified in a future iteration.
Low Fidelity Wireframes
23
Notes: • Floating action button (FAB) provides a UI to create flags (concerns), kudos, notes and files.
24
Low Fidelity Wireframes
Low Fidelity Wireframes
25
HIGH FIDELITY WIREFRAMES
26
Notes: • Search UI was made persistent on main screen. • It was determined there was no need to switch between semesters so the UI was removed. • Student Menu UI was moved to top right of screen. • Toggle UI introduced a way to change the time frame of recent activity. • It is in this iteration that Flags were introduced as Concerns and Issues as Opportunities.
High Fidelity Wireframes
27
Notes: • From the Student Menu, users can tap the student UI to access detailed information. • Display all students or particular class. • Attendance can be taken and saved to database by clicking the Save FAB. • E4S made it easy to email a group of students or their instructors.
28
High Fidelity Wireframes
High Fidelity Wireframes
29
Notes: • Student Search UI provides visual feedback to help guide the user quickly to the student they are locating.
30
High Fidelity Wireframes
Notes: • In this iteration I removed the navigation at the bottom that switches from the different types of student information. In later iterations the main menu is reintroduced and is made persistent on all screens. I felt this would be clearer to our users than using contextual menus.
High Fidelity Wireframes
31
32
High Fidelity Wireframes
Notes: • If the instructor had Jared in more than one class then the Select a Course UI would appear, otherwise it would not. • In future iterations this page was reworked to include the interaction templates for instructors.
High Fidelity Wireframes
33
Notes: • Application would be fully responsive. • At 720px and above, the menu moves to the top and the student list UI persists on the left.
34
High Fidelity Wireframes
High Fidelity Wireframes
35
TECHNICAL SPECIFICATIONS
36
In order to properly address each interaction within the application, I created a series of interaction flows that detailed how each microinteraction should behave. “The difference between a product you love and a product you tolerate is often the micro-interactions you have with it.� [6] This not only helped me give attention to the details that would deliver a better user experience but it allowed me to communicate more effectively to anyone that would assist in writing the code.
Technical Specifications
37
Sample Interaction Flow
38
Technical Specifications
Technical Specifications
39
Design • Adobe Illustrator was used to create some visual assets for the application. • I used Axure to create all the wireframes, prototypes and interaction flows.
Backend • A PL/SQL programs was written to nightly update the data that relates to students to individuals in their success network (coaches, advisors, administrators, etc.) This was put into place due to the complexity of the relationships in the ERP and drastically increased performance time for the end user. • An Oracle database trigger was put in place that inserted data into main concerns table when certain events happened such as a student having three unresolved concerns. • ColdFusion was used to provide access to database objects that could build dynamic pages and in some cases simply return JSON objects to the front end.
40
Technical Specifications
Frontend • The application is a single page app (SAP) that uses vanilla JavaScript and jQuery. • Media Queries are used to provide the different views depending on the viewport size of the device. • JQuery Mobile was used primarily for swipe events. • The basic architecture of the front end is such to provide the user the ability to interact as soon as they can after launching the application or certain screens. The design utilized AJAX calls to the server to populate sections of the UI that are not as critical to the user under given situations. • Infinite scrolling was implemented so that data is retrieved from the server only as the user needs it.
Technical Specifications
41
DELIVERABLES
42
It took just under three months to code the project once the design had been finalized and was delivered one month ahead of schedule. Including myself there was one other person that worked on the project.
Notes: • To make the navigation consistent, the Settings UI was moved with the rest of the navigational UIs as it is in the Desktop view. • When no data is retrieved, the E4S image is displayed.
Deliverables
43
* Screen shots are taken from the actual application. As such, sensitive data has been pixelated throughout ths section.
44
Deliverables
Notes: • Tapping on a card UI allows users to participate in the discussion of the concern.
Deliverables
45
Notes: • Student menu (left.) • When user taps on a student UI it opens the detail for that student (right) allowing access to information and allows the ability to raise concerns and kudos or create notes and attach files to the student’s record.
46
Deliverables
Notes: • Creating a concern for the selected student (right.) Note the template in use that helps set the tone for how the instructor interacts with that student.
Deliverables
47
Notes: • Student menu with options for emailing students and taking attendance.
48
Deliverables
Notes: • Student menu with options for emailing students and taking attendance. • Search UI was moved back into student list UI. This provided better consistency between the Desktop and Mobile versions.
Deliverables
49
RESOURCES
50
Footnotes [1] Kuh, George D. "How to Help Students Achieve." Chronicle of Higher Education, vol. 53, no. 41, 15 June 2007, p. B12. [2] Phillip Long and George Siemens, “Penetrating the Fog: Analytics in Learning and Education,” EDUCAUSE Review, September 12, 2011 er.educause.edu/~/media/files/article-downloads/erm1151.pdf [3] Waddington, R. Joseph, et al. "Improving Early Warning Systems with Categorized Course Resource Usage." Journal of Learning Analytics, vol. 3, no. 3, 01 Jan. 2016, pp. 263-290. [4] Sloan C. “Teenagers in the Ivory Tower: Engaging and Retaining Traditional College Students.” Change: The Magazine Of Higher Learning [serial online]. January 1, 2013;45(2):35-39. [5] Garcia Sánchez C, Gillings de González B, López Martínez C. “The Impact of Teacher-Student Relationships on EFL Learning.” How [serial online]. October 1, 2013;20(1):116-129. [6] Saffer, Dan. Microinteractions: Full Color Edition: Designing with Details (p. 3). O’Reilly Media. Kindle Edition.
Resources
51
Additional Reading Pfleging, Elizabeth. “An Evaluation of the Early Alert Program at Columbia College.” May 2002. EBSCOhost search.ebscohost.com.library.acaweb.org/login.aspx?direct=true&AuthType =ip,url,uid,cookie&db=eric&AN=ED478596&site=ehost-live. Macfadyen, Leah P. and Shane Dawson. "Mining LMS Data to Develop an 'Early Warning System' for Educators: A Proof of Concept." Computers & Education, vol. 54, no. 2, Feb. 2010, pp. 588-599.
52
Resources