"Seek" HCI Design Project

H541: FINAL PROJECT PART 4 PHASE 4: Final Report Delivery

Team Juliett: Joshua Hansen Echo Lu Pratik Nalawade

Table of Contents PHASE 4: Final Report Delivery Team Juliett: Table of Contents Problem Space and Requirements Gathering Domain Space Target Users The target users for the domain space are students, teachers and teaching assistant. #1 Interview Summary with Professor #2 Interview Summary with Teacher’s Assistant #3 Interview Summary with Student Questionnaire Key Findings from User Interviews Key Findings from Secondary Research Purpose and Goals Design (Conceptualization) Key Project Requirements Technical Requirements Functional & Design Requirements Brainstorm Key Scenarios iChat for Canvas Seek Chosen System Paper Prototype Link to Prototype: Paper Prototype Cognitive Walkthrough Walkthrough 1 Walkthrough 2 Walkthrough 3 Summary Recommendations Dynamic Prototype Link to Prototype: Dynamic Prototype Validation (Product Assessment)

Usability Study Findings Task Error Post-Task Questionnaire Interview Summary of Product Assessment Problems with the Product Recommendations for Future Development Appendix References

Problem Space and Requirements Gathering

Domain Space Our social computing problem lies in the domain space of education and communication between students, their peers, and professors. Currently, students can use multiple means of communication when trying to reach other students and their professors. Some of the methods of communication that exist are: university email, Canvas email, Canvas chat, Canvas announcements, Facebook, SMS text, and phone calls. University and Canvas email feels outdated and students have no way of knowing if their email is being read or ignored by the recipient. Canvas announcements are helpful if they address the particular problem the student is having, but if not then they become irrelevant. Facebook, text, and phone calls could work to bridge the gap of communication, but not everyone has Facebook, and giving out a phone number may feel like a violation of privacy to some students and professors.

One of the main problems with many of these services is that they aren’t real time.

Students have to wait hours and sometimes days to receive responses back from professors. While there may be certain issues that we won’t be able to address with our design solution, we do hope to create a workable solution that will benefit both students and professors.

Target Users The target users for the domain space are students, teachers and teaching assistant.

#1 Interview Summary with Professor

We interviewed a female graduate professor. The interview took place in one of her classrooms during class time, but during the group project time. We gathered some valuable information about her communication practices between herself and her students and TA. She receives anywhere from 30-40 emails a day, while 1-3 of those are from students. The best way for her to communicate with her students is through email, because it creates a digital trail that she can later refer to, if needed. It typically takes her anywhere from an hour to four days to respond back to student’s emails. The fluctuation in time is dependent on several factors including: severity of the problem, professor’s schedule, and miscommunication with her TA. Her TA is her ‘first line of defense’ from student’s emails. Sometimes though, she will have an email in her inbox that she doesn’t think she needs to respond to, because she thinks the TA will take care of it, but the TA has already looked at it and is unsure how to respond back. The interview provided some good findings that will help address our domain space’s problem.

#2 Interview Summary with Teacher’s Assistant The second interviewee is a 26 years old male second-year graduate student, he is currently a teaching assistant of a undergraduate level class with 19 students, it is his third semester working as a teaching assistant in his program. The interview took place in his off-campus apartment. Outside of class lecture, he has lab hours once a week, to help students with classwork. He also has office hours every week, students can also schedule appointment to meet with him individually in the library. He usually uses email to communicate with his students, and sometimes students ask him questions right after lectures since he attends every lecture class. When it is close to deadlines or during midterm or final week, he gets a lot of

emails in a week and stays longer in the classroom after lectures to answer questions. Some students use WeChat to ask him questions. If a class is highly depend on Canvas, he gets more emails from students about due dates, quizzes, online submissions, etc. Students tend to ask complicated questions in office hours and lab hours in person, like questions related to semester-long projects. Usually he answers emails in the same day or within the next day.

Students tend to ask similar questions in the beginning of the semester, like confusion about assignment instructions and class structure, and he will learn from these questions and find out about the need of the students and make announcements on Canvas so they will not ask similar questions again. If no student shows up during office or lab hours, he would do his own classwork, he does not answer emails during these hours. He usually answers emails during the night. He rarely uses Facebook messenger, never used Whatsapp, Telegram or Slack.

Sometimes he has to contact students himself, but most students take days or a week to reply his emails. He found that most students came to him just want to know if their work has the right result, but he cares more about if they fully understand the method and know how to get the right results correctly.

#3 Interview Summary with Student

The third interviewee is a 25 years old female second-year graduate student. The interview took place in her off-campus apartment. She does not communicate with her professors very often outside of class. This semester she only talked with her professors twice, both were right after class. Her questions for professors as a second year graduate student are usually about class projects. When she was a first year graduate student, she communicate with her professors more often outside of class, and her questions were mostly about how to adjust to the program with an unrelated undergraduate program, sometimes she asked for career advices and issues with teammate. Most of her professors in the program respond to her emails very quickly, the longest time for them to respond are a few days, but one of her professors took weeks to answer her emails. She uses email, Slack to communicate with her professors outside of class. Emails are for formal questions, and for schedule appointment to meet in person, and she prefers to meet in person than talking in emails; Slack are for informal questions and quick questions. One of her professors stated that he prefers to

use Slack as the major communication tool outside of class in the beginning of her program, so all his students use Slack to communicate with him. And all students in her program use Slack to communicate with each other, it is a kind of culture in her program. She also mentioned that the conferences related to her program that she attended all use Slack to communicate with attendees, like announcements, and she thinks it is a culture of the industry. She does not use Facebook messenger, but she had received messenger messages from students who does not know any of her contact information, and by only knowing her name, the quickest and easiest way for them to find her is Facebook messenger. She does not use Whatsapp, fellow students in her program who are from India always ask to add her on Whatsapp, but she does not like to use it. She never used Telegram.

Questionnaire Interview questions for student ●

Do you ever have the need to get communicate with the professor outside of class, and

if so, how often? ●

What are some of the questions you ask the professor?

●

How long does it take for you to get a response back from professors?

●

How difficult is it to communicate with the professor outside of class?

●

What means of communication do you use when you want to reach the professor?

●

Do you ever try and communicate with other students outside of class, and if so, what

methods do you use? ●

How long does it take to receive responses back from other students?

●

How familiar are you with messaging services like, facebook messenger, whatsapp,

telegram, or slack?

Interview questions for teacher ●

How do you communicate with students outside of class?

●

What do you find is the best way to communicate with students?

●

How often do you receive emails from students?

●

Do students try and communicate with you in other ways besides email, if so, what are

those ways? ●

How long, on average, does it take for you to reply back?

●

Do you get a lot of the same questions from multiple students?

●

Do you use your office hours to respond back to student inquiries?

●

How familiar are you with messaging services like, facebook messenger, whatsapp,

telegram, or slack? ●

Are there any issues that you have faced when communicating with students?

Key Findings from User Interviews ●

Teachers are very busy with other activities and may ignore emails till they have free

time ●

Teachers prefer to send information via email as it leaves a trail

●

The time to reply on emails is between 4 hours to 2-3 days

●

More emails are received during the start of semester and when a deadline is

approaching ●

Although students are familiar with major instant messaging applications they do not use

them to communicate with teacher until the teacher asks for it

Key Findings from Secondary Research In our secondary research, we have 2 scenarios one in which a survey was conducted to understand how text and instant messaging is viewed by the students as a form student-teacher communication [​1​]. The other scenario was where Instant messaging was implemented as a means of communication after which students and teachers were observed to understand if

Instant messaging was an effective means of communication [​2​]. These are the key findings from this research ●

Text and instant messaging is widely used in peer communication

●

Existing instant messaging(IM) applications are not used as a medium for student

teacher communication as students feel IM is informal

●

Students check texts and messages more often than Email

Purpose and Goals The goal of our design is to improve student-teacher communication. Although emails are widely used for communication, they are not very effective since same questions can be asked by multiple people and the teacher has to consolidate all the queries. The response time for emails can be as long as 2-3 days which defeats the purpose since most queries are sent before and deadline. This form of communication requires something more immediate like real time messaging but more organised like emails. Based on interviews and our secondary research it was found that Instant messaging is used a means of communication but hasn’t been effective in student-teacher communication. The reason being that these Instant messaging applications are used more for social communication and are reported to be distracting when used for academic purposes. Another problem is these applications are used for social and private communication where the method of communicating is very informal while student-teacher communication is a more formal communication.

With our design, we would like to achieve the following: ●

Create a system that is real-time like instant messaging applications but is primarily used

for student-teacher communication ●

Create a system that will help teachers respond to queries in a timely manner without

taking too much of their time ●

Improve the flow of information to reduce the load on Teacher and Teaching assistants

Design (Conceptualization) Key Project Requirements Based on the user interviews and secondary research we outlined the following goals for our system ●

Create a system that is real-time like instant messaging applications but is primarily used for student-teacher communication

●

Create a system that will help teachers respond to queries in a timely manner without taking too much of their time

●

Improve the flow of information to reduce the load on Teacher and Teaching assistants

For creating a system that meets those goals we envision a system with the following requirements

Technical Requirements ●

The system should be compatible across mobile and desktop computers

●

The system will need a server to facilitate exchange of messages

Functional & Design Requirements ●

The system should have separate functions for students and teachers

●

The system should allow teacher to communicate with a group of students

●

The system should allow teacher to send out announcements

●

The system should allow one to one teacher-student interaction

●

The system should be accessible to all students

●

Data across the system should not be ephemeral

●

The system should allow teachers and teaching assistants to delegate tasks

●

The system should allow teachers to define and implement office hours

●

The system should allow student and teachers to be part of multiple groups

Brainstorm

Key Scenarios iChat for Canvas

In this scenario students use a chat messaging service to ask TAs and the professor questions during class. While the professor is lecturing we see that, in the 1st sketch, the student has a question. Instead of interrupting the professor in the middle of a lecture, the student logs into Canvas. Canvas has a static chat box labeled iChat that stays open for the students while they are in class. The TA monitors iChat and answers any questions the students may have. All of the students can see what each other are typing, so if multiple students have the same question, only one student needs to ask the questions. This type of system would only work during class, but the professor could continuously lecture without being interrupted to

answer questions, unless the TA could not answer a particular question. If that circumstance were to occur, then a message would appear on the professor’s computer.

Seek

In this scenario students use an mobile app to communicate with instructors, teaching assistants and classmate during the semester. At the beginning of each semester, students sign up for the groups of each class, the instructor and teaching assistant will be the admins of these groups. Each week, instructor and/or teaching assistants will host office hours on Seek to answer questions. Conversation will be saved for other students who cannot show up online during office hours. Student also can have informal chat in the group. Keywords and tags can

be add to each question so it can be easily grouped and searched. Different channels for specific class subjects can be created by students, in order to discuss similar stuff.

Chosen System We chose to move forward with the Seek system. Seek fulfilled the most requirements out of all of our ideas. Seek will be a real-time messaging app, similar to Telegram or Slack. But instead of apps like Telegram or Slack, Seek will only be accessible to post in at certain times of day. We don’t want Seek to bombard or overwhelm the teacher’s assistant or the teacher. The teacher will set up times that ‘open’ up the app for students to type out questions. Also, to limit the less important questions, each student will only be allowed to ask 2 questions per week. We don’t want students using this app to ask when assignments are due, when they could find that information in the syllabus. The app will keep everything on record and allow for multiple discussions to be created, so that not all questions will go in one chat. Seek will also allow teachers to make short announcements that pop-up for students when they first open the app.

The TA will be the first responder for most of the questions, but if there are questions that the TA can’t answer, he or she can flag them so the teacher knows that they must take a look at them. Every technical and functional requirement will be addressed by the use of Seek.

Paper Prototype Paper Prototype was submitted as a separate file.

Scenario: You are a graduate student enrolled in three classes at IUPUI. In two classes you are a student and in another you are a teaching assistant. As a student you want to use the messaging app to ask a question in your human computer interaction practice class. You want to ask about the project requirements for assignments 2 and 3. After the question is asked, you want to check out the general channel to see if any relevant comments have been left by other students. After finding out that there are no relevant comments, you want to return back to your class selection.

Now you want to enter the class that you are a TA for. Find the question that hasn’t been answered yet and answer it by letting the student know that they are to add 2 pictures. When finished, return back to class selection and logout.

Cognitive Walkthrough Walkthrough 1 CWR Number: CW01

Product Name: Seek – Student Teacher communication

Task Name: Asking teacher, a question using the application

Date and Time of Study: 11/15/2016

Experimenters’ Names: Pratik Nalawade

Task Description Task is for a student user who has a doubt regarding an assignment for the human computer interaction practice class. The student wants to get it clarified from the teacher.

Task Action sequence 1.​ 2.​ 3.​ 4.​ 5.​

​User:

Open application System: Login screen is displayed ​User: Enter credentials and click login System: User’s enrolled classes are displayed ​User: Select a class System: Class details are displayed ​User: Add a new question System: Question page is displayed where user will enter the data ​User: Submit the question System: Question is displayed with other questions

Walkthrough 2 1.​ ​User: Open application System: Login screen is displayed CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the user was presented with the login screen

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes

2.​ ​User: Enter credentials and click login System: User’s enrolled classes are displayed CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the user can see their enrolled classes and the respective office hours

If the user does the right thing, will they know that they did the right

Yes

thing and that they are making progress towards their goal?

3.​ ​User: Select a class System: Class details are displayed CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the user can see their enrolled classes and the respective office hours

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes, the class name is displayed on top on selecting a class

4.​ ​User: Add a new question System: Question page is displayed where user will enter the data CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the plus icon is a good indicator that the user can add a new question

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes, the system displays questions page

5.​ ​User: Submit the question System: New question is displayed with other questions CW Question

Issue?

Notes

Will the user know what to do at this step?

No, the submit and cancel buttons are on top while user flow is top to bottom

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

No, Although the question is added to the list of questions there is no message indicating the action was successful

General Notes​: There is no unread message count on the screen. There is no indicator suggesting if the question has been answered or not for other users.

CWR Number: CW02

Product Name: Seek – Student Teacher communication

Task Name: Answering a question from a student

Date and Time of Study: 11/18/2016

Experimenters’ Names: Echo Lu

Task Description The task is generated for teaching assistant, to answer a question on Seek that a student asked.

Task Action Sequence 1.​

​User:

Open application System: Login screen is displayed 2.​ ​User: Enter credentials and click login System: User’s active classes are displayed, including both classes that the user is taking or teaching 3.​ ​User: Select a class System: Class details are displayed 4.​ ​User: Open up a posted question System: Question page is displayed where user can access the data 5.​ ​User: Submit the answer for targeted question System: Answer is displayed under the question 6. User: Go back to the previous page System: The button for unread message is unchecked

Walkthrough 3 1.​ ​User: Open application System: Login screen is displayed CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the user was presented with the login screen

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes

2.​ ​User: Enter credentials and click login System: User’s all active classes are displayed, including both classes that the user is teaching or taking CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the user can see all the active classes separated by names

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes

3.​ ​User: Select a class System: Class details are displayed CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes, the class name is displayed on top on selecting a class

4.​ ​User: Open up a unread posted question System: Question page is displayed where user can access the information CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes, the red button is a good indicator that the user can open up to see the new questions

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes, the system displays questions page

5.​ ​User: Post the answer to the question System: Answer is displayed after the question CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes

6.​ ​User: Go back to previous page System: The button for unread message is unchecked CW Question

Issue?

Notes

Will the user know what to do at this step?

Yes

If the user does the right thing, will they know that they did the right thing and that they are making progress towards their goal?

Yes, the system shows there is no more unread messages

Summary The core tasks for our application is 1) to post a question for the teaching assistant or instructor to answer and 2) to answer a question from a student. After the walkthroughs we have uncovered some usability problems. First of all, the ‘submit’ and ‘cancel’ button are on the top of the screen, the user would have to scroll the page to the top to use these buttons. Second, there is no pop-up system message telling the user that they have successfully done their tasks.

Third, there is no unread count message displayed anywhere on the screen so the user cannot tell how many messages they will be attend to and how many of them are left.

Recommendations 1.​ ​Moving the submit and cancel buttons on the questions page​: The buttons can be placed at the bottom so that the user can immediately submit after writing the body of the message 2.​ ​Toast Messages​: Once the user has performed an action, a toast message should appear indicating the result of the action. E.g. On submitting the question, a toast message should appear saying “Question submitted successfully” 3.​ ​Unread count: ​There should be an unread count against the group messages, so the user knows there are messages that he/she has not read.

Dynamic Prototype Link to Prototype: ​Dynamic Prototype

Validation (Product Assessment) Usability Study Findings Task Error The summary of the task error result shows that 1) unread messages are hard to find, because the symbol is not eye catching. This error has prevented evaluators from performing the main task in a reasonable amount of time. A more eye-catching symbol like number in red color could be used to replace the current one. 2) Where to see posted questions and where to

post new questions features takes some time to find. This error happened in relation to the first finding, if the first error could be fixed this error should not happen again. 3) Group feature is easy to locate but an evaluator suggested it be placed into its specific class, and the status of group subject submission is hard to find. Because of this, it can be hard for TAs to perform tasks, but different font color, background color, or check marks should be able to solve this problem.

Post-Task Questionnaire From the questionnaire we learned that there are many aspects that our design can be improved upon. Overall the evaluators find our design clean and pretty, especially the login screen. One of the users stated that they are unsure whether this design could solve our problem space. The reason why this evaluator said that is that our design still has some improvements to make and its functions cannot be fully experienced in our current prototype.

They have suggested some symbol displays changes and moving some functions to a more appropriate space. For example the group should be moved under each class. An inbox is suggested to be made in order to help the navigation. We found that very helpful.

Interview ​In the interviews, evaluators discussed their logged information in the task error sheet

and questionnaire in details which helped us better understand the problem of our current design and possible future changes. All evaluators expressed that it is a pleasureable experience to use the app because of its beautiful design. 75% of them believe our design could solve the problem space and are willing to use the app in real practice. Their biggest concern is the navigation of the app, which confused them when doing the task. We would take the suggestions to move group under each class, and change the unread message notice icon. We might add an inbox function for the messages. After redesign another round of evaluation would help us to find other problems that were not found during these evaluations. Besides that we found these evaluations were helpful and only minor changes should be made to our current design.

Summary of Product Assessment

Problems with the Product 1. User flow​: During our evaluations we found out that the user flow was not intuitive enough which caused the evaluator to keep searching for options to perform the task.

On logging in, the application presents the user with menu instead of the user dashboard; this particular flow was not appreciated by the evaluators.

2. Read/Unread distinction​: The current green and red indicators of read and unread questions are not clear to most of the evaluators. The indicators are also very small so it becomes difficult for the user to find unread questions when there is a large list of questions.

3. Consistency across application​: The menu on user dashboard and subject dashboard is activated by clicking on the “hamburger menu” icon while the menu is accessed via dropdown when user is writing question or sending message to students/teacher. This is inconsistent and only one interaction should be followed across the application. The flag indicator is an image while the button to flag the question is textual, this also needs to be consistent across the application.

Recommendations for Future Development 1. Inbox​: An inbox feature where the user can see his/her asked questions will make it easier for the user to keep a track of his queries. In current design the question may be lost in the list of other questions submitted by other users. This inbox feature could also indicated read and unread messages/questions using the current email client design guideline since user will be more familiar with it.

2. Improve user flow​: On logging into the application the user should be navigated to the user dashboard and not the menu. Actions that are relevant to a particular subject and most used should be moved from the side menu to the main screen so that they are easily accessible.

3. Tweak UI​: Although the current UI was appreciated by the evaluators , it is not consistent across the application. Text fields in group tab are hard to read and need to improved/replaced with relev 4. ant iconography. Iconography for the “flag” feature needs to more consistent across the application. Button design in the header section of message needs a redesign.

5. Onboarding​: The user should be presented with onboarding screens on his/her initial login. This will help the user to get familiar with user flow and terminologies used in the application.

Appendix Instructions to carry out study One page presentation outline Task Error Recording Sheet Online Questionnaire

References 1. Sharon Lauricella and Robin Kay, EXPLORING THE USE OF TEXT AND INSTANT MESSAGING IN HIGHER EDUCATION CLASSROOMS, Sept 2013, http://www.researchinlearningtechnology.net/index.php/rlt/article/view/19061 2. Wooseob Jeong, Instant Messaging in On-Site and Online Classes in Higher Education, Jan 2007, http://er.educause.edu/articles/2007/1/instant-messaging-in-onsite-and-online-classes-in-highereducation