Learning Systems at Quest Alliance January 14, 2013 Eva Miller, Design Fellow Design Impact
Š Design Impact 2012
Quest Alliance Creates programs and services to prepare youth for a better future: Quest is a non-governmental organization that designs job training solutions for young people who have been failed by India’s educational system. English language and vocational training matter, but the true heart of Quest’s approach is life skills and work readiness, with a growing emphasis on active learning methods. Students understand themselves better, grow in confidence, and see work and education as a path to greater goals for their lives. Works with partners to deliver programs and services: Quest Alliance does not want to be in the service delivery business and creates partnerships with industry, education, and vocational organizations to deliver their programs. They also train facilitators to assure Quest’s educational philosophy becomes reality in the classroom.
Quest Alliance offices in Bangalore, India
© Design Impact 2013
Create a system for learning that scales: Design structured information that works across all programs, from a single piece of active learning to an online environment that supports learners, facilitators, partners, and Quest staff
The project challenge
Š Design Impact 2013
ONE: Diary/Field Study to better understand the daily lives of our initial audience for a learning management system: facilitators
TWO: Create a content framework that encompasses Quest’s educational philosophy and clarifies active learning
Milestones from November 3 to January 15
THREE: Begin to define the features of the learning management system we’ll need
© Design Impact 2013
Identified… Facilitators who match the desired audience from earlier work with staff: “eager/overwhelmed teachers”
Three from the Equip Youth vocational education program and two from Skills to Succeed program Screened them and collected background information Created a paper diary for participants Asked them to keep the diary for 7 days and make 3 entries per day ONE: Diary/Field Study
© Design Impact 2013
Waited‌ We began in late October, but many participants had delays because of festivals, holidays, or planned breaks between program batches Some do not teach every day, so it took 2-3 weeks to get enough entries from them Facilitators found it hard to schedule a time for me to visit their classroom, review the diaries, and walk through a typical day for them (field study). This took an extra 6-7 weeks to do ONE: Diary/Field Study
Š Design Impact 2013
Emerging themes… Time. Some neglect home life, many come home late, too tired to prepare for the next day. Travel is often an issue and can take as much as 3-4 extra hours every day. Most report preparing over 2 hours a day. Product opportunities: structured materials follow a pattern and take less time to “decode.” In active learning, most of the “content” is provided by the students, which lessens the need to commit a lesson to memory ONE: Diary/Field Study
© Design Impact 2013
Emerging themes… Keeping students engaged. Students don’t come or come late, and many can’t focus long on the lessons Product opportunities: connect curriculum more firmly to work preparedness, use more active learning methods, and run student design sessions to improve curriculum and see what is most relevant to their lives.
ONE: Diary/Field Study
© Design Impact 2013
Emerging themes… Student-Facilitator bond. Students crave the facilitator’s attention and the relaxed classroom atmosphere. Product opportunities: create continuous feedback through the system to give students attention that promotes performance. Support communication through social tools that can use mobile technology, like SMS messaging, but collect conversations in the system.
ONE: Diary/Field Study
© Design Impact 2013
Emerging themes… Support. Some wonder if their organizations know or care what they are doing. Co-trainers lean heavily on each other. Product opportunities: create online community that gives facilitators more colleagues with whom to share issues and ideas. Allow peers to review each other’s work. Make key leaders at the partner organization part of this community or create more continuous reporting to put success stories in front of them ONE: Diary/Field Study
© Design Impact 2013
Emerging themes‌ Perfectionism and passion. All spend several hours every day to be wellprepared and take it hard when class goes poorly. All expressed their love for this work and how rewarding it can be. Product opportunities: we can reward passion by calling out extra effort and letting other facilitators see this appreciation publically. We can help reduce the pressure to be perfect by minimizing process-heavy curriculum and using more active learning, instead. ONE: Diary/Field Study
Š Design Impact 2013
There is still one more diary and field study to collect—the last facilitator finally started to make entries. I also feel we should have another male facilitator complete a diary and field study to balance out this group. But there may not be time to do this.
Most of the field study interviews with facilitators are transcribed. All need to be analyzed for patterns and insights that can be converted into a “Day in the Life” facilitator journey. This is an experience map that calls out pain points and opportunities for our product.
ONE: Diary/Field Study
© Design Impact 2013
•
Festival season, holidays, weddings, and end-of-year vacations often meant I could not make progress because people were away for extended periods. I did not foresee this and could not plan accordingly. I had to move ahead with other design activities while waiting for diaries and appointments for the field study visits
•
The loss of a month means trimming the project plan. This will remove the iterations of internal and external workshops to perfect our content model. These can continue, but the design work will have to move forward with initial findings only.
•
Facilitators seem ripe for burnout. Most in our study are new to this field, but already seem over-burdened, though some report they are getting more comfortable in this role. All have other responsibilities at their organizations besides delivering Quest classes.
Concerns at this point
© Design Impact 2013
Content Framework Design Sessions Giving shape to Quest’s educational philosophy
TWO: Content framework and active learning
Š Design Impact 2013
Researching… How competitors and related organizations and experts define life skills and active learning Quest’s own life skills, work readiness, English, and retail materials to see what they reveal about its educational goals This research helped us create a first draft of our content framework: Quest’s educational goals
TWO: Content framework and active learning
© Design Impact 2013
Defining… Before our workshop, we revisited the user profiles for whom we are creating our content framework and learning system. We decided to add more detail to our facilitator profiles. To do this, we used some of the early findings from our diary/field study. We’ll be able to create something richer once this study is complete.
Ritu is a more “traditional” teacher and not our emphasis for this system
TWO: Content framework and active learning
© Design Impact 2013
Defining… “Rashmi” is the eager/overwhelmed facilitator who comes to the work with fresh eyes and some idealism. She is our primary focus as a facilitator using the system.
TWO: Content framework and active learning
© Design Impact 2013
Defining… We added more detail to our student user profiles, too. We revisited earlier work from Quicksand for this. When we do a content workshop with students, we’ll refine and enrich these. “Sahir” is more entrepreneurial and impatient to get started on a business venture of his own. He is not our primary focus as a student using the system
TWO: Content framework and active learning
© Design Impact 2013
Defining… “Soma” is a vocational student who has a goal to get a job to help her family but discovers there’s a bigger world out there for her.
She is not our main emphasis as a student using the system.
TWO: Content framework and active learning
© Design Impact 2013
Defining… “Santosh” is the main student audience for our system. He arose from our earlier stakeholder workshop to align key Quest staff around what they want from a learning management system. Staff most want to support the “aimless student” like Santosh, who wants to earn money but doesn’t have a clear goal in life.
TWO: Content framework and active learning
© Design Impact 2013
Testing‌ How staff view the content framework: categories and educational goals
How they would rank various educational goals in importance for our main user How they would rate these goals in complexity for our main user How they would categorize these goals themselves
TWO: Content framework and active learning
Š Design Impact 2013
Refining… We changed the content framework slightly to reflect staff feedback But we discovered our proposed categories are largely the same as staff. We didn’t reveal our category names and use only colors and images to indicate categories.
TWO: Content framework and active learning
© Design Impact 2013
Prototyping… We created a workshop using our new content framework to brainstorm classroom activities
Staff began by experiencing active learning through the “Marshmallow Challenge:” teams have 18 minutes to build the tallest freestanding structure they can from spaghetti, tape, and string
TWO: Content framework and active learning
© Design Impact 2013
Prototyping… We demonstrated that active learning has a structure we can follow: plan, do, give feedback, reflect, and apply to the future Then asked staff to pluck a goal from each category in the content framework and brainstorm as many “active learning” ideas as they could.
Still analyzing this session, but the framework is mostly solid Staff will need more direction on active learning TWO: Content framework and active learning
© Design Impact 2013
•
Staff are responsible for various programs and see them as distinctive from others. The idea of a content framework as a single foundation for Quest’s curriculum design and active learning as the structure for delivering it means programs do not matter, except as wrappers for content. This is a big change from the way Quest has developed curriculum in the past.
•
Doing only one iteration of activity brainstorming may not be enough. But there is not time for more.
Concerns at this stage
© Design Impact 2013
Defining system features Beginning to see how the system will work
THREE: Defining the learning system
Š Design Impact 2013
Discovering pain points‌ Staff wanted to respond to Accenture, a partner who is willing to help launch a learning management system to support Quest and other organizations they work with To discover the features we wanted most, we worked through pain points with the current Moodle learning management system and talked about what features might help.
THREE: Defining the learning system
Š Design Impact 2013
Finding solutions‌ We wrote a feature list for Accenture that listed everything that addressed our pain points
We included some features from earlier discussions with Accenture To help make our issues more clear, we included benefits to the business and to our users
THREE: Defining the learning system
Š Design Impact 2013
Hi,#Selvi:##Your#account#
MyQuest Your+Course+page See#all...
Class+progress
Goals+and+styles TOP+3
i6 es Find+ac6v : Filter#by
On#a#Saturday#a+ ernoon,#Selvi#comes#home#and#decides#to# catch#up#on#her#student#evalua7ons
BOTTOM+3
EVALUATE udents Your+st (28)
ed# review ##All#un #List#(10) ####Quick ts#not en d ######Stud ntly#evaluate ce #########re
Your+p 213#ac7 1C10#of#
vi7 es
eers
ly recent ###Not# ted#(14) ua ####eval t#(1) reques #####By# #
... n ew A d d# # have yone# g# an # es in Do out#us 7 ps#ab i#class?#I# # ic ... mus #like#to would w# e#a#fe I#mad zers#that# .. gi great. ener #work# really
DISCU
SS
Adding clarity… We responded to various questions second version of the feature list
She#pulls#up#her#course#page# to#have# a#look.#Selvi’s#class# from Accenture withis#a doing#well#in#some#areas,#but#not#others.# MyQuest Ac6vity:++ Created+by:+ Added:+ Comments:#
Quick+List:++++<+Last+item++|++Next+item+>
Sing#an#angry#song Harita#V,#Raj#K Oct#30,#2012 8#comments#|#Add#comment
Hi,#Selvi:##Your#account#
We completed and added some scenarios from earlier work to show how users will interact with the learning system we want
It’s#a#song#about#anger#from#Harita#and#Raj.#She#remembers# this#one!#Selvi#laughs#again#at#how#clever#it#was.
THREE: Defining the learning system
© Design Impact 2013
Adding clarityâ&#x20AC;Ś We added a few sketches, too. We wanted to show that we want to include a visual indicator of progress to make it easy to see how well a student, facilitator, class, or center is doing over time. Seeing that story through data is persuasive to partners and motivates students and facilitators
THREE: Defining the learning system
Š Design Impact 2013
The Project Plan Where are we at now?
DESIGN ACTIVITIES AND TIMELINE
© Design Impact 2013
Design Activity
Roles
Time estimate
Review background materials and programs
User experience lead: Eva
Sept 3-7
Plan and give stakeholder workshop on the learning management product
User experience lead: Eva
Sept 10-21
Establish main audiences for the product as a user profile for now
DESIGN ACTIVITIES AND TIMELINE: Learning
Stakeholders: Aakash, Amitav, Abhijeet, Ayeesha, Augustine, Ali, Ashutosh, Jayashree, Nikita User experience lead: Eva
DONE DONE
Sept 10-21 DONE
Š Design Impact 2013
Design Activity
Roles
Time estimate
Write and share “Voice of the Business” findings
User experience lead: Eva
Sept 24-Oct 1
DONE
Select design team
User experience lead: Eva Business owner: Aakash User experience lead: Eva
Oct 2-12
DONE
User experience lead: Eva
Jan 10-Jan 30 STARTED
Diary/Field Study to understand facilitators’ daily lives Create “day in the life” snapshots of facilitators to guide product design Establish active learning framework: draft version, based on competitive analysis and audit of existing content
User experience lead: Eva Curriculum designer: Lara Content lead: Ali Internal workshop to get staff feedback on active User experience lead: Eva learning framework: rank by importance, complexity, and Product owners: Nikita and Abhijeet free sort for feedback on our categories. Analyze results Curriculum designer: Lara and adjust. Content lead: Ali Internal workshop to generate draft activities: use User experience lead: Eva framework. Create staff workshop activities to clarify Product owners: Nikita and Abhijeet active learning and brainstorm activities in art, music, Content leads: Ali, Leena drama, film, games, etc. Analyze results and adjust. Curriculum designer: Lara Write draft activities User experience lead: Eva Content leads: Ali, Leena Curriculum designer: Lara External workshop using draft activities with students: User experience lead: Eva lead 4-5 students through activity, then explore what Product owner: Nikita and Abhijeet might work better. Explore student challenges and needs. Content leads: Ali, Leena (May need interpreter) Curriculum designer: Lara
DESIGN ACTIVITIES AND TIMELINE: Defining
Oct 12-Jan 30 STARTED
Nov 14-Dec 11
DONE
Dec 11-20
DONE
Dec 21-Jan 15 STARTED
Jan 22-30
Jan 22-30
© Design Impact 2013
Design Activity
Roles
Time estimate
Create student personas
User experience lead: Eva
Feb 18-19
Write pilot activities using the best ideas from student workshops
User experience lead: Eva Content leads: Ali, Leena Curriculum designer: Lara User experience lead: Eva Product owner: Nikita Content leads: Ali, Leena Curriculum designer: Lara Technology lead: Venky User experience lead: Eva Product owner: Nikita User experience lead: Eva
Feb 18-22
User experience lead: Eva Product owner: Nikita Content lead: Ali User experience lead: Eva
Mar 12-15
User experience lead: Eva Visual/Interaction designer: Kaleidoscope (TBD)
Mar 25-Apr 1
Use pilot activities at selected training centers and observe
Participant survey to get student and facilitator feedback on pilot activities Analyze pilot results Adjust activities and finalize content plan: hand off to content team Create initial learning system wireframes, review with team and finalize Create remaining wireframes and interaction design, review with team and finalize
DESIGN ACTIVITIES AND TIMELINE: Model, test, and build
Feb 25-Mar 8
Mar 1-8 Mar 4-11
Mar 4-22
Š Design Impact 2013
Design Activity
Roles
Time estimate
Paper-based usability testing with facilitators and Quest staff
User experience lead: Eva Visual/Interaction designer: Kaleidoscope (TBD) Product owner: Abhijeet Content lead: Ali Technology lead: Venky User experience lead: Eva Visual/Interaction designer: Kaleidoscope (TBD) Back-end developer: Azri Solutions Content lead: Ali User experience lead: Eva Product owner: Abhijeet Content lead: Ali Technology lead: Venky Visual/Interaction designer: Kaleidoscope (TBD) Back-end developer: Azri Solutions User experience lead: Eva Visual/Interaction designer: Kaleidoscope (TBD) Back-end developer: Azri Solutions User experience lead: Eva Visual/Interaction designer: Kaleidoscope (TBD) Back-end developer: Azri Solutions
Apr 2-12
Create clickable prototype
Prototype usability testing with facilitators and staff
Begin development iterations and create final visual design Present development iterations at demos to team
Release (alpha)
DESIGN ACTIVITIES AND TIMELINE: Model, test, and build
Apr 8-19
Apr 22-26
Apr 29-May 31
May 6-May 31
June 3
Š Design Impact 2013
•
Functional specifications are not in the spirit of agile development and any list of necessary features should come from actual designs. But Accenture’s timeframe may differ from ours, and they may not be able to accommodate our needs, once we have the product sketches in more detail.
•
If timelines keep sliding, the next place to cut will be product development. We will end with tested system designs but won’t have time to write code. Quest will manage this work without a Design Fellow, so it will be important to make design documents clear for the next team to pick up.
Concerns at this stage
© Design Impact 2013
Thanks. Contact: Eva Miller eva.miller@d-impact.org www.d-impact.org
Š Design Impact 2012