MOOCs4Dev @ ITCILO: Fostering Development through Massive Open Online Courses

TABLE OF CONTENTS What are MOOCs?

-2

Why MOOCs4Dev?

-6

MOOCs Design

-8

MOOCs @ ITCILO

-9

Challenges of MOOCs4Dev

-17

1 Business Model

-17

2 Quality Assurance

-19

3 Access Barriers

-21

4 Facilitation and Teaching Support Quality

-22

MOOCs @ ITCILO Toolkit

-24

Further Recommendations

-25

The Way Forward

-25

WHY THIS GUIDE? What makes Massive Open Online Courses revolutionary is their availability and scalability. MOOCs have the potential to democratise education and foster development.

That’s why MOOCs @ ITCILO are in continuous expansion and evolution, counting already five MOOCs which were successfully implemented between 2015 and 2016.

This guide will provide a snapshot of MOOCs for development and the main challenges they face in developing countries, with a focus on ITCILO recent experience. ITCILO developed a Toolkit to foster best practice and continuous improvement in MOOCs provision.

1

Fostering development through Massive Open Online Courses

WHAT ARE MOOCS?

MOOC

MOOCs are Massive Open Online Courses. Online courses are courses that are delivered online, whereby students use the Internet to access a set of freely accessible online resources, to interact with other students and to submit assignments.

Online courses have a timeline; they are not only a collection of resources.

MOOCs differ from online courses, as they also are: Massive, as they have the capacity to be offered to a very large numbers of learners all over the world.

Open, as no tuition is charged, there is no screening of students, no prior qualification is required for entry and there is no predefined required level of participation, which is voluntary and dependent on the interested individual.

DEMOCRATIZING

lifelonglearning

MOOCs are online courses with various combinations of content, activities, peer-to-peer interactions, mentor interactions and tests. MOOCs integrate learning material with interactive coursework, quizzes, assignment submissions and games. In addition, self-paced learning leaves flexibility to the student. Some MOOCs are stand-alone courses for students in scattered locations, while blended or hybrid courses combine online and face-to-face components. If participants fulfil certain criteria, they often receive a certificate.

2

A BIT OF HISTORY

2015

3

MOOCs experienced high growth over a short period of time. As you can see from the table below, the number of MOOCs exceeded 4000 courses in 2016.

4

The platforms Coursera, edX, and Udacity are normally known as the ‘Big Three’ providers. The main features of the Big Three providers are highlighted in the following table: Coursera

Udacity

edX

Catalogue

Up to 660 courses, about 85 active at a given time.

38 active courses.

Up to 173 courses, about 30 active at a given time.

Partnership

Largest variety of university partners and others including: The Museum of Modern Art, The National Geographic Society and The World Bank.

Often develops content with corporate partners like Salesforce or 23andMe to ensure that the material is current.

Hosts classes from elite universities and number of schools in Asia.

Other partnerships: The IMF and the Linux Fund.

completion (“honour system”); fee-

‘Freemium’ model, with fee-based services such as: one-on-one coaching, personal evaluation of

Business Model

of completion (“honour system”); feecer

.

work.

5

work.

Format

Teachers work more independently and quality is variable.

Visiting teachers work within an established format.

Teachers work more independently and quality is variable.

Additional features

Scheduled model of learning; Transcripts in many different languages; iOS, Android and Kindle Fire apps.

Self-paced model of learning; Focus on programming and computer science classes; iOS and Android apps.

Scheduled model of learning; Focus on sciences and medicine; Some foreign language classes; No apps.

MOOCs integrate 4 main elements: (I) the connectivity of social networking, (II) the facilitation of acknowledged experts in a field of study, (III) a collection of freely accessible online resources; (IV) the active engagement of students who self-organize their participation.

There are various benefits of MOOCs, which include: enabling institutions to meet the growing need for flexible and innovative learning support; fostering workplace skill development within and outside institutions; and offering added value compared to conventional offline course provision due to the potential of MOOCs to bring down costs and offer a flexible course delivery.

WHY MOOCS4DEV? Education is at the heart of the Sustainable Development Goals.

In this context, the scope for MOOC to tackle the needs of the developing world has recently been explored. In developing countries, a small proportion of people are enrolled in higher education, while globalisation calls for a rapid catch-up process to take place. MOOCs can address the growing demand for education and training that currently remains unmet by the formal education system, thus opening up education in developing countries.

MOOCs4Dev are MOOCs oriented towards learners from developing countries, with modest exposure to online practices. They focus on topics and issues that have near-term impact in the context of sustainable development, such as crowdfunding (CF4Dev @ ITCILO). They remove entry requirements and financial barriers and offer flexibility through self-paced learning.

MOOCs4Dev @ ITCILO focused on topics which are in line with the overall areas of competencies of the Centre. They did not have entry requirements and were time bounded (usually up to 8 weeks), combining both self-paced and collaborative design.

Without education, there is no development’ John Roberts, Co-Founder and President of the Open University of West Africa

6

MOOCs4Dev @ ITCILO aim to tackle the main issues faced by learners in developing countries:

Access Widen access through multiple channels, tackling the needs of learners in remote locations or who cannot afford high-cost training.

Equity Target learners with little prior experience of the learning topic and put a strong emphasis on gender balance.

Quality Foster quality by building local capacity and extensive end-ofcourse evaluation.

The literature found that tangible benefits from MOOC participation are more likely to be reported by participants in developing countries with lower educational achievement and lower socioeconomic status. In particular, participants typically report tangible career and/ or educational benefits, which include getting a new job, starting a business or completing prerequisites for an academic program. It appears that there is a great scope for MOOCs4Dev to democratise education and substantially help the most disadvantaged.

7

MOOCS DESIGN

xMOOCs

cMOOCs

Pedagogical model

Traditional behaviourist approach, based on instructive learning. Mostly scheduled model of learning.

Connectivist pedagogy; self-paced model of learning, with learners flexibly building knowledge through social learning processes.

Features

Content-based, prescriptive learning.

Transmission of knowledge happens didactically, through learning objectives and pre-designed video lectures, activities and exercises. Social learning happens through forums or chat functions.

Learner-centred approach. Networks self-generating and distributing knowledge, whereby learners create course coherence and progression, in a decentralised process with frequent iteration. Learners engage in ‘selflearning’.

Pros

Scope for local training capacity and se blended learning. Cleargoals and assessments foster quality s can be used as a signalling tool.

The course is highly flexible and leaves space for learners’ ingenuity. Low cost pre-planning and content production.

Cons

Lower flexibility; high cost pre-planning, due to the need to develop a clear course structure and video production.

Unclear course goals and peer-to-peer assessment may be problematic for quality assurance.

When best to use

For learning domain knowledge that can be mastered through repetitive practice.

To allow learners to acquire higher-order creative skills.

Examples

Coursera, Udacity, EdX, Futurelearn

OpenLearning

Providers are increasingly putting emphasis on MOOCs design.

A major issue in MOOCs design is the pedagogical model of choice. Pedagogical models of MOOCs refer to how participants learn and include aspects such as: the chronology of resources, aims and direction of learning and focus.

A table which highlights the main differences between xMOOCs (eXtended courses) and cMOOCs (Connectivist courses) follows.

The vast majority of available MOOCs, including MOOCs @ ITCILO, are mainly based on the xMOOC approach.

Recommendation for ITCILO: employ a more mixed approach in MOOCs design, combining aspects of both instructor and learner-centred approaches.

8

WHY DID ITCILO GET STARTED WITH MOOCS?

MOOC@ ITCILO Since 2015, five MOOCs have been successfully run by the ITCILO:

MOOC Crowdfunding for Development (CF4Dev) May/June 2015

participants 803

MOOC CF4Dev (South Africa) April/May 2016

participants 300

MOOC Gamification for Development (Game4Dev) April/May 2016

participants 242

MOOC Tech@Work June/July 2016

participants 174

Crowdfunding MOOC for Caribbean Entrepreneurs (CMCE) Sept/Nov 2016

9

participants 298

11 April- 27 May 2016

GAMIFICATION 4 DEVELOPMENT

Foster inclusion of learners worldwide SDG 4 aims to ‘ensure inclusive and quality education for all and promote lifelong learning’.

In this context, MOOCs4Dev enable access to quality education for underserved learners, democratise education and promote lifelong learning internationally. Engaging in MOOCs provision contributes to institutional goals and is in line with the 2016-2020 Strategic Plan of the Centre. As improved outreach is one of the strategy focus areas highlighted by the Plan, MOOCs contribute to it by making ITCILO learning more accessible to the rest of the world.

Involve, engage, motivate!

Acquire reputation and visibility As a provider, ITCILO can engage in the development of a state-of-the-art product to increase its expertise breadth. MOOCs are marketing mechanism to showcase institutional expertise. E.g. Tech@Work MOOC, which was launched to contribute to the ILO Centenary Initiative and crowdsource resources on the role of technology in the world of work.

Promote innovation Innovation in technology-enhanced learning is one of the strategy focus areas highlighted by the 2016-2020 Strategic Plan; in this context, MOOCs provision fosters experimentation with innovative pedagogy. E.g. Game4Dev MOOC, which enabled to experiment with game-based e-Learning methodologies.

Create a professional network The creation of a MOOCs-based institutional network can foster strategic partnerships. E.g. the Crowdfunding MOOC for Caribbean Entrepreneurs (CMCE) was created in partnership with the World Bank Group.

dataset is in constant expansion. ITCILO aims to explore learning analytics practices to foster data-driven decision-making, as datadriven business is an important strategy focus area highlighted by the 2016-2020 Strategic Plan.

Form an alumni network Starting from the existing participant database, the Centre will be able to further engage in targeted promotion of its learning offering and to create professional networks of ITICLO alumni, e.g.

Communities of Practice.

Develop a sustainable business model Developing a sustainable and scalable business model is key, as providers increasingly focus on monetisation. MOOCs @ ITCILO focused on improving economies of scale through low-cost production and re-selling practices. E.g. the first CF4Dev MOOC was adapted for further roll out in South Africa and in the Caribbean.

Engage Activity Managers with technology and foster staff development MOOCs also provide a tool for better engaging Activity Managers with technology-enhanced learning and to promote innovation in staff development practices.

Explore learning analytics Having reached more than 1,800 learners worldwide, our participant

10

PEDAGOGICAL PRINCIPLES MOOCs @ ITCILO can be evaluated based on the following five fundamental principles from key instructional design theories and models (Margaryan et al., 2015).

1 Problem-centred: learning is promoted when learners acquire skills in the context of real-world problems. For example, the CMCE course required learners to develop their own crowdfunding campaign.

2 Activation: learning is promoted when learners activate new or existing knowledge as a foundation for new skills through activities.

For example, the Game4Dev MOOC incorporated game elements in the learning process.

3 Demonstration: learning is promoted when learners observe a demonstration of the skill to be learned.

4 Application: learning is promoted when learners apply their newly acquired skill to solve problems. All MOOCs @ ITCILO included module quizzes and knowledge checks which often used scenariobased questions.

11

5 Integration: learning is promoted when learners reflect on and discuss about their newly acquired skill. For example, the Tech@Work MOOC gave learners the opportunity to create blog articles, capstone projects and provide recommendations for future ITCILO action.

MOOCs @ ITCILO are very participatory and activity-oriented. Generally, MOOCs @ ITCILO are structured in modules where learning activities are action-oriented, focusing on different purposes such as ‘Read’, ‘Think’, ‘Create’, ‘Reflect’.

The following diagram provides a visual representation on where the ITCILO stands on each of the framework’s criteria.

Learning is augmented by audio-visuals, videos, and easy-to-follow activity templates. In addition, activity forums, networking forums and social media spaces enable peer-learning, exchange and the creation of new content.

Often at the end of the course learners are asked to submit a socalled capstone project which is an opportunity for learners to submit their own projects based on the knowledge gained in the course. The submission of a final capstone project is usually linked with awarding a certificate of participation. For example, in both CF4Dev and Game4Dev MOOCs learners submitted their own crowdfunding and gamification strategies at the end of the course.

In end-of-course surveys, 79% of respondents agree that the learning methods used were appropriate.

12

PARTICIPANTS’ BACKGROUND MOOCS @ ITCILO reached a total number of 1815 learners, most of whom from developing countries. Female participation was on average 49%. The corresponding percentage in the literature is typically much lower, at around 25%.

1815 learners

On average, 62% of participants were following a MOOC for the first time and 85% had no prior experience with the MOOC’s topic. This suggests that by offering MOOCs, ITCILO is widening access to its training offer through informal channels and is providing learning opportunities in areas of need. As the following graph shows, participants across the five ITCILO MOOCs represented a variety of organisations, which fosters diversity and inclusion.

49% female participation

However, social partners are hardly represented in the pool of participants, as MOOCs launched by ITCILO so far have covered topics of low relevance to social partners (e.g. crowdfunding, gamification).

Recommendation for ITCILO: develop a targeted action to improve representation of social partners in MOOCs. 13

62% of participants were following a MOOC for the first time

85% had no prior experience with the MOOC’s topic

14

PUBLIC INSTITUTION (NOT GOV)

COMPLETION A common criticism of MOOCs is that most of the people who sign up don’t complete them. Nevertheless, the completion criterion might not be suitable to assess the effectiveness of MOOCs. There is a need to get a better sense of learners’ expectations and motivations. Indeed, the literature shows that motivation to achieve certification predicts completion rates. Retention for students who intend to finish a MOOC is around 24%, while retention for all students is around 5%.

In addition, focusing on certification rates alone underestimates the desirability of activities like browsing and exploring courses, which MOOCs are designed to support.

In this context, the following diagram shows 5 typical behaviours of users identified for MOOCs @ ITCILO. Even if users who complete the course and obtain a certification are a small percentage of registered users, other usage patterns such as accessing the material, exploring the content and engaging with it are an important component of the impact that MOOCs have on learners.

Therefore, new metrics that go beyond completion rates are necessary to capture the diverse usage patterns in the data.

MOOCs @ ITCILO have an average completion rate of 11%, higher than completion rates normally found in the literature, which range from 5% to 7%. Certification criteria typically require a minimum level of participation in social and collaborative spaces and submission of a final project (Capstone Project).

15

MOTIVATION AND FEEDBACK In the literature, the top motivations of young people to engage with MOOCs are: • • • •

MOOCS @ ITCILO outperform the indicators typically found in the literature and therefore have a strong potential to foster equality and inclusion.

Gaining job-related skills, Obtaining a professional certification, Preparing for additional education, Finding a new job.

In this context, 60% of people taking part in an ITCILO MOOCs said the content was relevant to their job. 86% of respondents said they would use the skills acquired during the course.

82%

The learning objectives were achieved

88%

Satisfied with the quality of ITCILO MOOCs

93%

Would recommend the course to a colleague

85%

The course contents were appropriate

Recommendations for ITCILO: Expand evaluation surveys to gain more insight on motivation and tangible career and educational benefits from MOOCs; Explore alternative incentives for MOOC completion, going beyond certification (e.g. partial fellowships for completing additional courses).

16

CHALLENGES OF MOOCS4DEV Despite the great potential of MOOCs4Dev to democratise education worldwide, there are still four main challenges that need to be addressed to truly leverage their potential. These are: 1) the development of a sustainable business model, 2) the creation of a quality assurance framework, 3) tackling access barriers specific to developing countries and 4) assuring facilitation and teaching quality is contextualised and responds to local skillset demand.

1. Business Model The business model for MOOCs is still under development. MOOCs require substantial start up expenses, after which economies of scale kick in and implementation unit costs decrease significantly. In addition, the cost of re-running a MOOC is significantly lower than the initial development cost. The major cost drivers in the development and implementation of a tutor-based e-learning modality are: • Development of the content and learning activities (internal and/or external staff time) • Involvement of subject-matter expert for tutoring, pedagogical facilitation and technical/ administrative support teams (internal and/or external staff time) • The nature of the delivery platform (pre-existing internal platform, internally developed delivery platform, external provider) • Programming for special features such as virtual labs, simulations, or gamification • Analysis of platform data

17

• Partnership The estimated cost of a MOOC ranges from $35,000 to even $300,000, depending on the nature of the main cost drivers. For example, Udacity budgets $200,000 for each course it makes; edX budgets $250,000 per course plus another $50,000 each time the course is re-run. The University of Pennsylvania spends about $50,000 per course, while the University of Edinburgh spends approximately $45,000 on a course.

Case study from ILO: The ‘Workers‘ Rights in a Global Economy’ MOOC launched by the ILO Bureau for Workers’ Activities (ACTRAV) in cooperation with the Global Labour University launched in 2016 has the following cost structure: Activity Production and animation of 25 video lectures Two workshops

Cost

Total

600 x 25

15,000

10 people x 2 days x 2

24,000

+ 6 months coordination work + 9 Lectures (GLU network partners) + Implementation for 3000 subscribers: 10 weeks course tutor 15 hours a week; lecturers providing weekly summary response (2 hours per lecturer); lead professor (1 hour/week) TOTAL

€ 45,000 +

* Freemium model on IT Platform (free for audit, € 29 for participation, € 49

Recommendations for ITCILO: explore further potential revenue streams such as freemium models and peer-to-peer content development.

Based on the different experiences during the pilot period at ITCILO, the following cost structure with the following standard parameters was identified: Activity Activation of a tutor-based learning template

Cost

Total

1,050 1

,050 1,050

Standard development cost of a 5 week on-line learning course

25 P staff days x 600 EURO

15,000

Contract of the subject matter expert tutor which allocates 10 days of staff time in co-development of content and 10 days in expert tutoring during the implementation phase

20 consultant days x 400 EURO

8,000

Internal staff time for the pedagogical facilitation

10 P staff days x 600 EURO

6,000

Internal staff time for the organisational support

15 G staff days x 240 EURO

3,600

IT backstopping

6 days x 600 EURO

3,600

TOTAL

â‚Ź 37,250

The previous table shows that ITCILO has focused on cost-effective solutions in MOOCs provision. There are two key aspects of finding the right business model: financial sustainability and monetisation, which can be tackled by reducing costs or by finding potential revenue streams. The following table shows cost reduction actions taken by ITCILO in green and initiatives under development in red.

Opportunities for cost reduction/revenue streams

MOOCs @ ITCILO

Low-cost/no-cost partnership

E.g. The Game4Dev MOOC was implemented through partnerships with key UN agencies willing to collaborate either on the development or the facilitation of the course

Provide MOOCs on the institution’s existing platform

All ITCILO MOOCs were offered using the institutional eCampus platform

Use open source content/ existing material/ free social media tools/etc.

All ITCILO MOOCs were developed by aggregating content which was either developed in-house or freely available online

Use costrecording tools Re-selling model (selling tailored versions of previous MOOCs)

Freemium model (the product is free to a large extent but some users pay for additional services, e.g. formal

Video recording was made through free video software and mobile apps

for further roll out in South Africa and the Caribbean region, allowing to smooth out development costs and to re-use course materials

An upcoming MOOC co-developed by DELTA and ACTEMP will explore the use of this model by allowing learners to

fees)

18

2. Quality Assurance A major challenge to the sustainability of MOOCs in the long term is the issue of quality. A framework for Quality Assurance (QA) doesn’t exist yet and is needed. Quality assurance has two goals: ensure the institution’s goals for providing MOOCs are met and assure the goals of individual participants in a MOOC are met.

Based on MOOCs literature and on the ECBCheck framework for e-Learning Quality, ITCILO developed a quality assurance framework tailored to MOOCs4Dev. In the development of ITCILO Quality Assurance Framework, standard criteria for e-learning quality were taken into account and four key success factors for a MOOC advocated by Downes (2014) were added: • Autonomy • Diversity • Openness • Interactivity These four criteria are integrated in the framework and are especially relevant for evaluating MOOCs4Dev. The framework is made of three clusters of evaluation: Design, Implementation and Evaluation. Each cluster has a list of indicators.

For each indicator, three scores are possible: 0 for non completion; 1 for completion; 2 for outstanding completion.

19

• Design section - aspects such as the provision of pre-course information, specification of learning objectives and certification criteria, course content quality and participants’ diversity are assessed; in addition, openness and diversity criteria are included in this section.

• Implementation section - course design, usability, facilitation and interactivity are evaluated; in addition, autonomy and interactivity criteria are included in this section.

• Evaluation section - aspects such as the end-of-course evaluation process, certification rates, learners’ satisfaction and benefits reported from taking the course are assessed.

The following diagram shows the overall indicators and a rough quick scoring for MOOCs @ ITCILO.

Recommendations for ITCILO: the end-of-course evaluation could be improved by assessing tangible benefits reported by participants and by having a stronger process for integrating learners’ recommendations.

In addition, MOOCs certifications could be linked to further access to training or scholarship and their use as a screening tool could be explored.

DESIGN

Score

EVALUATION

Score

1

Encompassing evaluation process foreseen at the end of the course

1

2

Modules build progressively on each other

2

Process for integrating recommendations foreseen

2

1

Autonomy: learners can determine their own pace (compulsory/optional material)

2

2

Tutors are competent to facilitate and give feedback and learner support

Information on contact persons available

2

Content quality (coherence, logical sequence, flexibility) Diversity of participants (gender, country, age, organisation)

Pre-course information available (e.g. technical requirements/ workload/ language/ deadlines/ on criteria) Overall learning

Openness: no tuition is charged, no prior is required for entry and there is no

Score

IMPLEMENTATION

2

Course design suitable to learners and learning objectives (e.g. blended learning type)

level of participation Methodology explained

compared to available benchmark

2

2

Tangible career/educational in end-of-course survey

0

Assignments aligned with learning objectives

2

Learning objectives met

2

2

Usability standards met and content navigable

2

2

Interactivity and social networking possibility provided

2

TOT: 13/14

TOT: 13/14

Access to training/scholarship

General learner satisfaction with the course

MOOCs @ ITCILO perform very well in the design cluster.

Indeed, MOOCs @ ITCILO provide learners with prior information on the course, such as on certification criteria, learning objectives and various requirements. In addition, this report has shown that diversity of participants was achieved. In terms of implementation, MOOCs @ ITCILO strongly invested in competent facilitation and provision of individual feedback. Course design is suitable to learning objectives of learners in developing countries (e.g. CF4Dev). Assignments design is in line with learning objectives (e.g. the final Capstone Project). The following diagram provides a visual representation of ITCILO scores in each category.

1

2

TOT: 10/14

20

3. Access Barriers There is a set of issues, specific to developing countries, which creates practical limits to MOOCs access. Even if MOOCs are free other kinds of costs can occur, such as data and time costs. So that MOOCs can reach a wider group of learners, providers must be conscious of the barriers that prevent certain populations from gaining access. Some of these barriers are: • The level of prior knowledge, as the literature shows that learners who take MOOCs are typically highly educated and therefore MOOCs do not always reach all potential learners; • Digital literacy, as learners in developing countries are typically not accustomed to online learning platforms; • Linguistic constraints; • Gender, as country-specific social norms might lead to genderbased discrimination.

In MOOCs @ ITCILO, 85% learners did not have any prior knowledge of the topic. Also, 62% of respondents in end-of-course surveys found the schedule and workload of the course ‘just right’. 82% of participants found the navigation user-friendly and 78% found the user guide useful.

The gender balance was also achieved in MOOCs @ ITCILO, with 49% of participants being female.

The paucity of good telecommunication infrastructure outside of

21

urban settings is perhaps the most tangible challenge of MOOCs4Dev. Barriers such as limitations of public ITC, high cost of bandwidth, lack of high-speed broadband internet and landlines are a challenge for the democratising effect of MOOCs on educational opportunities in developing countries. In this context, the increasing relevance of mobile learning is a unique opportunity for expanding the reach of MOOCs4Dev, through technology options that work within low bandwidth scenarios and provide offline learning possibilities that are not unduly affected by local network conditions.

MOOCs @ ITCILO tackled network connectivity barriers by designing a workload table that minimizes online prerequisites. Many assignments could be completed offline, while mobile learning was encouraged, as many functions are doable on mobile (quizzes, social forum posting).

Learning resources were also available in different formats (video, text, online articles) to make sure everyone with an internet connection could access the course content. In addition, timely technical support was offered to those who experienced difficulties accessing resources. The eCampus platform is also mobile responsive, therefore the course can be accessed through mobile devices, although completion of assignments and reading of complex resources on mobile is not recommended.

Recommendations for ITCILO: MOOCs @ ITCILO are currently in English; the language offer could be expanded (e.g. by launching MOOCs in Spanish and French) by identifying project opportunities jointly with the Training Programmes, thereby tackling language barriers.

4. Facilitation and Teaching Support Quality Other often-mentioned criticisms of MOOCs are the insufficient number and training of teaching staff and the lack of real-world interaction. In MOOCs4Dev, foreign providers often fail to address the demands of local learners. Therefore, there is a need for local mentors, facilitators who know about the needs of specific learners in developing countries. Particularly in MOOCs4Dev, more emphasis needs to be placed on providing a truly culturally appropriate environment, developed in collaboration with local experts, so learners can better engage with the content. Localization and customization of knowledge to the needs of the developing countries is also important for protection of indigenous knowledge and the contribution to capacity building of higher education.

Blended learning, whereby classroom instruction is complemented by online or technology-based reinforcement and vice versa, is a critical part of the future of MOOCs4Dev. The literature has suggested that blended learning is more likely to succeed in developing regions and that a combination of video materials and facilitation drives student engagement. In developing countries, a major challenge is the shortage of qualified trainers, who are typically unfamiliar with online mentoring. In this context, the provision of quality facilitation training for trainers is key.

MOOCs @ ITCILO tackled challenges related to facilitation and teaching support quality by investing in strategic local partnerships

and in facilitation guidelines. The foreseen strategic plan also involves working together with national and local training providers to train facilitators using content developed by ITCILO. MOOCs can therefore strengthen the internal capacity of local institutions to reach a higher number of learners in the community. The Online Facilitation Guide is a tool created by ITCILO to improve facilitation coordination (see Toolkit). Usually MOOCs are led by a team of facilitators belonging to different agencies or institutions; therefore it is recommended to prepare ad-hoc checklists for everyone to be on track.

In addition, CMCE and CF4Dev ZA were created as locally tailored courses, with a further effort to contextualise material. These MOOCs also required the implementation of ad-hoc online tutoring where facilitators received 1 hour of coaching with regard to online facilitation tasks.

Finally, in all MOOCs a Facilitators’ Forum was also created for the different facilitators to share challenges and lessons learnt with the team.

In end-of-course surveys, 80% of participants found the tutors’ contribution satisfactory and 79% found the technical support efficient. In addition, the majority of participants found the course content relevant to their job and nearly all said they were likely to use the skills acquired during the MOOC..

Recommendations for ITCILO: expand the final evaluation to include questions on how well the MOOC was contextualised to address local skillset demand, in order to gain more insight on the success of the current localisation strategy.

22

23

AREA

Why is it useful?

Provides technical information on how to use the eCampus platform (e.g.

MOOCs @ ITCILO Toolkit The following section provides an overview of the tools and resources that were developed by ITCILO in the past two years. The MOOCs @ITCILO toolkit was built taking into account ITCILO Quality Assurance framework to make sure each phase of the MOOC development is sustained by ad-hoc materials and tools.

TOOL

User guide

DESIGN

Participant Guide

discussion forums).

It was rated useful by 78% of participants.

Provides detailed information on the course structure (e.g. methodology, criteria and target group).

Contact us at delta@itcilo.org if you wish to receive the extended version of the Toolkit or for additional information.

Promotional Strategy Plan

A promotional plan was launched to increase the reach of social media based advertisement of the CMCE MOOC. Following this, 30% of CMCE course participants heard about the course on social media.

Facilitators’ Forum

A shared space created for facilitators to report on aspects such as weekly trends, issues to follow-up on, top learners and general progress.

IMPLEMENTATION Online Facilitation Guide

Quality Assurance Framework

EVALUATION

Provides week-by-week instructions, checklists and timetables for online tutors and includes an Assignments Assessment Rubric with grading guidelines.

ITCILO QA framework developed in this report, divided in Design, Implementation and Evaluation areas.

MOOCs @ ITCILO score high on QA, in particular on Design and Implementation.

Expanded Evaluation Questions

Additional questions were added to standard evaluation survey questions.

Learning Analytics

A framework for evaluating learners' performance and behaviours in the course, beyond activity completion measures.

24

FURTHER RECOMMENDATIONS This report has shown that there is a large scope for MOOCs4Dev to democratise education. To do so, major challenges still need to be addressed. One of the most impactful recommendations would be to capitalise on the availability and inexpensiveness of mobile devices.

In countries such as Ghana, Cameroon, Nigeria, and Tanzania 80% or more of the population now owns mobile phones. This is why the expansion of mobile learning is so promising for developing countries.

constantly exploring innovative learning practices, such as gamification.

Applying pedagogical innovation on MOOCs delivery in developing countries has a strong potential to enable access for a growing number of learners and democratise education.

Combined with the integration of social media in MOOCs platforms, mobile learning could make MOOCs accessible for a wider community, leading to higher rates of participation and certification. The ITCILO has already explored mobile learning and the potential of mobile tools to create high quality, relevant and impactful learning experiences.

The Way Forward

As aforementioned, there is scope for pedagogical innovations to drive student engagement. Innovative practices in need of further exploration include blended learning and flipping the classroom (where teacher-student contact time usually used for lectures is used differently, e.g. for discussions, experiments, group-work, working with peers) and adaptive learning, i.e. learning material that adjusts automatically to the learning needs and abilities of learners to enable students to be served more engaging material based on their individual profiles.

New trends such as the increased prominence of self-paced learning models and gamification practices have also been identified as a motivating factor in continuing MOOC participation. ITCILO is

25

ITCILO is developing in-house expertise in MOOCs4Dev. MOOCs @ ITCILO are in continuous improvement, with innovative practices being constantly experimented with. To move forward, there are five main points of focus for future action.

26

Contacts & Authors:

Bibliography

DELTA (Distance Education Learning & Technology Applications)

Baggaley, J. (2014) Bridging Fields at a Critical Time. Journal of Learning for Development. Vol. 1(1).

Gaebel M. (2014) MOOCs: Massive Open Online Courses. European University Association.

Tom Wambeke Head of Programme, DELTA t.wambeke@itcilo.org

Hayes S. (2015) MOOCs and Quality: A Review of the Recent Literature. QAA MOOCs Network.

Hew K. F. & Cheung W. S. (2013) Students’ and instructors’ use of massive open online courses

Alessia Messuti Junior Programme Officer, DELTA a.messuti@itcilo.org

(MOOCs): Motivations and challenges. Educational research review.

Ho, A. D., Reich, J., Nesterko, S., Seaton, D. T., Mullaney, T., Waldo, J., & Chuang, I. (2014). HarvardX and MITx: The first year of open online courses (HarvardX and MITx Working Paper No. 1).

Francesca Bertolino Intern Consultant, DELTA

Hollands F. M. & Tirthali D. (2014) MOOCs: Expectations and Reality. Center for Benefit-Cost Studies of Education Teachers College, Columbia University.

Koutropoulos A. & Zaharias P. (2015) Down the Rabbit Hole: An initial typology of issues around the development of MOOCs. Current Issues in Emerging eLearning, Vol 2 (1).

Liyanagunawardena T.; Adams A. A.; Williams S. A. (2013) MOOCs: A Systematic Study of the Published Literature 2008-2012. The International Review of Research in Open and Distance Learning.

Liyanagunawardena T. & Williams S. (2013) The Impact and Reach of MOOCs: A Developing Countries’ Perspective. E-Learning Papers, n. 33.

Margaryan A.; Bianco M.; Littlejohn A. (2015) Instructional quality of Massive Open Online Courses (MOOCs). Computers & Education Vol. 80.

Online Course Report (2016) State of the MOOC 2016: A Year of Massive Landscape Change For Massive Open Online Courses.

Porter D. & Beale R. (2015) A Policy Brief on MOOCs. Commonwealth of Learning.

Porter D. (2014) MOOCs on M4D. Commonwealth of Learning.

Saltzman G. M. (2014) The Economics of MOOCs. The NEA 2014 Almanac of Higher Education.

Shah D. (2015) MOOCs in 2015: Breaking Down the Numbers. EdSurge.

UNESCO (2013) Introduction to MOOCs: Avalanche, Illusion or Augmentation? IITE Policy Brief.

UNESCO & Commonwealth of Learning (2016) Making Sense of MOOCs: A Guide for Policy-Makers in Developing Countries.

Wang Y. & Baker R. (2015) Content or platform: Why do students complete MOOCs? MERLOT Journal of Online Learning and Teaching, Vol. 11(1).

Yáñez C. E. F.; Nigmonova D; Panichpathom W. (2014) DeMOOCratization of Education? Massive Open Online Courses (MOOCs) and the opportunities and challenges for developing countries. Graduate Institute of International and Development Studies.

Zhenghao C.; Alcorn B.; Christensen G.; Eriksson N.; Koller D.; Emanuel E. J. (2015) Who’s Benefiting from MOOCs, and Why. Harvard Business Review.