E-Learn Special Edition : Learning Analytics

SPECIAL EDITION www.elearnmaga zine.com

P. 14

P. 20

P. 24

Interview:

Infographic:

Roundtable:

Niall Sclater - Ethical Concerns With Learning

10 Reasons Why Institutions Should

Experiments and Challenges with

Analytics: What You Should Be Aware of

Implement Learning Analytics

Learning Analytics Around the World

LEARNING ANALYTICS:

Using Data to Optimize the Learning Experience

Functioning as a collaborative teaching and learning community, E-Learn is a place for educators to share ideas, insights, perspectives, and practices for the purpose of improving student success.

Want to participate?

We'd like to hear from you.

Share your experience, perspective, or field of expertise through an interview, column, or article. Suggest our next topic of focus, get in touch with the E-Learn team.

Email elearn@blackboard.com for further information.

From the Editor THE TOPIC OF LEARNING ANALYTICS HAS BEEN

Find out how institutions can go about inter-

trending in recent years. Although it is still a

preting data, the possibilities for instructors,

growing field, with some countries adopting the

and how to identify institutional barriers.

technology more quickly than others and some just beginning to discover its possibilities, one

Discover how Dr. John Whitmer and a team of

thing is for sure: learning analytics is here to stay.

scientists are developing and enhancing the next wave of data-informed learning environment fea-

Educational institutions can use learning analytics

tures that will better serve the educators, learners,

to turn raw data into actionable insights to achieve

and institutions. Get insights on how democratiz-

results across the entire institution. From adminis-

ing analytics can benefit students and teachers,

trative staff to faculty and students, this education

and how institutions, educators and students will

insight can increase institutional performance and

engage with this technology in the near future.

help meet today’s biggest challenges in academia.

Consider the ethical concerns related to learning At Blackboard, our aim is to continuously provide

analytics, explore 10 reasons why higher edu-

you with the most current and relevant informa-

cation institutions should implement learning

tion about the topics that matter most in educa-

analytics today, review insights and experienc-

tion today. Learning analytics promise to engage

es from experts in three distinct regions, and

faculty and students, increase learning outcomes,

learn how Concordia University Wisconsin

and improve institutional success. With that in

is harnessing the power of learning analytics

mind, we have put together meaningful informa-

to identify at-risk students to help them succeed.

tion that can help you along this exciting journey.

We hope you find this information timely and We begin this issue with a piece featuring

valuable, and as always, we invite all members

Timothy Harfield, Blackboard’s senior product

of the teaching and learning community

marketing manager, and his thoughts on the

to share their experiences, best practices,

challenges and possibilities of learning analytics.

and insights for the benefit of the entire

As Harfield puts it, learning analytics cannot

community. If you’re interested in sharing

replace human judgment, but it can inform it.

your story, we’d like to hear from you.

Sincerely, The E-Learn Team

For more information, please contact: elearn@blackboard.com

EDI TO R

PR ODU C T I O N

Manuel Rivera

Carolina Pintor

C ON T R I BU T I N G E DI TO R S

W E B M AST E R

Katie Gallagher

María José Correa

Liliana Camacho S PEC I A L T H AN KS JOUR N A L I S M Katie Blot, Phill Miller, Priscila Zigunovas, Leonardo

Katie Gallagher,

Tissot, Cristina Wagner.

Carl Marrelli, Lynn Zingraf, Timothy Harfield,

PHOTO GR A PH Y

John Whitmer.

AFP

PR OY EC TOS S E M AN A

AR T A ND D ESI GN

Photography Editor Mario Inti García Mutis

TRiiBU Studio Camilo Higuera, Enny Rodríguez, Camila Mejía, Laura Naranjo, David Peña, Juana Molina.

Table of Contents Learning Analytics / February 2018

Learning Analytics: Possibilities and Challenges of Using Data Science in Higher Education...................... 4 Learning Analytics Amplifies Quality Teaching.........................10 Ethical Concerns With Learning Analytics: What You Should Be Aware of............... 14 10 Reasons Why Institutions Should Implement Learning Analytics........ 20 Experiments and Challenges with Learning Analytics Around the World...... 24 Concordia University Wisconsin: How to Achieve a Record Retention Rate................ 30

Š 2018 E-Learn. Some rights reserved. The views expressed in this magazine are those of the authors and do not reflect the opinions, policies or official positions of Blackboard. Statements about future plans or prospects are given on the date this and not intended to be a prediction of future events. We assume no obligation to update any statement at any time.

E - L E AR N

Sponsored by Blackboard

Timothy Harfield Senior Product Marketing Manager for Blackboard Analytics at Blackboard Inc.

4

E X PE R T A N A LYS I S

Learning Analytics: Possibilities and Challenges of Using Data Science in Higher Education Still a young field, learning analytics is being increasingly seen by higher education institutions as a powerful resource to inform decisions and achieve better learning results. by: priscila zigunovas

“WHAT GETS MEASURED GETS MANAGED.” THIS QUOTE,

of students in a timely manner,” says

attributed to Peter Drucker, the father of modern manage-

Harfield, who has a background in philosophy

ment, is often used by analytics researchers and thinkers in

and sociology and has published extensively on

reference to the power of data to inform decisions, a practice

how learning analytics can be used to promote

that many higher education institutions could benefit from.

student success along with humanistic values.

“In order to effectively manage our systems, our processes and the

However, most institutions are still

success of students, we need to have measurement, because with

far from exploring all the possibilities

that we have access to information and knowledge that we can

that learning analytics can offer.

use to control particular outcomes,” says Dr. Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics.

Great Expectations

The biggest change in education in recent years has been

Learning analytics, as a field and as a disci-

the increase in scale, which not only impacts the num-

pline, is still very young – about six years old.

ber of people institutions are able to reach, but also limits

Institutions and researchers are still work-

their ability to engage students face-to-face. Learning

ing with a great deal of experimentation.

analytics can play an important role in this challenge.

Over the past few years, some institutions “As education becomes increasingly scaled and asyn-

have invested in learning analytics with great

chronous, analytics becomes more important as a

success, while other early adopters were dis-

tool in support of high-quality teaching and learn-

appointed and are now skeptical about what

ing practices that are responsive to meet the needs

the field can offer, according to Harfield.

5

E - L E AR N

“One day we woke up and we had access to all kinds of data. Everybody got really excited. There was a lot of

Human Judgment is Still Essential

promise, but not a whole lot of clarity around what was really possible and how this access to data could ac-

Analytics is nothing more and nothing less

tually benefit institutions,” explains Harfield.

than the visual display of quantitated information, according to Harfield. However,

The novelty of the field and the lack of experience by practi-

capturing activity in the form of data and

tioners, institutions, vendors and researchers caused them

transforming that data into visual displays

not to realize which types of policies and practices would

of information, such as tables, charts and

be necessary to make a successful analytics initiative.

graphs, involves human judgment, and institutions have to take that into account.

Over time, since the innovation rate was not as rapid as institutions were expecting, the initial excite-

In the expert’s experience, the institutions that

ment turned into disappointment. As it turns out,

are most effective at working with learning an-

data alone isn’t sufficient to get the work done and

alytics are those with experienced and prudent

developing impactful data-informed initiatives re-

practitioners who carefully consider the data in

quires a great amount of dedication from institutions.

the context of deep knowledge about students, institutional practices and cultural factors.

Now that this first stage has passed, the future of learning analytics looks quite promising. “Right now, be-

“Learning analytics is not an opportunity for

cause institutions, media and vendors have developed

us to stop thinking, assessing and making deci-

much more realistic expectations, we know more and

sions, but it is an important artifact that needs

can start really getting the work done. We have a level of

to be considered along with a variety of other

maturity that is necessary to be realistic about what we

sources of knowledge, including the human

need to do in order to see real results,” says Harfield.

wisdom that comes with experience in order to solve particular problems,” Harfield says.

“Analytics is not a replacement for human

Learning analytics is trending and it seems to offer endless possibilities. Institutions and education professionals can benefit from this powerful technology, as long as they remember that working with data requires human wisdom.

judgment. It’s a form of information, but it still needs to be interpreted, and that interpretation requires human wisdom.”

Identifying Institutional Barriers Over time, institutions will be using increasingly more data as a base for their decisions and strategies. A major challenge that they have to face is not to be overwhelmed with the amount of data. That’s why it’s important to invest in structures like data warehouses, so that they have access to data when they need it.

6

PHOTOS: AFP Tami Chappell

E X PE R T A N A LYS I S

“However, once they have invested in that

The result of that is often a paradox: universities and

infrastructure, they need to forget about

colleges want students to be successful; however, be-

the data and focus on the questions, such

cause of the complexity of these institutions, they of-

as ‘What are the problems we need to

ten end up including systematic barriers to the success

solve as an institution?,’” says Harfield.

of the very students they want to see succeed.

He suggests beginning with those inquiries and then

Analytics can provide the data that institutions need to

thinking about how to translate them into questions

identify those barriers, and it can positively impact student

that can be answered using the available data, and

success but also institutional success and efficiency as well.

finally translating that data back into strategies that can actually inform and improve the specific

“Analytics allows institutions to see themselves

outcomes institutions are looking to achieve.

almost like in a mirror. It allows them to gain access to how the institution is functioning as a

“Let’s not forget that the university is a very

whole and to identify the way in which students

old institution and, as a result, it’s incredi-

are being systematically and disproportionate-

bly complex. It can be really challenging to

ly advantaged or disadvantaged because of how

navigate if you are a student,” says Harfield.

that institution functions,” explains Harfield.

7

E - L E AR N

Possibilities for Instructors

more likely to earn a grade C or higher compared to students who didn’t use the tool. 1

A way in which instructors are using analytics to improve their courses is by creating solutions to identify students at risk.

“Also, I have seen through research that we have

This could be automated by using predictive analytics,

done with the University of Michigan that the

through Blackboard Predict, and also through tools like the

benefits received by having access to this kind

Retention Center in Blackboard Learn 9.1. The later allows

of student-facing analytics disproportionate-

faculty to establish a threshold based on factors that they

ly affects lower performing students. So, it’s

consider important, and also to monitor students as needed.

actually helping exactly the kind of students we want to help keep on track,” tells Harfield.

“This type of access to information about student activity and the consequences of that activity for their success are less

From a pedagogical perspective, he explains,

important if you are in a small face-to-face class, but it becomes

learning analytics gives instructors an op-

more important as you enter into online courses and large class-

portunity to create interesting assignments

es, where that face-to-face interaction is lost,” says Harfield.

that require students to, for example, reflect upon the analytics that they are seeing.

Proactive advising using predictive analytics is a trend that will be even more present in the future, as it allows instructors and

“We know that simply presenting information

institutions to identify students at risk before they go off-track.

to students doesn’t make a difference in their behavior, but what does have an impact are

Harfield recalls that the reason why traditional approach-

opportunities to actively reflect on that data

es to academic advising increasingly don’t work is because

and what it means for them,” says Harfield.

students that are in most need of that advising — usually low income, first generation and minority students

Future Perspectives

— are exactly the ones that are the least likely to actively seek out help from support systems on their own.

Although there are still many questions to be answered regarding the use of data in

“By using predictive analytics, we are able to identify

education, there are also several opportu-

students at risk early, before they fail the class, before

nities for pedagogical innovation in the use

they have dropped out,” suggests Harfield. These students

of learning analytics that instructors, profes-

can be invited for a conversation with professors, student success

sors, teachers and coaches have yet to explore.

professionals, academic advisors or coaches in order for them to understand the potential barriers that students are facing and

In the future, Harfield says he would like to see more

to develop strategies to help them overcome these difficulties.

reflection and research done on the most effective way to leverage these new analytic technologies.

Another way in which teachers can use analytics is giving students access to their own information, which fosters a sense

“I’m really looking forward to seeing, as tech-

of self-regulated learning. Research has found interesting

nology advances, how we will be able to adapt

results in that area. John Fritz, from University of Maryland,

our strategies, our approaches and our thinking

Baltimore County (UMBC), has found that students who used

about pedagogy to make the most effective use

a feedback tool called “Check my Activity” were 1.92 times

of those technologies in support of students.”

8

E X PE R T A N A LYS I S

Understand How Analytics Technology Works Blackboard’s solution for using data to drive innovation in education is called Analytics for Learn, and it is designed to help administrators, instructional designers and faculty to identify students at risk and optimize university learning environments for greater rates of student success.

Analytics for Learn offers three important features: data aggregation, the ability to perform longitudinal analysis, and the ability to combine data from multiple sources. Gain a better understanding of each concept below.

1

2

3

DATA AGGREGATION

LONGITUDINAL ANALYSIS

COMBINING DATA FROM

In the learning management system (LMS), every student click and behavior is recorded. “These produce huge complex tables that are extremely challenging to write queries against. Also, as you write these complex queries, you run the risk of putting an enormous burden or strain on that operational system,” explains Harfield. When Analytics for Learn is used, that stream of data is pulled into another system and then transformed, so that information can be easily accessed.

Since retaining data requires a great deal of storage, most learning management systems have a tendency to hold information for a short period, usually one or two years. “But by storing this information in a separate warehouse or system, such as Blackboard Analytics for Learn, it’s possible to capture information and analyze trends across multiple years,” explains Harfield.

NEW VERSION

Version 4.3.5 of Analytics for Learn offers support for Rubrics and Tool Detail.

MULTIPLE SOURCES

Analytics for Learn makes it possible to combine information not only from the learning management system but also from the student information system.

“That allows us to do really interesting things, to understand learning behaviors in light of demographic factors and other information that is only stored within the student information system,” says Harfield. Some institutions are integrating data from other learning tools and systems as well.

SOURCE 1. Fritz, J. (2013, April 30). Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course Designs. Retrieved November 8, 2017, from https://www.educause.edu/ir/library/pdf/ERB1304.pdf.

9

Sponsored by Blackboard

Learning Analytics Amplifies Quality Teaching

PHOTO: AFP John Hefti

E - L E AR N

As research and implementation on learning analytics advances, it is possible to catch a glimpse of a future in which data-informed features will become essential to help instructors and students make better decisions. by: priscila zigunovas

LEARNING ANALYTICS AND STUDENT BEHAVIORAL

make decisions about how to develop new features,

data insights are a new form of knowledge that

refine existing features, as well as develop new dedicat-

was not available before educational technologies

ed embedded analytics features,” Whitmer explains.

were so deeply integrated in the classroom. At least that’s the perception of Dr. John Whitmer,

Evolution in Data Collection

Blackboard’s Analytics and Research Director.

Before learning analytics, due to the difficulty accessWhitmer manages a team of data scientists

ing behavioral data at scale, the research that campus-

distributed across the globe who work on

es conducted about student use of academic technology

research and analysis. With advanced degrees

usually relied on student interviews and surveys, which

in statistics, training in machine learning

aren’t always accurate. The response rates are low and the

and computer science, as well as experience

students who respond are a biased sample, often reflect-

with e-learning, these scientists think about

ing power users or people with extreme opinions.

and provide answers to questions about how institutions, students and instructors use these

The field of learning analytics is creating and apply-

products in collaboration with Blackboard Prod-

ing new statistical techniques to help researchers

uct Management and Product Design teams.

to deeply understand what is happening in class.

“We look at how both faculty and stu-

“They give us this new form of knowledge about what hap-

dents use our products in order to help

pened during a class that allows us to create better learning

10

E X PE R T A N A LYS I S

John Whitmer Blackboard’s Learning Analytics and Research Director

materials and learning experiences, as well as

Blackboard Learn (in particular) and their class grade. In

interact with students while the class is still

fact, there is a very small relationship between the two.1

going on, while there is still time to intervene, maybe even change the success of a student

“There is often an assumption in learning analytics that if

within that same class,” says Whitmer.

you could get information about what students do and how frequently they use the platform, then you would have a

Next, learn about the main research

magic wand that would completely reveal students’ success

findings by Whitmer’s team, as well as

and factors around student learning. But when we look at it

innovative features that can make a dif-

on a large scale, we find that this is not true,” he explains.

ference for teachers and students.

Instead, they found that it was important to look deep-

Can Learning Analytics Provide a Formula for Student Success?

er into course design and the specific uses of Blackboard Learn. “For example, students that spend a larger amount of time looking at their grades tend to do better than stu-

Sometimes the biggest discovery is that a

dents who spend less time doing that,” says Whitmer.

hypothesis cannot be confirmed. One of the most interesting research findings from

In classes that use assessments or discussion forums frequently, the

Whitmer’s team is that, when looking at scale

amount of time students spend on those activities is directly related

across all courses and institutions, there isn’t

to their grade. In the end, what matters most is centered around

a strong relationship between student use of

student effort, pedagogical approaches and instructional practices.

11

E - L E AR N

How Can Democratizing Analytics Benefit Teachers and Students?

Educational research is often done by a small group of experts, and historically, the results only circulate around administrative and leadership positions within an institution. These leaders can choose to disseminate the information or not.

“But often teachers and students have a lot of questions, and they are very interested in data and information that can help them make decisions,” points out Whitmer. Democratizing

Blackboard’s Learning Analytics Research team investigates how institutions, students and instructors all over the world really use the company’s products. Their findings help Blackboard refine and develop new features.

analytics means making data and insights accessible to them as well as to other stakeholders that can benefit from it. students not participating in certain types of Ethics is also something that needs to be carefully considered,

activities? Looking at those relationships between

according to Whitmer. “That is an important part of what

activity in student learning at a very detailed

Blackboard does, to ensure that the powerful techniques we can

level, looking at discussion forums, looking at

apply are aligned with what we should do in terms of the ethical

your lecture materials, and reviewing notes,

obligations that we have to students. And those obligations are

instructors can get a sense of what students are

different around the globe, given different laws and regula-

actually doing in their course and how it relates to

tion and cultural practices around student data,” he says.

the success they have,” recommends Whitmer.

How Can a Teacher Begin to Use Learning Analytics?

A Blackboard initiative can make this process much easier in the future. With embedded analytics, Blackboard is integrating analytics

Fundamentally, learning analytics means that the

within the workflow of what teachers are already

teacher can study students’ actions and how frequent-

doing, a unique approach in the market.

ly they access learning materials and activities.

“Teachers often do not have the time within their “Before learning analytics, we did not know, or at least

busy lives and teaching schedules to go off and

we did not know at large scale, what happened while

think separately about these questions. So, instead

students were in class, how often they studied, how

of going to a separate reporting area or a menu

they studied, how often they opened the materials, how

for analytics, Blackboard is embedding an-

they interacted with them,” points out Whitmer.

alytics within the workflows the teachers are natively engaging in,” explains Whitmer.

The expert suggests that teachers should start using analytics by identifying the most important ques-

In practice, that means that when instructors

tions that they need evidence in order to answer.

log into the LMS, they can receive summary updates about how their students are doing and

“For example, are my most successful students accessing, par-

which ones may need attention; when teachers

ticipating in their learning activities, or are my least successful

are grading, they get information about student

12

E X PE R T A N A LYS I S

participation; when instructors are reviewing discussion fo-

What Will Happen Next?

rums, they get information about the quality of student posts, for example. “We bring the analytics to them,” says Whitmer.

Do Students Notice Notifications?

• More Sophisticated Solutions, Better Insights - Learning analytics is still at its early stages. As education technologies become more and more integrated, the amount of data

The possibility of enabling automatic notifications for

available for analytics will increase, as well as

students based on their performance and activity is an

new analysis techniques to derive meaning

innovative feature in the Blackboard Learn Ultra Ex-

from this data. The solutions for institutions,

perience. These alerts are embedded directly within the

educators and students tend to become

course, and when they are clicked, a student receives

more robust and sophisticated over time.

detailed information as well as a suggested action.2

• More Critical Customers - As learning According to Whitmer, this feature has been dis-

analytics becomes more common, students and

cussed for some time by educators, but institutions

faculty will have more experience and become

have been hesitant in providing notifications directly

critical consumers of these solutions. They will

to students because they are concerned about poten-

be able to distinguish, for example, between

tial negative repercussions. After all, individuals receive

learning analytics and conventional reporting.

so many alerts and notifications daily that distraction and lack of focus are becoming serious problems.3

• More Effectiveness - Learning analytics will be increasingly used to help instructors,

However, research found that sending rule-based

administrators and students to make decisions.

notifications to students is beneficial and recommend-

Also, students will be more successful in

ed. “We have found that students are very interested in

achieving their educational goals. This doesn’t

receiving these notifications and alerts, and they access

mean that analytics will replace human

them much more frequently than they do other types of

decision-making. “We don’t see a scenario in

e-mails or notifications they get from educational insti-

which analytics is anything close to replacing

tutions,” says Whitmer. “Students want to know how

human judgment,” says Whitmer. “In a sense,

they are doing, and how they are doing relative

learning analytics is augmented intelligence.”

to their peers, both positive and negative.”

SOURCES

In that sense, learning analytics can be intended to identify what is most important and put that in the center of the student learning experience, so they are better able to pay attention and to focus.

“Students can have their own ideas about what they need by going through learning analytics and finding successful or unsuccessful patterns in the past. We are able to identify these behaviors and practices and then bring them to students’ attention,” says Whitmer.

1. Whitmer, J. (2016, March 18). Research in progress: Learning analytics at scale for Blackboard Learn. Retrieved November 22, 2017, from http://blog.blackboard. com/research-in-progress-learning-analytics-at-scale.

2. Whitmer, J., Nasiatka, D., & Harfield, T. (2017, July 26).

Do notifications get noticed? New study finds that embedded alerts to students promote action at high rates. Retrieved November 22, 2017, from http://blog.blackboard. com/new-study-notifications-promote-student-action 3. Rosen, L. (2012, April 09). Attention Alert! Study on Distraction Reveals Some Surprises. Retrieved November 22, 2017, from https:// www.psychologytoday.com/blog/rewired-the-psychology-technology/201204/attention-alert-study-distraction-reveals-some..

13

E - L E AR N

Ethical Concerns With Learning Analytics: What You Should Be Aware of Educational institutions that implement a learning analytics strategy can collect significant student data and create reports based on student activity. But is it ethical to use students’ data? Is communicating to students how their data is being used an ethical responsibility of educators? Lastly, are institutions taking proper care of privacy and student rights? Current discussions regarding learning analytics and ethics in the education community have been studying the implications of dealing with such data. This is a complex issue, and there are a variety of concerns that should be considered, especially when innovation in learning analytics continues to be developed at a rapid pace. by: cristina wagner glen fruin, scotland, united kingdom

IN THIS INTERVIEW WITH E-LEARN, CONSULTANT

Niall Sclater explains that learning analyt-

Main Ethical Issues Related to Learning Analytics

ics faces a combination of ethical, legal, and logistical issues, and talks about some of the

•

Poor Quality or Insufficient Data - “Trying to do

ethical concerns that surround student data and

analytics when you do not have much data, or it is

learning analytics. Sclater has been working with

not good quality, is a really difficult and pointless

innovation in online and distance education

exercise. A lot of work needs to be done in cleaning the data

for more than 25 years. He has also worked on

set and compensating for that,” says Sclater. One example

many collaborative national and internation-

of this is something called ‘enmeshed identities’, where

al projects related to the use of technology to

students are working together online and the data cannot

enhance learning and teaching, and is the author

differentiate between the person who is authenticated and the

of the book “Learning Analytics Explained.”

other members of the group. “In learning analytics, the two

14

I N T E RV I E W

PHOTOS: AFP Andy Buchanan

Niall Sclater Consultant and Director at Sclater Digital

15

E - L E AR N

main data sources are generally the learning management system (LMS) and the student information system (SIS), which capture only a tiny bit of the learning that takes place and some of the contextual information. Ideally, we need more data points as well as accurate data.”

•

Invalid Analytics - Not only does the data need to be high quality, the analytics also need to be valid. “One of the main rationales behind predictive learning analytics is that there is a relationship between a student’s engagement in their learning activities and their subsequent success. Generally, the more students engage in the learning process, the more likely they are to complete their course and get a better grade. But engagement is not the same as success, and quite often people confuse the concepts of causation and correlation”, Sclater explains.

•

Loss of Autonomy in Decision Making - This is a frequently discussed ethical issue, particularly with adaptive learning systems which continually change the learning based on how students are performing. “Some people worry that it might ‘spoon-feed’ the students too much with automated suggestions, therefore make the learning process less demanding.”

•

•

Gaming the System - Students may do something to try to

Students’ Behavior - Another issue is related

improve their scores or their engagement. In one institution,

to when students change their behavior, either

for example, Sclater heard about a student inserting their

consciously or unconsciously, if they know

ID card into the library entry system a number of times,

that their activities are being continuously

trying to improve their library engagement score. “This

monitored. “If the institution is monitoring the

is an example of unforeseen outcomes that you can have

e-book the student is reading, is that going to

when students are aware that you are measuring things.”

change his or her behavior? That might result in the student improving their learning, but it

16

•

Obligation to Act - Finally, Sclater points out an-

also might increase stress levels. Some students

other ethical matter he thinks is important: whether

might even decide not to participate in some

there is an obligation on the institution to act on the

activities at all because they feel uneasy.”

basis of the analytics. “If you have a lot of data

I N T E RV I E W

In others, it is when they have not submitted an assignment on time. And, in others, there

Adopting learning analytics strategies may come with some ethical and legal concerns. Educational institutions should be prepared to deal these appropriately.

are regular points in which personal tutors or student advisers will look at where the students are at, four times per semester, for example.”

In addition, institutions should define the different types of intervention, such as an automatic reminder sent to the student, a question, a prompt, an invitation to meet with a tutor, or even

on the students, is there an ethical obliga-

a supportive message. “All these measures can be

tion to do something with it? If it is possible

included in an intervention plan,” he says. Insti-

to stop a student from dropping out, should

tutions should also think about the frequency and

you not be obliged to use that data?,” he asks.

timing of interventions for them to be effective.

Data Should Be Used Carefully

One of the biggest issues for institutions, however, is about how to get staff to change

Sclater clearly states that students should be aware of the

their working practices, and to understand

type of data that is being collected about them, and what is

that analytics is either going to help their daily

being done with it. He mentions that the law in some coun-

activities, or make an impact on the students.

tries says that if any user asks about what the institution is

Lastly, Sclater points out: “There is not much

storing about them, they need to be able to tell the student

point in carrying out interventions unless you

exactly what it is and what they are doing with it. “I do not

are going to evaluate the success of them.”

think there are many universities and colleges yet that are in a position to do that,” he affirms. “The only way forward is to make sure you understand all the data that is

In Learning Analytics, Students Are Individuals, Not Numbers

being collected about individual students, and be in a position to gather it together and provide it quickly

It is very easy to look at a dashboard or some

to any student that asks for it,” Sclater suggests.

other kind of analytics and detect a failing student without considering that he or she might have all

Is There an Ideal Way to Carry Out Interventions With Students?

sorts of potential reasons for that prediction, such as facing an illness or other personal issues, finding the concepts too difficult to understand, or having

Learning analytics is only constructive when accompanied

a demanding job and not having enough time to

by interventions to change student behavior or improve the

study properly. “The only way to find that out may

course. The decision about when to make an intervention

be to bring a human into the process,” Sclater says.

varies according to different institutional processes. Sclater believes that institutions should define what he calls

This means treating students like humans, not

“a trigger,” meaning what precipitates that inter-

numbers. “Many universities are using an-

vention. “In some institutions, it might be the student

alytics as a way of identifying the student

has not logged into the system for two weeks, for example.

who might need a subsequent conversation

17

E - L E AR N

with a person.” Greater data sources can help to find out more about the student and tailor the intervention to their

Developing a Learning Analytics Policy

circumstances. “But you are never going to get the whole picture about a student. For many of them, meeting a person to chat

Sclater says that educational institutions

through their issues is the only way forward,” Sclater alerts.

should develop an institutional policy agreed by the relevant stakeholders in the institu-

Educators Should Focus on the Positive

tion, including students. He suggests some of the main topics the policy should include:

One of the main concerns for educators is that, if analytics shows •

What kind of data is being collected

•

Who is responsible for the overall initiative

•

How the institution is dealing with transparency - making it clear what you are doing and what kind of consent you are gathering from students

•

What is being done about confidentiality and securing the data

•

How to ensure the data and the analytics are valid

•

How students will have access to their personal data

journey here. I think the potential is huge for this.”

•

How interventions will be carried out

How to Manage Student Data

•

How you are going to make sure that there are no, or minimized, adverse impacts on the students

students their position among the class percentile, it could potentially demotivate some of them. According to Sclater, while many students want to see that they are doing well compared to their peers, being told or seeing that they are likely to fail could demotivate others. “It is important to tell students early in the course to get into the habit of checking these analytics,” he says. “And then maybe if they are not doing well, the appropriate thing may be for that student to pull out of the course and enroll in a different one. I do not think that the possibility of demotivating a student is a good reason not to show students how they are predicted to perform.”

For Sclater, the best way to provide this information to students is still a matter for investigation. “If we can get the computer systems to understand more about what motivates the individual person, then we may be able to tailor the messages, and the kind of information we present to the students more appropriately.” He is excited by the possibilities that personalization presents. “We are only at the beginning of a

The legal requirements concerning students’ data vary from country to country. According to Sclater, some of the matters that institutions should pay attention to are

He also recommends the development of a student

being clear about what data is being collected and

guide to learning analytics that answers what stu-

why you’re doing that, the capacity to anonymize

dents might want to know (he has developed a model

the data, ensuring that the student has the right to

student guide that can be downloaded here https://

have their data erased if they request that, and de-

analytics.jiscinvolve.org), and of a more in-depth,

veloping careful access controls. ensuring that the

technical document that presents how the analytics

student has the right to have their data erased if they

are actually working. Algorithmic transparency, he

request that, and developing careful access controls.

suggests, is going to be increasingly important.

18

I N T E RV I E W

6 Key Factors to Check if Your Institution is Ready to Build a Learning Analytics Strategy Niall Sclater gives some tips for universities and colleges to make sure they are ready to develop a learning analytics project and to deal with the relevant issues:

1

3

5

LEADERSHIP

Someone in the institution has to lead and promote the initiative. The person should be senior enough to give it credibility and should have a high-level knowledge about learning analytics.

INVESTMENT

The institution should look into the resources that are going to be used in analytics, whether they are for buying software, linking up data sources into the analytics software and so on, and also look at costs in implementing learning analytics, such as the staff costs.

IT

Besides the technology that is going to be adopted, there are different data sources that need to be cleaned and put into the right format.

Also, there’s the need to make sure the staff have skills in handling, interpreting and visualizing the data.

2

CULTURE

Institutions need to build a vision for the initiative around their institutional vision, making learning analytics a part of their strategic plan.

4

6

STRUCTURE AND GOVERNANCE

“Learning analytics projects cut across existing roles and power structures in an organization, as different people need to be involved, and individuals can be quite concerned that their perceived ownership of data or influence is threatened somehow by another project coming in,” says Niall.

PREPARE TEACHERS AND LEARNERS

They need to understand some of the processes of adopting learning analytics and be aware of its impacts and advantages.

19

E - L E AR N

10 Reasons Why Institutions Should Implement Learning Analytics Learning analytics turns raw data into valuable strategic information, offering insights to improve teaching, learning, and the environments in which they occur. We’ve gathered 10 reasons why learning analytics has the power to improve the educational experience. by: cristina wagner

1

i n f o g r a p h i c : t r iib u Studio

PROMOTE REFLECTION AND SELF-REGULATED LEARNING

Student-facing reports help learners increase their performance by providing them with information about their course activity and performance compared to others. Systems like this can provide students with signals about how well they are doing in general, as well as information about specific areas they need to work on. By making learning analytics available to students, institutions can promote self-regulated learning. This practice not only improves student outcomes but also fosters a mindset that may help learners increase their post–graduation success.1

20

2

IDENTIFY STUDENTS AT RISK OF POOR PERFORMANCE

By using data to identify students who are likely to struggle in a course, proactive advising can help students in a timely manner. Learning analytics surfaces relevant information about student engagement and performance that is key for academic advisors and student success specialists.¹ This way, professionals can reach out and intervene at an earlier stage than would otherwise be possible and help learners persist in spite of challenges.2 For example, using predictive analytics tools like Blackboard Predict, faculty and advisors can start identifying at-risk students before the second week in the semester.⁸

I N FO G R A PH I C

3

UNDERSTAND THE IMPACT OF INSTRUCTIONAL DESIGN PATTERNS ON STUDENT PERFORMANCE

Learning analytics can help identify the most successful instructional design style by program and course. By correlating tool use by faculty and students with specific learning outcomes, institutions can identify and scale high impact practices to assist specific program and curricular goals.¹⁰

4

INCREASE REPORTING EFFICIENCY

Colleges and universities must fulfill a wide variety of institutional reporting requirements. Many of these concern educational quality and the impact of educational technology investments. By capturing data on learning activity, and automating report creation, learning analytics technology can increase the accuracy and efficiency of institutional reporting in a way that frees analysts to think beyond mere reporting and also use the same data in support of other strategic initiatives on campus. For example, California Baptist University Online used Analytics for Learn to automate an attendance tracking process that was otherwise labor-intensive and prone to error.¹¹ Because of an increase in institutional efficiency, CBU Online is also able to start using that same information to identify atrisk students and intervene with targeted interventions to keep more students on track for graduation.

5

OPTIMIZE ASSESSMENTS

Learning analytics allows instructors to identify assignments that are highly associated with a student’s final grade and those that tend to correlate with a sudden drop in performance and course withdrawals. Instructors can quickly identify the assessments that are working and those that might need revision.

21

E - L E AR N

6

MEASURE TOOL ADOPTION AND IMPACT

With access to data on learning environment activity, institutions can get a realistic picture of how the system is being used and increase its adoption where it is likely to make the biggest impact.¹² Institutions can also assess the use of third party tools, which, in some cases, might only be used by a few faculty members and do not justify their high costs. This process is important to establish an educational technology investment value, and also for instructional designers and faculty developers to look for evidence that supports assumptions about best practices.²

7

IMPROVE FACULTY DEVELOPMENT

With access to student engagement information, LMS usage and instructional design practices, learning analytics helps faculty developers provide evidence in support of high impact instructional practices.

Access to information about faculty behavior before and after professional development activities also helps institutions to evaluate the impact of their faculty development initiatives.¹³ Instructors are also able to identify and promote effective teaching practices, as they can immediately address gaps and challenges and adjust their courses accordingly.³ Data can also help identify areas in need of improvement by the instructor to facilitate enhanced instructor-student interactions in the educational environment.⁹

8

NUDGE STUDENTS TO IMPROVE LEARNING BEHAVIORS

Learning analytics can be used to improve student success by providing students with relevant performance information and alerting them in advance whether they are at risk of not passing a course. When students can see how they are performing, it increases their sense of agency over their own learning and empowers them.8 Blackboard Learn Ultra has the ability to automate notifications embedded directly within the course. This way, students can receive more detailed information as well as a suggested action, allowing them to keep their performance on track and improve the areas in which they are having difficulties.⁴

22

I N FO G R A PH I C

9

UNDERSTAND THE IMPACT OF ASSIGNMENTS ON DISCUSSION BOARD ENGAGEMENT

Learning analytics can help instructors understand individual learners’ discussion forum activity, as well as critical thinking and the originality of their contributions. They can also test the impact that specific discussion board assignments have on the amount and quality of student participation in the online environment. With insights gained from learning analytics tools like X-Ray Learning Analytics, instructors can create activities to keep students engaged.⁵

10

IDENTIFY AND ELIMINATE SOURCES OF SYSTEMATIC INEQUITY

Access to longitudinal data about relative student outcomes across different sections of the same course can help institutions to identify areas in which the course environment itself is creating barriers to success. For example, institutions often see significant grade variance between sources taught online and in person. At many institutions, faculty grading patterns differ widely between course sections. Pierce College⁶ and Indian River State College⁷ are examples of institutions that have benefited from access to this kind of data.

SOURCES 1. Blackboard. Nine things you can do with Blackboard Analytics for Learn [PDF].

2. Harfield, Timothy. From LMS Reporting to Learning Analytics: Exciting updates to Analytics for Learn.

Retrieved November 17, 2017, from http://blog. blackboard.com/from-lms-reporting-to-learning-analytics-exciting-updates-to-analytics-for-learn.

3. Everhart, Deb. Learning Analytics: The Future is Now. Retrieved November 17, 2017, from https://edtechdigest.wordpress.com/2012/05/10/learning-analytics-the-future-is-now 4. Whitmer, John. Do notifications get noticed?

New study finds that embedded alerts to students promote action at high rates. Retrieved November 17, 2017, from http://blog.blackboard.com/ new-study-notifications-promote-student-action 5. Blackboard. X-Ray Learning Analytics. Retrieved November 17, 2017, from http://www.blackboard.com/ resources/pdf/datasheet-xray-rev20170228_002_.pdf 6. Osborn, Eliana. Dashboards for Success. Retrieved November 17, 2017, from https://www.insidehighered. com/digital-learning/article/2017/03/15/pierce-college-uses-data-dashboards-improve-graduation-rates 7. Blackboard. Learn how Indian River State College grew their online enrollments by 35% in two years. Retrieved November 17, 2017, from http://pages.blackboard.com/campaign/analytics/indian-river-state-college-case-study.aspx 8. Rattiner, Marlen. Walking the line of predictive analytics in higher education. Retrieved December, 2nd 2017, from http://blog.blackboard.com/walking-the-line-predictive-analytics-higher-education/.

9. John, A., Mansureh, K., Sandra, N., & Theresa, K. (n.d.). Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. Retrieved November 16, 2017, from http://files.eric.ed.gov/fulltext/EJ1105911.pdf 10. Harfield, Timothy. Analytics for Learn: Using data science to drive innovation in higher education. Retrieved December 4, 2017, from http://blog.blackboard.com/analytics-for-learn-data-science-innovation-higher-education/ 11. Simpson, Rich. Using analytics to track non-attending students. Retrieved December4 2017, from http://blog.blackboard.com/using-analytics-to-track-non-attending-students/ 12. Harfield, Timothy. Analytics From LMS Reporting to Learning Analytics: Exciting updates to Analytic. Retrieved December 4, 2017, from http://blog. blackboard.com/from-lms-reporting-to-learning-analytics-exciting-updates-to-analytics-for-learn/ 13. Mead, M. (Director). (2017, June 13). Learning analytics and faculty development [Video file]. Retrieved December 4, 2017, from https:// www.youtube.com/watch?v=BSJKVU6Ltys

23

E - L E AR N

Sponsored by Blackboard

Experiments and Challenges with Learning Analytics Around the World

Dragan Gasevic Professor and Sir Tim O’Shea Chair in Learning Analytics and Informatics and Co-Director, Centre for Research in Digital Education, The University of Edinburgh, Scotland

Since the first learning analytics conference, held in Canada in 2011,1 the field has been developing rapidly. Some of the trending areas in research are teacher and learner dashboards, predictive analytics and automatic feedback. There is also rising use of natural language processing technologies and multimodal learning analytics. E-Learn interviewed experts from North America, Latin America and Europe in order to understand these trends and to find out how they are experimenting with novel uses of data in education, as well as their perspective on the future of learning analytics.. by: priscila zigunovas

LEARNING ANALYTICS IS AN EMERGING FIELD, AND ITS ADOPTION BY

help make their students’ educational experiences

higher education institutions around the world is very uneven.

better, but there is not as much action yet as one

The United States, United Kingdom, Canada and Australia are

might hope,” Wise explains. “I think everybody

considered leaders in this domain, while regions such as Latin

feels a little behind the curve of what is possible.

America are still at an initial stage of exploring its possibilities.

They think everyone else is doing these spectacular things, but (with a few notable exceptions)

In the United States, there is great awareness and inter-

most universities are all working through the same

est in learning analytics by higher education institutions,

set of issues. The reason is that there are a variety

says Alyssa Wise, associate professor of learning sciences &

of barriers that need to be overcome to put robust

educational technology and director of New York Universi-

learning analytics systems into place,” she adds.

ty’s Learning Analytics Research Network (NYU-LEARN).

These barriers include technical and infrastructural “If you talk to leaders at almost any higher education institution

challenges, and important questions around data

in the country, you will see that they know about learning

stewardship and access to data. “One of the biggest

analytics, and they are excited about the idea of using data to

challenges we face with learning analytics projects

24

R O U N DTA B L E

The United States, United Kingdom, Canada and Australia are considered leaders in learning analytics, while regions such as Latin America are still at an initial stage of exploring its possibilities.

how analytics can be beneficial for institutions, and the lack of strategic ability, that is, not having institutional leaders that can identify those critical issues and how learning analytics can help. A third barrier would be related to privacy protection and ethics, which is a main issue in specific countries.

PHOTO: AFP Andy Buchanan

“Countries such as Germany have had, historically, through World War II, a huge abuse of their private data, and thus people are much more concerned with respect to ethics and privacy,” explains Gasevic. “What we are seeing, since we are running a major project called SHEILA,2 is that students are aware of the ethical and privacy protection questions and they have high expectations that institutions will protect their privacy, but at the same time, they also have big expectations that their data will be used.” is that we have a great idea for a project and all of the stakeholders are on board, but nobody

In Latin America, a recent study revealed what researchers

knows who is able to give permission to look at the

already knew: adoption of learning analytics in Latin Ameri-

different data sources we need. This has not been

can countries is still at very initial stages, according to Xavier

something that universities have had to deal with

Ochoa, professor and director of the Teaching and Learn-

before because nobody has asked,” explains Wise.

ing Technologies Research Group at the Escuela Superior Politécnica del Litoral (ESPOL), in Guayaquil, Ecuador.

In Europe, according to Dragan Gasevic, professor and chair in learning analytics and informatics

“There are few institutions that are committed to implement-

at The University of Edinburgh, the majority of

ing learning analytics in their core processes, but there’s a

institutions are aware of learning analytics, and

lot of interest. I haven’t talked with any administrator that

there are some small experiments taking place.

is not very willing to try learning analytics at their institution,” says Ochoa. According to him, the lack of research-

Possible barriers for adoption include shortage

ers and education professionals with expertise in learning

of capacity or knowledge to actually understand

analytics is the main barrier for adoption in the region.

25

E - L E AR N

See next some of the most interesting experiments that are taking place at institutions around the world.

Predictive Analytics: An Upward Trend Alyssa Wise, from New York University, is involved in several different projects that work with proactive support for students through predictive modeling. In predictive analytics, a computational model is created based on students who have previously taken a course, and then applied to students who are beginning the course in order to identify those who are likely to have difficulties.

“The accuracy of the model we have is quite good. We can identify as early as the first day of class who are the students likely to have trouble in a course. The challenging thing is: what do we do about it?” she says.

It is evident through this example that being able to predict difficulties isn’t enough. Researchers and education professionals still need to figure out how to best use this information to support teaching and learning.

In one of the projects, Wise explains, the model shows that student difficulties in introductory college mathematics courses

induce stereotype threat, and thus become a

are linked to inadequate prior preparation. “In this particular

self-fulfilling prophecy,” adds Wise. “Instead, it

model the path toward action is actually straightforward. We

is important to stay positive and focus on what

can offer these students learning resources and mastery driven

actions they need to take to be successful.”

practice sets to help them get up to speed on the skills and concepts that they may not be strong enough in at the start,” she says.

According to the expert, NYU has started to build teacher dashboards that give instructors

However, in other cases, the model can predict who will have

an overview of their students’ digital activities.

trouble, but the factors used to predict are not very clearly

This can be used both to evaluate particular class

actionable. “In this case, we know which students are going

elements and identify students that may need

to have challenges, but we do not know why or how. So, it’s a

particular attention. “The next step is to study how

much more difficult challenge to figure out what to do with the

instructors use these new sources of information

model. Do we show it to the instructors? How do the instruc-

to inform their teaching and how we can better

tors contact the students and what do they say?” says Wise.

support them in this process of data-informed pedagogical decision-making,” Wise describes.

“We have to be careful because you do not want to tell students at the start of a course that you are expecting them to do poorly.

This illustrates why a great deal of attention is

This has the potential to be demotivating, and in some cases

now being paid to how institutions are putting

26

R O U N DTA B L E

learning progression, their time management and their misconceptions,” explains Gasevic.

This empowers teachers so they can get much deeper insight into what is happening in their classrooms, especially in blended or flipped models. And more importantly, instructors can also provide personalized individual feedback at scale.

Alyssa Wise Associate Professor of Learning Sciences & Educational Technology and Director, NYU Learning Analytics Research Network (NYU-LEARN), New York University, United States

A project in which Gasevic is involved aims to accomplish exactly that. Project OnTask began in 2016 led by The University of Sydney in collaboration with University of Technology Sydney, University of New South Wales, University of South Australia, University of Texas at Arlington, and The University of Edinburgh.

PHOTO: AFP Dominick Reuter

“With learning analytics and support of the OnTask software, instructors are able to provide personalized feedback that scales up highly. For the amount of work to support 3 to 5 students, they are able to personalize feedback for hundreds or thousands of students”, Gasevic explains. The project’s results show increase in student performance, satisfaction, and learning process.

The connection between learning analytics and learning design is another important topic in the field. Learning analytics can help data and models into practice in ways that are eth-

teachers to reflect on the effectiveness of their learning design.

ical and useful to support teaching and learning.

“For example, when I am teaching a course, I can re-

Feedback Loops: Empowering Students and Teachers

flect on what is working in my course, or what is not working, and based on that I can make certain changes in my design and, in little time, start making that or-

Establishing or improving feedback loops between

chestration much more effectively,” explains Gasevic.

teachers and learners is the main gain of learning analytics according to Dragan Gasevic, from The

Gasevic and his team developed an award-winning software

University of Edinburgh. That can be especially help-

called LOCO-Analyst, a tool that aims at providing teachers

ful in larger classes, sometimes with hundreds of

with feedback on the relevant aspects of the learning pro-

students, which makes it impossible for a teacher to

cess taking place in a web-based learning environment.

interact with all of them at a more personalized level.

“When we developed LOCO-Analyst we were trying to “Learning analytics helps us get the

analyze different types of digital traces generated by

insight into the learning patterns

learners as they were using and interacting with digital

of every one of our students, in or-

resources on the web and inside of their learning envi-

der to understand, for example, their

ronments, such as Blackboard Learn,” he explains.

27

E - L E AR N

According to the expert, one of the function-

there, there aren’t traces of these activities. Maybe you have

alities that is most appreciated by teachers

the end results of what they did, but a great deal of informa-

offers critical insight not only at the level of

tion is lost if you only analyze the end result,” he explains.

particular resources, but at the conceptual level, and the way students are actually construct-

Multimodal learning analytics exploit audio and video

ing knowledge around certain concepts. The

recordings, what students write, look at, and say, and all this

insights into different acts of social interaction

information is used to get a clear picture of the learning process.

that happen among students are also valued.

“Humans are multimodal in nature, we capture Gasevic sees a fairly big disconnect in terms

information through all our senses, so we

of what is actually needed by teachers and

are trying to make the computer do some-

students, and what is currently out there. “What

thing similar,” explains Ochoa.

is happening in reality is that most teachers and students do not understand the content

According to him, as technology became more accessible

in their dashboards. I attribute that to major

over the recent years, the development of multimodality

problems, and the first problem is related to

became a reality. Two factors that have enabled the field of

the lack of proper ethnographic studies trying

multimodal learning analytics are the development of artifi-

to understand the teachers and the students

cial intelligence and the availability of very cheap sensors.

in the way they can embed and use analytics inside of their specific tasks,” says Gasevic.

“Imagine that you can have a device that costs almost nothing with a camera and a microphone,” describes

Beyond the LMS: Multimodal Learning Analytics

Ochoa. “With this type of sensors and the right software, it’s possible to analyze, for example, student posture, create transcripts of what the student is saying, or capture emo-

At the beginning, learning analytics worked ex-

tions in the student’s voice. So, now, computers are able

clusively with data from online tools, like learn-

to see, are able to hear, and we are trying to exploit those

ing management systems or online games. But

capabilities to better understand the learning process.”

what about the learning that is happening outside the computer? How could data from the re-

This expert estimates that 30% of studies on learning ana-

al world, such as in classrooms, student groups,

lytics currently include some kind of multimodality. That

or even when students are doing their home-

could be an indication of institutions realizing that they need

work, be captured and measured? This branch

to be more holistic when looking at learning processes.

is called multimodal learning analytics and it’s Xavier Ochoa’s personal field of research.

One example of a multimodal analytics project developed by ESPOL is a tutor environment where students

“Multimodal learning analytics arise natu-

can practice giving presentations. Students enter a room,

rally out of the need to understand learning

close the door, and then they are greeted by a virtu-

where it is happening. For example, there is

al audience to whom they can present their slides.

a lot of learning that happens when students work together, trying to solve a problem or an

“The system will analyze their posture, their gaze, the

exercise in a course. If there is no computer

volume of their voice, if they are stammering, and it

28

R O U N DTA B L E

What’s Trending?

• Technical and theoretical advances are making it possible to collect better and richer data.

• This can be done not only through online tools, but also through clickers, sensors and motion detectors that collect data from the real world. That’s multimodal learning analytics.

• Providing automatic, personalized feedback through student behavior analysis is becoming more common, especially for large classes.

• Natural language processing technologies are being used to analyze student-created products. This allows instructors to go beyond multiple-choice questions and

PHOTO: AFP Rodrigo Buendia

respond to students’ creative works as well.

• Researchers also want to discover how analytics can help understand students’

emotions, not just their cognitive state.

Xavier Ochoa Full Professor, Director of the Information Technologies Center, Escuela Superior Politécnica del Litoral (ESPOL), Ecuador

• A challenge that researchers currently face is that tools have to be designed to work in

specific learning contexts, but at the same time they need to be general enough to be used

will analyze their slides, if they have too much text or

across different courses and even institutions.

the font is too small, and will give an automatic feedback on the presentation,” describes Ochoa.

• Learning analytics will be increasingly used to support decision making for

On that same area, ESPOL is using multimodal learning

students, instructors, administrators,

analytics to understand what differentiates experts from

institutions and even governments.

non-experts while they are trying to solve a problem. That’s a type of system where multimodal learning analytics is used to provide feedback not for the student, but for the instructor.

“We can try to understand if a student has reached mastery in a skill just by looking at him or her. That’s exactly what a professor would do, observe student behavior,” compares Ochoa.

SOURCES 1. LAK11. (2011). 1st International Conference on Learning Analytics and Knowledge 2011. Retrieved November 15, 2017, from https://tekri.athabascau.ca/analytics/program.

2. SHEILA. (n.d.). Using data wisely for education futures.

Retrieved November 16, 2017, from http://sheilaproject.eu.

29

E - L E AR N

Sponsored by Blackboard

Elizabeth Polzin

PHOTO: AFP Darren Hauck

Assistant Vice President of Academics for Student Success at Concordia University Wisconsin

30

C U STO M E R S N A P S H OT

Concordia University Wisconsin: How to Achieve a Record Retention Rate Identifying at-risk students and helping them out. That’s what the use of Blackboard intelligence and learning analytics solutions has brought to Concordia University Wisconsin (CUW). by: leonardo tissot mequon, wisconsin, united states

BEFORE THE USE OF THIS TECHNOLOGY, IF STUDENTS

collected this past year helped us better understand

were struggling with learning, the university had

the challenges our students were encountering and

to rely on faculty reports. And several times, when

aided us in much earlier intervention,” she says.

it came to this point, it was already too late for students to get back on track and save their semesters.

“Now, we are able to make an early intervention,”

In Search of a More Personalized Experience

says Elizabeth Polzin, Assistant Vice President of Academics for Student Success at CUW.

At CUW, students are not just “a number.” “We are proud to say we know students’ names at CUW. The ability to use

The work started about a year ago, first limited to

analytics to better serve our students provides them with a

on-campus undergraduate students. That was a

personalized experience. Rather than waiting until the end

precaution measure – making sure they were getting

of the semester to determine what went wrong, we have to

reliable data. And the results have been great so far.

be able to take note when they stop attending class or begin to fall behind in their coursework,” Elizabeth points out.

Last fall, CUW had a retention rate of 72%. A year later, the university experi-

The most important endeavor is to find out how

enced a 10% increase, reaching a record

students are developing in their academic com-

82% retention rate. Along with the

mitments while there are still opportunities

involvement of administration and fac-

for recovery if they are doing poorly, not just

ulty, analytics also was a big part of this

at the end of the semester when it is too late.

result, Elizabeth believes. “The data we

That is what academic support is there for.

31

E - L E AR N

“With the Blackboard learning analytics tools, this year has been

and trying to formulate solutions – from finan-

a bit of an explosion for us. We just started to dabble with what

cial to mental health. If a student disappears for

types of reports we could create, and better strategize our out-

a while, it’s possible – through the attendance

reach. I think it’s one of the reasons we saw a big jump on our

feature – to determine if they haven’t been

retention rate: we had the opportunity to provide personalized

to class. With that type of real time informa-

outreach before the semester was over,” the assistant VP says.

tion, a university staff member can follow up with the student to determine what support

THE IMPACT OF LEARNING ANALYTICS AT CUW

they may need. Instead of using automated emails, we’re able to provide individualized

RETENTION RATE

outreach from someone the student knows.”

Building a Better Relationship Between Faculty and Students

81.7% 71.62%

(New Record)

2016 2017

The use of data analysis has improved relationships between faculty and students due to a new referral system for professors. In the past, when a student started to struggle, faculty may have felt as though they had to deal with the situation on their own. However, that’s not something all professors feel comfortable with. Now, there is a system in

Offering Better Quality Outreach

place where faculty can refer students they are concerned about, and an academic support team

Based on the data provided, CUW is working with dashboards

is able to use analytics to address students’ needs.

that can be viewed by each of their advisors, with focus on student performance and a variety of risk factors. For example, “advisors can now easily view whether a student has had a significant drop

A Way to Provide Revenue for Universities

on their GPA from the prior term, if they are registered for 18 or more hours but have a low GPA, or if they have had a significant

More than offering students and faculty the best

drop in term GPA from their cumulative GPA” among other data.

possible experience, learning analytics has also been impacting CUW’s revenue through academic

The dashboards help the university envision how to

support. “It’s a better use of time and human re-

support students more efficiently – even the ones

sources in general. As a supervisor, I prefer to not

who need outreach at the beginning of the semester,

have my team spinning their wheels in a direction

before they even start struggling with their educa-

where they end up in a dead end. Using analytics

tion. “With a review of identified risk factors at the

gives us focus and direction as to the best way

beginning of the term, our intervention can start

to serve our students,” Elizabeth reflects. “The

much earlier than it has in the past,” Elizabeth says.

bottom line is: through the use of analytics, we know that our decisions are informed,

CUW also has an intervention team to help students, such as

and the results we’ve seen are a positive

getting to know what types of problems they are experiencing

consequence of that,” she concludes.

32

Over the past four years, we've focused on collaboration with the teaching and learning community by sharing:

Visit E-Learn online and stay up-to-date with online educational trends. ideas

perspectives

insights

practices

Get all content related to learning analytics here

Read our stories and see how they can benefit you and your institution:

www.elearnmagazine.com 33

“The only way forward is to make sure you understand all the data that is being collected about individual students, and be in a position to gather it together and provide it quickly to any student that asks for it.”

Niall Sclater Consultant and Director at Sclater Digital

“Analytics is not a replacement for human judgment. It’s a form of information, but it still needs to be interpreted, and that interpretation requires human wisdom.”

Timothy Harfield Senior Product Marketing Manager for Blackboard Analytics

“At Condordia University Wisconsin, students are not just a number. We are proud to say we know students’ names.”

Elizabeth Polzin Assistant Vice President of Academics for Student Success at Concordia University Wisconsin