SPECIAL EDITION www.elearnmaga zine.com
P. 14
P. 20
P. 24
Interview:
Infographic:
Roundtable:
Niall Sclater - Ethical Concerns With Learning
10 Reasons Why Institutions Should
Experiments and Challenges with
Analytics: What You Should Be Aware of
Implement Learning Analytics
Learning Analytics Around the World
LEARNING ANALYTICS:
Using Data to Optimize the Learning Experience
Functioning as a collaborative teaching and learning community, E-Learn is a place for educators to share ideas, insights, perspectives, and practices for the purpose of improving student success. Want to participate? We'd like to hear from you. Share your experience, perspective, or field of expertise through an interview, column, or article. Suggest our next topic of focus, get in touch with the E-Learn team.
Email elearn@blackboard.com for further information.
From the Editor THE TOPIC OF LEARNING ANALYTICS HAS BEEN
Find out how institutions can go about inter-
trending in recent years. Although it is still a
preting data, the possibilities for instructors,
growing field, with some countries adopting the
and how to identify institutional barriers.
technology more quickly than others and some just beginning to discover its possibilities, one
Discover how Dr. John Whitmer and a team of
thing is for sure: learning analytics is here to stay.
scientists are developing and enhancing the next wave of data-informed learning environment fea-
Educational institutions can use learning analytics
tures that will better serve the educators, learners,
to turn raw data into actionable insights to achieve
and institutions. Get insights on how democratiz-
results across the entire institution. From adminis-
ing analytics can benefit students and teachers,
trative staff to faculty and students, this education
and how institutions, educators and students will
insight can increase institutional performance and
engage with this technology in the near future.
help meet today’s biggest challenges in academia. Consider the ethical concerns related to learning At Blackboard, our aim is to continuously provide
analytics, explore 10 reasons why higher edu-
you with the most current and relevant informa-
cation institutions should implement learning
tion about the topics that matter most in educa-
analytics today, review insights and experienc-
tion today. Learning analytics promise to engage
es from experts in three distinct regions, and
faculty and students, increase learning outcomes,
learn how Concordia University Wisconsin
and improve institutional success. With that in
is harnessing the power of learning analytics
mind, we have put together meaningful informa-
to identify at-risk students to help them succeed.
tion that can help you along this exciting journey. We hope you find this information timely and We begin this issue with a piece featuring
valuable, and as always, we invite all members
Timothy Harfield, Blackboard’s senior product
of the teaching and learning community
marketing manager, and his thoughts on the
to share their experiences, best practices,
challenges and possibilities of learning analytics.
and insights for the benefit of the entire
As Harfield puts it, learning analytics cannot
community. If you’re interested in sharing
replace human judgment, but it can inform it.
your story, we’d like to hear from you.
Sincerely, The E-Learn Team
For more information, please contact: elearn@blackboard.com
EDI TO R
PR ODU C T I O N
Manuel Rivera
Carolina Pintor
C ON T R I BU T I N G E DI TO R S
W E B M AST E R
Katie Gallagher
María José Correa
Liliana Camacho S PEC I A L T H AN KS JOUR N A L I S M Katie Blot, Phill Miller, Priscila Zigunovas, Leonardo
Katie Gallagher,
Tissot, Cristina Wagner.
Carl Marrelli, Lynn Zingraf, Timothy Harfield,
PHOTO GR A PH Y
John Whitmer.
AFP
PR OY EC TOS S E M AN A
AR T A ND D ESI GN
Photography Editor Mario Inti García Mutis
TRiiBU Studio Camilo Higuera, Enny Rodríguez, Camila Mejía, Laura Naranjo, David Peña, Juana Molina.
Table of Contents Learning Analytics / February 2018
Learning Analytics: Possibilities and Challenges of Using Data Science in Higher Education...................... 4 Learning Analytics Amplifies Quality Teaching.........................10 Ethical Concerns With Learning Analytics: What You Should Be Aware of............... 14 10 Reasons Why Institutions Should Implement Learning Analytics........ 20 Experiments and Challenges with Learning Analytics Around the World...... 24 Concordia University Wisconsin: How to Achieve a Record Retention Rate................ 30
Š 2018 E-Learn. Some rights reserved. The views expressed in this magazine are those of the authors and do not reflect the opinions, policies or official positions of Blackboard. Statements about future plans or prospects are given on the date this and not intended to be a prediction of future events. We assume no obligation to update any statement at any time.
E - L E AR N
Sponsored by Blackboard
Timothy Harfield Senior Product Marketing Manager for Blackboard Analytics at Blackboard Inc.
4
E X PE R T A N A LYS I S
Learning Analytics: Possibilities and Challenges of Using Data Science in Higher Education Still a young field, learning analytics is being increasingly seen by higher education institutions as a powerful resource to inform decisions and achieve better learning results. by: priscila zigunovas
“WHAT GETS MEASURED GETS MANAGED.” THIS QUOTE,
of students in a timely manner,” says
attributed to Peter Drucker, the father of modern manage-
Harfield, who has a background in philosophy
ment, is often used by analytics researchers and thinkers in
and sociology and has published extensively on
reference to the power of data to inform decisions, a practice
how learning analytics can be used to promote
that many higher education institutions could benefit from.
student success along with humanistic values.
“In order to effectively manage our systems, our processes and the
However, most institutions are still
success of students, we need to have measurement, because with
far from exploring all the possibilities
that we have access to information and knowledge that we can
that learning analytics can offer.
use to control particular outcomes,” says Dr. Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics.
Great Expectations
The biggest change in education in recent years has been
Learning analytics, as a field and as a disci-
the increase in scale, which not only impacts the num-
pline, is still very young – about six years old.
ber of people institutions are able to reach, but also limits
Institutions and researchers are still work-
their ability to engage students face-to-face. Learning
ing with a great deal of experimentation.
analytics can play an important role in this challenge. Over the past few years, some institutions “As education becomes increasingly scaled and asyn-
have invested in learning analytics with great
chronous, analytics becomes more important as a
success, while other early adopters were dis-
tool in support of high-quality teaching and learn-
appointed and are now skeptical about what
ing practices that are responsive to meet the needs
the field can offer, according to Harfield.
5
E - L E AR N
“One day we woke up and we had access to all kinds of data. Everybody got really excited. There was a lot of
Human Judgment is Still Essential
promise, but not a whole lot of clarity around what was really possible and how this access to data could ac-
Analytics is nothing more and nothing less
tually benefit institutions,” explains Harfield.
than the visual display of quantitated information, according to Harfield. However,
The novelty of the field and the lack of experience by practi-
capturing activity in the form of data and
tioners, institutions, vendors and researchers caused them
transforming that data into visual displays
not to realize which types of policies and practices would
of information, such as tables, charts and
be necessary to make a successful analytics initiative.
graphs, involves human judgment, and institutions have to take that into account.
Over time, since the innovation rate was not as rapid as institutions were expecting, the initial excite-
In the expert’s experience, the institutions that
ment turned into disappointment. As it turns out,
are most effective at working with learning an-
data alone isn’t sufficient to get the work done and
alytics are those with experienced and prudent
developing impactful data-informed initiatives re-
practitioners who carefully consider the data in
quires a great amount of dedication from institutions.
the context of deep knowledge about students, institutional practices and cultural factors.
Now that this first stage has passed, the future of learning analytics looks quite promising. “Right now, be-
“Learning analytics is not an opportunity for
cause institutions, media and vendors have developed
us to stop thinking, assessing and making deci-
much more realistic expectations, we know more and
sions, but it is an important artifact that needs
can start really getting the work done. We have a level of
to be considered along with a variety of other
maturity that is necessary to be realistic about what we
sources of knowledge, including the human
need to do in order to see real results,” says Harfield.
wisdom that comes with experience in order to solve particular problems,” Harfield says. “Analytics is not a replacement for human
Learning analytics is trending and it seems to offer endless possibilities. Institutions and education professionals can benefit from this powerful technology, as long as they remember that working with data requires human wisdom.
judgment. It’s a form of information, but it still needs to be interpreted, and that interpretation requires human wisdom.”
Identifying Institutional Barriers Over time, institutions will be using increasingly more data as a base for their decisions and strategies. A major challenge that they have to face is not to be overwhelmed with the amount of data. That’s why it’s important to invest in structures like data warehouses, so that they have access to data when they need it.
6
PHOTOS: AFP Tami Chappell
E X PE R T A N A LYS I S
“However, once they have invested in that
The result of that is often a paradox: universities and
infrastructure, they need to forget about
colleges want students to be successful; however, be-
the data and focus on the questions, such
cause of the complexity of these institutions, they of-
as ‘What are the problems we need to
ten end up including systematic barriers to the success
solve as an institution?,’” says Harfield.
of the very students they want to see succeed.
He suggests beginning with those inquiries and then
Analytics can provide the data that institutions need to
thinking about how to translate them into questions
identify those barriers, and it can positively impact student
that can be answered using the available data, and
success but also institutional success and efficiency as well.
finally translating that data back into strategies that can actually inform and improve the specific
“Analytics allows institutions to see themselves
outcomes institutions are looking to achieve.
almost like in a mirror. It allows them to gain access to how the institution is functioning as a
“Let’s not forget that the university is a very
whole and to identify the way in which students
old institution and, as a result, it’s incredi-
are being systematically and disproportionate-
bly complex. It can be really challenging to
ly advantaged or disadvantaged because of how
navigate if you are a student,” says Harfield.
that institution functions,” explains Harfield.
7
E - L E AR N
Possibilities for Instructors
more likely to earn a grade C or higher compared to students who didn’t use the tool. 1
A way in which instructors are using analytics to improve their courses is by creating solutions to identify students at risk.
“Also, I have seen through research that we have
This could be automated by using predictive analytics,
done with the University of Michigan that the
through Blackboard Predict, and also through tools like the
benefits received by having access to this kind
Retention Center in Blackboard Learn 9.1. The later allows
of student-facing analytics disproportionate-
faculty to establish a threshold based on factors that they
ly affects lower performing students. So, it’s
consider important, and also to monitor students as needed.
actually helping exactly the kind of students we want to help keep on track,” tells Harfield.
“This type of access to information about student activity and the consequences of that activity for their success are less
From a pedagogical perspective, he explains,
important if you are in a small face-to-face class, but it becomes
learning analytics gives instructors an op-
more important as you enter into online courses and large class-
portunity to create interesting assignments
es, where that face-to-face interaction is lost,” says Harfield.
that require students to, for example, reflect upon the analytics that they are seeing.
Proactive advising using predictive analytics is a trend that will be even more present in the future, as it allows instructors and
“We know that simply presenting information
institutions to identify students at risk before they go off-track.
to students doesn’t make a difference in their behavior, but what does have an impact are
Harfield recalls that the reason why traditional approach-
opportunities to actively reflect on that data
es to academic advising increasingly don’t work is because
and what it means for them,” says Harfield.
students that are in most need of that advising — usually low income, first generation and minority students
Future Perspectives
— are exactly the ones that are the least likely to actively seek out help from support systems on their own.
Although there are still many questions to be answered regarding the use of data in
“By using predictive analytics, we are able to identify
education, there are also several opportu-
students at risk early, before they fail the class, before
nities for pedagogical innovation in the use
they have dropped out,” suggests Harfield. These students
of learning analytics that instructors, profes-
can be invited for a conversation with professors, student success
sors, teachers and coaches have yet to explore.
professionals, academic advisors or coaches in order for them to understand the potential barriers that students are facing and
In the future, Harfield says he would like to see more
to develop strategies to help them overcome these difficulties.
reflection and research done on the most effective way to leverage these new analytic technologies.
Another way in which teachers can use analytics is giving students access to their own information, which fosters a sense
“I’m really looking forward to seeing, as tech-
of self-regulated learning. Research has found interesting
nology advances, how we will be able to adapt
results in that area. John Fritz, from University of Maryland,
our strategies, our approaches and our thinking
Baltimore County (UMBC), has found that students who used
about pedagogy to make the most effective use
a feedback tool called “Check my Activity” were 1.92 times
of those technologies in support of students.”
8
E X PE R T A N A LYS I S
Understand How Analytics Technology Works Blackboard’s solution for using data to drive innovation in education is called Analytics for Learn, and it is designed to help administrators, instructional designers and faculty to identify students at risk and optimize university learning environments for greater rates of student success.
Analytics for Learn offers three important features: data aggregation, the ability to perform longitudinal analysis, and the ability to combine data from multiple sources. Gain a better understanding of each concept below.
1
2
3
DATA AGGREGATION
LONGITUDINAL ANALYSIS
COMBINING DATA FROM
In the learning management system (LMS), every student click and behavior is recorded. “These produce huge complex tables that are extremely challenging to write queries against. Also, as you write these complex queries, you run the risk of putting an enormous burden or strain on that operational system,” explains Harfield. When Analytics for Learn is used, that stream of data is pulled into another system and then transformed, so that information can be easily accessed.
Since retaining data requires a great deal of storage, most learning management systems have a tendency to hold information for a short period, usually one or two years. “But by storing this information in a separate warehouse or system, such as Blackboard Analytics for Learn, it’s possible to capture information and analyze trends across multiple years,” explains Harfield.
NEW VERSION
Version 4.3.5 of Analytics for Learn offers support for Rubrics and Tool Detail.
MULTIPLE SOURCES
Analytics for Learn makes it possible to combine information not only from the learning management system but also from the student information system. “That allows us to do really interesting things, to understand learning behaviors in light of demographic factors and other information that is only stored within the student information system,” says Harfield. Some institutions are integrating data from other learning tools and systems as well.
SOURCE 1. Fritz, J. (2013, April 30). Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course Designs. Retrieved November 8, 2017, from https://www.educause.edu/ir/library/pdf/ERB1304.pdf.
9
Sponsored by Blackboard
Learning Analytics Amplifies Quality Teaching
PHOTO: AFP John Hefti
E - L E AR N
As research and implementation on learning analytics advances, it is possible to catch a glimpse of a future in which data-informed features will become essential to help instructors and students make better decisions. by: priscila zigunovas
LEARNING ANALYTICS AND STUDENT BEHAVIORAL
make decisions about how to develop new features,
data insights are a new form of knowledge that
refine existing features, as well as develop new dedicat-
was not available before educational technologies
ed embedded analytics features,” Whitmer explains.
were so deeply integrated in the classroom. At least that’s the perception of Dr. John Whitmer,
Evolution in Data Collection
Blackboard’s Analytics and Research Director. Before learning analytics, due to the difficulty accessWhitmer manages a team of data scientists
ing behavioral data at scale, the research that campus-
distributed across the globe who work on
es conducted about student use of academic technology
research and analysis. With advanced degrees
usually relied on student interviews and surveys, which
in statistics, training in machine learning
aren’t always accurate. The response rates are low and the
and computer science, as well as experience
students who respond are a biased sample, often reflect-
with e-learning, these scientists think about
ing power users or people with extreme opinions.
and provide answers to questions about how institutions, students and instructors use these
The field of learning analytics is creating and apply-
products in collaboration with Blackboard Prod-
ing new statistical techniques to help researchers
uct Management and Product Design teams.
to deeply understand what is happening in class.
“We look at how both faculty and stu-
“They give us this new form of knowledge about what hap-
dents use our products in order to help
pened during a class that allows us to create better learning
10
E X PE R T A N A LYS I S
John Whitmer Blackboard’s Learning Analytics and Research Director
materials and learning experiences, as well as
Blackboard Learn (in particular) and their class grade. In
interact with students while the class is still
fact, there is a very small relationship between the two.1
going on, while there is still time to intervene, maybe even change the success of a student
“There is often an assumption in learning analytics that if
within that same class,” says Whitmer.
you could get information about what students do and how frequently they use the platform, then you would have a
Next, learn about the main research
magic wand that would completely reveal students’ success
findings by Whitmer’s team, as well as
and factors around student learning. But when we look at it
innovative features that can make a dif-
on a large scale, we find that this is not true,” he explains.
ference for teachers and students. Instead, they found that it was important to look deep-
Can Learning Analytics Provide a Formula for Student Success?
er into course design and the specific uses of Blackboard Learn. “For example, students that spend a larger amount of time looking at their grades tend to do better than stu-
Sometimes the biggest discovery is that a
dents who spend less time doing that,” says Whitmer.
hypothesis cannot be confirmed. One of the most interesting research findings from
In classes that use assessments or discussion forums frequently, the
Whitmer’s team is that, when looking at scale
amount of time students spend on those activities is directly related
across all courses and institutions, there isn’t
to their grade. In the end, what matters most is centered around
a strong relationship between student use of
student effort, pedagogical approaches and instructional practices.
11
E - L E AR N
How Can Democratizing Analytics Benefit Teachers and Students? Educational research is often done by a small group of experts, and historically, the results only circulate around administrative and leadership positions within an institution. These leaders can choose to disseminate the information or not. “But often teachers and students have a lot of questions, and they are very interested in data and information that can help them make decisions,” points out Whitmer. Democratizing
Blackboard’s Learning Analytics Research team investigates how institutions, students and instructors all over the world really use the company’s products. Their findings help Blackboard refine and develop new features.
analytics means making data and insights accessible to them as well as to other stakeholders that can benefit from it. students not participating in certain types of Ethics is also something that needs to be carefully considered,
activities? Looking at those relationships between
according to Whitmer. “That is an important part of what
activity in student learning at a very detailed
Blackboard does, to ensure that the powerful techniques we can
level, looking at discussion forums, looking at
apply are aligned with what we should do in terms of the ethical
your lecture materials, and reviewing notes,
obligations that we have to students. And those obligations are
instructors can get a sense of what students are
different around the globe, given different laws and regula-
actually doing in their course and how it relates to
tion and cultural practices around student data,” he says.
the success they have,” recommends Whitmer.
How Can a Teacher Begin to Use Learning Analytics?
A Blackboard initiative can make this process much easier in the future. With embedded analytics, Blackboard is integrating analytics
Fundamentally, learning analytics means that the
within the workflow of what teachers are already
teacher can study students’ actions and how frequent-
doing, a unique approach in the market.
ly they access learning materials and activities. “Teachers often do not have the time within their “Before learning analytics, we did not know, or at least
busy lives and teaching schedules to go off and
we did not know at large scale, what happened while
think separately about these questions. So, instead
students were in class, how often they studied, how
of going to a separate reporting area or a menu
they studied, how often they opened the materials, how
for analytics, Blackboard is embedding an-
they interacted with them,” points out Whitmer.
alytics within the workflows the teachers are natively engaging in,” explains Whitmer.
The expert suggests that teachers should start using analytics by identifying the most important ques-
In practice, that means that when instructors
tions that they need evidence in order to answer.
log into the LMS, they can receive summary updates about how their students are doing and
“For example, are my most successful students accessing, par-
which ones may need attention; when teachers
ticipating in their learning activities, or are my least successful
are grading, they get information about student
12
E X PE R T A N A LYS I S
participation; when instructors are reviewing discussion fo-
What Will Happen Next?
rums, they get information about the quality of student posts, for example. “We bring the analytics to them,” says Whitmer.
Do Students Notice Notifications?
• More Sophisticated Solutions, Better Insights - Learning analytics is still at its early stages. As education technologies become more and more integrated, the amount of data
The possibility of enabling automatic notifications for
available for analytics will increase, as well as
students based on their performance and activity is an
new analysis techniques to derive meaning
innovative feature in the Blackboard Learn Ultra Ex-
from this data. The solutions for institutions,
perience. These alerts are embedded directly within the
educators and students tend to become
course, and when they are clicked, a student receives
more robust and sophisticated over time.
detailed information as well as a suggested action.2
• More Critical Customers - As learning According to Whitmer, this feature has been dis-
analytics becomes more common, students and
cussed for some time by educators, but institutions
faculty will have more experience and become
have been hesitant in providing notifications directly
critical consumers of these solutions. They will
to students because they are concerned about poten-
be able to distinguish, for example, between
tial negative repercussions. After all, individuals receive
learning analytics and conventional reporting.
so many alerts and notifications daily that distraction and lack of focus are becoming serious problems.3
• More Effectiveness - Learning analytics will be increasingly used to help instructors,
However, research found that sending rule-based
administrators and students to make decisions.
notifications to students is beneficial and recommend-
Also, students will be more successful in
ed. “We have found that students are very interested in
achieving their educational goals. This doesn’t
receiving these notifications and alerts, and they access
mean that analytics will replace human
them much more frequently than they do other types of
decision-making. “We don’t see a scenario in
e-mails or notifications they get from educational insti-
which analytics is anything close to replacing
tutions,” says Whitmer. “Students want to know how
human judgment,” says Whitmer. “In a sense,
they are doing, and how they are doing relative
learning analytics is augmented intelligence.”
to their peers, both positive and negative.” SOURCES
In that sense, learning analytics can be intended to identify what is most important and put that in the center of the student learning experience, so they are better able to pay attention and to focus. “Students can have their own ideas about what they need by going through learning analytics and finding successful or unsuccessful patterns in the past. We are able to identify these behaviors and practices and then bring them to students’ attention,” says Whitmer.
1. Whitmer, J. (2016, March 18). Research in progress: Learning analytics at scale for Blackboard Learn. Retrieved November 22, 2017, from http://blog.blackboard. com/research-in-progress-learning-analytics-at-scale. 2. Whitmer, J., Nasiatka, D., & Harfield, T. (2017, July 26). Do notifications get noticed? New study finds that embedded alerts to students promote action at high rates. Retrieved November 22, 2017, from http://blog.blackboard. com/new-study-notifications-promote-student-action 3. Rosen, L. (2012, April 09). Attention Alert! Study on Distraction Reveals Some Surprises. Retrieved November 22, 2017, from https:// www.psychologytoday.com/blog/rewired-the-psychology-technology/201204/attention-alert-study-distraction-reveals-some..
13
E - L E AR N
Ethical Concerns With Learning Analytics: What You Should Be Aware of Educational institutions that implement a learning analytics strategy can collect significant student data and create reports based on student activity. But is it ethical to use students’ data? Is communicating to students how their data is being used an ethical responsibility of educators? Lastly, are institutions taking proper care of privacy and student rights? Current discussions regarding learning analytics and ethics in the education community have been studying the implications of dealing with such data. This is a complex issue, and there are a variety of concerns that should be considered, especially when innovation in learning analytics continues to be developed at a rapid pace. by: cristina wagner glen fruin, scotland, united kingdom
IN THIS INTERVIEW WITH E-LEARN, CONSULTANT
Niall Sclater explains that learning analyt-
Main Ethical Issues Related to Learning Analytics
ics faces a combination of ethical, legal, and logistical issues, and talks about some of the
•
Poor Quality or Insufficient Data - “Trying to do
ethical concerns that surround student data and
analytics when you do not have much data, or it is
learning analytics. Sclater has been working with
not good quality, is a really difficult and pointless
innovation in online and distance education
exercise. A lot of work needs to be done in cleaning the data
for more than 25 years. He has also worked on
set and compensating for that,” says Sclater. One example
many collaborative national and internation-
of this is something called ‘enmeshed identities’, where
al projects related to the use of technology to
students are working together online and the data cannot
enhance learning and teaching, and is the author
differentiate between the person who is authenticated and the
of the book “Learning Analytics Explained.”
other members of the group. “In learning analytics, the two
14
I N T E RV I E W
PHOTOS: AFP Andy Buchanan
Niall Sclater Consultant and Director at Sclater Digital
15
E - L E AR N
main data sources are generally the learning management system (LMS) and the student information system (SIS), which capture only a tiny bit of the learning that takes place and some of the contextual information. Ideally, we need more data points as well as accurate data.” •
Invalid Analytics - Not only does the data need to be high quality, the analytics also need to be valid. “One of the main rationales behind predictive learning analytics is that there is a relationship between a student’s engagement in their learning activities and their subsequent success. Generally, the more students engage in the learning process, the more likely they are to complete their course and get a better grade. But engagement is not the same as success, and quite often people confuse the concepts of causation and correlation”, Sclater explains.
•
Loss of Autonomy in Decision Making - This is a frequently discussed ethical issue, particularly with adaptive learning systems which continually change the learning based on how students are performing. “Some people worry that it might ‘spoon-feed’ the students too much with automated suggestions, therefore make the learning process less demanding.” •
•
Gaming the System - Students may do something to try to
Students’ Behavior - Another issue is related
improve their scores or their engagement. In one institution,
to when students change their behavior, either
for example, Sclater heard about a student inserting their
consciously or unconsciously, if they know
ID card into the library entry system a number of times,
that their activities are being continuously
trying to improve their library engagement score. “This
monitored. “If the institution is monitoring the
is an example of unforeseen outcomes that you can have
e-book the student is reading, is that going to
when students are aware that you are measuring things.”
change his or her behavior? That might result in the student improving their learning, but it
16
•
Obligation to Act - Finally, Sclater points out an-
also might increase stress levels. Some students
other ethical matter he thinks is important: whether
might even decide not to participate in some
there is an obligation on the institution to act on the
activities at all because they feel uneasy.”
basis of the analytics. “If you have a lot of data
I N T E RV I E W
In others, it is when they have not submitted an assignment on time. And, in others, there
Adopting learning analytics strategies may come with some ethical and legal concerns. Educational institutions should be prepared to deal these appropriately.
are regular points in which personal tutors or student advisers will look at where the students are at, four times per semester, for example.” In addition, institutions should define the different types of intervention, such as an automatic reminder sent to the student, a question, a prompt, an invitation to meet with a tutor, or even
on the students, is there an ethical obliga-
a supportive message. “All these measures can be
tion to do something with it? If it is possible
included in an intervention plan,” he says. Insti-
to stop a student from dropping out, should
tutions should also think about the frequency and
you not be obliged to use that data?,” he asks.
timing of interventions for them to be effective.
Data Should Be Used Carefully
One of the biggest issues for institutions, however, is about how to get staff to change
Sclater clearly states that students should be aware of the
their working practices, and to understand
type of data that is being collected about them, and what is
that analytics is either going to help their daily
being done with it. He mentions that the law in some coun-
activities, or make an impact on the students.
tries says that if any user asks about what the institution is
Lastly, Sclater points out: “There is not much
storing about them, they need to be able to tell the student
point in carrying out interventions unless you
exactly what it is and what they are doing with it. “I do not
are going to evaluate the success of them.”
think there are many universities and colleges yet that are in a position to do that,” he affirms. “The only way forward is to make sure you understand all the data that is
In Learning Analytics, Students Are Individuals, Not Numbers
being collected about individual students, and be in a position to gather it together and provide it quickly
It is very easy to look at a dashboard or some
to any student that asks for it,” Sclater suggests.
other kind of analytics and detect a failing student without considering that he or she might have all
Is There an Ideal Way to Carry Out Interventions With Students?
sorts of potential reasons for that prediction, such as facing an illness or other personal issues, finding the concepts too difficult to understand, or having
Learning analytics is only constructive when accompanied
a demanding job and not having enough time to
by interventions to change student behavior or improve the
study properly. “The only way to find that out may
course. The decision about when to make an intervention
be to bring a human into the process,” Sclater says.
varies according to different institutional processes. Sclater believes that institutions should define what he calls
This means treating students like humans, not
“a trigger,” meaning what precipitates that inter-
numbers. “Many universities are using an-
vention. “In some institutions, it might be the student
alytics as a way of identifying the student
has not logged into the system for two weeks, for example.
who might need a subsequent conversation
17
E - L E AR N
with a person.” Greater data sources can help to find out more about the student and tailor the intervention to their
Developing a Learning Analytics Policy
circumstances. “But you are never going to get the whole picture about a student. For many of them, meeting a person to chat
Sclater says that educational institutions
through their issues is the only way forward,” Sclater alerts.
should develop an institutional policy agreed by the relevant stakeholders in the institu-
Educators Should Focus on the Positive
tion, including students. He suggests some of the main topics the policy should include:
One of the main concerns for educators is that, if analytics shows •
What kind of data is being collected
•
Who is responsible for the overall initiative
•
How the institution is dealing with transparency - making it clear what you are doing and what kind of consent you are gathering from students
•
What is being done about confidentiality and securing the data
•
How to ensure the data and the analytics are valid
•
How students will have access to their personal data
journey here. I think the potential is huge for this.”
•
How interventions will be carried out
How to Manage Student Data
•
How you are going to make sure that there are no, or minimized, adverse impacts on the students
students their position among the class percentile, it could potentially demotivate some of them. According to Sclater, while many students want to see that they are doing well compared to their peers, being told or seeing that they are likely to fail could demotivate others. “It is important to tell students early in the course to get into the habit of checking these analytics,” he says. “And then maybe if they are not doing well, the appropriate thing may be for that student to pull out of the course and enroll in a different one. I do not think that the possibility of demotivating a student is a good reason not to show students how they are predicted to perform.” For Sclater, the best way to provide this information to students is still a matter for investigation. “If we can get the computer systems to understand more about what motivates the individual person, then we may be able to tailor the messages, and the kind of information we present to the students more appropriately.” He is excited by the possibilities that personalization presents. “We are only at the beginning of a
The legal requirements concerning students’ data vary from country to country. According to Sclater, some of the matters that institutions should pay attention to are
He also recommends the development of a student
being clear about what data is being collected and
guide to learning analytics that answers what stu-
why you’re doing that, the capacity to anonymize
dents might want to know (he has developed a model
the data, ensuring that the student has the right to
student guide that can be downloaded here https://
have their data erased if they request that, and de-
analytics.jiscinvolve.org), and of a more in-depth,
veloping careful access controls. ensuring that the
technical document that presents how the analytics
student has the right to have their data erased if they
are actually working. Algorithmic transparency, he
request that, and developing careful access controls.
suggests, is going to be increasingly important.
18
I N T E RV I E W
6 Key Factors to Check if Your Institution is Ready to Build a Learning Analytics Strategy Niall Sclater gives some tips for universities and colleges to make sure they are ready to develop a learning analytics project and to deal with the relevant issues:
1
3
5
LEADERSHIP
Someone in the institution has to lead and promote the initiative. The person should be senior enough to give it credibility and should have a high-level knowledge about learning analytics.
INVESTMENT
The institution should look into the resources that are going to be used in analytics, whether they are for buying software, linking up data sources into the analytics software and so on, and also look at costs in implementing learning analytics, such as the staff costs.
IT
Besides the technology that is going to be adopted, there are different data sources that need to be cleaned and put into the right format. Also, there’s the need to make sure the staff have skills in handling, interpreting and visualizing the data.
2
CULTURE
Institutions need to build a vision for the initiative around their institutional vision, making learning analytics a part of their strategic plan.
4
6
STRUCTURE AND GOVERNANCE
“Learning analytics projects cut across existing roles and power structures in an organization, as different people need to be involved, and individuals can be quite concerned that their perceived ownership of data or influence is threatened somehow by another project coming in,” says Niall.
PREPARE TEACHERS AND LEARNERS
They need to understand some of the processes of adopting learning analytics and be aware of its impacts and advantages.
19
E - L E AR N
10 Reasons Why Institutions Should Implement Learning Analytics Learning analytics turns raw data into valuable strategic information, offering insights to improve teaching, learning, and the environments in which they occur. We’ve gathered 10 reasons why learning analytics has the power to improve the educational experience. by: cristina wagner
1
i n f o g r a p h i c : t r iib u Studio
PROMOTE REFLECTION AND SELF-REGULATED LEARNING
Student-facing reports help learners increase their performance by providing them with information about their course activity and performance compared to others. Systems like this can provide students with signals about how well they are doing in general, as well as information about specific areas they need to work on. By making learning analytics available to students, institutions can promote self-regulated learning. This practice not only improves student outcomes but also fosters a mindset that may help learners increase their post–graduation success.1
20
2
IDENTIFY STUDENTS AT RISK OF POOR PERFORMANCE
By using data to identify students who are likely to struggle in a course, proactive advising can help students in a timely manner. Learning analytics surfaces relevant information about student engagement and performance that is key for academic advisors and student success specialists.¹ This way, professionals can reach out and intervene at an earlier stage than would otherwise be possible and help learners persist in spite of challenges.2 For example, using predictive analytics tools like Blackboard Predict, faculty and advisors can start identifying at-risk students before the second week in the semester.⁸
I N FO G R A PH I C
3
UNDERSTAND THE IMPACT OF INSTRUCTIONAL DESIGN PATTERNS ON STUDENT PERFORMANCE
Learning analytics can help identify the most successful instructional design style by program and course. By correlating tool use by faculty and students with specific learning outcomes, institutions can identify and scale high impact practices to assist specific program and curricular goals.¹⁰
4
INCREASE REPORTING EFFICIENCY
Colleges and universities must fulfill a wide variety of institutional reporting requirements. Many of these concern educational quality and the impact of educational technology investments. By capturing data on learning activity, and automating report creation, learning analytics technology can increase the accuracy and efficiency of institutional reporting in a way that frees analysts to think beyond mere reporting and also use the same data in support of other strategic initiatives on campus. For example, California Baptist University Online used Analytics for Learn to automate an attendance tracking process that was otherwise labor-intensive and prone to error.¹¹ Because of an increase in institutional efficiency, CBU Online is also able to start using that same information to identify atrisk students and intervene with targeted interventions to keep more students on track for graduation.
5
OPTIMIZE ASSESSMENTS
Learning analytics allows instructors to identify assignments that are highly associated with a student’s final grade and those that tend to correlate with a sudden drop in performance and course withdrawals. Instructors can quickly identify the assessments that are working and those that might need revision.
21
E - L E AR N
6
MEASURE TOOL ADOPTION AND IMPACT
With access to data on learning environment activity, institutions can get a realistic picture of how the system is being used and increase its adoption where it is likely to make the biggest impact.¹² Institutions can also assess the use of third party tools, which, in some cases, might only be used by a few faculty members and do not justify their high costs. This process is important to establish an educational technology investment value, and also for instructional designers and faculty developers to look for evidence that supports assumptions about best practices.²
7
IMPROVE FACULTY DEVELOPMENT
With access to student engagement information, LMS usage and instructional design practices, learning analytics helps faculty developers provide evidence in support of high impact instructional practices. Access to information about faculty behavior before and after professional development activities also helps institutions to evaluate the impact of their faculty development initiatives.¹³ Instructors are also able to identify and promote effective teaching practices, as they can immediately address gaps and challenges and adjust their courses accordingly.³ Data can also help identify areas in need of improvement by the instructor to facilitate enhanced instructor-student interactions in the educational environment.⁹
8
NUDGE STUDENTS TO IMPROVE LEARNING BEHAVIORS
Learning analytics can be used to improve student success by providing students with relevant performance information and alerting them in advance whether they are at risk of not passing a course. When students can see how they are performing, it increases their sense of agency over their own learning and empowers them.8 Blackboard Learn Ultra has the ability to automate notifications embedded directly within the course. This way, students can receive more detailed information as well as a suggested action, allowing them to keep their performance on track and improve the areas in which they are having difficulties.⁴
22
I N FO G R A PH I C
9
UNDERSTAND THE IMPACT OF ASSIGNMENTS ON DISCUSSION BOARD ENGAGEMENT
Learning analytics can help instructors understand individual learners’ discussion forum activity, as well as critical thinking and the originality of their contributions. They can also test the impact that specific discussion board assignments have on the amount and quality of student participation in the online environment. With insights gained from learning analytics tools like X-Ray Learning Analytics, instructors can create activities to keep students engaged.⁵
10
IDENTIFY AND ELIMINATE SOURCES OF SYSTEMATIC INEQUITY
Access to longitudinal data about relative student outcomes across different sections of the same course can help institutions to identify areas in which the course environment itself is creating barriers to success. For example, institutions often see significant grade variance between sources taught online and in person. At many institutions, faculty grading patterns differ widely between course sections. Pierce College⁶ and Indian River State College⁷ are examples of institutions that have benefited from access to this kind of data.
SOURCES 1. Blackboard. Nine things you can do with Blackboard Analytics for Learn [PDF]. 2. Harfield, Timothy. From LMS Reporting to Learning Analytics: Exciting updates to Analytics for Learn. Retrieved November 17, 2017, from http://blog. blackboard.com/from-lms-reporting-to-learning-analytics-exciting-updates-to-analytics-for-learn. 3. Everhart, Deb. Learning Analytics: The Future is Now. Retrieved November 17, 2017, from https://edtechdigest.wordpress.com/2012/05/10/learning-analytics-the-future-is-now 4. Whitmer, John. Do notifications get noticed? New study finds that embedded alerts to students promote action at high rates. Retrieved November 17, 2017, from http://blog.blackboard.com/ new-study-notifications-promote-student-action 5. Blackboard. X-Ray Learning Analytics. Retrieved November 17, 2017, from http://www.blackboard.com/ resources/pdf/datasheet-xray-rev20170228_002_.pdf 6. Osborn, Eliana. Dashboards for Success. Retrieved November 17, 2017, from https://www.insidehighered. com/digital-learning/article/2017/03/15/pierce-college-uses-data-dashboards-improve-graduation-rates 7. Blackboard. Learn how Indian River State College grew their online enrollments by 35% in two years. Retrieved November 17, 2017, from http://pages.blackboard.com/campaign/analytics/indian-river-state-college-case-study.aspx 8. Rattiner, Marlen. Walking the line of predictive analytics in higher education. Retrieved December, 2nd 2017, from http://blog.blackboard.com/walking-the-line-predictive-analytics-higher-education/. 9. John, A., Mansureh, K., Sandra, N., & Theresa, K. (n.d.). Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. Retrieved November 16, 2017, from http://files.eric.ed.gov/fulltext/EJ1105911.pdf 10. Harfield, Timothy. Analytics for Learn: Using data science to drive innovation in higher education. Retrieved December 4, 2017, from http://blog.blackboard.com/analytics-for-learn-data-science-innovation-higher-education/ 11. Simpson, Rich. Using analytics to track non-attending students. Retrieved December4 2017, from http://blog.blackboard.com/using-analytics-to-track-non-attending-students/ 12. Harfield, Timothy. Analytics From LMS Reporting to Learning Analytics: Exciting updates to Analytic. Retrieved December 4, 2017, from http://blog. blackboard.com/from-lms-reporting-to-learning-analytics-exciting-updates-to-analytics-for-learn/ 13. Mead, M. (Director). (2017, June 13). Learning analytics and faculty development [Video file]. Retrieved December 4, 2017, from https:// www.youtube.com/watch?v=BSJKVU6Ltys
23
E - L E AR N
Sponsored by Blackboard
Experiments and Challenges with Learning Analytics Around the World
Dragan Gasevic Professor and Sir Tim O’Shea Chair in Learning Analytics and Informatics and Co-Director, Centre for Research in Digital Education, The University of Edinburgh, Scotland
Since the first learning analytics conference, held in Canada in 2011,1 the field has been developing rapidly. Some of the trending areas in research are teacher and learner dashboards, predictive analytics and automatic feedback. There is also rising use of natural language processing technologies and multimodal learning analytics. E-Learn interviewed experts from North America, Latin America and Europe in order to understand these trends and to find out how they are experimenting with novel uses of data in education, as well as their perspective on the future of learning analytics.. by: priscila zigunovas
LEARNING ANALYTICS IS AN EMERGING FIELD, AND ITS ADOPTION BY
help make their students’ educational experiences
higher education institutions around the world is very uneven.
better, but there is not as much action yet as one
The United States, United Kingdom, Canada and Australia are
might hope,” Wise explains. “I think everybody
considered leaders in this domain, while regions such as Latin
feels a little behind the curve of what is possible.
America are still at an initial stage of exploring its possibilities.
They think everyone else is doing these spectacular things, but (with a few notable exceptions)
In the United States, there is great awareness and inter-
most universities are all working through the same
est in learning analytics by higher education institutions,
set of issues. The reason is that there are a variety
says Alyssa Wise, associate professor of learning sciences &
of barriers that need to be overcome to put robust
educational technology and director of New York Universi-
learning analytics systems into place,” she adds.
ty’s Learning Analytics Research Network (NYU-LEARN). These barriers include technical and infrastructural “If you talk to leaders at almost any higher education institution
challenges, and important questions around data
in the country, you will see that they know about learning
stewardship and access to data. “One of the biggest
analytics, and they are excited about the idea of using data to
challenges we face with learning analytics projects
24
R O U N DTA B L E
The United States, United Kingdom, Canada and Australia are considered leaders in learning analytics, while regions such as Latin America are still at an initial stage of exploring its possibilities.
how analytics can be beneficial for institutions, and the lack of strategic ability, that is, not having institutional leaders that can identify those critical issues and how learning analytics can help. A third barrier would be related to privacy protection and ethics, which is a main issue in specific countries.
PHOTO: AFP Andy Buchanan
“Countries such as Germany have had, historically, through World War II, a huge abuse of their private data, and thus people are much more concerned with respect to ethics and privacy,” explains Gasevic. “What we are seeing, since we are running a major project called SHEILA,2 is that students are aware of the ethical and privacy protection questions and they have high expectations that institutions will protect their privacy, but at the same time, they also have big expectations that their data will be used.” is that we have a great idea for a project and all of the stakeholders are on board, but nobody
In Latin America, a recent study revealed what researchers
knows who is able to give permission to look at the
already knew: adoption of learning analytics in Latin Ameri-
different data sources we need. This has not been
can countries is still at very initial stages, according to Xavier
something that universities have had to deal with
Ochoa, professor and director of the Teaching and Learn-
before because nobody has asked,” explains Wise.
ing Technologies Research Group at the Escuela Superior Politécnica del Litoral (ESPOL), in Guayaquil, Ecuador.
In Europe, according to Dragan Gasevic, professor and chair in learning analytics and informatics
“There are few institutions that are committed to implement-
at The University of Edinburgh, the majority of
ing learning analytics in their core processes, but there’s a
institutions are aware of learning analytics, and
lot of interest. I haven’t talked with any administrator that
there are some small experiments taking place.
is not very willing to try learning analytics at their institution,” says Ochoa. According to him, the lack of research-
Possible barriers for adoption include shortage
ers and education professionals with expertise in learning
of capacity or knowledge to actually understand
analytics is the main barrier for adoption in the region.
25
E - L E AR N
See next some of the most interesting experiments that are taking place at institutions around the world.
Predictive Analytics: An Upward Trend Alyssa Wise, from New York University, is involved in several different projects that work with proactive support for students through predictive modeling. In predictive analytics, a computational model is created based on students who have previously taken a course, and then applied to students who are beginning the course in order to identify those who are likely to have difficulties. “The accuracy of the model we have is quite good. We can identify as early as the first day of class who are the students likely to have trouble in a course. The challenging thing is: what do we do about it?” she says. It is evident through this example that being able to predict difficulties isn’t enough. Researchers and education professionals still need to figure out how to best use this information to support teaching and learning. In one of the projects, Wise explains, the model shows that student difficulties in introductory college mathematics courses
induce stereotype threat, and thus become a
are linked to inadequate prior preparation. “In this particular
self-fulfilling prophecy,” adds Wise. “Instead, it
model the path toward action is actually straightforward. We
is important to stay positive and focus on what
can offer these students learning resources and mastery driven
actions they need to take to be successful.”
practice sets to help them get up to speed on the skills and concepts that they may not be strong enough in at the start,” she says.
According to the expert, NYU has started to build teacher dashboards that give instructors
However, in other cases, the model can predict who will have
an overview of their students’ digital activities.
trouble, but the factors used to predict are not very clearly
This can be used both to evaluate particular class
actionable. “In this case, we know which students are going
elements and identify students that may need
to have challenges, but we do not know why or how. So, it’s a
particular attention. “The next step is to study how
much more difficult challenge to figure out what to do with the
instructors use these new sources of information
model. Do we show it to the instructors? How do the instruc-
to inform their teaching and how we can better
tors contact the students and what do they say?” says Wise.
support them in this process of data-informed pedagogical decision-making,” Wise describes.
“We have to be careful because you do not want to tell students at the start of a course that you are expecting them to do poorly.
This illustrates why a great deal of attention is
This has the potential to be demotivating, and in some cases
now being paid to how institutions are putting
26
R O U N DTA B L E
learning progression, their time management and their misconceptions,” explains Gasevic. This empowers teachers so they can get much deeper insight into what is happening in their classrooms, especially in blended or flipped models. And more importantly, instructors can also provide personalized individual feedback at scale.
Alyssa Wise Associate Professor of Learning Sciences & Educational Technology and Director, NYU Learning Analytics Research Network (NYU-LEARN), New York University, United States
A project in which Gasevic is involved aims to accomplish exactly that. Project OnTask began in 2016 led by The University of Sydney in collaboration with University of Technology Sydney, University of New South Wales, University of South Australia, University of Texas at Arlington, and The University of Edinburgh.
PHOTO: AFP Dominick Reuter
“With learning analytics and support of the OnTask software, instructors are able to provide personalized feedback that scales up highly. For the amount of work to support 3 to 5 students, they are able to personalize feedback for hundreds or thousands of students”, Gasevic explains. The project’s results show increase in student performance, satisfaction, and learning process. The connection between learning analytics and learning design is another important topic in the field. Learning analytics can help data and models into practice in ways that are eth-
teachers to reflect on the effectiveness of their learning design.
ical and useful to support teaching and learning. “For example, when I am teaching a course, I can re-
Feedback Loops: Empowering Students and Teachers
flect on what is working in my course, or what is not working, and based on that I can make certain changes in my design and, in little time, start making that or-
Establishing or improving feedback loops between
chestration much more effectively,” explains Gasevic.
teachers and learners is the main gain of learning analytics according to Dragan Gasevic, from The
Gasevic and his team developed an award-winning software
University of Edinburgh. That can be especially help-
called LOCO-Analyst, a tool that aims at providing teachers
ful in larger classes, sometimes with hundreds of
with feedback on the relevant aspects of the learning pro-
students, which makes it impossible for a teacher to
cess taking place in a web-based learning environment.
interact with all of them at a more personalized level. “When we developed LOCO-Analyst we were trying to “Learning analytics helps us get the
analyze different types of digital traces generated by
insight into the learning patterns
learners as they were using and interacting with digital
of every one of our students, in or-
resources on the web and inside of their learning envi-
der to understand, for example, their
ronments, such as Blackboard Learn,” he explains.
27
E - L E AR N
According to the expert, one of the function-
there, there aren’t traces of these activities. Maybe you have
alities that is most appreciated by teachers
the end results of what they did, but a great deal of informa-
offers critical insight not only at the level of
tion is lost if you only analyze the end result,” he explains.
particular resources, but at the conceptual level, and the way students are actually construct-
Multimodal learning analytics exploit audio and video
ing knowledge around certain concepts. The
recordings, what students write, look at, and say, and all this
insights into different acts of social interaction
information is used to get a clear picture of the learning process.
that happen among students are also valued. “Humans are multimodal in nature, we capture Gasevic sees a fairly big disconnect in terms
information through all our senses, so we
of what is actually needed by teachers and
are trying to make the computer do some-
students, and what is currently out there. “What
thing similar,” explains Ochoa.
is happening in reality is that most teachers and students do not understand the content
According to him, as technology became more accessible
in their dashboards. I attribute that to major
over the recent years, the development of multimodality
problems, and the first problem is related to
became a reality. Two factors that have enabled the field of
the lack of proper ethnographic studies trying
multimodal learning analytics are the development of artifi-
to understand the teachers and the students
cial intelligence and the availability of very cheap sensors.
in the way they can embed and use analytics inside of their specific tasks,” says Gasevic.
“Imagine that you can have a device that costs almost nothing with a camera and a microphone,” describes
Beyond the LMS: Multimodal Learning Analytics
Ochoa. “With this type of sensors and the right software, it’s possible to analyze, for example, student posture, create transcripts of what the student is saying, or capture emo-
At the beginning, learning analytics worked ex-
tions in the student’s voice. So, now, computers are able
clusively with data from online tools, like learn-
to see, are able to hear, and we are trying to exploit those
ing management systems or online games. But
capabilities to better understand the learning process.”
what about the learning that is happening outside the computer? How could data from the re-
This expert estimates that 30% of studies on learning ana-
al world, such as in classrooms, student groups,
lytics currently include some kind of multimodality. That
or even when students are doing their home-
could be an indication of institutions realizing that they need
work, be captured and measured? This branch
to be more holistic when looking at learning processes.
is called multimodal learning analytics and it’s Xavier Ochoa’s personal field of research.
One example of a multimodal analytics project developed by ESPOL is a tutor environment where students
“Multimodal learning analytics arise natu-
can practice giving presentations. Students enter a room,
rally out of the need to understand learning
close the door, and then they are greeted by a virtu-
where it is happening. For example, there is
al audience to whom they can present their slides.
a lot of learning that happens when students work together, trying to solve a problem or an
“The system will analyze their posture, their gaze, the
exercise in a course. If there is no computer
volume of their voice, if they are stammering, and it
28
R O U N DTA B L E
What’s Trending? • Technical and theoretical advances are making it possible to collect better and richer data.
• This can be done not only through online tools, but also through clickers, sensors and motion detectors that collect data from the real world. That’s multimodal learning analytics.
• Providing automatic, personalized feedback through student behavior analysis is becoming more common, especially for large classes.
• Natural language processing technologies are being used to analyze student-created products. This allows instructors to go beyond multiple-choice questions and
PHOTO: AFP Rodrigo Buendia
respond to students’ creative works as well.
• Researchers also want to discover how analytics can help understand students’
emotions, not just their cognitive state.
Xavier Ochoa Full Professor, Director of the Information Technologies Center, Escuela Superior Politécnica del Litoral (ESPOL), Ecuador
• A challenge that researchers currently face is that tools have to be designed to work in
specific learning contexts, but at the same time they need to be general enough to be used
will analyze their slides, if they have too much text or
across different courses and even institutions.
the font is too small, and will give an automatic feedback on the presentation,” describes Ochoa.
• Learning analytics will be increasingly used to support decision making for
On that same area, ESPOL is using multimodal learning
students, instructors, administrators,
analytics to understand what differentiates experts from
institutions and even governments.
non-experts while they are trying to solve a problem. That’s a type of system where multimodal learning analytics is used to provide feedback not for the student, but for the instructor. “We can try to understand if a student has reached mastery in a skill just by looking at him or her. That’s exactly what a professor would do, observe student behavior,” compares Ochoa.
SOURCES 1. LAK11. (2011). 1st International Conference on Learning Analytics and Knowledge 2011. Retrieved November 15, 2017, from https://tekri.athabascau.ca/analytics/program. 2. SHEILA. (n.d.). Using data wisely for education futures. Retrieved November 16, 2017, from http://sheilaproject.eu.
29
E - L E AR N
Sponsored by Blackboard
Elizabeth Polzin
PHOTO: AFP Darren Hauck
Assistant Vice President of Academics for Student Success at Concordia University Wisconsin
30
C U STO M E R S N A P S H OT
Concordia University Wisconsin: How to Achieve a Record Retention Rate Identifying at-risk students and helping them out. That’s what the use of Blackboard intelligence and learning analytics solutions has brought to Concordia University Wisconsin (CUW). by: leonardo tissot mequon, wisconsin, united states
BEFORE THE USE OF THIS TECHNOLOGY, IF STUDENTS
collected this past year helped us better understand
were struggling with learning, the university had
the challenges our students were encountering and
to rely on faculty reports. And several times, when
aided us in much earlier intervention,” she says.
it came to this point, it was already too late for students to get back on track and save their semesters. “Now, we are able to make an early intervention,”
In Search of a More Personalized Experience
says Elizabeth Polzin, Assistant Vice President of Academics for Student Success at CUW.
At CUW, students are not just “a number.” “We are proud to say we know students’ names at CUW. The ability to use
The work started about a year ago, first limited to
analytics to better serve our students provides them with a
on-campus undergraduate students. That was a
personalized experience. Rather than waiting until the end
precaution measure – making sure they were getting
of the semester to determine what went wrong, we have to
reliable data. And the results have been great so far.
be able to take note when they stop attending class or begin to fall behind in their coursework,” Elizabeth points out.
Last fall, CUW had a retention rate of 72%. A year later, the university experi-
The most important endeavor is to find out how
enced a 10% increase, reaching a record
students are developing in their academic com-
82% retention rate. Along with the
mitments while there are still opportunities
involvement of administration and fac-
for recovery if they are doing poorly, not just
ulty, analytics also was a big part of this
at the end of the semester when it is too late.
result, Elizabeth believes. “The data we
That is what academic support is there for.
31
E - L E AR N
“With the Blackboard learning analytics tools, this year has been
and trying to formulate solutions – from finan-
a bit of an explosion for us. We just started to dabble with what
cial to mental health. If a student disappears for
types of reports we could create, and better strategize our out-
a while, it’s possible – through the attendance
reach. I think it’s one of the reasons we saw a big jump on our
feature – to determine if they haven’t been
retention rate: we had the opportunity to provide personalized
to class. With that type of real time informa-
outreach before the semester was over,” the assistant VP says.
tion, a university staff member can follow up with the student to determine what support
THE IMPACT OF LEARNING ANALYTICS AT CUW
they may need. Instead of using automated emails, we’re able to provide individualized
RETENTION RATE
outreach from someone the student knows.”
Building a Better Relationship Between Faculty and Students
81.7% 71.62%
(New Record)
2016 2017
The use of data analysis has improved relationships between faculty and students due to a new referral system for professors. In the past, when a student started to struggle, faculty may have felt as though they had to deal with the situation on their own. However, that’s not something all professors feel comfortable with. Now, there is a system in
Offering Better Quality Outreach
place where faculty can refer students they are concerned about, and an academic support team
Based on the data provided, CUW is working with dashboards
is able to use analytics to address students’ needs.
that can be viewed by each of their advisors, with focus on student performance and a variety of risk factors. For example, “advisors can now easily view whether a student has had a significant drop
A Way to Provide Revenue for Universities
on their GPA from the prior term, if they are registered for 18 or more hours but have a low GPA, or if they have had a significant
More than offering students and faculty the best
drop in term GPA from their cumulative GPA” among other data.
possible experience, learning analytics has also been impacting CUW’s revenue through academic
The dashboards help the university envision how to
support. “It’s a better use of time and human re-
support students more efficiently – even the ones
sources in general. As a supervisor, I prefer to not
who need outreach at the beginning of the semester,
have my team spinning their wheels in a direction
before they even start struggling with their educa-
where they end up in a dead end. Using analytics
tion. “With a review of identified risk factors at the
gives us focus and direction as to the best way
beginning of the term, our intervention can start
to serve our students,” Elizabeth reflects. “The
much earlier than it has in the past,” Elizabeth says.
bottom line is: through the use of analytics, we know that our decisions are informed,
CUW also has an intervention team to help students, such as
and the results we’ve seen are a positive
getting to know what types of problems they are experiencing
consequence of that,” she concludes.
32
Over the past four years, we've focused on collaboration with the teaching and learning community by sharing:
Visit E-Learn online and stay up-to-date with online educational trends. ideas
perspectives
insights
practices
Get all content related to learning analytics here
Read our stories and see how they can benefit you and your institution:
www.elearnmagazine.com 33
“The only way forward is to make sure you understand all the data that is being collected about individual students, and be in a position to gather it together and provide it quickly to any student that asks for it.”
Niall Sclater Consultant and Director at Sclater Digital
“Analytics is not a replacement for human judgment. It’s a form of information, but it still needs to be interpreted, and that interpretation requires human wisdom.”
Timothy Harfield Senior Product Marketing Manager for Blackboard Analytics
“At Condordia University Wisconsin, students are not just a number. We are proud to say we know students’ names.”
Elizabeth Polzin Assistant Vice President of Academics for Student Success at Concordia University Wisconsin