Will Analytics transform Education?
A critical view on the data we gather about the learners

Effective use of data is vital for success in today’s business world. In education, Analytics (or Learning Analytics) is becoming a hot topic, promising to disrupt and transform education and learning. In this overview article we do a short detour to the business world for some examples of business analytics; look at how education have approached the phenomenon; explore some practices; and raise some concerns about the downside of this trend.

The most spectacular example of business use of consumer data is the US chain store Target’s analysis of changes in a customer’s life, e.g. finding out whether or not a customer is pregnant[1], with the aim to send them coupons for certain products they will need. Internationally, Amazon’s use of real-time recommendations to give users feedback almost instantly and to create a personal shopping experience for each and every customer[2] is well known. Business analytics, however goes beyond targeting consumers. Oakland Athletics’[3] uses predictive analytics to sign baseball players with great potential, so that they are able to make them successful for a lower cost. Similarly, companies like Netflix, Wal-Mart, Facebook and Google, have also been analysing consumer behaviour for years to expand their markets and sales. The way data has been used to gain a competitive advantage in businesses and other sectors has attracted policy makers, funders, and also educators and administrators in the education sector.

Making sense of analyticsIn education, slightly different definitions of Analytics have emerged. Educause has emphasised the aspect of prediction of analytics[4], while JISC CETIS has highlighted the aim of developing actionable insights[5]. In the 2012 Learning Analytics and Knowledge Conference the distinction between Learning Analytics (LA) and the purposes it serves was stressed[6]. An infographic of LA, developed by Andrianes Pinantoan at Open College, explains what it is and how it works (see end of this article). In some cases, people use analytics and learning analytics interchangeably in their pulications and research papers.

Probably, it is fair to say that the core concept of analytics (or learning analytics) is not completely new to many educators and it has been in existence for several decades in education theory and practices. On one hand, good teaching practice has, for a long time, involved recording information with pen and paper for the analysis and reflection of this data to inform further action and interventions relating to individual pupils or classes; on the other hand, researchers in the educational data mining field have a long history in developing tools and techniques to make use of data to improve teaching and learning or education on the whole. However, there is a gap between what technology promises to do and what people can do with existing data sources and tools in reality. It is only in recent years, due to the increasing use of technology in education, that more and more personal information and detailed records on learning activities and assessment have become available in Virtual Learning Environments (VLEs) and other systems. The development of new techniques and tools that lower the technical and cost barrier of undertaking such analysis makes it possible for educators to gain an insight from various data sources to achieve efficiency and effectiveness and improve students’ performances.

Analytics in practiceAnalytics in education is still at its early stage and most of the work in this area is conceptual and comprises small scale funded projects. At the moment, most educational data-related projects are under the umbrella of analytics or learning analytics and include tracking student learning, providing early alerts and interventions to improve retention and operations. The following are some examples of analytics applications (platforms and specific tools) and organisational initiatives in education:

1. Adaptive testing, tracking and reporting:Most large corporations such as Pearson, Blackboard and Desire2Learn have invested heavily in analytics to capture a significant amount of data, including the time spent on a resource, frequency of posting, number of logins, etc. The Khan Academy, a free web site where students can access thousands of tutorial videos covering hundreds of subjects, also provides a completely online learning environment and incorporates learning analytics to enable teachers to assess progress and focus on an individual student, including:  progress summary, daily activity report, class goals report, progress report, student activity report, student focus report, etc[7]. By using various analytics tools, students can review their learning progress and teachers are also supported in how to personalise learning for students in need for more help in specific areas.

2. Analytics tools for early alert, intervention and collaboration:Some analytical tools have been developed to track students’ academic performances by integrating their data collected from a variety of information management systems, allowing educators to assess the risk, initiate early interventions and support collaborative learning. For example, the Signals project at Purdue University utilizes the data collected from student information systems, learning management systems, and the grade book for a specific course to track students’ performances and identify at-risk students in real time. The LOCO-Analystprovides teachers with charts, graphs, and other data representations that help them see how their students are performing and how students interact with one another in web-based learning environments to help the teacher determine how to engage their students online. Social Networks Adapting Pedagogical Practice (SNAPP), a network visualization tool developed by researchers at the University of Wollongong, can analyse students’ interactions in a forum and display it in a visualised diagram which help teachers to identify the key connections and disconnected students and support collaborative learning in a web-based learning environment.

3. Analytics projects for institutional efficiency and effectiveness:There are a number of institutional analytics initiatives which enable institutions to measure their business and operational performance, and improve the effectiveness of operations, including admission management and drop-out prevention, resource management, financial planning, etc. For example, the Student Experience Traffic Lighting (SETL) project at the University of Derby, integrates data from different systems to provide an overview of a student’s level of engagement with the institution and is proactive in identifying students at risk of withdrawing to improve retention rate; The Enhancing Student Centred Administration for Placement Experience (ESCAPES) project at the University of Nottingham focuses on improving the management of its placement process for both staff and students by providing unified placement processes across the institution and extra facilities for data reporting to improve administrative efficiency and more effective management of relationships with students whilst they are on placement.

Concerns on analytics in education:Not surprisingly, funders, educators and administrators are expressing great excitement over the promises and possibilities of using analytics in education. However, a number of unanswered questions, concerns, and hesitations suggest that there is a need to move analytics forward with caution and reflection. It is agreed that education is a complicated system and cannot really be run as business. Similarly, learning is a complex social activity and all technologies, regardless of how innovative or advanced they may be, remain unable to capture the full scope and nuanced nature of learning. Gardner Campbell is concerned that analytics might encourage a more reductive approach towards learning which is dangerous for promoting deeper learning and meaningful learning. Melanie Booth has warned that analytics needs to be complemented with well guided teaching, learning and assessment principles, otherwise its measures may be meaningless. Some early analytics projects also found that it was dangerous to make decisions about student learning based solely on a set of data, as it is essential to understand the context of the data and developing the relationship between the tutor and student. Simon Buckingham Shum and Rebecca Ferguson (2011)[8] identified some of the concerns on using analytics in teaching and learning as:

  • Not genuinely promoting meaningful learning.
  • Devaluing the role of teachers/mentors, who will never be replicated by machine intelligence.
  • Disempowering learners by making them rely on continuous machine feedback rather than developing their own meta-cognitive and learning-to-learn skills.
  • Devaluing the essence of deep learning and scholarship, through assigning short-term financial value to superficial indicators.
  • Raising ethical issues about the use, re-use and merge learner data, both public and private

Data, by itself, does not mean anything and it depends on human interpretation and intervention. Analytics may provide a valuable insight into a student’s learning, but if teachers do not take actions to intervene, then it will not help to improve the academic performances. Furthermore, contextual data that are not always found in learning management systems, should also be recorded and considered, in order to provide full pictures of students’ learning and performance.

Undoubtedly, with the growing adoption of technology in education and with better data gathering and analysis tools available, analytics will play a significant role in improving administration, research, teaching and learning, and resource provision in education. However, technology cannot act alone to radically disrupt and transform education and learning, educators need to have the knowledge, skills and tools for using analytics in practice to make informed decisions and take action. Analytics poses a new challenge and provides opportunities for educators and technologists to work together to explore new tools and techniques to use data effectively, to bring real changes in teaching and learning and to transform the accountability, efficiency and relevance of education.

[2]4 Ways to Harness the Power of Real-Time Analytics, http://blog.kissmetrics.com/real-time-analytics/

[4]Accordingto EDUCAUSE, “Analytics is the use of data, statistical analysis, and explanatory and predictive models to gain insights and act on complex issues”. (2012)

[5]JISC CETIS has coined a working definition of analytics as “Analytics is the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data”  (CETIS, 2012), This definition sees analytics as three activities: 1) data provision, 2) interpretation and visualisation and 3) actions based on insights.

[6]Learning Analytics and Knowledge Conference (2012) defined learning analytics as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”

[7]. Exploring the Khan Academy’s use of Learning Data and Learning Analytics, http://www.emergingedtech.com/2012/04/exploring-the-khan-academys-use-of...

[8]Social Learning Analytics, http://tinyurl.com/9kt3cjp


Editor for this story: Tore Hoel