What are Learning Analytics?

The term Learning analytics refers to “the measurement, collection, analysis and reporting of data about learners and their contexts (1), for purposes of understanding and optimizing learning (2) and the environments in which it occurs”.

• “data about learners and their contexts” (1)
  • the emergence of digital learning platforms in (higher) education makes the learning process of students much more visible than roughly 20 years ago. After all, these platforms are responsible for the greater share of information transfer and communication between lecturers and students, and they store all that activity;
  • ‘digital traces’ left by (groups of) students give lecturers a better view on how their students interact with the learning contents they upload to the learning platform.
• “for purposes of understanding and optimising learning” (2)
  • the learning analytics or analyses of students’ learning process within the digital learning environment can help lecturers to better understand, and to optimise said processes;
  • it is a general rule only to perform analyses at a group level. Lecturers are not allowed to focus on individual students, not even out of curiosity, e.g. to check individual students’ activity logs;
  • Ghent University has drawn up a code of conduct for the different users of ‘education data’. Anyone who wants to analyse education data (i.e.: anyone who applies learning analytics) must follow the rules stipulated in this code of conduct with respect for the students’ privacy. 

Use of Ufora Education Data by lecturers for Policy-making, Education Improvement, Student Coaching and Student Support

What analyses can lecturers perform based on the data available on Ufora? Where to find those data and analyses, and what to learn from them? This Education Tip provides the answers. 

 

Student Percentages: Content Visitors/Viewers and Average Time Spent

This analysis shows:

• an overview of how many students have viewed a particular content item (learning activity);
• the total number of students who can view that particular content item. With some calculations you can derive a % from this;
• the average time students spent on this content item

This analysis allows you to make an estimate of how actively your student group is engaging with your course unit. Have you noticed that some content items have hardly, or not at all been visited? 

• then the content item is either not visible to the students or they cannot find their way there.
• check the content item’s settings.
• Tip! If there are interim ‘tests’ between different content items, you can also check how many students complete the test.

You can find this analysis on Ufora:

Contents > Table of Contents > Related Tools > View Reports

 

Ufora Tests Analyses

Ufora Tests provide meaningful information, regardless of whether you use the test for exercises or for actual assessments.

User Statistics

This analysis shows:

• the average test score of a particular student group;
• the score distribution: the percentage of the students scoring more, or less than half the marks  

This analysis teaches you:

• to estimate how well or how badly the student group has mastered the content.

You can find this analysis on Ufora:

Ufora tools > Tests > Choose the test for which you want to see the analysis and click ‘v’ next to the name of the test > Statistics

 

Question Statistics

This analysis shows:

• the student group’s average score per question 
• the distribution of scores per test question: the percentage of the students scoring more, or less than half the marks

This analysis helps you to assess the quality of the question, regardless of whether it is used as an exercise or assessment:

• a question with an average score tests the student’s knowledge well;
• a question that is answered correctly by 100% of the students is too easy and does not contribute much to the assessment.
• a question that is not answered correctly by any student is too difficult.

 

In addition to the average score, this analysis provides standard deviation, discrimination index and point-biserial correlation.

• discrimination index
  • a good question distinguishes between high-achieving students and low-achieving students. Ideally, the students who score high on the test in its entirety also score high on question X, while the students who score low on the test also get question X wrong. A question that distinguishes between the high-scoring and low-scoring students is said to have a high discrimination index;
  • imagine that a question is answered poorly by the group of high-scoring students and is answered correctly by the group of low-scoring students. This means that something is wrong with the question.
• The point-biserial correlation is the Pearson correlation for dichotomous variables.

 

Question Details

This analysis enables you to check the % of students to have chosen each of the answer options in a multiple choice question:

• a ‘distractor’ (i.e. a wrong answer option) that is chosen as the correct answer by none of the students is not a good distractor. This means that the answer option is probably too obviously incorrect;
• when only the correct answer is selected by the students, all other distractors are not good either. This means that all the other answer options are too obviously wrong as well; 
• ideally, each distractor is ‘reasonably’ chosen as the correct answer by the students. If that is the case, then there is real doubt among students, and they have to analyse the answer options and weigh them against each other. This way you thoroughly test underlying theoretical views.

Use of Ufora Education Data by Lecturers for Assessment Purposes

 

Under certain circumstances, and only for assessment purposes, it is possible to perform education data analyses at the level of individual students. However, this is only allowed if the analysis can demonstrate whether or not a student has attained a specific learning objective. 

Do not... 

assess students based on the time and duration of their activity on the digital learning platform. 

Do… 

assess students based on e.g. whether or not they have through a specific learning pathway, whether or not they have uploaded an assignment, posted a reaction on the discussion forum, uploaded a video clip, etc... 

The above data can be used in the context of both formative and summative assessment. Important to take into account, however, is the fact that lecturers must always inform their students of which data will be used, and how it will be taken into account in the marking. This informaton can either be given in class, at the start of term, and/or through a message on the digital learning platform. The section "Detailed Clarification of Assessment Methods" on the course sheet should also mention the use of learning analytics.

 

Ufora Course in Learning Analytics

A collaboration between the faculty of Medicine and Health Sciences and the faculty of Economics and Business Administration has resulted in an inspiring course on Learning Analytics. In this course you will find out more about

• what exactly learning analytics is
• the different types of learning analytics
• how to use them to optimise your work.

Get inspired: https://ufora.ugent.be/d2l/le/discovery/view/course/86534 (in Dutch)

Want to Know More?

1st International Conference on Learning Analytics and Knowledge, Banff, Alberta, February 27–March 1, 2011, as cited in George Siemens and Phil Long, "Penetrating the Fog: Analytics in Learning and Education," EDUCAUSE Review, vol. 46, no. 5 (September/October 2011)