Data and E-Learning Part 6: Limitations of Data

We have seen in this series of articles how we can use data to answer questions we have. We have seen the power of combining datasets to answer more complicated questions and the different data types we have at our disposal. Data is not all-powerful though; in this article we are going to take a different tack and provide a reality check on some of the limitations of data.

Limitations

Data is a tool that we can use to improve student outcomes, however, it does not do this on its own. It is one tool that is available to us, but it is a tool that must be used correctly if it is to have an impact. We will look at what data, on its own, cannot reasonably be expected to do.

Improve student outcomes (in isolation)

Data can be an effective tool for improving student outcomes if you use the data to improve those outcomes. Data by itself will not improve them; you need to identify what the data is telling you and then take action for the improvements to occur. This could be by implementing strategies to assist students that you have identified are struggling or are in high-risk groups. You could make changes to areas of the programme that the data has shown to be weak. Perhaps you will provide additional support at specific points in the course. The data can highlight what action you need to take to improve student outcomes and you must then take that action for it to have an effect.

Guarantee the future

Data can be used to make forecasts about the future. What it cannot do is guarantee that future. The data may tell you that the average completion rate for a course is 80%. This may reasonably allow you to expect a similar completion rate for the coming year, but you cannot be sure that that will be the case. Historical data provides an indication of possible future performance but does not guarantee it. Data should inform your decision-making process, but you should not assume blindly what happened in the past will happen again in the future.

Make up for poor course design

Data can help identify issues and provide pointers to better ways of doing things. Data by itself cannot make up for a poorly designed course. The greatest data analysis system in the world won’t make up for a course that is buggy, contains the wrong information, or does not sufficiently engage learners to allow them to achieve their learning objectives. You should definitely collect data to assess the quality of your offerings, just don’t expect that the data alone is enough to make up for the shortcomings in course design that it uncovers.

Implications

The usefulness of data is limited by its underlying quality and the quality of the analysis performed on it. Bad data in will equal bad data out. Taking good data and drawing bad conclusions from it will be similarly ineffective. To make data work for you it is essential that you analyse the data to correctly understand it and then take appropriate action based on this analysis to make the desired improvements. Data is most effective when it is analysed correctly and then put into action.

To overcome the limitations of data you should do the following:

• Be clear on what questions you want to answer with your data
• Collect the data that is required to answer these questions
• Analyse the data in an appropriate fashion
• Draw sound conclusions from the data
• Identify what you will do based on these conclusions
• Take action!
• Collect further data to determine the impact of your actions on achieving your overall student objectives.

Data is an effective tool for getting the most out of your Learning Management System provided you understand the limitations we have discussed in this article. We will turn our attention in the next article to getting data out of a Moodle installation.

You can view the previous posts in this series here:

• Part 1: The Gold Mine
• Part 2: Data Types
• Part 3: Understanding Your Data
• Part 4: Developing Questions
• Part 5: Combining Datasets

• About
• Latest Posts

Jeff Mitchell

P&C Reporting and Data Analyst at Fisher & Paykel Appliances

Jeff is passionate about the role of learning and development, and has a specific interest in how people and organisations can be developed in order to achieve their potential. Jeff has a keen interest in information technology and specifically data analysis and the e-learning space.

Latest posts by Jeff Mitchell (see all)

• Version Control – Git GUI – 7th March 2023
• Version Control – Git Command Line Tool – 7th February 2023
• Version Control – Creating Repositories – 7th January 2023