When talking shop with other Moodlers, one common pain point lies with Document Converter plugins. This is my story.
Introduced with Moodle 3.1, Document Converters allow Moodle to convert common file types like Microsoft Word or OpenOffice that are uploaded as Assignment submissions to PDF. This is so the annotation feedback plugin can be used on a wider variety of student submissions. Naturally, many professors enjoy the flexibility and convenience of annotating without requiring students to manage submitting their assignments in the correct file format (talk about herding cats!)
From an administration standpoint, it’s important to note that Moodle does not perform document conversion itself. Rather, the converter plugins allow integration with a separate utility, which can be run on the same server or elsewhere. When a student adds a file to a submission, an entry is created in the Moodle database in the mdl_assignfeedback_editpdf_queue table (I’m sure there’s more to it, but this is a handy way to check how many conversions your Moodle site is running). A scheduled task runs – every 15 minutes by default – to convert any newly-submitted files to PDF so they can be marked up. If a professor happens to visit the grading screen of a submission before it can be processed, they will have to wait while the file is converted ad hoc. This might take anywhere from a few seconds to several minutes depending on the document and the speed of the conversion process.
Worst of all is when an instructor is working with queued submissions of a file type that isn’t compatible with the conversion process, as this often causes lag and long page loads. One of my users reported wait times of up to 2 minutes trying to grade an assignment that student submitted .zip files to before they could save grades and feedback. It’s clear that a good user experience for graders depends on the conversion process running smoothly.
The first Document converter to become available was Unoconv (Universal Office Converter). Unoconv is about as Open Source and Linux-y as it gets: it’s a command line program that runs a special version of LibreOffice to convert files.
When I took over as the Moodle admin at my school, I mostly knew Unoconv by it’s reputation, which is… bad (really bad.) My predecessor had so much trouble keeping unoconv running reliably that one of his last acts before leaving the university was to just disable document conversion altogether. In the wider Moodle-verse, things didn’t seem much better. Much of the sentiment seemed to be that unoconv was difficult and prone to causing headaches. I’ll let Lafayette College’s Charles Fulton (a developer and Moodler of some renown) summarize the vibe:
My users were missing the annotation capability on Word files, but I was not optimistic about using unoconv. Suffice it to say, it was not long before I started looking for alternatives.
Google Drive Converter – Making Progress
A second Document converter for Google Drive was released with Moodle 3.3 in 2017. The Google Drive document converter was added by the Moodle core team as an easier plug and play solution. Check out this YouTube video from Moodle Moot for a great breakdown of the how document conversion works in Moodle and what problems the team was trying to solve.
My main hesitation in using the Google solution was privacy. The way this converter works is to use a Google system account, the same as used to enable oAuth sign-in and Google Drive integration. For document conversion, it sends the file to the system account’s Google Drive, converts to PDF, downloads the result, and then deletes the original upload.
My university is a lapsed Google school (now on O365), so I wasn’t sure I would be able to get a proper, institution owned Google account set-up, and I didn’t want to use a consumer Google account as it would not be subject to the same FERPA-compliant terms – who knows what type of data mining would be done on student work? But, after working with our IT staff, I was able to get a service account under our Google Apps for Edu domain.
The set-up was easy – Moodle provides good documentation on how to set-up things on the Google side, and then in Moodle it’s plug and play with your oAuth account. I was quite pleased with the Google solution. It was an easy win for a new Moodle admin.
Things continued smoothly for well over a year – aside from some blurriness on the converted documents I had no complaints with the Google converter. Then….COVID-19 and online instruction happened. I’m not sure what the root cause was: the increase in web traffic to our Moodle servers, a sizable uptick in assignments being submitted, Google throttling on their end, or some combination of the three, but all of a sudden our conversions could not keep pace. The queue of pending conversions kept growing. Moreover, the scheduled task for conversions started to fail repeatedly, causing Moodle to add a “Fail delay” which has to be manually cleared, compounding the problem and making lots of extra work for yours truly. Meanwhile, professors already stressed by the current state of affairs began to experience significant wait times or performance degradation while trying to use Moodle for grading.
I tried what I could to remedy the situation: I shortened the interval between scheduled task runs from 15 minutes to 5, deleted the backlog in the queue hoping the process could stay caught up after the initial rush, and even set Annotation Feedback to be off by default on new assignments site wide (guess what: even if an assignment has PDF Annotate turned OFF, Moodle still adds its submissions to the conversion queue). Honestly this was the last thing I needed during all online instruction in a pandemic so within a week I decided to take another look at alternatives.
When researching alternatives, I found two new file converter plugins of interest in the Moodle plugins database. Onedrive and LibreLamba. I quickly ruled out OneDrive as it seems to only be compatible with consumer grade Microsoft accounts, which again raises privacy issues. LibreLamba, however, seemed to offer the best of both worlds. It uses a slick cloud-based solution but allows the end-user to maintain ownership and privacy of student data. The plugin works by allowing us to set-up and use our own conversion process in AWS with serverless Lamba functions and object storage via S3 buckets.
This sounds a bit scarier than it is. All that it requires to set-up are some basic command line skills and a working knowledge of AWS administration – but is far less complicated to manage than unoconv. The plugin is from CatalystIT, who have done lots of great work in the Moodle community and they really did well making this plugin Moodle admin friendly.
With high hopes, I followed the guide on the LibreLambda GitHub page. Creating a user in AWS, installing the Moodle plugins, and running the script to get things set-up was straightforward and painless – only a little more involved than setting up the Google version. After letting the process run for a few days, I’m impressed. It’s fast, efficient, and critically seems to be keeping up with the surge in online assignments at my school. So far looks to be superior to the Google Drive converter. Thank you, Catalyst! I sincerely hope they are able to keep this plugin updated for new versions of Moodle!