Moodle PDF Conversion Plugins: Good, Bad & Ugly

When talking shop with other Moodlers, one common pain point lies with Document Converter plugins. This is my story.

Introduced with Moodle 3.1, Document Converters allow Moodle to convert common file types like Microsoft Word or OpenOffice that are uploaded as Assignment submissions to PDF. This is so the annotation feedback plugin can be used on a wider variety of student submissions. Naturally, many professors enjoy the flexibility and convenience of annotating without requiring students to manage submitting their assignments in the correct file format (talk about herding cats!)

From an administration standpoint, it’s important to note that Moodle does not perform document conversion itself. Rather, the converter plugins allow integration with a separate utility, which can be run on the same server or elsewhere. When a student adds a file to a submission, an entry is created in the Moodle database in the mdl_assignfeedback_editpdf_queue table (I’m sure there’s more to it, but this is a handy way to check how many conversions your Moodle site is running). A scheduled task runs – every 15 minutes by default – to convert any newly-submitted files to PDF so they can be marked up. If a professor happens to visit the grading screen of a submission before it can be processed, they will have to wait while the file is converted ad hoc. This might take anywhere from a few seconds to several minutes depending on the document and the speed of the conversion process.

Worst of all is when an instructor is working with queued submissions of a file type that isn’t compatible with the conversion process, as this often causes lag and long page loads. One of my users reported wait times of up to 2 minutes trying to grade an assignment that student submitted .zip files to before they could save grades and feedback. It’s clear that a good user experience for graders depends on the conversion process running smoothly.

Unoconv

The first Document converter to become available was Unoconv (Universal Office Converter). Unoconv is about as Open Source and Linux-y as it gets: it’s a command line program that runs a special version of LibreOffice to convert files. 

When I took over as the Moodle admin at my school, I mostly knew Unoconv by it’s reputation, which is… bad (really bad.) My predecessor had so much trouble keeping unoconv running reliably that one of his last acts before leaving the university was to just disable document conversion altogether. In the wider Moodle-verse, things didn’t seem much better. Much of the sentiment seemed to be that unoconv was difficult and prone to causing headaches.  I’ll let Lafayette College’s Charles Fulton (a developer and Moodler of some renown) summarize the vibe:

My users were missing the annotation capability on Word files, but I was not optimistic about using unoconv. Suffice it to say, it was not long before I started looking for alternatives.

Google Drive Converter – Making Progress

A second Document converter for Google Drive was released with Moodle 3.3 in 2017. The Google Drive document converter was added by the Moodle core team as an easier plug and play solution. Check out this YouTube video from Moodle Moot for a great breakdown of the how document conversion works in Moodle and what problems the team was trying to solve.

My main hesitation in using the Google solution was privacy. The way this converter works is to use a Google system account, the same as used to enable oAuth sign-in and Google Drive integration. For document conversion, it sends the file to the system account’s Google Drive, converts to PDF, downloads the result, and then deletes the original upload. 

My university is a lapsed Google school (now on O365), so I wasn’t sure I would be able to get a proper, institution owned Google account set-up, and I didn’t want to use a consumer Google account as it would not be subject to the same FERPA-compliant terms – who knows what type of data mining would be done on student work? But, after working with our IT staff, I was able to get a service account under our Google Apps for Edu domain.

The set-up was easy – Moodle provides good documentation on how to set-up things on the Google side, and then in Moodle it’s plug and play with your oAuth account. I was quite pleased with the Google solution. It was an easy win for a new Moodle admin.

The #pivot

Things continued smoothly for well over a year – aside from some blurriness on the converted documents I had no complaints with the Google converter. Then….COVID-19 and online instruction happened. I’m not sure what the root cause was: the increase in web traffic to our Moodle servers, a sizable uptick in assignments being submitted, Google throttling on their end, or some combination of the three, but all of a sudden our conversions could not keep pace. The queue of pending conversions kept growing. Moreover, the scheduled task for conversions started to fail repeatedly, causing Moodle to add a “Fail delay” which has to be manually cleared,  compounding the problem and making lots of extra work for yours truly. Meanwhile, professors already stressed by the current state of affairs began to experience significant wait times or performance degradation while trying to use Moodle for grading.

I tried what I could to remedy the situation: I shortened the interval between scheduled task runs from 15 minutes to 5, deleted the backlog in the queue hoping the process could stay caught up after the initial rush, and even set Annotation Feedback to be off by default on new assignments site wide (guess what: even if an assignment has PDF Annotate turned OFF, Moodle still adds its submissions to the conversion queue). Honestly this was the last thing I needed during all online instruction in a pandemic so within a week I decided to take another look at alternatives.

Enter LibreLambda

When researching alternatives, I found two new file converter plugins of interest in the Moodle plugins database. Onedrive and LibreLamba. I quickly ruled out OneDrive as it seems to only be compatible with consumer grade Microsoft accounts, which again raises privacy issues. LibreLamba, however, seemed to offer the best of both worlds. It uses a slick cloud-based solution but allows the end-user to maintain ownership and privacy of student data. The plugin works by allowing us to set-up and use our own conversion process in AWS with serverless Lamba functions and object storage via S3 buckets.

This sounds a bit scarier than it is. All that it requires to set-up are some basic command line skills and a working knowledge of AWS administration – but is far less complicated to manage than unoconv. The plugin is from CatalystIT, who have done lots of great work in the Moodle community and they really did well making this plugin Moodle admin friendly.

Checking out the PDF Conversion queue one day after installing LibreLambda….24 hours before this it was at over 1250.

With high hopes, I followed the guide on the LibreLambda GitHub page. Creating a user in AWS, installing the Moodle plugins, and running the script to get things set-up was straightforward and painless – only a little more involved than setting up the Google version. After letting the process run for a few days, I’m impressed. It’s fast, efficient, and critically seems to be keeping up with the surge in online assignments at my school. So far looks to be superior to the Google Drive converter. Thank you, Catalyst! I sincerely hope they are able to keep this plugin updated for new versions of Moodle!

Photo by Fleur on Unsplash

Ben Kahn
Connect

Latest posts by Ben Kahn (see all)
blank

Ben Kahn

Ben works as a systems administrator, systems architect, instructional designer, and technologist at the University of Portland, managing Open Source tools for learning, including enterprise Moodle & WordPress installs.

6 thoughts on “Moodle PDF Conversion Plugins: Good, Bad & Ugly

  • 11th May 2020 at 7:34 pm
    Permalink

    Hi Ben, I’m very comfortable with mysql. I finally made a couple of queries to identify waste records in database. Also find the moosh command to purge them. BUT, I’m not 100% sure about the Moodle database datamodel, so I’ll keep all this “in the fridge” to test it once course is over. I don’t want to mess it up with COVID-19 all around and more than 400 students depending on Moodle 😉

    Reply
  • 3rd May 2020 at 8:34 am
    Permalink

    Thanks Ben. I’ve disabled the Scheduled Task “Prepare Submissions for Annotation.”. I’m now trying to find the way to delete all unnecessary converted documents to get rid of 5 GB of space.

    Reply
    • 11th May 2020 at 4:24 pm
      Permalink

      Hi Oscar, I’m not sure how you would be able to do this….there might be a way if you are clever enough with MySQL queries. If you can find a list of the file IDs you want to delete, you can use Moosh to purge them: https://moosh-online.com/commands/

  • 2nd May 2020 at 12:16 am
    Permalink

    Interesing post.
    As you said “even if an assignment has PDF Annotate turned OFF, Moodle still adds its submissions to the conversion queue”. I got into trouble with this. Annotate PDF is consuming huge amount of this space in my small Moodle Server.
    Did you find a way to delete all unuseful pdf anotations?

    Regards

    Reply
    • blank
      2nd May 2020 at 6:50 am
      Permalink

      Hi Oscar, I didn’t find a way to do this. If you want to keep unneeded files from being converted into PDFs you can disable the Scheduled Task “Prepare Submissions for Annotation.” However, that would mean that any graders who are using PDF annotation would have to wait for the files to convert when they visit the assignment grading screen. and this can be undesirable. You may just want to look at disabling the Annotate PDF Feedback plugin altogether if your server can’t handle the space.

  • 20th April 2020 at 9:24 am
    Permalink

    Fascinating post Ben !
    We find with smaller clients that Unoconv works well, but I think with organisations the size of a University, and the huge number of submissions, it doesn’t sound like it scales to that size well.
    Like you, I can’t ethically recommend or use Google for this – total privacy should always be our default starting point.
    I haven’t used LibreLambda yet.
    But now I know it’s from our friends at Catalyst (the “other” Moodle Partner in New Zealand) I will have a look – they are a great company, and we have lots of friends there 🙂

    Reply

Add a reply or comment...