Behat – part one

16th May 2021 Gareth Barnard

Introduction

Running, let alone writing the Moodle acceptance tests (docs.moodle.org/dev/Running_acceptance_test) is a bit tricky and complicated, and they can take ages. But what if there was a way to speed things up? In this first of a two part post, we’ll look at getting up and running with running tests one at a time. In part two, we’ll look at running tests in parallel and cover any questions raised in the comments of this part.

Disclaimers

Firefox® is a registered trademark of the Mozilla Foundation – www.mozilla.org/en-US/foundation/trademarks/policy/.
Ubuntu® is a registered trademark of Canonical Ltd – ubuntu.com/legal/intellectual-property-policy.
Moodle™ is a registered trademark of ‘Martin Dougiamas’ – moodle.com/trademarks.

Other names / logos can be trademarks of their respective owners. Please review their website for details.

I am independent from the organisations listed above and am in no way writing for or endorsed by them.

The information presented in this article is written according to my own understanding, there could be technical inaccuracies, so please do undertake your own research.

References

The thought

When I started to run the acceptance tests, I noticed that the hard disk was doing a lot of activity, the screen was flashing away as the ‘browser’ was being controlled automatically. So as the database does not need to be kept then why not optimise it to be faster by using a Ramdisk (en.wikipedia.org/wiki/RAM_drive) instead? And why use a GUI installation? So run the browser ‘headless’ (en.wikipedia.org/wiki/Headless_software) too.

Starting position

The technical detail in this post begins at the point where you have a working 64bit Linux (in my case Ubuntu) server running an installed LAMP and Moodle installation (Moodle 3.10). This is covered in places such as on the MoodleBites Server Administrator course (www.moodlebites.com/mod/page/view.php?id=3212) and docs.moodle.org/310/en/Step-by-step_Installation_Guide_for_Ubuntu.

My install is running in ‘/var/www/moodle’ and ‘/var/moodledata’ on Ubuntu 20.04 server with 4GB of ram, no GUI as a virtual machine, allocated two of the CPU’s cores with no execution cap. For the purpose of screenshots I’m running a second Ubuntu virtual machine with a GUI. Both of them on VirtualBox (www.virtualbox.org) with the network adapter set as ‘Bridged’ (to the WiFi adaptor) so that the IP addresses are allocated by my router, and thus it knows and can send data between the real and virtual machines.

The actual machine I’m using has an Intel core i5 9th gen, i5-9500 CPU @ 3.00GHz with 16GB of ram.

So from this point onwards ‘http://REALIPOFMACHINE/moodle’ works with having gone through the Moodle install process and created the ‘config.php’ file successfully. The database is called ‘moodle’, with an username of ‘moodle’ and a database password of ‘yourdbpass’.

Getting Behat up and initialised

This is my interpretation of the Moodle instructions (docs.moodle.org/dev/Running_acceptance_test):

In your home folder create a ‘behat’ folder and change directory into it.

Get and install Chrome:

wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo apt install ./google-chrome-stable_current_amd64.deb

Get the Linux Chrome driver from sites.google.com/a/chromium.org/chromedriver then download, extract the zip and copy ‘chromedriver’ via SSH to server in the ‘behat’ folder, then put chromedriver (with sudo) into /usr/local/bin:

sudo cp chromedriver /usr/local/bin/

Then make sure go has +rx and u +rwx – i.e. owned by root with rwxr-xr-x permissions:

sudo chmod 0755 /usr/local/bin/chromedriver

Get Selenium server with Java:

sudo apt install default-jre

Go to ‘www.seleniumhq.org/download’ and get the stand-alone server, i.e. find the link and in the ‘behat’ folder:

wget https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar

As we are running headless browser, we need ‘xvfb’ (en.wikipedia.org/wiki/Xvfb):

sudo apt install xvfb

Now we are ready to configure Moodle to use what we’ve installed, so:

sudo mkdir /var/behatdata
sudo chown -R www-data:www-data /var/behatdata

Add in Moodle’s config.php file, just before $CFG->directorypermissions:

sudo -u www-data nano /var/www/moodle/config.php

$CFG->behat_wwwroot = 'http://localhost/moodle';
$CFG->behat_prefix = 'bht_';
$CFG->behat_dataroot = '/var/behatdata';

$CFG->behat_profiles = [
    'default' => [
        'browser' => 'chrome',
        'extensions' => [
            'Behat\MinkExtension' => [
                'selenium2' => [
                    'browser' => 'chrome',
                    'wd_host' => 'http://localhost:4444/wd/hub'
                ]
            ]
        ]
    ]
];

This tells Moodle about the Behat data folder, the ‘different’ localhost wwwroot and what url Selenium will be running under (it drives the Chrome browser).

Now we can initialise the tests, so in ‘/var/www/moodle’:

sudo -u www-data php admin/tool/behat/cli/init.php

Once completed then we can login to Moodle and navigate to ‘Site Administration > Development > Acceptance Testing’, where we should see the step definitions (this helps with writing tests):

If they are not listed then you’ll get an informational error message instead and it means that the initialisation has failed.

Running tests

Now that everything is in place, we can run some tests, however the whole lot takes ages, so we’ll only run the topics course format tests ‘/course/format/topics/tests/behat’. For this we need two terminals, one for Selenium and the other for running the tests themselves:

In the ‘behat’ folder:

xvfb-run java -jar selenium-server-standalone-3.141.59.jar -port 4444

In /var/www/moodle:

sudo -u www-data vendor/bin/behat --config /var/behatdata/behatrun/behat/behat.yml --format=pretty --out=report_behat.txt --tags format_topics

When when ‘behat’ finishes you can see the time it took with:

tail report_behat.txt

Ramdisk configuration

I found it a bit tricky to change where the installed MySQL server was looking for its database, so instead after reading some articles, decided to use ‘Docker’ (www.docker.com) to run a MySQL instance as that allowed me to easily specify it on the command line. So first we need to install and configure Docker:

sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Find release of Ubuntu name from ‘/etc/apt/sources.list’ and use that with:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo apt update
apt-cache policy docker-ce
sudo apt install docker-ce
sudo systemctl status docker

To save typing ‘sudo’ every time we type ‘docker’:

sudo usermod -aG docker ${USER}

Then we need to relogin to take effect, then we can get MySQL:

docker pull mysql/mysql-server

and see it in the list of images:

docker images

Now create ramdisk:

sudo mkdir /mnt/ramdisk
sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk

and see it in the list of devices:

df -h

listed at the bottom

In a new terminal, run Docker MySQL with it:

docker run --name=test-mysql -p 6603:3306 --env="MYSQL_ROOT_PASSWORD=yourdbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql-server

and once up and running we will need to use another terminal and run:

docker inspect test-mysql

to get the ‘Gateway IP’ (in this case ‘172.17.0.1’) for ‘moodle’ user, then at each startup of the machine we will (after mounting the ramdisk), create the database for Moodle to use:

docker exec -it test-mysql mysql -uroot -pyourdbpass

CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'172.17.0.1' IDENTIFIED BY 'yourdbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'172.17.0.1' WITH GRANT OPTION;
exit

We now need to tell Moodle to use that database instead:

sudo -u www-data nano /var/www/moodle/config.php

and comment out / add:

//$CFG->dbhost = 'localhost';
$CFG->dbhost = '172.17.0.1'; // Docker Gateway IP from 'docker inspect test-mysql'.

$CFG->dboptions = array (
//'dbport' => '',
'dbport' => '6603', // Docker MySQL

Then check Moodle would install and connect to the database:

http://REALIPOFMACHINE/moodle

Running with Ramdisk

As before, we need to initialise the tests, so in ‘/var/www/moodle’:

sudo -u www-data php admin/tool/behat/cli/init.php

then run them:

If Selenium is not already running, in the ‘behat’ folder:

xvfb-run java -jar selenium-server-standalone-3.141.59.jar -port 4444

In /var/www/moodle:

sudo -u www-data vendor/bin/behat --config /var/behatdata/behatrun/behat/behat.yml --format=pretty --out=report_behat.txt --tags format_topics

Again when ‘behat’ finishes you can see the run with:

tail report_behat.txt

And hopefully you should be able to see a speed increase:

With before on the left and after on the right.

When you’ve finished, you can shutdown the machine, optionally stop and remove the Docker MySQL image beforehand:

docker stop test-mysql
docker rm test-mysql

Running again

After starting up the server, you’ll need to recreate the Ramdisk (the mount point already exists):

sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
docker start test-mysql

or if removed (with ‘docker rm test-mysql’):

docker run --name=test-mysql -p 6603:3306 --env="MYSQL_ROOT_PASSWORD=yourdbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql-server

Then create the moodle user and DB:

docker exec -it test-mysql mysql -uroot -pyourdbpass

CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'172.17.0.1' IDENTIFIED BY 'yourdbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'172.17.0.1' WITH GRANT OPTION;
exit;

Then you’ll be able to initialise and run tests as before. Please refer to the Moodle documentation for this.

Troubleshooting

When switching between the local and Docker sites, then you might get in a situation where one of the sites is not accessible, in that case, just purge the caches on the command line:

sudo php /var/www/moodle/admin/cli/purge_caches.php

Conclusion

My configuration is not the only one you can use, its just the one I’ve worked out for me. It might be possible to get this to work without Selenium (docs.moodle.org/dev/Running_acceptance_test#Run_tests_directly_in_Chrome.2C_with_no_Selenium), but I’ve not tried that yet.

I’m certainly not a Linux / Docker / Behat expert, but this seems to work for me!

In part two, we’ll look at running tests in parallel and cover any questions raised in the comments of this part.

What do you think? Please let me know in the comments.

About
Latest Posts

Gareth Barnard

Developer at HRDNZ

Gareth is a developer of numerous Moodle Themes including Essential (the most popular Moodle Theme ever), Foundation, and other plugins such as course formats, including Collapsed Topics.

Latest posts by Gareth Barnard (see all)

Static publishing – 16th April 2025
Network fonts – 16th March 2025
Static translate – 16th February 2025

3 thoughts on “Behat – part one”

Pingback: Behat revisited - ElearningWorld.org
Pingback: Behat - part two - ElearningWorld.org
Stuart R Mealor

16th May 2021 at 10:30 am

Wow – fascinating !
Thanks so much for the screenshots – that really helped me get a better idea of what you are discussing.
Behat has always been a very confusing area for me, but I feel I’ve got a better understanding now at least 🙂