Behat revisited

Introduction

In the first two parts (one and two), I covered getting up and running with running acceptance tests sequentially, using a Ramdisk to speed things up, running tests in parallel, specifying a custom theme to test on and writing your own tests. Over a year later we revisit Behat to speed up the process of orchestrating the environment and to understand a bit more about what is going on. To understand this post, I recommend that you read the first two parts beforehand.

Disclaimers

Ubuntu® is a registered trademark of Canonical Ltd – ubuntu.com/legal/intellectual-property-policy.

Moodle™ is a registered trademark of ‘Martin Dougiamas’ – moodle.com/trademarks.

Other names / logos can be trademarks of their respective owners. Please review their website for details.

I am independent from the organisations listed above and am in no way writing for or endorsed by them.

The information presented in this article is written according to my own understanding, there could be technical inaccuracies, so please do undertake your own research.

References

• Running acceptance tests – moodledev.io/general/development/tools/behat/running
• Writing acceptance tests – moodledev.io/general/development/tools/behat/writing
• CIDR – en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
• Docker Use bridge networks – docs.docker.com/network/bridge/
• Docker Networking with standalone containers – docs.docker.com/network/network-tutorial-standalone/
• Hat icon on featured image: www.iconfinder.com/icons/7671701/hat_winter_summer_beach_icon and creativecommons.org/licenses/by/3.0/ – The image of the bee is my own.

Starting position

This is an installed and configured Moodle installation with Apache and MySQL already installed. In the case here, Ubuntu 22.04 where I’ve:

• Configured the OS to use PHP 7.4 instead of 8.
• Installed Docker and pulled the ‘mysql-server’ image in the same way as part one.
• Created the Ramdisk mount point.

Improvements

It was frustrating me that each time I needed to find out the IP address of the ‘Gateway’ of the Docker network, the connection between the ‘host’ and the MySQL container that we use in the creation of the ‘moodle’ database user, such as the default of ‘172.17.0.1’ being:

CREATE USER 'moodle'@'172.17.0.1' IDENTIFIED BY 'dbpass';

What I wanted is something more certain and defined. After reading: docs.docker.com/network/network-tutorial-standalone/, docs.docker.com/network/bridge/ and en.wikipedia.org/wiki/Classless_Inter-Domain_Routing. I asked my Mum for a number between 1 and 255, and came up with the IP address ‘192.168.33.1’, with ‘33’ being the number she chose. Now we can create and label a new Docker bridge network:

docker network create --subnet "192.168.33.0/24" --gateway "192.168.33.1" --driver "bridge" behatbridge

and see what’s been created with:

docker network inspect behatbridge

Now when we create the MySQL user, we’ll have:

CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';

then when we run:

docker run --name=behat-mysql -p 6603:3306 --network behatbridge --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql-server

and with Moodle’s config.php file having:

$CFG->dbhost    = '127.0.0.1';
$CFG->dbname    = 'moodle';
$CFG->dbuser    = 'moodle';
$CFG->dbpass    = 'dbpass';
$CFG->prefix    = 'mdl_';
$CFG->dboptions = array (
  'dbpersist' => 0,
  //'dbport' => '3306', // Local MySQL
  'dbport' => '6603', // Docker MySQL
  'dbsocket' => '',
  'dbcollation' => 'utf8mb4_unicode_ci',
);

Then Moodle is connecting via ‘localhost’ (127.0.0.1) to the Docker ‘host’ on port ‘6603’, which the MySQL container understands to be ‘192.168.33.1’ and its running as ‘192.168.33.2’. Use ‘docker network inspect behatbridge’ to see this.

Another thing I found is that between OS restarts that Moodle would get confused and need it’s data folder clearing as this no longer matched the database as that was being lost, so before the container is stopped then:

sudo -u www-data php admin/tool/behat/cli/util.php –drop

needs to be run to tidy things up. And I’m not completely sure on the next point, but as Docker containers are created from the image and then when stopped have a stored state, then I think that MySQL could be storing state information in file(s) outside of the database located in ‘/var/lib/mysql’ and volume mapped to the ramdisk. And thus the container needs to be removed too, luckily there is the command line option ‘–rm’ that will do this for us when we stop the container instead of stopping then removing with:

docker stop behat-mysql
docker rm behat-mysql

Now with the Docker bridge in place and reading up on Bash scripting:

• medium.com/techtofreedom/4-ways-to-run-mysql-queries-in-bash-or-linux-command-line-110301f5f67c
• stackoverflow.com/questions/43099116/error-the-input-device-is-not-a-tty
• askubuntu.com/questions/939583/best-practices-on-using-sudo-in-a-bash-script
• www.howtogeek.com/410299/how-to-pause-a-bash-script-with-the-linux-sleep-command/
• www.cyberciti.biz/open-source/command-line-hacks/linux-run-command-as-different-user/

I developed a script to speed things up.

The automated Bash script

After a bit of trial and error, I’ve come up with ‘ball.sh’:

#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '
#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '\007' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '\007' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '\007' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '\007' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '\007' && echo 'Termination complete'
7' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '
#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '\007' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '\007' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '\007' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '\007' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '\007' && echo 'Termination complete'
7' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '
#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '\007' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '\007' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '\007' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '\007' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '\007' && echo 'Termination complete'
7' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '
#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '\007' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '\007' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '\007' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '\007' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '\007' && echo 'Termination complete'
7' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '
#!/bin/bash
# Behat 'all' script.
# Copyright 2022 G J Barnard.
# License 'GNU GPL v3 or later', see 'http://www.gnu.org/copyleft/gpl.html'.
echo 'Mount ramdisk'
mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
echo 'Run docker behat-mysql'
docker run --name=behat-mysql -d -p 6603:3306 --network behatbridge --rm --env="MYSQL_ROOT_PASSWORD=dbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql
sleep 15
echo -en '\007' && echo 'Create DB'
docker exec -i behat-mysql mysql -uroot -pdbpass <<BEHATMYSQL
CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'moodle'@'192.168.33.1' IDENTIFIED BY 'dbpass';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'192.168.33.1' WITH GRANT OPTION;
BEHATMYSQL
echo -en '\007' && echo 'Init Behat suite'
cd /var/www/moodle
sudo -u www-data php admin/tool/behat/cli/init.php
echo -en '\007' && echo 'Init complete, you can now run Selenium and tests'
read -p 'Press any key to terminate'
sudo -u www-data php admin/tool/behat/cli/util.php --drop
echo -en '\007' && echo 'Database dropped, stopping docker behat-mysql'
docker stop behat-mysql
echo -en '\007' && echo 'Termination complete'
7' && echo 'Termination complete'

Which you run with ‘sudo ./ball.sh’, which I realise is probably not a good idea but I couldn’t find another way to run php as the ‘www-data’ user.

The script:

• Mount’s the ramdisk.
• Runs the ‘behat-mysql’ docker container with the ‘behatbridge’ network, mapping the ports, detached to free up the console ‘-d’ and will remove the container when stopped ‘–rm’.
• Waits fifteen seconds to allow everything to settle and then beeps and puts out a message to say that it’s creating the database.
• Creates the database and user.
• Changes directory to the ‘/var/www/moodle’ folder (where Moodle is installed as per the previous first post).
• Initialises the Behat suite.
• Then pauses until you press a key.
• Pressing a key, then drops the test suite tables and files and then stops and removes the ‘behat-mysql’ container.

Running

Now were almost all set to run some tests, so start Selenium:

java -jar selenium-server-4.4.0.jar standalone --port 4444

and then for example, we can run the topics format tests:

sudo -u www-data php admin/tool/behat/cli/run.php –tags="@format_topics"

Note: I’m not covering parallel runs or headless here.

A little bit further

At this point I became a bit more curious and wondered how the web browser is being controlled by the driver that Selenium controls. As the Behat test suite reads the test, interprets it, sends commands to Selenium, which interprets and executes them on the browser via the driver to return a result back to Behat to then confirm success of each step in the test.

As Selenium is running on port ‘4444’ on ‘localhost’ then we can use ‘Wireshark’ (www.wireshark.org and www.elearningworld.org/what-is-a-byte-anyway/) to see that we actually have JSON (www.json.org/json-en.html) being used:

with ‘xpath’ (developer.mozilla.org/en-US/docs/Web/XPath), that we can find in the Behat PHP code, such as ‘section’ in the static ‘$moodleselectors’ array in ‘/lib/behat/classes/partial_named_selector.php’:

'section' => <<<XPATH
.//li[contains(concat(' ', normalize-space(@class), ' '), ' section ')][./descendant::*[self::h3]
    [normalize-space(.) = %locator%][contains(concat(' ', normalize-space(@class), ' '), ' sectionname ') or
    contains(concat(' ', normalize-space(@class), ' '), ' section-title ')]] |
.//div[contains(concat(' ', normalize-space(@class), ' '), ' sitetopic ')]
    [./descendant::*[self::h2][normalize-space(.) = %locator%] or %locator% = 'frontpage']
XPATH

which I think is the same ‘xpath’ we can see in the Wireshark packet capture because of ‘sitetopic’.
and so we can then understand a little more about what is actually going on.

Conclusion

I’m really pleased I’ve been able to make an automated script and delve deeper into how Behat works. Revisiting something that’s been bugging you for a while and learning more about it is satisfying.

What do you think? Please let me know in the comments.

• About
• Latest Posts

Gareth Barnard

Developer at HRDNZ

Gareth is a developer of numerous Moodle Themes including Essential (the most popular Moodle Theme ever), Foundation, and other plugins such as course formats, including Collapsed Topics.

Latest posts by Gareth Barnard (see all)

• Moodle 4 Security – A Review – 16th April 2024
• A little bit #fab – 16th March 2024
• Static framework – 16th February 2024