ElearningWorld.org

For the online learning world

MoodleTechnical

Behat — “Given” for the win!

Behat — “Given” for the win!

I’ve been using Behat for about six years, primarily with Moodle but also from the ground up with an Angular / Laravel project. Over this time I’ve made some mistakes and learned a fair bit about how to optimize my tests.

The biggest optimizations I’ve made have come from understanding the appropriate usage of “Given”, “When” and “Then”. The official documentation does a good job of explaining this here: https://docs.behat.org/en/v2.5/guides/1.gherkin.html#steps

When I first started writing tests my number one objective was to simply write a test that mimicked the user interaction with the system. Although this sounds like common sense, almost obvious, it leads to inefficient tests. Why? Because you end up testing the same thing over and over again.

Let me give you a Moodle example that I see quite a lot:

@javascript
Scenario: Test condition
# Basic setup.
Given I log in as “teacher1
And I am on “Course 1” course homepage with editing mode on

OK, so what is wrong with the above? Well for a start “I log in as” is a user action, i.e. the user has to do something. If it’s a user action then it’s a “When” and not a “Given”. In this case, it requires the user to visit the site login page, enter their credentials, and then hit the login button. And all of those user actions TAKE TIME.

There is only one instance where the user action to log in should be tested and that is in a test specifically targeting that feature — i.e. the ability to log in.

If I’m writing a test to make sure assignments can be graded, there is absolutely no reason for me to also test logging in via a form. That bit of functionality should already have been tested.

OK so how should the test look instead?

@javascript
Scenario: Test condition
# Basic setup.
Given I am logged in as “teacher1
And I am on “Course 1” course homepage with editing mode on

It looks almost the same but I hope you spotted the change. Instead of using “I log in as” (which is an action) we are now using the pre-requisite “I am logged in as” which is truly a given. How so? Well because it’s past tense — we aren’t asking the user to do anything. What we are saying is that after this step, this should be the state of play.

If we actively monitor the Behat feature running in Chrome (i.e. not in headless mode), we wouldn’t see the user log in. What we would see instead is the user simply arrive at the site home page logged in.

Note to Moodle devs — At the time of writing “I am logged in as” is not a step that you can use in Moodle out of the box (although I will be filing a ticket to add it).

Let’s take a look at how the “I am logged in as” step might work at the PHP end:

/**
* This is much better than using “I log in as step” since it cuts out the form
* filling steps.
*
*
@Given I am logged in as :username
*
@param string $username
*/
public function fast_login(string $username, ?array $urlparams = []): void {
$urlparams = array_merge($urlparams, [‘username’ => $username]);
$url = new moodle_url(‘/lib/tests/behat/fastlogin.php’, $urlparams);
$this->execute(‘behat_general::i_visit’, [$url]);
}

You can see that the new fast_login step requires the user to visit a URL (lib/tests/behat/fastlogin.php), so isn’t this a “when” and not a “given”? Well not really because we aren’t asking the user to interact. Let’s take a look at what the lib/tests/behat/fastlogin.php endpoint would do:

<?php
// Standard Moodle comment would go here.

/**
* Fast login end point for BEHAT TESTS ONLY.
*
*
@package theme_cfz
*
@author Guy Thomas
*
@copyright 2021 Class Technologies Inc. {@link https://www.class.com/}
*
@license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/
require(__DIR__.’/../../../config.php’);

$behatrunning = defined(‘BEHAT_SITE_RUNNING’) && BEHAT_SITE_RUNNING;
if (!$behatrunning) {
die;
}

$username = required_param(‘username’, PARAM_ALPHANUMEXT);
// Note – with behat, the password is always the same as the username.
$password = $username;

$failurereason = null;
$user = authenticate_user_login($username, $password, true, $failurereason, false);
if ($failurereason) {
error_log(“Failed to login as behat step for $username with reason: ” . $failurereason);
throw new Exception($failurereason);
}
if (!complete_user_login($user)) {
throw new Exception(“Failed to login as behat step for $username”);
}

$redirecturl = optional_param(‘redirecturl’, null, PARAM_URL);
$redirecturl = $redirecturl ?? $CFG->wwwroot;

if (optional_param(‘forceeditmode’, false, PARAM_INT)) {
$sesskey = sesskey();
$url = new moodle_url($redirecturl);
$url->param(‘edit’, 1);
$url->param(‘sesskey’, $sesskey);
$redirecturl = $url.”;
}

redirect($redirecturl);

You might not have a PHP interpreter built into your brain, so I’ll explain what’s happening in this endpoint —

First, we make sure that this endpoint can only ever be utilized by Behat:

$behatrunning = defined(‘BEHAT_SITE_RUNNING’) && BEHAT_SITE_RUNNING;
if (!$behatrunning) {
die;
}

Then we log in as the user using the required username (the password is always the same as the username in a Behat test):

$username = required_param(‘username’, PARAM_ALPHANUMEXT);
// Note – with behat, the password is always the same as the username.
$password = $username;

$failurereason = null;
$user = authenticate_user_login($username, $password, true, $failurereason, false);
if ($failurereason) {
error_log(“Failed to login as behat step for $username with reason: ” . $failurereason);
throw new Exception($failurereason);
}
if (!complete_user_login($user)) {
throw new Exception(“Failed to login as behat step for $username”);
}

Finally, if we have a successful login, we can then redirect the user straight to the page they need to be on:

$redirecturl = optional_param(‘redirecturl’, null, PARAM_URL);
$redirecturl = $redirecturl ?? $CFG->wwwroot;

if (optional_param(‘forceeditmode’, false, PARAM_INT)) {
$sesskey = sesskey();
$url = new moodle_url($redirecturl);
$url->param(‘edit’, 1);
$url->param(‘sesskey’, $sesskey);
$redirecturl = $url.”;
}

redirect($redirecturl);

We’ve effectively removed ALL of the user interaction. AND because our login endpoint supports a “redirecturl” parameter, we can condense steps in our tests to make them even faster. For example, we can write a custom PHP step that logs us in AND puts us on a specific course page in edit mode.

This has no detrimental effects on the test because we don’t need to test logging in AND navigating to a course AND turning edit mode on. All of that stuff should be handled in a feature made specifically to test that stuff.

Where is the proof?

I modified the following feature to use “I am logged in as”:

mod/assign/tests/behat/grading_status.feature

I ran the test 6 times pre-modification and 6 times post-modification. The average speed pre-modification was 5 minutes 32 seconds. The average speed post-modification was 5 minutes 26 seconds. This is a speedup of 6 seconds. You might be thinking that such a small improvement isn’t worth it. Don’t forget “I log in as” is peppered across the moodle code base.

Conclusion

Make your Behat tests more efficient by using “Given” steps appropriately. In short:

Stop testing the same things over and over again for no reason.

Source: https://brudinie.medium.com/feed

blank
Follow me

blank

ElearningWorld Admin

Site administrator

Add a reply or comment...