Static publishing
Last time in the series on Hugo, we looked at translation, in this post we’ll continue with our site and look at how we can easily update our site.
Disclaimers
Names / logos can be trademarks of their respective owners. Please review their website for details. I am independent from the organisations mentioned and am in no way writing for or endorsed by them. The information presented in this article is written according to my own understanding, there could be inaccuracies, so please do undertake your own research. The featured image is my copyright, please don’t use without my permission.
Methodology
In the first post in this series, ‘Static’ (www.elearningworld.org/static) we generated our site with the hugo command ‘hugo –minify –cleanDestinationDir’ to generate our website to the ‘public’ folder. This we then deployed to a Raspberry Pi running the ‘Apache’ web server. Though I was not specific on how you got the files to the Pi, as I used ‘WinSCP’ (winscp.net/eng/index.php) to copy them to a temporary folder then copy using the correct ‘www-data’ user.
A whole site copy is okay for when we’re first generating the site, but what happens when we want to update it? Do we really want to remove all of the files and replace them, effectively taking down the site for a moment (though your procedures could have a ‘Performing maintenance’ page that you employ) or is there a better way? What if we could only alter what has changed? Surely that would be better and more efficient? If indeed the answer is ‘Yes’, then what we need is a way to ‘mirror’ what we generate (and check locally, ‘Running locally’ in the ‘Static’ post) with what the web server serves.
Example setup
To take things further and be more realistic the web site now uses ‘https’ as the protocol, as you would have with a hosting provider. However as the site is ‘static’ then its need might not be strictly required, will all depend on your content and the user’s interaction.
For this example though, you can setup Apache on the Pi to serve the site with ‘https’. To recap, the deployed site was in ‘/var/www’ folder which is owned by the ‘www-data’ user (a no-login account) and then this was stated in ‘/etc/apache2/sites-available/000-default.conf’ (though I didn’t specifically specify that in the post). The files were then copied with ‘sudo -u www-data cp -R /home/pi/transfer/elw/*’ to ensure that the user and group ownership is correct. Since then I’ve rebuilt my Pi and the folder is now ‘/var/www/hugo’, and this is what will be used here.
Firstly we need to do the following shell commands:
sudo a2enmod ssl sudo a2ensite default-ssl sudo mkdir /etc/apache2/ssl
Which will enable ‘SSL’ in Apache and create a folder for us to have our SSL Certificate and Key. My Pi is called ‘matilda’ and I’ve configured that name in the ‘hosts’ file to reference the IP of the Pi. Next we generate the SSL Certificate and Key with:
sudo openssl req -config /etc/ssl/openssl.cnf -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/matilda.key -out /etc/apache2/ssl/matilda.crt
Filling in the requested details, but critically ‘Common name’ needs to be ‘matilda’. Next edit:
sudo nano /etc/apache2/sites-enabled/default-ssl.conf
And add:
SSLCertificateFile /etc/apache2/ssl/matilda.crt SSLCertificateKeyFile /etc/apache2/ssl/matilda.key
Below the existing entries of the same name, then comment those out with a hash:
#SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem #SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
Then reload Apache with:
sudo service apache2 reload
Which we can then test with:
openssl s_client -connect matilda:443
Then on our machine running Hugo, in our ‘hugo.toml’ file, change ‘baseURL’ to:
baseURL = 'https://matilda/'
Rebuild with:
hugo --minify --cleanDestinationDir
And copy the files to ‘/var/www/hugo/’ with the ‘www-data’ user as before. If you didn’t have a method for doing this, then read on and the same transfer process will work here.
All being well, then we’ll have the starting point of:
With the site data from the last post about translation: github.com/gjb2048/hugo-elw/tree/Feb_2025.
Publishing tools
On Linux, we can use ‘Rsync’ (rsync.samba.org) which allows us to connect to a remote machine and transfer our files. In this example I’m using SSH without a password – www.raspberrypi.com/documentation/computers/remote-access.html#enable-the-ssh-server and www.raspberrypi.com/documentation/computers/remote-access.html#configure-ssh-without-a-password.
Ensure that ‘both’ your source and destination machines have ‘rsync’ installed for it to work. You may need to contact your hosting provider to check this.
Using the following references:
- download.samba.org/pub/rsync/rsync.1
- www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories
- superuser.com/questions/671011/how-to-arbitrarily-map-user-group-ownerships-in-rsync
- unix.stackexchange.com/questions/240814/rsync-with-different-user/546296#546296
I’ve deduced the command:
rsync -avzP . pi@matilda:/var/www/hugo/ --delete-during --rsync-path='sudo -u www-data rsync'
That we can use, where I’m updating the site from github.com/gjb2048/hugo-elw/tree/Feb_2025 to github.com/gjb2048/hugo-elw/tree/Apr_2025 which adds a new ‘Publishing’ page:
The output of the command gives:
Where we can see that I’ve rebuilt the site, then only the changes have been sent over given the number of bytes sent (“sent 12,140 bytes” at the bottom). This we can tell if I remove all of the files on the Pi and repeat the same command:
Where now ‘1,616,024’ bytes have been sent, and hence this is another way of deploying our site in the first instance. To breakdown what:
rsync -avzP . pi@matilda:/var/www/hugo/ --delete-during --rsync-path='sudo -u www-data rsync'
does using download.samba.org/pub/rsync/rsync.1 as a reference. The ‘-avzP’ options breaks down into:
‘-a’ being ‘archive’ which is the options of “rlptgoD” being:
‘-r’ being ‘recursive’ thus ‘recurse into directories’.
‘-l’ being ‘links’ thus ‘copy symlinks as symlinks’.
‘-p’ being ‘perms’ thus ‘preserve permissions’.
‘-t’ being ‘times’ thus ‘preserve modification times’.
‘-g’ being ‘group’ thus ‘preserve group’.
‘-o’ being ‘owner’ thus ‘preserve owner’.
‘-D’ being ‘devices’ thus ‘preserve device files’.
then:
‘-v’ being ‘verbose’ thus ‘increase verbosity’.
‘-z’ being ‘compress’ thus ‘compress file data during the transfer’.
‘-P’ being both ‘partial’, ‘keep partially transferred files’ and ‘progress’, ‘show progress during transfer’. Which now raises the question if partially transferred files are actually a good thing? Looking at unix.stackexchange.com/questions/48298/can-rsync-resume-after-being-interrupted then actually perhaps not as we won’t want a broken file nor having temporary files around so we can improve our command to have:
--progress –append-verify
being:
rsync -avz --progress --append-verify . pi@matilda:/var/www/hugo/ --delete-during --rsync-path='sudo -u www-data rsync'
an example in a moment! For the rest of the command:
‘. pi@matilda:/var/www/hugo/’ specifies the source and destination (with login credentials).
‘–delete-during’ being ‘receiver deletes during the transfer’
‘–rsync-path=’sudo -u www-data rsync’’ being ‘specify the rsync to run on remote machine’ which ensures that the files and folders have the ‘www-data’ username and group for Apache to serve them with as we’re using the ‘pi’ account to login as we can’t use ‘www-data’ being a no login account. This assumes that ‘pi’ has ‘sudo’ capability, a topic for another time perhaps.
However… testing the updated command gives:
Which is clearly problematic when updating, so investigating further on other sites:
- www.cyberciti.biz/faq/rsync-resume-partially-transferred-downloaded-files-option
- medium.com/neural-engineer/efficient-file-transfers-with-rsync-how-to-resume-interrupted-transfers-f9bb818376d0
- ostechnix.com/how-to-resume-partially-downloaded-or-transferred-files-using-rsync
Therefore using ‘–progress’ is actually a good thing as long as we look at the output of rsync and make follow up decisions as appropriate. Those sites additionally state what ‘–append’ and ‘–append-verify’ do, in that they are used with ‘–partial’. This just goes to demonstrate the importance of testing and understanding for yourself what a command does. Therefore, the command actually is:
rsync -avzP . pi@matilda:/var/www/hugo/ --delete-during --rsync-path='sudo -u www-data rsync'
But what about Windows? What are options there? Two tools that I already use that have file transfer mirror capability are WinSCP and SyncBackFree:
Where protocols like SFTP (en.wikipedia.org/wiki/SSH_File_Transfer_Protocol) can be used, and others too. But what if we wanted to use ‘rsync’ given what we know now? There is of course the Windows Subsystem for Linux (learn.microsoft.com/en-us/windows/wsl/install) but I’m already using VirtualBox (www.virtualbox.org), so wouldn’t want a conflict and additionally don’t want it running on my SSD with swap being on the same drive and causing excessive ware, I have a HDD for that.
The ‘rsync’ download page (rsync.samba.org/download.html) mentions ‘Cygwin’ (www.cygwin.com), so I decided to give that a go. After installing adding ‘rsync’ (plus ssh too):
But I found that it was failing, even after changing the versions to match:
Searching on the web I found that ‘Git for Windows’ (gitforwindows.org) could be conflicting, so specifying the correct ‘ssh’ with the ‘-e’ option gives:
However the connection was still asking for the password even though at this point ‘ssh-add’ was working, so I tried in a new terminal:
At this point I gave up the fight and thought about ‘Git for Windows’ itself as if uses ‘Ming-w64’ (www.mingw-w64.org), which contains ‘MSYS2’ (www.msys2.org), a ‘a modified version of Cygwin’. I use ‘Git for Windows’ as I develop my Moodle plugins on Windows and employ TortoiseGit (tortoisegit.org) as a GUI to save time. Looking at unix.stackexchange.com/questions/595247/gitbash-mingw64-bash-rsync-command-not-found and stackoverflow.com/questions/75752274/rsync-for-windows-that-runs-with-git-for-windows-mingw-tools. I found that after downloading ‘rsync-3.2.7-2-x86_64.pkg.tar.zst’ from repo.msys2.org/msys/x86_64, then extracting (with 7Zip – www.7-zip.org) the files in their folder structure to where Git for Windows is installed, in my case ‘D:\Program Files\Git\’ that I needed ‘libxxhash-0.8.3-1-x86_64.pkg.tar.zst’ too given the error message ‘rsync’ gave (‘msys-xxhash-0.dll’ was missing), that at last it worked!:
And to prove that ssh is being used:
The private key file ‘MATILDA’ was generated as an exported OpenSSH key from the Putty generator (www.chiark.greenend.org.uk/~sgtatham/putty/latest.html) which also gives you the public key for ‘pasting into OpenSSH authorized_keys file’. This could be the subject of another post. In the mean time www.raspberrypi.com/documentation/computers/remote-access.html#ssh gives instructions for generating and using a key pair with and on the Pi – the same principles apply to the keys generated by Putty. There is also www.ssh.com/academy/ssh/public-key-authentication. You may get a message about ‘Known hosts’, this is to do with the identity of the server as that has a private / public key pair that is used to confirm that it is actually the server you think it is. Please see www.ssh.com/academy/ssh/command#using-the-linux-client, www.ssh.com/academy/ssh/host-key and www.ssh.com/academy/ssh/sshd_config for more information.
Upcoming
In the next post about Hugo, I intend to look at render hooks.
Conclusion
I hope that you’ve been able to understand how to incrementally publish your Hugo site.
You can get all of the code for this post from: github.com/gjb2048/hugo-elw/tree/Apr_2025 – please do have a go and tell me what you think. What other tools could be used?
- Static publishing – 16th April 2025
- Network fonts – 16th March 2025
- Static translate – 16th February 2025
