Introduction
Background
We first need to define our terms.
Hugo already builds sites with one quick and easy command.
The automated website building this post refers to is several additional steps required for website publishing after that process in one particular workflow. Every person’s or team’s workflow for publishing a website is different. The workflow we’ll automate in this post includes the following:
- rebuilding a specific website with Hugo
- deleting all files and subdirectories in the website publishing directory except one specified file
- copying all files from the Hugo website project directory to the publishing directory
We’ll handle each step in turn and then bring them all together.
Prerequisite skills
Since we’ll be building Bash scripts, [a basic knowledge of Bash]((https://loopedline.com/post/bash-getting-started-working-with-the-shell/) and of automating Bash scripts is assumed. The emphasis here is on basic knowledge - no advanced knowledge is required and you can get up to speed quickly with the two posts linked above.
Development Setup: Creating a test directory
Before we start developing our script, you may wish to first create a new directory in which to test these commands.
This will make it easier to see the effects of the changes we’re making and will reduce the risk of impacting files we don’t want to alter or delete.
Since we will be deleting files and folders recursively this is particularly important.
Development Setup: Creating test files and subdirectories
After creating a test directory we’ll want to create test files and subdirectories.
We’ll need test files and directories for both the website publishing directory and the Hugo website project directory. We can just copy the real directories into our test directory.
Once our main test directory, subdirectories, and files are setup we’re ready to proceed.
Rebuilding a website with Hugo
As already mentioned, Hugo offers one command to quickly and simply rebuild a site.
The site building command as well of the basics of building your first Hugo website have been discussed in previous posts, so we we’ll assume that knowledge.
What we’ll develop here is a script to automatically build a specific website in a specific directory. This will allow us to run a script from any working directory and build that site in another directory. This in turn allows us to include the site building in a larger workflow without having to navigate between different working directories.
#!/bin/bash
# VARIABLES
directoryofhugoproject=/home/user/TEMP/test-directory/hugo-project-directory
# MAIN program flow
echo
echo "Building Hugo site at $directoryofhugoproject"
cd $directoryofhugoproject
hugo
echo
Let’s break down the elements of that command.
#!/bin/bash - This must be the first line of the shell script, and as Linuxize summarizes it, “This sequence of characters (#!) is called shebang and is used to tell the operating system which interpreter to use to parse the rest of the file.”
directoryofhugoproject=/home/user/TEMP/test-directory/hugo-project-directory - Here we assign the path to our test Hugo website directory to a variable named “directoryofhugoproject”.
echo - There are several echo commands with no arguments throughout the script. These print a new line, which helps to format our output and make it more legible.
echo "Building Hugo site at $directoryofhugoproject" - Print the literal string “Building Hugo site at " and the contents of the variable $directoryofhugoproject, which we assigned earlier, in order to provide some additional information to the user.
cd $directoryofhugoproject - Change the working directory to the path we specified in the variable $directoryofhugoproject.
hugo - The Hugo command to rebuild the website in the working directory.
That is all we need.
Once you update the variable directoryofhugoproject to the path of your Hugo website project you can use this script to rebuild your site from any working directory.
Deleting all files and folders in the website publishing directory except one specified file
Since Hugo rebuilds all pages as necessary to incorporate any changes, we want to replace all the old website files with the new files that Hugo just built.
We don’t merely want to copy and overwrite these files, however, because that would leave in place any files that were deleted by Hugo during the rebuild.
One additional consideration is that for this workflow we have one website file, other.html, that is built by another process and not by Hugo. Consequently, we want to leave it intact and not delete it when we delete the other files.
We’ll build to that functionality in several steps.
Deletion Step 1: Deleting all files except one
This answer from StackExchange provides an excellent place to start.
WARNING: This script will delete (almost) all files in a directory, so use it only in a directory that contains files you want to delete. Now is a good time to reconfirm that you are in the test-directory you created. You may also wish to create a backup copy of those files to make it easier to repeat testing.
Navigate to the website publishing directory and run the following script.
#!/bin/bash
# VARIABLES
filetonotdelete=other.html
# MAIN program flow
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose
Let’s break down the new elements of that command.
filetonotdelete=other.html - Here we assign the name of the file that we want to protect from deletion to a variable named filetonotdelete.
ls --hide=$filetonotdelete - List the contents of the working directory except the file listed in the variable $filetonotdelete.
| - Pipe the results of the command on the left as input to the command on the right.
xargs --delimiter='\n' - The xargs command “converts input from standard input into arguments to a command” according to Wikipedia. We use the --delimiter option to tell it to separate arguments based on the newline character, which is represented by \n. This takes the output piped to it from the previous ls command and passes that output as arguments to the following rm command.
rm --verbose - Remove (that is, delete) the files listed by the previous ls command. The --verbose option causes the command to output the names of the files it is deleting.
In summary, the command lists all files in the working directory (except the file named in the filetonotdelete variable), passing that list to the xargs command, which, in turn, passes that list as arguments to the rm, which, finally deletes those files.
Deletion Step 2: Deleting all files and subdirectories except one
The above script is a good start, but it doesn’t yet delete subdirectories or their contents. We need to make one small modification to accomplish that.
WARNING: This script will delete (almost) all files in a directory as well as subdirectories and all their contents, so use it only in a directory that contains files and subdirectories you want to delete. You need to be especially cautious when using the rm command with the –recursive option.
#!/bin/bash
# VARIABLES
filetonotdelete=other.html
# MAIN program flow
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
Let’s look at the new element of that command.
--recursive - This option tells the rm command to also delete subdirectories and all of their contents.
In summary, the script deletes all files (except one specified file), all subdirectories, and the contents of all subdirectory of the working directory.
Deletion Step 3: Deleting the contents of a specific directory
Now we will modify this script to work for a specified directory so that it can be called from anywhere. This not only simplifies it’s use, but also offers some protection from accidentally calling it in the wrong directory and deleting the wrong files.
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# MAIN program flow
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
Let’s break down the new elements of that script.
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-site-directory - Assign the path to our test publishing website directory to a variable named directoryofwebsiteforpublishing.
cd $directoryofwebsiteforpublishing - Change the working directory to the path we specified in the variable $directoryofwebsiteforpublishing.
In summary, the script deletes all files (except one specified file), all subdirectories, and the contents of all subdirectories within a specified directory.
Deletion Step 4: Wrapping program logic in a function
Now we will wrap the main program flow in a function to make it easier to incorporate into the larger workflow.
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# FUNCTION deletefilesanddirectories - deletes all files and directories in a specified directory except for a specified file
deletefilesanddirectories() {
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
}
# MAIN program flow
deletefilesanddirectories
Let’s break down the new elements of that script.
deletefilesanddirectories() { . . . } - This wraps the logic that was previously in the “MAIN program flow” section in a function called deletefilesanddirectories.
deletefilesanddirectories - This calls the function called deletefilesanddirectories and executes its commands.
In summary, this script accomplishes the same things as the previous script, but wraps the logic in a function, which will make it easier to refer to that logic by calling the function as we add additional elements to the script in the following steps.
Deletion Step 5: Informing the user
Now that we have the deletion functionality working, we want to ensure that the user knows what files will be deleted. So, we’ll incorporate some output to notify the user.
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# FUNCTION deletefilesanddirectories - deletes all files and directories in a specified directory except for a specified file
deletefilesanddirectories() {
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
}
# FUNCTION indent - indent a line before outputting
indent() { sed 's/^/ /'; }
# FUNCTION warningwithprompt - prints a warning that the script will delete files and subdirectories and prompts the user before continuing
warningwithprompt() {
cd $directoryofwebsiteforpublishing
echo
echo "This script will delete the following files, subdirectories, and ALL subdirectory contents:"
echo
ls --hide=$filetonotdelete -1 --color | indent
echo
}
# MAIN program flow
warningwithprompt
#deletefilesanddirectories
Let’s examine the new elements of that script.
indent() { sed 's/^/ /'; } - A function that indents a line with two spaces before outputting the result. The logic here is from this StackOverflow answer.
warningwithprompt() { . . . } - A function to provide output about the files that are about to be deleted. (We will add a user prompt later, which is why the function is named warningwithprompt.)
ls --hide=$filetonotdelete -1 --color | indent -
Similar to the logic in the earlier command that deletes files, this displays the names of the files and folders that would be deleted. The -1 option calls for outputting one file per line. The --color option preserves the color output of the ls command when it is piped, which makes it easier to interpret. The indent command calls the indent function.
warningwithprompt - Calls the warningwithprompt function.
#deletefilesanddirectories - The deletefilesanddirectories call is commented out so that we can focus on the changes to output without actually deleting files while testing.
In summary, this script outputs a warning to the user about the files and folders that will be deleted.
Deletion Step 6: Prompting the user for permission to continue
Now that we can output a warning to the user, we’ll want to prompt the user to ask if they wish to continue.
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# FUNCTION deletefilesanddirectories - deletes all files and directories in a specified directory except for a specified file
deletefilesanddirectories() {
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
}
# FUNCTION indent - indent a line before outputting
indent() { sed 's/^/ /'; }
# FUNCTION warningwithprompt - prints a warning that the script will delete files and subdirectories and prompts the user before continuing
warningwithprompt() {
cd $directoryofwebsiteforpublishing
echo
echo "This script will delete the following files, subdirectories, and ALL subdirectory contents:"
echo
ls --hide=$filetonotdelete -1 --color | indent
echo
read -n1 -p "Do you wish to proceed? [y/n] " response
# Convert response to lowercase
response=${response,,}
echo
echo
}
# MAIN program flow
warningwithprompt
warningwithprompt
case $response in
y|ye|yes)
echo Executing script.
echo
#deletefilesanddirectories
echo
echo Script completed.
;;
*)
echo Action cancelled.
;;
esac
Let’s examine the new elements of that script.
Note that several components of the these new elements are based on this and this StackOverflow answer.
read -n1 -p "Do you wish to proceed? [y/n] " response - Read user input from the command line and store it in the response variable. The -p option prints the literal string that follows it, “Do you wish to proceed? [y/n] “. The -n1 option tells the read command to only accept 1 character before returning the result.
response=${response,,} - This is a Bash parameter expansion that modifies the case of the alphabetic characters in the parameter response. Since this uses the ,, expansion, all matching alphabetic character in response will be converted to lowercase. And since this omits the pattern variable, every charcter will match. Therefore, all alphabetic character in response will be converted to lowercase.
case $response in . . . esac - This is the case syntax for Bash and tests the response variable against the following cases sequentially.
y|ye|yes) . . . ;; - This is the first option in the case command that the response variable is compared against. It is checking if response is equal to “y” or “ye” or “yes”. If it matches any of those three it executes the commands contained within the ) and the ;;. Since the clause is terminated with a ;;, if there is a match, no further matches will be searched for. (NOTE: The “ye' and “yes” options will not be used in the current script because the -n1 option in the read command means only one character will be read. These are kept in for flexibility if the -n1 option is removed in a later script.)
*) . . . ;; - This is the second option in the case command that the response variable is compared against. Because of the wildcard * this will always evaluate as true.
In summary, this script outputs a warning to the user about the files and folders that will be deleted, prompts the user for a response, awaits the response, and then outputs one of two messages to the user depending on the response.
Deletion Step 7: Prompting the user for permission to continue
Now all we need to do to complete this script is to uncomment the call to the deletefilesanddirectories function.
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# FUNCTION deletefilesanddirectories - deletes all files and directories in a specified directory except for a specified file
deletefilesanddirectories() {
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
}
# FUNCTION indent - indent a line before outputting
indent() { sed 's/^/ /'; }
# FUNCTION warningwithprompt - prints a warning that the script will delete files and subdirectories and prompts the user before continuing
warningwithprompt() {
cd $directoryofwebsiteforpublishing
echo
echo "This script will delete the following files, subdirectories, and ALL subdirectory contents:"
echo
ls --hide=$filetonotdelete -1 --color | indent
echo
read -n1 -p "Do you wish to proceed? [y/n] " response
# Convert response to lowercase
response=${response,,}
echo
echo
}
# MAIN program flow
warningwithprompt
case $response in
y|ye|yes)
echo Executing script.
echo
deletefilesanddirectories
echo
echo Script completed.
;;
*)
echo Action cancelled.
;;
esac
In summary, this script outputs a warning to the user about the files and folders that will be deleted, prompts the user for a response, awaits the response, and then outputs one of two messages to the user depending on the response. If the user enters a “y” the script deletes all files (except one specified file), all subdirectories, and the contents of all subdirectory of a specified directory. However, if the user enters any character except “y” the script terminates.
By updating the variables directoryofwebsiteforpublishing and filetonotdelete to the relevant values for our production files, we have a script that will delete all the files of the specified directory except the one file we want to preserve.
Copying all files from the Hugo website directory to the publishing directory
Since we can delete all files (except one protected file) in our website publishing directory, we now want to repopulate the folder with updated files from the Hugo website directory. We’ll look at a script to automatically copy those files from one folder to the other. The basic logic of this script is from this StackOverflow answer.
#!/bin/bash
# VARIABLES
directoryofhugoprojectfiles=/home/user/TEMP/test-directory/hugo-project-directory/public
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
# FUNCTION copyfilesanddirectories - copy all files and directories in a specified directory to another specified directory
copyfilesanddirectories() {
cp --recursive --verbose $directoryofhugoprojectfiles/. $directoryofwebsiteforpublishing/
}
# MAIN program flow
copyfilesanddirectories
Let’s examine the new elements of that script.
directoryofhugoprojectfiles=/home/user/TEMP/test-directory/hugo-project-directory/public - Note the important distinction between the directoryofhugoproject and directoryofhugoprojectfiles variables.
The Hugo project is in the directory:
- /home/user/TEMP/test-directory/hugo-project-directory
Which is stored in the variable:
- directoryofhugoproject
While the website files the project produces are in the directory:
- /home/user/TEMP/test-directory/hugo-project-directory/public
Which is stored in the variable:
- directoryofhugoprojectfiles
cp --recursive --verbose $directoryofhugoprojectfiles/. $directoryofwebsiteforpublishing/ - This calls the cp command, which copies the contents of the first directory into the second directory. The --recursive option causes the command to copy all subdirectories and their contents. The --verbose option causes the command to output the name of each file being copied. As the StackOverflow answer explains, the dot at the end of the first directory path instructs the cp command “to copy the contents of the current directory, not the directory itself. This method also includes hidden files and folders.”
In summary, this script copies all of the contents of first directory into the second directory and outputs the name of each file as it does so.
Putting it all together
We have all three components of functionality necessary to accomplish our objective. Now we’ll bring them together. Each script could be run individually in sequence, but to make the process simpler and to further reduce the possibility of introducing human error, we’ll create one script to call them each in turn. Alternatively, we could combine the logic of all three scripts into one file, but that would make maintenance more difficult and prevent us from running each script separately if the need ever arose.
Also, to enhance flexibility and to reduce the risk of unexpected behavior if the scripts are modified in the future, we’ll alter each script to remove the hardcoded directory and file references and instead assign those values by passing parameters. We’ll then hardcode those values in the calling script so we don’t have to re-enter them every time the main script is run.
sitebuild_01_rebuildsite.sh
#!/bin/bash
# VARIABLES
directoryofhugoproject=$1
# MAIN program flow
echo
echo "Executing $BASH_SOURCE"
echo
echo "Building Hugo site at $directoryofhugoproject"
cd $directoryofhugoproject
hugo
echo
sitebuild_02_deletefiles.sh
#!/bin/bash
# VARIABLES
directoryofwebsiteforpublishing=$1
filetonotdelete=$2
# FUNCTION deletefilesanddirectories - deletes all files and directories in a specified directory except for a specified file
deletefilesanddirectories() {
cd $directoryofwebsiteforpublishing
ls --hide=$filetonotdelete | xargs --delimiter='\n' rm --verbose --recursive
}
# FUNCTION indent - indent a line before outputting
indent() { sed 's/^/ /'; }
# FUNCTION warningwithprompt - prints a warning that the script will delete files and subdirectories and prompts the user before continuing
warningwithprompt() {
cd $directoryofwebsiteforpublishing
echo "This script will DELETE the contents of $directoryofwebsiteforpublishing"
echo
echo "This script will DELETE the following files, subdirectories, and ALL subdirectory contents:"
echo
ls --hide=$filetonotdelete -1 --color | indent
echo
read -n1 -p "Do you wish to proceed? [y/n] " response
# Convert response to lowercase
response=${response,,}
echo
echo
}
# MAIN program flow
echo
echo "Executing $BASH_SOURCE"
echo
warningwithprompt
case $response in
y|ye|yes)
echo Executing script.
echo
deletefilesanddirectories
echo
echo Script completed.
echo
;;
*)
echo Deletion cancelled.
echo
exit
;;
esac
sitebuild_03_copyfiles.sh
#!/bin/bash
# VARIABLES
directoryofhugoprojectfiles=$1
directoryofwebsiteforpublishing=$2
# FUNCTION copyfilesanddirectories - copy all files and directories in a specified directory to another specified directory
copyfilesanddirectories() {
echo "Copying files."
echo
echo "FROM:"
echo "$directoryofhugoprojectfiles"
echo
echo "TO:"
echo "$directoryofwebsiteforpublishing"
echo
cp --recursive $directoryofhugoprojectfiles/. $directoryofwebsiteforpublishing/
}
# MAIN program flow
echo
echo "Executing $BASH_SOURCE"
echo
copyfilesanddirectories
sitebuild_00_publishsite.sh
#!/bin/bash
# VARIABLES
directoryofhugoproject=/home/user/TEMP/test-directory/hugo-project-directory
directoryofhugoprojectfiles=/home/user/TEMP/test-directory/hugo-project-directory/public
directoryofwebsiteforpublishing=/home/user/TEMP/test-directory/publishing-directory
filetonotdelete=other.html
# FUNCTION sb_rebuildsite - run the sitebuild_01_rebuildsite.sh script with additional output
sb_rebuildsite() {
echo
echo
echo ---------- Step 1: Rebuild site - START ----------
source sitebuild_01_rebuildsite.sh $directoryofhugoproject
echo ---------- Step 1: Rebuild site - END ----------
echo
echo
}
# FUNCTION sb_deletefiles - run the sitebuild_02_deletefiles.sh script with additional output
sb_deletfiles() {
echo
echo
echo ---------- Step 2: Delete files - START ----------
source sitebuild_02_deletefiles.sh $directoryofwebsiteforpublishing $filetonotdelete
echo ---------- Step 2: Delete files - END ----------
echo
echo
}
# FUNCTION sb_copyfiles - run the sitebuild_03_copyfiles.sh script with additional output
sb_copyfiles() {
echo
echo
echo ---------- Step 3: Copy files - START ----------
source sitebuild_03_copyfiles.sh $directoryofhugoprojectfiles $directoryofwebsiteforpublishing
echo ---------- Step 3: Copy files - END ----------
echo
echo
}
# MAIN program flow
echo
echo "Executing $BASH_SOURCE"
echo
sb_rebuildsite && sb_deletfiles && sb_copyfiles
Let’s examine the new elements of these scripts.
echo "Executing $BASH_SOURCE" - Output the path and filename of the script preceded by the literal string “Executing “.
exit - Exit the script and the parent script (if any). If the user chooses not to delete existing files, the script won’t proceed to copying files in the next step.
source sitebuild_01_rebuildsite.sh $directoryofhugoproject - The source command runs the script sitebuild_01_rebuildsite.sh, passing the parameter $directoryofhugoproject to it.
sb_rebuildsite && sb_deletfiles && sb_copyfiles - The && operator runs the next command in sequence only if the previous command returns TRUE. This means that the following commands won’t run until the previous command completes successfully.
There were also several changes and additions to output meant to aid with debugging and making functionality clear to the user.
Conclusion
We now have the option of running each script individually to execute each portion of functionality, or running a single script to complete each in turn.