Build automation for Godot

2024-06-03 by Gabriel C. Lins

Build automation for Godot

When starting a new project, we know that one of the most important things is getting it out for people to test as soon as possible. This is especially important with games: you have to make sure you're not pursuing something that's uninteresting, and it's also useful to elicit ideas from people from early on to avoid major redesigns.

Publishing the game on Itch.io is very simple: you zip your web page, Wasm app and any scripts that go along with it, and it gets deployed as an iframe in your game page. For downloadable games like Windows apps, the process is similar: upload a zip with your game build and it'll be published. It can get very tedious though, since building from your game engine is always a few clicks, then zipping it correctly and browsing your file system to upload it can be very repetitive and prone to small mistakes.

This is the main driver behind me making a very simple build script for my latest Godot project. Soon I thought of other benefits, such as automatically downloading the correct editor version in any machine where I want to work, or sharing with my developer friends and allowing them to do the same. Here's how I did it:

Writing the script

Step 0: Universal tips

Here are some things I would recommend regardless of whether you're following this tutorial step by step or just gathering ideas for later.

Choosing a scripting tool

I think picking a language you're familiar with or already have installed is not really that important. Even though it's one of the first considerations when making a maintainable tool, this is ultimately a very small script, so it's a good opportunity to learn to do it in a language you're not as familiar with.

Ruby is my third-most familiar programming language after JavaScript and Python. I've worked with it for a few years and I use it almost daily at my current job. However, it's been a long time since I used it for a personal project, let alone small-scale scripting instead of Rails apps, so I decided to go with it.

Suggestions

If you're a novice coder, or not dead set on any particular scripting tool or language, here are the ones I would recommend.

  • Python. Quite obviously the best scripting language here – it's mature, has massive amounts of documentation and Stack Overflow answers for anything, runs on any system, has the best stdlib of any scripting language, and you most likely already have it installed.
  • JavaScript + Deno. Node.js is the de facto standard used everywhere, but it's a little annoying to make CLIs with it in my opinion. Deno offers much more succint ways to use the command line, supports TypeScript from the get-go, and overall makes you write much less code.
  • Make + Shell scripts. Probably the fastest to write if you already know how to, since you can just type in your commands and get everything done. Personally I'd have to do a lot of research to figure out how to do some of the things in this tutorial, so I chose a different but similar option.
  • Ruby + Rake. This is what we'll use in this tutorial. I already laid out my reasons above, but if you want a reason why I consider it a good option overall, that's because Rake allows you to easily create and compose command line tasks without worrying too much about refactoring. This is pretty much true about all the options here, so it really comes down to preference.

At the end of this article I'll be providing the files so you can use these scripts yourself, so if you just want a quickstart, feel free to scroll down!

Break it down in steps

Whatever language you end up choosing, remember that scripts like these break a lot, especially if you're using it all the time in different environments, sharing with other people etc. Because the script will be broken down into tiny tasks, we can easily refactor each of them as we need to, add in-between steps etc.

Make it easy to replicate

If you're writing your own script, I heavily recommend doing it using Docker. I didn't do it because I was only developing games on Windows, but as soon as I wanted to make a change while on the road with a Mac I had to set everything up manually.

I'll port this to Docker myself eventually and then write an update to this article. With Docker I can build in any platform using a Linux script, so that's a better time investment than having platform-specific code. If you want to do that, I've laid the groundwork here, but this is a Windows-only solution for now.

Step 1: Setting up Rake

Since we'll use Ruby, we'll set up Rake to run our command-line tasks. First we make sure Ruby 3 is installed in our system. You can download it from ruby-lang.org or get it using your favourite package manager. I use rbenv-win to manage my versions but if you're only using Ruby for this project that would be overkill.

rbenv is not the most recommended tool for Ruby version management nowadays and I mostly use it out of habit. If you're serious about your version management and environment isolation shenanigans, you'll find better modern tools like asdf, but I haven't got around to using that yet.

After you've made sure Ruby is installed and accessible in your system, you can initialise your environment by going to your project directory and running:

bundle init
bundle add rake

This will create a Gemfile and a Gemfile.lock. These are the package manager files that will allow other developers (or you on other machines) to use this Rake script. Each time you clone your repository you can then run bundle install to get Rake and any other packages you need.

Step 2: Downloading Godot from GitHub

Looking at the URL the Godot official website sends us to when we click on Download, we can see that all Godot versions are hosted on GitHub. Looking at https://api.github.com/repos/godotengine/godot-builds/releases, we can see a list of all releases available in JSON format (large file warning!).

Our first Rake task will be rake download, which downloads our favoured Godot version from there and places it in a nice little folder for later access.

We'll use httparty to make HTTP requests simpler.

bundle add httparty

Now, create a Rakefile and open it with your favoured text editor. Make it look like this:

# Rakefile

task :download do
  puts 'Hello world!'
end

Now, run it from the command line to make sure Rake works:

$ rake download
Hello, world!

Great. Now we can make the download task actually do what we want it to do. From the GitHub API above, we can see that each version has a tag_name and a list of assets containing the download URLs for multiple platforms. The tag_name is the version we'll develop in. In the case of my preexisting project, that's 4.2.1-stable. Let's add a few packages and set some constants at the top of our Rakefile that we'll use later in multiple scripts:

require 'fileutils'
require 'httparty'
require 'json'

RELEASES_URL = 'https://api.github.com/repos/godotengine/godot-builds/releases'
GODOT_VERSION = '4.2.1-stable'
EDITOR_PATH = '.editor/'

Now we can update our download task to get that JSON file and find the URL of our platform's release. In my case, that's win64.

task :download do
  # Fetch the releases and find our own version by "tag_name"
  github_data = JSON.parse HTTParty.get(RELEASES_URL).body
  version = github_data.find { |i| i['tag_name'] == GODOT_VERSION }

  # Fail if the version was not found
  exit 1 unless version

  # This is the filename for win64 as seen by exploring the JSON manually
  # You can also find it in the Releases page where you can download these from your browser
  asset_name = "Godot_v#{GODOT_VERSION}_win64.exe.zip"

  # Find the asset with the correct name within the version we located
  asset = version['assets'].find { |i| i['name'] == asset_name }

  # Get the asset's download URL
  dl_url = asset['browser_download_url']
  puts "Fetching #{dl_url}"
end

At this point, we can run the task again and see that dl_url was successfully assigned.

$ rake download
Fetching https://github.com/godotengine/godot-builds/releases/download/4.2.1-stable/Godot_v4.2.1-stable_win64.exe.zip

The next thing to do is put FileUtils to use by creating our editor directory and placing the downloaded file there.

  # ...
  dl_url = asset['browser_download_url']
  puts "Fetching #{dl_url}"

  # Remove the directory if it exists, then recreate
  # This makes sure it's clean for future downloads
  FileUtils.rm_r EDITOR_PATH
  FileUtils.mkdir_p EDITOR_PATH

  # We then open a file called "godot.zip" within that folder
  open(EDITOR_PATH + 'godot.zip', 'wb') do |f|
    # Set the file to binary mode
    f.binmode

    print 'Starting download...'

    # Keep track of the total bytes
    # This is just to have some responsiveness in the CLI
    total = 0

    # These HTTParty options make sure the download works
    # stream_body also allows us to write chunks to disk and update the CLI
    HTTParty.get(dl_url, stream_body: true, follow_redirects: true) do |chunk|
      total += chunk.size

      # Use print instead of puts so it doesn't print a newline
      print "\r#{total} bytes downloaded"

      # Write the chunk to file
      f.write(chunk)
    end
  end

This is our first step done! Running this should place Godot in .edit/godot.zip. You can open it with your preferred archive manager and make sure all files are in there. Fortunately Godot doesn't need installing so we can just leave it there and it won't conflict with any other EXEs we have previously downloaded.

Step 2: Extracting the files

Well, we did it! We downloaded Godot. However, it's all zipped up. We'll have to extract that using rubyzip. In your command line, run:

bundle add rubyzip

Now that we have installed rubyzip, we can require it at the top of our Rakefile and create a new task called extract:

require 'rubyzip'

# ... after the download task

task :extract do
  puts "Extract works!"
end

Again, we can run rake extract to make sure the command was set up correctly.

Now, we can use rubyzip to extract the archive, then delete it using FileUtils:

task :extract do
  # Open the file we created before...
  Zip::File.open("#{EDITOR_PATH}/godot.zip") do |zip|
    # zip is an iterable list of files inside the archive
    zip.each do |file|
      puts file.name

      # Extract each file to the corresponding path outside the zip
      zip.extract(file, "#{EDITOR_PATH}/#{file.name}")
    end
  end

  # Remove the zip to save disk space
  FileUtils.rm "#{EDITOR_PATH}/godot.zip"
end

This task should now succesfully extract Godot into .editor. You can run rake extract to make sure it works, then open the EXE and see that the whole process succeeded.

We can now combine both tasks into an install task by appending the following to our Rakefile:

task install: %w[download extract]

This tells Rake to run download and extract in order whenever we run rake install.

Step 3: Testing

The first thing we'd like to do is make sure we know how to run Godot from the command line. They have very nice documentation about this, so we can just use one of their nice little commands to run our project in dev mode.

.\.editor\Godot_v4.2.1-stable_win64.exe --path .

If everything was done correctly (and you already had a Godot project in this path in the first place), this command boots up your game. However, it's very long-winded, and we have to remember the arguments each time. This is why we have Rake.

# after other constants
GODOT_PATH = ".editor/Godot_v#{GODOT_VERSION}_win64_console.exe"

# ...

task :test do
  `#{GODOT_PATH} --path .`
end

If you're not familiar with Ruby idioms, backtick strings are commands to be run sequentially in a shell. Therefore we can just copy what we just typed into our terminals, but we replace the path to Godot with a constant because we'll need it again later.

Running rake test should have the same exact result as the command we just ran.

We can also add a similar command to spin up our project using the Godot executable we just downloaded, which is useful whenever you clone your project in a new machine.

task :dev do
  `#{GODOT_PATH} --editor --path .`
end

Step 4: Automating the build

Now is where it gets interesting. This is why I did all of this. In my normal workflow, these are the steps I usually follow to publish a new version of my project:

  1. I open the Godot editor if I wasn't just working on the project. I then select the project I want to export from the list of all my projects.
  2. I click on Project>Export..., then, if I have previously exported to the platforms I want, click Export All...
  3. For each platform I exported to, I have to:
    1. Go into its folder, remove the previous export ZIP
    2. Select all files, create a ZIP
    3. Move it to the directory with all releases so I can upload it on Itch.io faster
  4. Go to Itch.io, edit my game, and upload all ZIPs.

We're going to cut steps #1 through #3. Since we already have a Godot copy in a predictable folder and can run commands using it, the next steps are pretty straightforward.

First, we create a PRESETS constant in our Rakefile with each preset name as found in export_presets.cfg in your project root. In my case, it looks like this:

PRESETS = ['Web', 'Windows Desktop']

The first thing we'll do in this task is remove old build artifats. Generally they're overwritten, but some old files can stay behind if there's no replacement to them, so it's always a good practice:

task :clean do
  puts "Removing old build artifacts..."

  # Make sure the folder exists
  FileUtils.mkdir_p ".build"
  FileUtils.rm_r ".build"
end

Then, we create an export task that recreates them and zips each export with its own name.

Note: This step assumes you've already set up export configuration in Godot and set the export path in each project to .build/{preset_name}/{file_name}. You can update these manually in export_presets.cfg if they already exist, otherwise create the presets in the editor and make sure then names and paths match.

task :export do
  # For each preset in our constant...
  PRESETS.each do |preset|
    # Recreate the export directory in .build
    # IMPORTANT: This needs to match the path in export_presets.cfg
    FileUtils.mkdir_p ".build/#{preset}"

    puts "Exporting project for #{preset}..."

    # Run the headless export command
    `#{GODOT_PATH} --headless --export-debug "#{preset}"`

    puts "Creating a ZIP archive..."

    # Create the zip in .;build`
    Zip::File.open(".build/#{preset}.zip", create: true) do |zip|
      # For each file in the export directory, add it to the zip
      Dir[".build/#{preset}/*"].each do |filename|
        zip.add(filename.split('/')[-1], filename)
      end
    end
  end
end

We can then combine these two tasks into build:

task build: %i[clean export]

And done! Whenever you're ready to release your game, you can now just run rake build from your project root and send your files to Itch.io! What's more, if you want to set up your project in a different computer, you can just run rake install and that'll also be done for you automagically.

Lastly, don't forget to ignore the build artifacts and Godot executable if you're using Git:

# at the end of .gitignore

*.tmp
.build/
.editor/

Going Beyond: CI/CD

I haven't set the project up for continuous deployment, so I still have to manually pick up my zips from the filesystem and upload them to Itch.io. Itch does provide an API to update games automatically, so going forward I would like to combine that and GitHub Actions to get my game automatically updated with every commit to the main branch.

This goes hand in hand with my other goal of rewriting the Windows-specific portions of this script for Linux and running it on Docker. That will make sure all tasks run in open-source environments such as GitHub Actions and GitLab CI.

When I have done all that, I'll come back here and tell everyone here!

The code

If you want to fast-track your setup and just clones my script into your machine and tweak it, feel free to do so! It's hosted on GitHub so you can easily clone it for yourself.