While Xojo lacks a command line compiler (Feedback case 3215), there is a IDE Communicator tool available (Documentation here). This tool shows you how to talk to Xojo and pass commands or IDE scripts. The IDE Scrips are scripts to run within the IDE and do various things like changing constants and trigger several builds with different configurations. For example we have for one project a script to build three variants of an application like a Demo, a Standard and a Pro version.
Our Test application gets the folder with all Xojo example projects for the plugin as input. It may take several hours to build them all, so a progress bar is really useful for us. If this runs in a virtual machine in the background, it doesn't even interfere with us using the computer for work.
We load each project by launching Xojo with the project file as document to open. If Xojo is running already, the project loads. We wait a second and then have an AppleScript to send ESC key press to Xojo in case there is a deprecation dialog showing. Next we call IDECommunicator command line tool via Shell class. We run our test script, which is just a text file with some commands for Xojo to do. This includes changing the app name, so all test builds have the same application file name. Next we build the project with BuildApp() function. We print the result in the script, so via shell class we can later read the output. Finally the script closes the script. When the shell finishes, we can write the result in a log file. If the result was success, we can delete the project and otherwise, we need to put it in the list to check later manually. We also check built apps what frameworks are used, so we can show that in the plugin documentation.
With the tests using Xojo 2019r1 we got projects updated to build in the newer Xojo version. We fixed a couple of things like changed syntax, graphics object access and removed Mac Carbon classes in our plugin for the upcoming 19.2 release. Enjoy!