Continuous Integration Tools

Last updated on 26th March 2024

During development it will likely become useful to make usage of automated testing tools for both Unit and Regression testing. Because the Simudyne SDK allows for multiple run configurations the ability to create scripts for certain configurations lends itself to automatic testing. Below are two examples of what some usage of CI tools would look like.

Main.java Setup

Below is a sample model class that makes usage of various demo models and runner configurations. It has been setup to look for command line arguments in order to determine which mode it should run in, as well as configuration for number of ticks/runs/agents.

Note: You will need to make sure that the input for numAgents exists in your model if following this code example.

Main.java

package sandbox;

import sandbox.demos.SimudyneSIR.SimudyneSIR;
import sandbox.demos.TumorGrowthSimulator.TumorGrowthModel;
import sandbox.demos.cda.CDAModel;
import sandbox.demos.creditCard.CreditCardModel;
import sandbox.demos.forestFire.ForestFireModel;
import sandbox.demos.mortgage.MortgageModel;
import sandbox.demos.schelling.SchellingModel;
import sandbox.demos.tokyo.TokyoModel;
import sandbox.demos.trading.TradingModel;
import sandbox.demos.volatilityModel.VolatilityModel;
import simudyne.core.Model;
import simudyne.core.exec.runner.ModelRunner;
import simudyne.core.exec.runner.MultirunController;
import simudyne.core.exec.runner.RunnerBackend;
import simudyne.core.exec.runner.definition.BatchDefinitionsBuilder;
import simudyne.core.exec.runner.definition.ModelSamplerDefinitionsBuilder;
import simudyne.core.exec.runner.definition.Scenario;
import simudyne.core.exec.runner.definition.ScenarioDefinitionsBuilder;
import simudyne.nexus.Server;

import java.time.Duration;
import java.time.Instant;

public class Main {
  public static void main(String[] args) {
    Instant startTime = Instant.now();
    System.out.println("Start Time " + startTime);

    String fallBackMode = "BATCH"; // BATCH, SCENARIO, SAMPLER, CONSOLE
    Class fallbackModel = sandbox.testing.housing.MortgageModel.class;
    int runs = (args.length > 1) ? Integer.parseInt(args[0]) : 1;
    long ticks = (args.length > 2) ? Long.parseLong(args[1]) : 100;
    long agents = (args.length > 3) ? Long.parseLong(args[2]) : 1000;

    try {
      @SuppressWarnings("unchecked")
      Class<? extends Model> modelClass = (args.length > 3) ? (Class<? extends Model>) Class.forName(args[3]) : fallbackModel;
      RunnerBackend runnerBackend = RunnerBackend.create();
      ModelRunner modelRunner = runnerBackend.forModel(modelClass);
      String mode = (args.length > 4) ? args[4] : fallBackMode;

      switch (mode) {
        case "BATCH":
          {
            System.out.println("Running in Batch Mode");
            BatchDefinitionsBuilder runDefinitionBuilder;
            if (args.length > 3) {
              runDefinitionBuilder =
                  BatchDefinitionsBuilder.create().forRuns(runs).forTicks(ticks).withInput("numAgents", agents);
            } else {
              runDefinitionBuilder = BatchDefinitionsBuilder.create().forRuns(runs).forTicks(ticks);
            }
            modelRunner.forRunDefinitionBuilder(runDefinitionBuilder);
            modelRunner.run().awaitOutput();
            exit(startTime);
          }
        case "SCENARIO":
          {
            System.out.println("Running in Scenario Mode");
            Scenario runDefinitionBuilder =
                ScenarioDefinitionsBuilder.create().createScenario("Test").forRuns(runs).forTicks(ticks);
            modelRunner.forRunDefinitionBuilder(runDefinitionBuilder.done());
            modelRunner.run().awaitOutput();
            exit(startTime);
          }
        case "SAMPLER":
          {
            System.out.println("Running in Model Sampler Mode");
            ModelSamplerDefinitionsBuilder runDefinitionBuilder =
                ModelSamplerDefinitionsBuilder.create().forRuns(runs).forTicks(ticks);
            modelRunner.forRunDefinitionBuilder(runDefinitionBuilder);
            modelRunner.run().awaitOutput();
            exit(startTime);
          }
        case "CONSOLE":
          {
            System.out.println("Running in Console Mode");
            Server.register("Trading Model", TradingModel.class);
            Server.register("Mortgage Model", MortgageModel.class);
            Server.register("Credit Card Model", CreditCardModel.class);
            Server.register("Continuous Double Auction Model", CDAModel.class);
            Server.register("Volatility Model", VolatilityModel.class);
            Server.register("Chain Bankruptcy Model", TokyoModel.class);
            Server.register("S.I.R. Model", SimudyneSIR.class);
            Server.register("Tumor Growth Model", TumorGrowthModel.class);
            Server.register("Schelling Segregation Model", SchellingModel.class);
            Server.register("Forest Fire Model", ForestFireModel.class);

            // Start the server.
            Server.run();
          }
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

  public static void exit(Instant startTime) {
    Instant finishTime = Instant.now();
    long timeElapsed = Duration.between(startTime, finishTime).getSeconds();
    System.out.println("Finish Time " + finishTime);
    System.out.println("Elapsed Time (seconds) " + timeElapsed);
    System.exit(0);
  }
}

Scripting Runs

Now that we've setup our Main.java to accept command line arguments we have the ability to cleany run multiple types of simulations for testing purposes. Below is a sample shell script that tests the model and ramps up number of agents, ticks, and runs.

MortgateModel.sh

#!/bin/bash
cd ..
# Baseline
mvn compile exec:java -Dexec.args="1 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
# Agent
mvn compile exec:java -Dexec.args="1 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="1 10 1000 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="1 10 10000 sandbox.testing.advanced3.MortgageModel BATCH"
# Tick
mvn compile exec:java -Dexec.args="1 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="1 100 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="1 1000 100 sandbox.testing.advanced3.MortgageModel BATCH"
# Runs
mvn compile exec:java -Dexec.args="1 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="10 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="100 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="1000 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
mvn compile exec:java -Dexec.args="10000 10 100 sandbox.testing.advanced3.MortgageModel BATCH"
exit

Azure Devops

Please refer to Deploy on Azure for a visual guide on how to add a Github project to Azure Devops. However while that page will describe how to get your model on a Azure Web Service, this will cover what your .yaml file should look like for testing with the above setup.

azure-pipelines.yml

# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java

trigger:
- master
- staging

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    options: '--settings settings.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: false
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    goals: 'compile'

- task: DownloadSecureFile@1
  name: license
  inputs:
    secureFile: 'azure_pipelines.license'

- task: DownloadSecureFile@1
  name: maven_settings
  inputs:
    secureFile: 'settings.xml'

- script: |
    mkdir -p ~/.simudyne
    cp $(license.secureFilePath) /home/vsts/.simudyne/azure.license
    cp $(maven_settings.secureFilePath) $(System.DefaultWorkingDirectory)/settings.xml
  displayName: 'Setup Files'

- task: ShellScript@2
  inputs:
    scriptPath: MortgageModel.sh

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)'
    Contents: '**.log'
    TargetFolder: $(Build.ArtifactStagingDirectory)
  displayName: 'Copy Logs'

Secure Files

As the settings.xml contains login information, and the license file is also meant to be obscured you'll notice that both are secure files. See [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/secure-files?view=azure-devops) for a guide on how to add secure files to Azure Devops.

Jenkins CI

Another CI tool is Jenkins. Below is a simple example that uses Maven and Java 8, informs the user some initialization information and runs our above simple script. It also includes credential files that must be configured via Jenkins correctly as shown here.

A simple JenkinsFile example

pipeline {
    agent any
    tools { 
        maven 'Maven 3.3.9' 
        jdk 'jdk8' 
    }
    stages {
        stage ('Initialize') {
            steps {
                sh '''
                    echo "PATH = ${PATH}"
                    echo "M2_HOME = ${M2_HOME}"
                ''' 
            }
        }

        stage ('Build') {
            withCredentials([
              file(credentialsId: 'license_file', variable: 'LICENSE_FILE'),
              file(credentialsId: 'settings_file', variable: 'MVN_SETTINGS')
            ]) steps {
				sh "cp --force $LICENSE_FILE licenseKey; cp --force $MVN_SETTINGS settings.xml"
                sh '/opt/script-directory/MortgageModel.sh'
            }
        }
    }
}

Atomic Debug Logging System Dynamics