Build Your First Model

Introduction

This exercise is designed to help you define, design, and implement your own model in the Simudyne SDK. Whether you are new to the agent-based modelling field, or a seasoned veteran, it is recommended that new users start here when building the first of their own models in order to build a strong foundation in the core concepts and functionalities of the technology.

If you have never built a model in the Simudyne SDK before, please read the below sections before beginning this exercise:

1. About Simudyne SDK
2. Graph Computation Approach

It is also recommended that new users begin with some tutorials, sample models, and challenges before attempting this exercise:

1. Tutorials
2. Sample Models
3. Challenges

Estimated time to complete: ~1 day

Sections & topics covered:

1. Iterative Model Building
2. Repeatable Models
3. Use Case Definition
4. Model Design & Class Organisation
5. Implementation
6. Interactive Console Mode
7. Headless (BackendRunner) Mode
8. Input & Output
9. Analyzing Results with Jupyter or another data science tool

The objective of this exercise is to select a system of interest and build the first iteration of an agent-based model in the Simudyne SDK.

At the end of this exercise you will have a simulation and data science workflow that includes the following:

1. VCS: GitHub repo
2. Agent-based model
3. Interactive console model runner
4. Headless model runner
5. Input data channels
6. Output data channels
7. Advanced analytics capabilities (Jupyter Notebook)

If you'd like to get a trial license to the Simudyne SDK, you can follow the link below to get access.

Get Access to the SDK

Each section builds upon the previous and should be followed in order.

Iterative Model Building

Iterative and incremental development practices are key to Agile software development. The goal is to break down the development of a large application into smaller chunks. Code is designed, developed and tested in cycles. Each iteration sees the addition of new features to enhance the overall functionality of the application. At each stage of development, the user should have a functional application.

Building agent-based models has a number of parallels to developing software. Building these types of models calls for the adoption of a software developer's mindset.

Rather than trying to build a large and complex model in one attempt, modellers should ensure that sensible model output can be obtained at all stages of model development. Doing so requires the modeller to start by implementing a stripped down version of the desired model. Model complexity is layered on incrementally, ensuring that each iteration of the model produces sensible output.

This type of workflow is supported by Git version control. Each incremental increase in model complexity, such as the introduction of a new agent type or the addition of a new method to an agent's behaviour, should be tested thoroughly. Once the modeller is satisfied, the new version of the model can be committed to version history. Changes which break the model's functionality can simply be rolled-back.

Branches support experimental model development. Modellers can create multiple branches from a model's master branch, testing new concepts and functionalities in each one. When a modeller is satisfied with functionality in a branch, they can merge that branch with the model's master branch.

We've provided a starter project that includes a shell model for ease of getting started and setting up VCS before you start building.

Repeatable Models

An important part of building models is that their results should be repeatable if the same inputs are given to them. The Simudyne SDK creates models that rely on numerical seeds that are randomly generated. A number of tools are provided for random number generation, but if a model is built properly, it should still be deterministic with respect to its seed.

It is easy to unintentionally introduce other sources of randomness into the model besides its numerical seed. The output of a properly built model should only change under two conditions: if its input parameters change or if its numerical seed is purposely changed. For the same model running with the same seed, the only source of change should be the input parameters.

To learn more about how to ensure determinism in your model, check out Determinism

Use Case Definition

If you are looking to identify a use case or want to assess the suitability of one you have selected to the agent-based approach, check out What is ABM?

If you already have a model that you want to implement and have defined specifications for that model, you can skip to the next section.

Once you have identified the system you wish to model, the next objective is to design a simplified version that begins to reproduce the dynamics of your real-world system.

Components to identify:

1. Emergent / Reactive Properties
2. Agent Types
3. Agent Behaviors & Interactions
4. Environment

Let's start with an example and look at how we approach simulating an equity market. The example being used in this section is the Synthetic Market Tutorial.

An equity market is made up of an exchange and market participants. We can visualize the structure of a market as below:

The market is made up of the different types of traders that can represent anything from individuals / retail traders, to the tier 1 banks and large hedge funds.

Additionally, the exchange has a complex set of protocols as well as an order book and matching engine.

Let's identify and simplify each component from the list above and use the equity market example for each one.

1. Emergent / Reactive Properties

Emergent properties and phenomena arise when micro-level interactions produce a macro-level effect. In the case of the equity market, the price formation process is an emergent property generated by the varied buying and selling strategies of the market participants and the market mechanism.

A key advantage of an agent-based simulation is the ability to capture the non-linear dynamics of a complex adaptive system. Understanding how the micro leads to the macro can provide powerful insights into how one might optimise for or against a particular macro outcome.

Using the equity market example, emergent properties include: price, volatility, fire sales, etc. Let's say we select price as the feature to reproduce first.

The price is dictated by the buying and selling of the shares. Thus, we will need to simulate the buying and selling of shares by our virtual traders to generate a simple price formation process.

Components to Identify (Equity Market Example):

1. Emergent / Reactive Properties = Price
2. Agent Types
3. Agent Behaviors & Interactions
4. Environment

2. Agent Types

Now that a desired emergent / reactive property has been selected we want to identify the types of agents involved in the system and that contribute to the process.

Agent types should be defined based on their unique relationships and capabilities for interaction.

For simplification purposes, we want to create the most undifferentiated version of our agent types. In the equity market example, the markets consist of an ecosystem of traders deploying different strategies at global exchanges. The simplified model needs just 1 basic trader type and 1 simple exchange.

We could represent fundamental traders, momentum traders, market makers, etc. in our model. However, the first step will be creating a basic trader type agent that sends simple orders to an exchange. The complexity of the trader and the order will be layered in after the mechanics of the market have been defined.

Components to identify (Equity Market Example):

1. Emergent / Reactive Properties = Price
2. Agent Types = Trader + Exchange
3. Agent Behaviors & Interactions
4. Environment

3. Agent Behaviors & Interactions

Agents are capable of reacting to their environment and interacting with other agents in the system. An agent-based simulation may consist of numerous complex sets of interactions and behaviors. The simplest form of interaction should be captured first. This interaction should be driven by 1-2 variables.

In the equity market example, the trader agents and exchange agents both have behaviors:

Trader Behaviors: decide when to buy / sell, send buy order, send sell order, receive market information Exchange Behaviors: receive orders from traders, match orders / calculate price.

Notice that these behaviors are simple, and their logic will also be very simple in the beginning. More on the specifics of the equity market example can be found in the Behaviors Section of the Synthetic Market Tutorial.

Components to identify (Equity Market Example):

1. Emergent / Reactive Properties = Price
2. Agent Types = Trader + Exchange
3. Agent Behaviors & Interactions = Traders Trade + Exchange Calculate Price + Exchange Publish Price
4. Environment

4. Environment - physical, social, economical, etc

Agents are programmed to interact with their environment and with other agents. The environment may be a spatial location of an agent relative to other agents, or it may provide a set of dynamically determined features, like the price of an equity or changes in interest rates.

In the equity market example, the price of the equity, the policies at the exchange, the liquidity, real-world events, etc. are all elements of the environment.

To simplify the equity market environment, the price at the exchange will represent our main environment variable as this will be the minimum requirement to drive the trader behaviors.

Components to identify (Equity Market Example):

1. Emergent / Reactive Properties = Price Formation Process
2. Agent Types = Trader + Exchange
3. Agent Behaviors & Interactions = Traders Trade + Exchange Calculate Price + Exchange Publish Price
4. Environment = Price at the Exchange

If your environment contains spatial elements, the below models may provide code or insight into how to account for space. (The Simudyne SDK does not contain an explicit set of features for simulating 2D space):

1. Tumor Growth Model
2. Forest Fire Tutorial
3. Schelling Tutorial

Once you have completed identifying the 4 key components of your system, you are ready to design the implementation of your model.

Model Design & Class Organization

Now that you have defined the basic components of your model, the objective of this section is to design the structure of your code following some best practices in the Simudyne SDK before you begin your implementation.

Only some best practices are mentioned in this section, more information that may be relevant to your model can be found in the Reference.

Groups of Agents

The agents within models can be arranged in groups where each group of agents represents a larger entity, such as a pool of investors. These groups can be connected together by creating links between agents that span the groups. When two groups are being connected together, one group is considered the "source", while the other is the "target". When two groups are connected together, the usual layout of those connections is to link every agent in the source group with every agent in the target group.

If the number of agents in the source group is represented by M, and the number of agents in the target group is represented by N, it takes M x N links to connect the two groups completely. This works well for small models, where there are less than one million agents in total, but the number of connections can skyrocket as the number of agents increases. The large number of connections can cause major slowdowns in model execution.

Actions and Sequences

Models can quickly grow to have many actions with splits in a single sequence. Adding the implementation logic for agents in the same place can make the sequence of actions hard to follow.

Actions should be created by functions inside agents. The functions should have clear names indicating the action's purpose. With purposeful naming, the sequence of steps becomes a sequence of clear action names, rather than a sequence of code that is hard to read.

Messages

Messages should be defined in a shared message class so they can be used by the entire model. Individual messages should be defined as static classes inside the larger message class. Do not define messages inside agents, inside a model, or as many separate classes. This can lead to code duplication and difficulty in class maintenance.

The Simudyne SDK provides built-in classes for messages which contain a single primitive type (integer, long, double, float, boolean) or an empty body. Modellers need only extend these built-in classes to get all the functionality they need.

public class Messages {

    // Custom message classes that extend the built-in message type for doubles
    public static class Bid extends Message.Double {
    }

    public static class Vacancy extends Message.Double {
    }
}

If a message needs to pass multiple different primitive types, but no complex objects or behaviour, extend the Simudyne SDK Message object.

// Message class for price quotes, which passes only primitive types.
public class Messages {
    public static class PriceQuoteMessage extends Message {
        double price;
        double reducedPrice;
    }
}

// Class demonstrating usage of price quote messages.
public class Seller extends Agent<GlobalState> {
    public static Action<Seller> sendPriceQuote() {
        return Action.create(
                Seller.class,
                seller -> {
                    seller
                            .getLinks(Links.BankLink.class)
                            .send(
                                    Messages.PriceQuoteMessage.class,
                                    message -> {
                                        message.price = 3;
                                        message.reducedPrice = 2.3;
                                    };
                    );
                };
        );
    }
}

To send a complex data type, extend the Simudyne SDK Message.Object<T> generic object. Provide the name of the complex data type object in place of the "T" variable. When an agent receives a message containing a complex data type object, that agent has access to the complex data type's functions.

// Message class for price quotes which passes a complex data object.
public class Messages {
    public static class PriceQuoteMessage extends Message.Object<PriceQuote> {
    }
}

// Class demonstrating usage of complex price quote messages.
public class Seller extends Agent<GlobalState> {
    public static Action<Seller> sendPriceQuote() {
        return Action.create(
                Seller.class,
                seller -> {
                    seller
                            .getLinks(Links.BankLink.class)
                            .send(Messages.PriceQuoteMessage.class, new PriceQuoteMessage(...))
                };
        );
    }
}

Agents And Inheritance

When multiple agent types need to share some common functionality, modellers can reduce code duplication and build better defined agents using inheritance. Extend the Simudyne SDK Agent class with the base class, and then the derived classes can be implemented with their own unique functionalities. In the following example, an Accommodation agent is the base for two derived agent types, Hotel and Flat. All of these agent types can be used just like other agents which don't implement inheritance.

// Accommodation base class.
public class Accommodation extends Agent<GlobalState> {
    public int size;

    public Action<Accommodation> sendVacancyMessage() {
        return Action.create(
                Accommodation.class,
                accommodation -> {
                    // Send message if vacant.
                };
        );
    }
}

// Flat derived class.
public class Flat extends Accommodation {
    public Action<Flat> registerNewHousehold() {
        return Action.create(
                Flat.class,
                flat -> {
                    // Register a new flat.
                };
        );
    }
}

// Hotel derived class.
public class Hotel extends Accommodation {
    public Action<Hotel> registerBooking() {
        return Action.create(
                Hotel.class,
                hotel -> {
                    // Make a new booking.
                };
        );
    }
}

// Model containing actions for Accommodation, Hotel, and Flat.
public class myModel extends AgentBasedModel<GlobalState> {
    @Override
    public void step() {
        run(
                Accommodation.sendVacancyMessage(),
                Household.requestHome(),
                Split.create(
                        Flat.registerNewHousehold(),
                        Hotel.registerBooking()
                );  
        );
    }
}

External Classes

External classes should be used for functionality that is not specific to agents. This functionality may be needed by multiple agents or by the model, but should not be unique to a specific agent type. A data loader or distribution builder could both be reasonable uses for these external classes. Methods in these classes should be static so that they can be accessed from anywhere in a model.

// External distribution class.
public class Distribution {
    public static EmpiricalDistribution loadDistribution() {
        // Load a distribution.
    }
}

// Flat agent class which uses the distribution class.
public class Flat extends Agent<GlobalState> {
    public Action<Flat> registerNewHousehold() {
        return Action.create(
                Flat.class,
                flat -> {
                    EmpiricalDistribution distribution = Distribution.loadDistribution();
                };
        );
    }
}

Once you have a plan and design for how you wish to implement your model you are ready to move on to the next section.

Implementation

Now that you have designed and specified the structure of your model and all its components, the objective of this section is to implement the logic and compile your code.

The starter project provided above can be used as a template, it contains many of the structures and best practices mentioned in previous sections.

If at any point you get stuck with the implementation of your model, hop into our Community Discord Server and ask a Simudyne Dev for help!

Interactive Console Mode

Now that you have successfully implemented your model and are able to pull it up in the console, the objective of this section is to experiment with the features and develop a useful dashboard to visualise the dynamics of your simulation.

The below sections contain relevant information on how to design the layout of your console:

1. Interactive Console
2. Outputting Data

There is also a public demos site that contains live simulation demos that can be run and may inspire your layout:

The console layer is a valuable feature during the conceptual phase of model building. Once the model is sufficiently developed, and its behavior needs to be analysed, the SDK is run and deployed in headless mode for performance and scalability.

core-abm.serialize.agents=true
core-abm.serialize.links=true
core-abm.serialize.accumulators=true

Headless (BackendRunner) Mode

The console is a powerful prototyping, debugging, and high-level analysis tool. It is not intended to replace the popular and advanced data science packages in Python, or other advanced analytics platforms and tools. The Simudyne SDK is also designed to be run headless as one of the key advantages of the software is its ability to swap out different backends for distributing your models using Spark or Akka.

Configuring the SDK to run headless will be the first step towards getting the SDK hooked up to your Jupyter Notebook.

The objective of this section is to learn how to bypass the console layer for running your experiments at speed and scale.

Relevant sections that provide more detail on the possible configurations of this run mode can be found in the resources below:

1. Run & Deploy
2. Multirun Setup
3. Monte Carlo

A separate Main.java class should be created to contain the headless runner code, an example has been provided below:

public class MainHeadless {
    public static void main(String[] args) {
        //Warp in try catch
        try {
            //set up a ModelRunner
            RunnerBackend runnerBackend = RunnerBackend.create();
            ModelRunner modelRunner = runnerBackend.forModel(MyModel.class);

            //Define a set of batch runs using BatchDefinitionsBuilder
            BatchDefinitionsBuilder runDefinitionBuilder =
                    BatchDefinitionsBuilder.create()
                            .forRuns(100) // a required field, must be greater than 0.
                            .forTicks(1000); // a required field, must be greater than 0.

            modelRunner.forRunDefinitionBuilder(runDefinitionBuilder);

            // To run the model and wait for it to complete
            modelRunner.run();

        } catch (RuntimeException e) {
            System.out.println(Arrays.toString(e.getStackTrace()));
            e.printStackTrace();
        }
    }
}

The above example uses the BatchDefinitionsBuilder, however there are other tools for setting up experiments in the SDK using ModelSamplerDefinitionsBuilder or the ScenarioDefinitionsBuilder. More on the differences between these features can be found in Multirun Setup.

Now that you have a headless main class and an interactive console main, data outputting can be set up for importing and exporting data.

Data Input & Output

Your models can output and input data regardless of interactive console or headless running mode. The objective of this section is to set up any input data channels (if they exist) and configure data outputs, for analysing results.

The focus on this section will be on output data. Input data can be ingested in multiple ways and is less general and more specific to your model depending on whether your data is related to the network, initial agent states, interactions, etc.

More information on input data can be found in the following sections:

1. Input & Output
2. Data Management
3. Loading Data

Once configured, the Simudyne SDK automatically outputs simulation data in a structured and labelled format.

Data can be exported in the following formats:

• Parquet
• JSON
• CSV
• MySQL
• Hive via Parquet
• H2 and JDBC

Data can either be exported automatically using the model schema, via custom Simudyne SDK I/O channels, or via a user-defined method.

Automatically exporting data is the fastest way to begin studying your agents and the system.

The following configurations can be added to your simudyneSDK.properties file to set up automatic exporting via the model schema.

Each configuration below will also require core.export-path=output to be added to the properties file. output can be swapped with the relevant file path or desired directory name which will be created in the projects root directory.

Parquet

core.parquet-export.enabled=true
feature.interactive-parquet-output=true

JSON

core.json-export.enabled=true

CSV

core.csv-export.enabled=true

MySQL

core.export.username=default
core.export.password=work12345
core.sql-export.enabled=true
core.sql-export-path=jdbc:mysql://localhost:3306/sdk

H2

core.export.username=default
core.export.password=work12345
core.hive-export.enabled=false
core.hive-export-path=hive2://localhost:10000/default

Once you have added one of the above options you will see data exported in the designated folder.

For more information on data outputting, check out the following sections:

1. Data Outputting
2. Output Channels
3. Output Export Options

Now that data exporting has been configured and the state of every agent and the network are being dumped at each step, the model is ready for analysis in a data science tool or workflow.

Analyzing Results with Jupyter or Another Data Science Tool

Hooking up your simulation to your preferred data science toolkit can be as simple as setting your data export path to a location that your Jupyter Notebook or data science toolkit can access.

Additionally, and because the models built in the Simudyne SDK are packaged as a Java JAR file, it is possible to run the Simudyne SDK using a Python subprocess where you can parameterize and run models from a notebook. More information on this topic can be found in the Jupyter Notebook Tutorial.