Repeatable Models

Repeatability in Models

An important part of building models is that their results should be repeatable if the same inputs are given to them. The Simudyne SDK creates models that rely on numerical seeds that are randomly generated. A number of tools are provided for random number generation, but if a model is built properly, it should still be deterministic with respect to its seed.

It is easy to unintentionally introduce other sources of randomness into the model besides its numerical seed. The output of a properly built model should only change under two conditions: if its input parameters change or if its numerical seed is purposely changed. For the same model running with the same seed, the only source of change should be the input parameters.

Enforcing Determinism

The easiest way to enforce determinism is to use the random number generator provided from the model context. The seed will be the same for all instances of this model for the entire run.

public class myModel extends AgentBasedModel<GlobalState> {
    public void setup() {
        SeededRandom pnrg = getContext().getPRNG();
        pnrg.generator.nextDouble();
    }
    ...
}

Randomness Health Checks

Health checks are implemented in the SDK to warn modellers if an unanticipated source of randomness may be present. This is achieved by internally running and testing the output of a registered model when starting the server.

Health checks can be disabled by setting the value of the config field nexus-server.health-check in the SimudyneSDK.properties file. This value is a boolean, true by default, so setting it to false will disable health checks. To learn more about model configurations related to health checks, click here.

Unintentional Ways to Introduce Randomness

User-Defined Random Generator

A random number generator defined by the user will not use the seed defined in a model's configuration. This generator's seed is not controlled, so determinism is not enforced.

Random r = new Random();
r.nextBoolean();

Variable Source for Seed

A model cannot create repeatable results if its numeric seed varies between simulation runs. In this example, a different time would be computed each time the model is called, so the seed would change every time.

long seed = System.currentTimeMillis();
SeededRandom prng = SeededRandom.create(seed);
prng.generator.nextDouble();

Modifying Global State from Agents

Although the global state can be accessed and updated by any agent using the getGlobals function, doing so may result in non-deterministic behavior. Modifying global variables can have unforeseen effects on other parts of a model. In this example, the agent updates the global state with its own value. Later in the code, the agent accesses this variable again to send a message to its neighbor. However, in the meantime, this variable could have been updated again. There is no certainty about its state. This example is trivial, but updating the GlobalState from an agent is usually considered bad practice.

boolean isTrueLocal = true;
getGlobals().isTrueGlobal = isTrueLocal;
...
// Later in the agent, undefined behavior!
getLinks(BooleanLinks.class).send(BooleanMessage.class, getGlobals().isTrueGlobal);

Relying on Message Ordering

Agents and messages are not processed in a specific order, so relying on message order will create randomness. In this example, the loan request fulfillment depends on the ordering of messages. If some agents send a request, there is no certainty about which request will be accepted. This dependency on apparent ordering can be found in other places. Links, IDs of agents, and updates of the global state do not possess a proper order.

int liquidity = 1000;
List<LoanRequest> allLoanRequest = getMessagesOfType(LoanRequest.class);
for(LoanRequest m: messages) {
    if(m.getBody() < liquidity) {
        liquidity = liquidity - m.getBody();
        getLinksTo(m.getSender()).send(LoanResponse.class, true);
    } else {
        getLinksTo(m.getSender()).send(LoanResponse.class, false);
    }
}