2.5.2

24th September 2022

This is a major relase of the software which focuses on improvements to data output, simulation control, determinism and debugging, scenario manipulation, and more. Please note that while 2.5.0 is available it is highly recommended to use 2.5.2 due to various fixes for both Spark/Hive configuration, Scala compatibility versioning, and console fixes.

Upgrading

Upgrading can be done by changing the simudyne.version setting in your pom.xml to 2.5.2. You MUST also switch any instance in your pom.xml where a library like simudyne-core-abm_2.11 exists to simudyne-core-abm_2.12 - This is because as of version 2.4 the SDK is now using Scala 12. If you have been using 2.4.0 or 2.4.1 you will likely not need to make this change.

Breaking Changes

There are a few breaking changes in version 2.5 that may affect your existing models:

• If you have been using the @ModelSettings annotation. Most notably the ticks and macroStep have been removed in favor of the new end for working with new simulation control clock
• The property name for parquet output and hive configurations have been made generic to accomodate multiple output formats. core.parquet-export-path is now core.export-path and core.export.username/core.export.password instead of hive-export

Most other features have been additive function wise and should have no discernable impact.

Changes

The following changes have been broken into the main X categories below.

Simulation Control Clock

We've provided a change to how models process start, interim, and end states. This allows for both better control, and easier readability in composed sequence flows of models. For more info please refer to the modelling sections particularly Model Interface

• Addition of firstStep, lastStep, kickOff functions that act as run but at corresponding times
• Addition of done function for actions to be called once model has ended, and finish to prematurely end the model
• Inclusion in REST API for new @ModelSettings

Atomic Logging

Atomic Logging is a new process by which actions and messaging can be set to automatically record to an external file every 'atomic' action that occurs. This will allow for greater debugging, and is tied into the Determinsim health check which has seen some changes. See Atomic Logging for more.

• Addition of Atomic Logging functionality creating 5 SDK controlled and 1 user controlled/writeable automatic logging enabled with a function call.
• Updating the Determinism health check to verify variables/agents/links and to include this info if the determinism health check fails which agents/variable does not match.
• Including Atomic Logging that if enabled will compare the resulting log files and inform which line has a mismatch
• Addition of Ordered Messages and functions for send/receive - previously this would only be possible by a user defined message with additional sort/filters.
• Ability to modify the length of the Determinism health check to be longer than default 3 ticks, this to support control clock changes in functionality such as kickOff

Enhanced Data Export

The Simudyne SDK previously was able to handle ingestion of JSON/CSV data to populate agents/network, and output to Parquet or Parquet format on HIVE, with other data available only via the REST API in JSON. We've now added multiple output formats as standard that will work the same as Parquet in terms of serializing structures or working with output channels.

• Ability to output to local JSON files
• Ability to output to CSV files
• Ability to output to MySQL tables
• Fixing of a bug where certain data formats (such as Int) would be treated as GenericRecord instead of ValueRecords
• Functionality to control timing of default outputs to instead of every tick to work for only first/kickoff/last ticks as desired. For certain models this should vastly lower size footprint if not required.
• Change of JDBCSource to allow for SQL Statements instead of a single table for ability to ingest from multiple joins as needed

Scenario Console Rework

As part of changes in 2.4.1 we made some changes to improve how scenarios work with the console in Interactive mode. Typically scenarios are meant to be used/defined with the Scenario Runner, but as the console remains an easy way to debug and see visually changes in parameters this required some changes.

• Ability to load and save local scenario JSON files
• Backend retention of stored scenarios and modifying scenario input to builder
• Adding Seed # to Scenario REST API
• Allowing Scenario Runner to skip a failed run and inform user rather than halt program
• Adding ScenarioDefinitionsBuilder to Java API (previously only existed in Scala)

Other

• Warning for missing properties file
• Fixing error where output files would terminate empty
• Fixing error where simulation ID not used in new 2.4 output directory names
• Ability to set initial value for Accumulators
• Functionality to return N agents from a set of links
• Various helper functions added to PRNG for ease of use
• Internal Pipeline tooling upgraded to allow for easier regression and model testing from Development work
• Additional demo models included
• Redhat OpenShift testing
• Ability to install offline has been streamlined