TARDIS - The Transparent Adaptive Resource Dynamic Integration System

Welcome to the TARDIS documentation!

The TARDIS resource manager enables the dynamic integration of resources provided by different resource providers into one overlaybatch system. TARDIS relies on COBalD - the Opportunistic Balancing Daemon in order to balance opportunistic resources. Hence, TARDIS is implemented as a COBalD service.

Installation

The TARDIS resource manager is available via PyPI, so you can use pip to install TARDIS and all its dependencies.

python3 -m pip install cobald-tardis

Configuration of COBalD

In order to run TARDIS a COBalD configuration is needed. Details about the available options and syntax can be found in the COBalD component configuration documentation.

The pools to be managed by TARDIS are created by a factory function create_composite_pool(), which is registered as a COBalD YAML plugin and, therefore, can be included using the !TardisPoolFactory YAML tag.

Example configuration

pipeline:
  # Makes decision to add remove resources based utilisation and allocation
  - !LinearController
    low_utilisation: 0.90
    high_allocation: 0.90
    rate: 1
  # Limits the demand for a resource
  - !Limiter
    minimum: 1
  # Log changes
  - !Logger
    name: 'changes'
  # Factory function to create composite resource pool
  - !TardisPoolFactory
    configuration: 'tardis.yml'

Configuration of TARDIS

In addition to the above mentioned COBalD configuration a TARDIS configuration is necessary as well, which is provided to the factory function create_composite_pool() via its configuration parameter.

The TARDIS YAML configuration supports the following sections:

Section	Short Description	Requirement
Plugins	Configuration of the Plugins to use (see Plugins)	Optional
BatchSystem	The overlay batch system to use (see BatchSystemAdapter)	Required
Sites	List of sites to create (see Generic Site Configuration)	Required
Site Sections	Configuration options for each site (see Generic Site Configuration)	Required

Example configuration

Plugins:
  SqliteRegistry:
    db_file: drone_registry.db

BatchSystem:
  adapter: FakeBatchSystem
  allocation: 1.0
  utilisation: !TardisPeriodicValue
               period: 3600
               amplitude: 0.5
               offset: 0.5
               phase: 0.
  machine_status: Available

Sites:
  - name: Fake
    adapter: FakeSite
    quota: 8000 # CPU core quota

Fake:
  api_response_delay: !TardisRandomGauss
                      mu: 0.1
                      sigma: 0.01
  resource_boot_time: !TardisRandomGauss
                      mu: 60
                      sigma: 10
  MachineTypes:
    - m1.infinity
  MachineTypeConfiguration:
    m1.infinity:
  MachineMetaData:
    m1.infinity:
      Cores: 8
      Memory: 16
      Disk: 160

Unified Configuration

Alternatively a unified COBalD and TARDIS configuration can be used. In this case, the TARDIS part of the configuration is represented by a tardis MappingNode.

Warning

In case of the unified configuration you can currently not use the yaml tag !TardisPoolFactory to initialize the pool factory, please use the COBalD legacy object initialisation __type__: tardis.resources.poolfactory.create_composite_pool instead!

Example configuration

pipeline:
  # Makes decision to add remove resources based utilisation and allocation
  - !LinearController
    low_utilisation: 0.90
    high_allocation: 0.90
    rate: 1
  # Limits the demand for a resource
  - !Limiter
    minimum: 1
  # Log changes
  - !Logger
    name: 'changes'
  # Factory function to create composite resource pool
  - __type__: tardis.resources.poolfactory.create_composite_pool
tardis:
    Plugins:
      SqliteRegistry:
        db_file: drone_registry.db

    BatchSystem:
      adapter: FakeBatchSystem
      allocation: 1.0
      utilization: !TardisPeriodicValue
                   period: 3600
                   amplitude: 0.5
                   offset: 0.5
                   phase: 0.
      machine_status: Available

    Sites:
      - name: Fake
        adapter: FakeSite
        quota: 8000 # CPU core quota

    Fake:
      api_response_delay: !TardisRandomGauss
                          mu: 0.1
                          sigma: 0.01
      resource_boot_time: !TardisRandomGauss
                          mu: 60
                          sigma: 10
      MachineTypes:
        - m1.infinity
      MachineTypeConfiguration:
        m1.infinity:
      MachineMetaData:
        m1.infinity:
          Cores: 8
          Memory: 16
          Disk: 160

Start-up your instance

To start-up your instance you can run the following command:

python3 -m cobald.daemon cobald.yml

However, it is recommended to start COBalD using systemd as decribed in the COBalD Systemd Configuration documentation.

Running your instance in Docker

For your convenience and to try out COBalD/TARDIS ready to use docker containers are provided via Dockerhub. To start a demo setup managing fake resources and a fake batch system simply run the following command.

docker run matterminers/cobald-tardis:latest

To run your own COBalD/TARDIS configuration you can run the following command.

docker run --mount type=bind,source=</your_path/cobald.yml>,target=/srv/cobald.yml matterminers/cobald-tardis:latest

To use the Sqlite Database to persistently store the state of your resources you can export an entire path to the container by running the following command.

docker run -v </your_path>:/srv matterminers/cobald-tardis:latest

Note

This path must contain a valid cobald.yml file. The persistent database will then be stored on the local machine.

Running your instance using HTCondor as overlay batch system in Docker

For your convenience and for systems without pre-compiled HTCondor support a ready to use docker container is provided via Dockerhub.

You can configure COBalD/TARDIS as described in this documentation and bind mount the directory containing the configuration into the containers /srv directory.

Note

This path must contain at least a valid cobald.yml file.

docker run -v $PWD/configuration:/srv matterminers/cobald-tardis-htcondor:latest

HTCondor Token Support

Starting with the production release series 9.0, HTCondor introduces a new security configuration, which is no longer host-based. The security configuration is now user-based and requires authentication to access the HTCondor pool. This is also true for read-only operations such as condor_status. Therefore, this docker image supports the IDTOKENS authentication method introduced with the HTCondor 9.0 series.

In order to use ID tokens, add any tokens provided by the operator of the overlay batch system to a tokens.d directory and bind mount it to /etc/condor/tokens.d. HTCondor is automatically using them to authenticate against the pool.

docker run -v $PWD/configuration:/srv -v $PWD/tokens.d:/etc/condor/tokens.d matterminers/cobald-tardis-htcondor:latest

Note

Since COBalD/TARDIS uses the condor_status command, the token added needs at least the ALLOW_READ privilege to access the HTCondor Collector and to query the status of resources.

In addition, COBalD/TARDIS uses the condor_drain command to release under utilized resources. Therefore, a second token to access the HTCondor StartD of the Drone is needed.

Usually, both tokens are provided by the operator of the HTCondor overlay batch system.

TARDIS - The Transparent Adaptive Resource Dynamic Integration System

Welcome to the TARDIS documentation!

Installation

Configuration of COBalD

Configuration of TARDIS

Unified Configuration

Start-up your instance

Running your instance in Docker

Running your instance using HTCondor as overlay batch system in Docker

HTCondor Token Support

Indices and tables