API Reference¶
Chiminey Stage APIs¶
mytardis
– MyTardis APIS¶
The MyTardis module provides functions that allow publishing Chiminey results to a connected MyTardis System, allowing the online storing, access and sharing capabilities of data and metadata.
Datastructures¶
paramset
Module Functions and Constants¶
Example:
def _get_exp_name_for_make(settings, url, path):
return str(os.sep.join(path.split(os.sep)[-2:-1]))
def _get_dataset_name_for_make(settings, url, path):
return str(os.sep.join(path.split(os.sep)[-1:]))
self.experiment_id = mytardis.create_dataset(
settings=mytardis_settings,
source_url=encoded_d_url,
exp_id=self.experiment_id,
exp_name=_get_exp_name_for_make,
dataset_name=_get_dataset_name_for_make,
experiment_paramset=[],
dataset_paramset=[
mytardis.create_paramset("remotemake/output", [])]
)
storage
– Storage APIS¶
This package provides a file-like api for manipulating local and remote files and functions at locations specified by platform instances.
Datastructures¶
Module Functions and Constants¶
sshconnection
– Manipulation of Remote Resources¶
Datastructures¶
ssh_client
Module Functions and Constants¶
compute
– Execution Of Remote Commands¶
Using an open ssh_client connector from sshconnector, these commands execute commands remotely on the target server.
Datastructures¶
Module Functions and Constants¶
messages
– Logging communication for Chiminey¶
This package, modelled off the django logging module, posts status messages for the BDP system. There are two classes of context for the user of this API:
- Within stage implementation, messages will be displayed within the status field in the job list UI.
- During job submission, messages will be displayed on the redirected page as a header banner.
Messages are processed by a separate high-priority queue in the celery configuration. Note that message delivery may be delayed due to celery priority or db exclusion on the appropriate context, so this function should not be used for real-time or urgent messages.
Datastructures¶
By convention, error messages are final messages in job execution to indicate fatal error (though job might be restarted via admin tool) and success is used to describe final successful execution of the job.
Module Functions and Constants¶
Send a msg at the required priority level, as per the django logging module.
Uses contextid field of settings to determine which running context to assign messages.
run_settings
– Contextual Namespace for Chiminey¶
The current state of any BDP calculation is represented as a context model instance, with an associated run_settings instance. This run_setting serves as input, output and scratch symbol table for all job execution.
The contextual namespace is used for numerous purposes in the BDP, including:
- Input parameters for Directive submission UI dialog
- Single source of truth for building settings dicts for BDP API modules.
- Job execution state
- Diagnostics and visualisation of job progress
- Stage triggering and scheduling during directive execution
Conceptually run_settings is a set of parameter sets each of which is conformant to a predefined schemas, that are defined in the admin tool.
run_settings is a two-level dictionary, internally serialised to models as needed.
Datastructures¶
context
A two level dictionary made up of schema keys and values. Conceptually, this structure is equivalent to a two-level python dictionary, but should be accessed via the API below. For example,
{ http://acme.edu.au/schemas/stages : { “key”: id, “id”: 3} }
Keys are concatenation of schema namespace and name, for example: http://acme.edu.au/schemas/stages/key
Module Functions and Constants¶
-
chiminey.runsettings.
getval
(context, key)¶ Retrieves the current value of the key within the run setttings context.
Parameters: - context – current context
- key – the key to search
Raises: SettingNotFoundException if the schema part of the key is not in the context
Returns: the value for the key
-
chiminey.runsettings.
setval
(context, key, value)¶ Sets the value of the key within the run settings context.
Parameters: - context – current context
- key – the key to search
Throws: runsettings.SettingNotFoundException if the schema part of key is not in the context.
Throws: runsettings.IncompatibleTypeException if the value is not compatible with the type of the key, according to the associated schema definition.