Skip to content

Instantly share code, notes, and snippets.

@argenos

argenos/README.md

Last active May 28, 2020 10:43
Show Gist options
  • Save argenos/594286167e58d16fddb2c633aee91879 to your computer and use it in GitHub Desktop.
Save argenos/594286167e58d16fddb2c633aee91879 to your computer and use it in GitHub Desktop.
Download ZIP
Using sacred and omniboard to run experiments
Raw
README.md

Running experiments

Create the omniboard container:

docker-compose up

Through the command line you can now run an experiment, and override some of its configuration, e.g:

python3 experiment_script.py  # Runs the default configuration 
python3 experiment_script.py  with short_duration  # Runs the experiment with duration=5 
python3 experiment_script.py with dataset_with_obstacles # Runs the experiment with a different dataset
python3 experiment_script.py with short_duration dataset_with_obstacles # Combines the two above

You can view the results in the omniboard dashboard at 127.0.0.1:9000. You can change this in the docker-compose.yaml file.

References

Raw
docker-compose.yaml
version: '3'
services:
mongo:
image: mongo
ports:
- 127.0.0.1:27017:27017
environment:
MONGO_INITDB_ROOT_USERNAME: sample
MONGO_INITDB_ROOT_PASSWORD: password
MONGO_INITDB_DATABASE: db
expose:
- 27017
networks:
- omniboard
omniboard:
image: vivekratnavel/omniboard:latest
command: ["--mu", "mongodb://sample:password@mongo:27017/db?authSource=admin"]
ports:
- 127.0.0.1:9000:9000
networks:
- omniboard
depends_on:
- mongo
orca:
image: quay.io/plotly/orca
ports:
- 9091:9091
network_mode: host
networks:
omniboard:
Raw
experiment_script.py
from sacred import Experiment
from sacred.observers import MongoObserver
from ingredients import data_ingredient
# Add the ingredients to your experiment
ex = Experiment('dataset_generation', ingredients=[data_ingredient])
observer = MongoObserver(url=f'mongodb://sample:password@localhost:27017/?authMechanism=SCRAM-SHA-1',
db_name='db')
ex.observers.append(observer)
# You can append your own logger
# ex.logger = _logger
# Sacred recommends limiting the verbosity of the experiments
# See https://sacred.readthedocs.io/en/latest/collected_information.html#live-information
# Uncomment the next line if using -l DEBUG or if there is too much output
# ex.captured_out_filter = lambda captured_output: "Output capturing turned off."
@ex.config
def config_experiment():
name = "Experiment name"
description = "A description of what the experiment does"
# Any parameters relevant to your experiment
num_robots = 5
map_name = 'brsu-small'
window = 45
tolerance = 15
start_positions = [1, 2, 3, 4, 5]
duration = 60
locations = []
split = None
weight = 'weight'
@ex.named_config
def short_duration():
duration = 5
tolerance = 0.5
window = 1
@ex.automain
def run_experiment(_config, _run, _log):
# Your main goes here
# You can show some measurements during runtime:
_run.log_scalar('error.robot%i' % robot.guid, error, time_step)
# Add artifacts (e.g. a file with results for post-processing)
ex.add_artifact('/results/results.yaml', content_type='dict')
Raw
ingredients.py
from sacred import Ingredient
data_ingredient = Ingredient('dataset')
@data_ingredient.config
def cfg():
file_name = 'dataset_1.yaml'
@data_ingredient.named_config
def dataset_with_obstacles():
file_name = 'dataset1_obstacles.yaml'
@data_ingredient.capture
def load_data(file_name):
data = load_yaml_file('/path/to/file', file_name)
return data
@data_ingredient.command
def stats(file_name):
data = load_data()
tasks = data.get('tasks')
print("Total tasks: %i" % len(tasks))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment