Comet Command-Line Utilities

When you install comet_ml, you also install a collection of utilities for the command-line interface (CLI).

  • comet upload - for uploading OfflineExperiments
  • comet optimize - for easy running of Optimizer scripts in parallel or serial
  • comet python - for injecting "import comet_ml" into your scripts
  • comet offline - for exploring offline experiment ZIP files

You can interactively get general help on these utilities with:

comet --help

and specific help with any of the following commands:

comet upload --help
comet optimize --help
comet python --help
comet offline --help

You can also easily see the comet_ml version by using:

comet --version

The rest of this page describes these utilities.

comet upload

The comet upload utility is used to uploading OfflineExperiments to Comet.ml. Consider the following command line:

$ comet upload /tmp/comet/5da271fcb60b4652a51dfc0decbe7cd9.zip

This is a script that is installed when you installed comet_ml. If that fails for any reason, you can try this more direct invocation using the same Python that you used when running your script:

$ python -m comet_ml.scripts.comet_upload /tmp/comet/5da271fcb60b4652a51dfc0decbe7cd9.zip

Don’t forget to include your API Key and update the experiment path to the one displayed at the end of your OfflineExperiment script run. For more details on configuring Python, please see Comet config file.

Sending multiple offline experiments is easy. To do so, execute the same comet upload command as before, but just replace the path to your experiment like so:

$ comet upload /path/to/*.zip

or:

$ python -m comet_ml.scripts.comet_upload /path/to/*.zip

Debugging

If you encounter any bugs with either the OfflineExperiment class or uploading, please run the uploader with the following:

$ COMET_LOGGING_FILE_LEVEL=debug \
    COMET_LOGGING_FILE=/tmp/comet.debug.log \
    COMET_API_KEY=MY_API_KEY \
    comet upload /path/to/experiments/*.zip

or:

$ COMET_LOGGING_FILE_LEVEL=debug \
    COMET_LOGGING_FILE=/tmp/comet.debug.log \
    COMET_API_KEY=MY_API_KEY \
    python -m comet_ml.scripts.comet_upload /path/to/experiments/*.zip

The debug logs will be in /tmp/comet.debug.log. This log will show details on all of the steps of the process. If you still have problems, please share this file with us via the Comet.ml Slack channel.

comet optimize

The comet optimize is a utility for running the Comet.ml optimizer in parallel or in serial. The format of the command line is:

$ comet optimize [options] [PYTHON_SCRIPT] OPTIMIZER

where OPTIMIZER is a JSON file, or an optimizer id.

PYTHON_SCRIPT is a regular Python file that takes an optimizer config file, or optimizer ID. If PYTHON_SCRIPT is not included, then an optimizer is created and the optimizer id is displayed.

Positional arguments:

  • PYTHON_SCRIPT - the name of the script to run
  • OPTIMIZER - optimizer JSON file or optimizer ID

Optional arguments:

  -h, --help            show this help message and exit
  -j PARALLEL, --parallel PARALLEL
                        number of parallel runs
  -t TRIALS, --trials TRIALS
                        number of trials per parameter configuration
  -e EXECUTABLE, --executable EXECUTABLE
                        Run using an executable other than Python
  -d DUMP, --dump DUMP  Dump the parameters to given filename

Note that comet optimize requires having your COMET_API_KEY pre-configured in one of the many ways possible, for example in an environment variable, or in your .comet.config file.

Examples of calling comet optimize:

$ export COMET_API_KEY=a287c4e3374d3645f3465346cc5
$ export COMET_OPTIMIZER_ID=$(comet optimize opt.json)
$ comet optimize script.py opt.json
$ comet optimize -j 4 script.py opt.json

To use an executable other than python, use -e, like so:

$ comet optimize -e "run-on-cluster.sh" script.py opt.json

comet python

The comet python utility is used to execute a Python script and import comet_ml automatically.

Although you still need to import comet_ml in your script, you do not need to import comet_ml before your machine learning libraries anymore.

Usage:

comet python [-h] [-p PYTHON] [-m MODULE] python_script

Positional arguments:

  • python_script: the python script to launch

Optional arguments:

  -h, --help            show this help message and exit
  -p PYTHON, --python PYTHON
                        Which Python interpreter to use
  -m MODULE, --module MODULE
                        Run library module as a script

comet offline

The comet offline utility is used to explore offline experiment archives.

Usage:

comet offline [-h] [--csv] [--section SECTION] [--level LEVEL]
                   [--name NAME] [--output OUTPUT] [--raw-size]
                   [--no-header]
                   archives [archives ...]

This command line displays summaries of an offline experiments:

$ comet offline *.zip

You may also display the ZIP details in a CSV (Comma-Separated Value) format. This format shows an experiment's data in a row format in the following order:

  • Workspace
  • Project
  • Experiment
  • Level
  • Section
  • Name
  • Value

where:

  • Workspace: the name of a specific workspace, or DEFAULT
  • Project: the name of a specific project, or "general"
  • Experiment: the experiment key for this experiment
  • Level: detail, maximum, or minimum
  • Section: metric, param, log_other, etc.
  • Name: name of metric, param, etc.
$ comet offline --csv *.zip

You may use the optional flags --level, --section, or --name to filter the rows. That is, if you use this command line:

$ comet offline --level detail *.zip

Note that when you use --level, --section, or --name then that implies --csv.

Positional arguments:

  • archives: the offline experiment archives to display

Optional arguments:

  -h, --help         show this help message and exit
  --csv              output details in csv format
  --section SECTION  output specific section in csv format, including param,
                     metric, log_other, data, etc.
  --level LEVEL      output specific summary level in csv format, including
                     minimum, maximum, detail
  --name NAME        output specific name in csv format, including items like
                     loss, acc, etc.
  --output OUTPUT    output filename for csv format
  --raw-size         Use bytes for file sizes
  --no-header        Use this flag to suppress CSV header