Python API

API

The API class is used as a Python interface to the Comet.ml Python API.

You can use an instance of the API() class to quickly and easily access all of your logged information at comet.ml, including metrics, parameters, tags, and assets.

Example calls to get workspace, project, and experiment data:

  • API.get(): gets all of your personal workspaces
  • API.get(WORKSPACE): gets all of your projects from WORKSPACE
  • API.get(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
  • API.get_experiment(WORKSPACE, PROJECT_NAME, EXPERIMENT_ID): get an APIExperiment
  • API.get_experiment("WORKSPACE/PROJECT_NAME/EXPERIMENT_ID"): get an APIExperiment
  • API.get_experiments(WORKSPACE): get all APIExperiments in WORKSPACE
  • API.get_experiments(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
  • API.get_experiments(WORKSPACE, PROJECT_NAME, PATTERN): get all APIExperiments in WORKSPACE/PROJECT/PATTERN

Examples:

>>> import comet_ml
>>> comet_api = comet_ml.api.API()

## Return all of my workspace names in a list:
>>> comet_api.get()

## Get an APIExperiment:
>>> experiment = comet_api.get("cometpublic/comet-notebooks/example 001")

## Get metrics:
>>> experiment.get_metrics("train_accuracy")

The API instance also gives you access to the low-level Python API function calls:

>>> comet_api.delete_experiment(experiment_key)

For more usage examples, see Comet Python API examples.

API.server_url


API.__init__

__init__(self, rest_api_key=None, cache=True, version="v2")

Application Programming Interface to the Comet Python interface.

Args:

  • rest_api_key: Optional. Your private COMET_REST_API_KEY.
  • cache: Bool, whether to cache on values or not.
  • version: Optional. The version of the REST API to use.

Note: rest_api_key may be defined in environment (COMET_REST_API_KEY)

or in a .comet.config file.

Example:

>>> from comet_ml.api import API
>>> comet_api = API(rest_api_key="08ac6a75a2be4d7c9aac2c39e0004f6e")
>>> comet_api.get("my-workspace")
['project1', 'project2', ...]

API.archive_experiment

archive_experiment(self, experiment_key)

Archive one experiment.

Args:

  • experiment_key: the experiment ID to archive

API.archive_experiments

archive_experiments(self, experiment_keys)

Archive list of experiments.

Args:

  • experiment_keys: the experiment IDs to archive

API.clear_cache

clear_cache(self)

Used when cache is on, but you have added/changed data outside of this API instance.

Note: you could also just start with no cache.

>>> api = API(cache=False)

Or, if you had started with cache, turn it off:

>>> api = API(cache=True)
>>> api.do_cache(False)

API.create_project

create_project(self, workspace, project_name, project_description=None, public=False)

Create a project.


API.delete_experiment

delete_experiment(self, experiment_key)

Delete one experiment.


API.delete_experiments

delete_experiments(self, experiment_keys)

Delete list of experiments.

Args:

  • experiment_keys: a list of experiment keys to delete.

API.delete_project

delete_project(self, workspace=None, project_name=None, project_id=None, delete_experiments=False)

Delete a project.

Args:

  • workspace: the name of the workspace (required if project_id not given)
  • project_name: the name of the project (required if project_id not given)
  • project_id: the project id (required, if workspace and project name not given)
  • delete_experiments: if True, delete all of the experiments, too

API.do_cache

do_cache(self, *endpoints)

Cache the given endpoints.

Example:

>>> from comet_ml.api import API
>>> comet_api = API()
>>> comet_api.do_cache("experiments", "projects")

API.do_not_cache

do_not_cache(self, *endpoints)

Do not cache the given endpoints.

Example:

>>> from comet_ml.api import API
>>> comet_api = API()
>>> comet_api.do_not_cache("experiments", "projects")

API.gen_experiments

gen_experiments(self, workspace, project_name=None, pattern=None)

Get APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.


API.get

get(self, workspace=None, project_name=None, experiment=None)

Get the following items:

  • list of workspace names, given no arguments
  • list of project names, given a workspace name
  • list of experiment names/keys, given workspace and project names
  • an experiment, given workspace, project, and experiment name/key

workspace, project_name, and experiment can also be given as a single string, delimited with a slash.


API.get_account_details

get_account_details(self)

Return the username and the default workspace name for the authorized user.

Returns:

{
'userName': 'USERNAME',
'defaultWorkspaceName': 'WORKSPACE',
}

-------------------

### API.get_archived_experiment

```python
get_archived_experiment(self, workspace, project_name, experiment)

Get a single archived APIExperiment by workspace, project, experiment.


API.get_archived_experiments

get_archived_experiments(self, workspace, project_name=None, pattern=None)

Get archived APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.


API.get_experiment

get_experiment(self, workspace, project_name, experiment)

Get a single APIExperiment by workspace, project, experiment.


API.get_experiment_by_id

get_experiment_by_id(self, experiment)

Get an APIExperiment by experiment id.


API.get_experiments

get_experiments(self, workspace, project_name=None, pattern=None)

Get APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.


API.get_metrics_for_chart

get_metrics_for_chart(self, experiment_keys, metrics, independent=True, full=False)

Get multiple metrics from a set of experiments. This method is designed to make custom charting easier.

Args:

  • experiment_keys: a list of experiment keys
  • metrics: a list of metric names (e.g., "loss")
  • independent: Bool, get independent results?
  • full: Bool, fetch the full result?

Returns: a dictionary of experiment keys with the following

structure:

>>> from comet_ml.api import API
>>> comet_api = API()
>>> comet_api.get_metrics_for_chart([experiment_key1, experiment_key2, ...],
  ["loss"])
{EXPERIMENT_KEY: {
'experiment_key': EXPERIMENT_KEY,
'steps': STEPS,
'epochs': None,
'metrics': [
  {'metricName': 'loss',
  'values': [VALUE, ...],
  'steps': [STEP, ...],
  'epochs': [EPOCH, ...],
  'timestamps': [TIMESTAMP, ...],
  'durations': [DURATION, ...],
  }]
}, ...}

API.get_project

get_project(self, workspace, project_name)

Return the details of a project in a workspace.

Args:

  • workspace: String, the name of the workspace
  • project_name: String, the name of the project

Returns a dict of project details if the workspace/project exists, otherwise None.

Examples:

>>> api.get_project("workspace", "project-name")
{'projectId': 'project-id',
'projectName': 'project-name',
'ownerUserName': 'user name',
'projectDescription': 'my description',
'workspaceName': 'workspace',
'numberOfExperiments': 14,
'lastUpdated': 1571747775420,
'public': False}

>>> api.get_project("workspace", "non-existent-project-name")
None

API.get_project_by_id

get_project_by_id(self, project_id)

Return the details of a project given its project id.

Args:

  • project_id: String, the ID of the project

Returns a dict of project details if the project_id exists, otherwise None.

Examples:

>>> api.get_project_by_id("2727432637263")
{'projectId': '2727432637263',
'projectName': 'project name',
'ownerUserName': 'user name',
'projectDescription': 'my description',
'workspaceName': 'workspace',
'numberOfExperiments': 14,
'lastUpdated': 1571747775420,
'public': False}

>>> api.get_project_by_id("non-existent-project-id")
None

API.get_project_notes

get_project_notes(self, workspace, project_name)

Get the notes of a project.

Args:

  • workspace: String, the name of the workspace
  • project_name: String, the name of the project

Returns: a string

Example:

>>> api.get_project_notes("my-workspace", "my-project")
"These are my project-level notes"

API.get_projects

get_projects(self, workspace)

Return the details of the projects in a workspace.

Args:

  • workspace: String, the name of the workspace

Returns a list of project details in workspace.


API.get_query_variables

get_query_variables(self, workspace, project_name)

Return the query variables of a project in a workspace. Used with API.query().

Args:

  • workspace: String, the name of the workspace
  • project_name: String, the name of the project

Returns a list of QueryVariables, like:

[Metadata('user_name'),
Metadata('start_server_timestamp'),
...]

API.get_workspaces

get_workspaces(self)

Return a list of names of the workspaces for this user.


API.query

query(self, workspace, project_name, query, archived=False)

Perform a query on a workspace/project to find matching APIExperiment. Queries are composed of

Args:

  • workspace: String, the name of the workspace
  • project_name: String, the name of the project
  • query: a query expression (see below)
  • archived: (optional boolean), query the archived experiments if True
((QUERY-VARIABLE OPERATOR VALUE) & ...)

# or:

(QUERY-VARIABLE.METHOD(VALUE) & ...)

where:

QUERY-VARIABLE is Metric(NAME), Parameter(NAME), Other(NAME), Metadata(NAME), or Tag(VALUE).

OPERATOR is any of the standard mathematical operators ==, <=, >=, !=, <, >.

METHOD is between(), contains(), startswith(), or endswith().

You may also place the bitwise ~ not operator in front of an expression which means to invert the expression. Use & to combine additional criteria. Currently, | (bitwise or) is not supported.

VALUE can be any query type, includeing string, boolean, double, datetime, or timenumber (number of seconds). None and "" are special values that mean NULL and EMPTY, respectively. Use API.get_query_variables(WORKSPACE, PROJECT_NAME) to see query variables and types for a project.

When using datetime, be aware that the backend is using UTC datetimes. If you do not receive the correct experiments via a datetime query, please check with the web UI query builder to verify timezone of the server.

query() returns a list of matching APIExperiments().

Examples:

# Find all experiments that have an acc metric value > .98:
>>> api.query("workspace", "project", Metric("acc") > .98)
[APIExperiment(), ...]

# Find all experiments that have a loss metric < .1 and
# a learning_rate parameter value >= 0.3:
>>> loss = Metric("loss")
>>> lr = Parameter("learning_rate")
>>> query = ((loss < .1) & (lr >= 0.3))
>>> api.query("workspace", "project", query)
[APIExperiment(), ...]

# Find all of the experiments tagged "My simple tag":
>>> tagged = Metric("My simple tag")
>>> api.query("workspace", "project", tagged)
[APIExperiment(), ...]

# Find all experiments started before Sept 24, 2019 at 5:00am:
>>> q = Metadata("start_server_timestamp") < datetime(2019, 9, 24, 5)
>>> api.query("workspace", "project", q)
[APIExperiment(), ...]

# Find all experiments lasting more that 2 minutes (in seconds):
>>> q = Metadata("duration") > (2 * 60)
>>> api.query("workspace", "project", q)
[APIExperiment(), ...]

Notes:

  • Use ~ for not on any expression
  • Use ~QUERY-VARIABLE.between(2,3) for values not between 2 and 3
  • Use (QUERY-VARIABLE == True) for truth
  • Use (QUERY-VARIABLE == False) for not true
  • Use (QUERY-VARIABLE == None) for testing null
  • Use (QUERY-VARIABLE != None) or ~(QUERY-VARIABLE == None) for testing not null
  • Use (QUERY-VARIABLE == "") for testing empty
  • Use (QUERY-VARIABLE != "") or ~(QUERY-VARIABLE == "") for testing not empty
  • Use Python's datetime(YEAR, MONTH, DAY, HOUR, MINUTE, SECONDS) for comparing datetimes, like Metadata("start_server_timestamp") or Metadata("end_server_timestamp")
  • Use seconds for comparing timenumbers, like Metadata("duration")
  • Use API.get_query_variables(WORKSPACE, PROJECT_NAME) to see query variables and types.

Do not use 'and', 'or', 'not', 'is', or 'in'. These are logical operators and you must use mathematical operators for queries. For example, always use '==' where you might usually use 'is'.


API.restore_experiment

restore_experiment(self, experiment_key)

Restore one experiment.

Args:

  • experiment_key: the experiment ID to restore

API.set_project_notes

set_project_notes(self, workspace, project_name, notes)

Set the notes of a project. Overwrites any previous notes.

Args:

  • workspace: String, the name of the workspace
  • project_name: String, the name of the project notes: String, the full notes

Returns: a JSON message

Example:

>>> api.set_project_notes("my-workspace", "my-project",
...                       "These are my project-level notes")
{'msg': 'saved', 'code': 200, 'data': None, 'sdk_error_code': 0}

API.stop_experiment

stop_experiment(self, experiment_key)

Stop a running experiment.

Args:

  • experiment_key: the experiment ID

Example:

import comet_ml

# Start an online experiment:
experiment = comet_ml.Experiment()

# Perhaps somewhere else, while experiment
# is running:
api = comet_ml.API()
api.stop_experiment(experiment.id)

API.update_cache

update_cache(self)

Deprecated: Use API.clear_cache()


API.update_project

update_project(self, workspace, project_name, new_project_name=None, description=None, public=None)

Update the metadata of a project by project_name and workspace.

Args:

  • workspace: name of workspace
  • project_name: name of project
  • new_project_name: new name of project (optional)
  • description: new description of project (optional)
  • public: new setting of visibility (optional)

Example:

>>> api_experiment.update_project("mywork", "oldproj",
...     new_project_name="newproj", description="desc",
...     public=True)

API.update_project_by_id

update_project_by_id(self, project_id, new_project_name=None, description=None, public=None)

Update the metadata of a project by project_id.

Args:

  • project_id: project id
  • new_project_name: new name of project (optional)
  • description: new description of project (optional)
  • public: new setting of visibility (optional)

Example:

>>> api_experiment.update_project_by_id("2627523253623",
...     new_project_name="newproj", description="desc",
...     public=True)

API.use_cache

use_cache(self, cache=None)

Turn cache on/off or return cache.

Examples:

>>> from comet_ml.api import API
>>> comet_api = API()
>>> comet_api.use_cache(False)
>>> comet_api.use_cache()
False

>>> comet_api.use_cache(True)
>>> comet_api.use_cache()
True