Python Panels SDK API¶

This page documents the comet_ml.API for Comet Python Panels.

Other Python Panel Resources:

API ¶

The API class is used as a Python interface to the comet-ml.com Python Panels API.

You can use an instance of the API() class to quickly and easily access all of your logged information at comet-ml.com, including metrics, parameters, tags, and assets.

Example calls to get workspace, project, and experiment data:

API.get(): gets all of your personal workspaces
API.get(WORKSPACE): gets all of your projects from WORKSPACE
API.get(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
API.get_experiment(WORKSPACE, PROJECT_NAME, EXPERIMENT_KEY): get an APIExperiment
API.get_experiment("WORKSPACE/PROJECT_NAME/EXPERIMENT_KEY"): get an APIExperiment
API.get_experiments(WORKSPACE): get all APIExperiments in WORKSPACE
API.get_experiments(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
API.get_experiments(WORKSPACE, PROJECT_NAME, PATTERN): get all APIExperiments in WORKSPACE/PROJECT/PATTERN

Examples:

```python

import comet_ml api = comet_ml.api.API()

Return all of my workspace names in a list:¶

api.get()

Get an APIExperiment:¶

experiment = api.get("cometpublic/comet-notebooks/example 001")

Get metrics:¶

experiment.get_metrics("train_accuracy") ```

Within Python Panels, the API() class is limited only to read methods. To log new data to Comet, see the Python SDK.

API.get¶

python get(self, workspace=None, project_name=None, experiment=None)

Get the following items:

list of workspace names, given no arguments
list of project names, given a workspace name
list of experiment names/keys, given workspace and project names
an experiment, given workspace, project, and experiment name/key

workspace, project_name, and experiment can also be given as a single string, delimited with a slash.

API.get_experiment¶

python get_experiment(self, experiment_key)

Get a single APIExperiment by workspace, project, experiment.

API.get_experiment_asset¶

python get_experiment_asset(self, experiment_key, asset_id, return_type="raw")

Get an asset, given the asset_id.

Args:

asset_id: the asset ID
return_type: the type of object returned. Default is "binary". Options: "binary" or "text"

Example:

```python

api_experiment.get_asset("298378237283728", return_type="json") {...} ```

API.get_experiment_asset_by_name¶

python get_experiment_asset_by_name(self, experiment_key, file_name, return_type=None)

Get an asset by name.

API.get_experiment_asset_list¶

python get_experiment_asset_list(self, experiment_key, asset_type="all")

Get a list of assets associated with the experiment.

Args:

asset_type: Optional String, type of asset to return. Can be "all", "image", "histogram_combined_3d", "video", or "audio".

Returns a list of dictionaries of asset properties, like:

```python

from comet_ml.api import API api = API() api.get_experiment_asset_list(key) [{'fileName': 'My Filename.png', 'fileSize': 21113, 'runContext': None, 'step': None, 'link': 'https://www.comet-ml.com/api/asset/download?experimentKey=KEY&assetId=ASSET_ID', 'createdAt': 1565898755830, 'dir': 'assets', 'canView': False, 'audio': False, 'video': False, 'histogram': False, 'image': True, 'type': 'image', 'metadata': None, 'assetId': ASSET_ID}, ...]

api.get_experiment_asset_list(key, "image") [{'fileName': 'My Filename.png', 'fileSize': 21113, 'runContext': None, 'step': None, 'link': 'https://www.comet-ml.com/api/asset/download?experimentKey=KEY&assetId=ASSET_ID', 'createdAt': 1565898755830, 'dir': 'assets', 'canView': False, 'audio': False, 'video': False, 'histogram': False, 'image': True, 'type': 'image', 'metadata': None, 'assetId': ASSET_ID}, ...] ```

API.get_experiment_by_id¶

python get_experiment_by_id(self, experiment)

API.get_experiment_by_key¶

python get_experiment_by_key(self, experiment_key)

Get an APIExperiment by experiment key.

API.get_experiment_by_name¶

python get_experiment_by_name(self, experiment_name)

Get an APIExperiment by experiment key.

API.get_experiment_code¶

python get_experiment_code(self, experiment_key)

Get the associated source code for this experiment.

Example:

```python

api.get_experiment_code(key) 'import comet_ml experiment = comet_ml.Experiment() experiment.end()' ```

API.get_experiment_curves¶

python get_experiment_curves(self, experiment_key)

Get all curve assets of an experiment as bytes.

Example:

```python

api.get_experiment_curves(key) [...] ```

API.get_experiment_graph¶

python get_experiment_graph(self, experiment_key)

Alias of APIExperiment.get_model_graph()

Get the associated graph/model description for this experiment.

Example:

```python

api_experiment.get_model_graph() {"class_name": "Sequential", ...} ```

API.get_experiment_html¶

python get_experiment_html(self, experiment_key)

Get the HTML associated with the experiment.

API.get_experiment_images¶

python get_experiment_images(self, experiment_key)

Get all image assets of an experiment as bytes.

Example:

```python

api.get_experiment_images(key) [...] ```

API.get_experiment_installed_packages¶

python get_experiment_installed_packages(self, experiment_key)

Get the associated installed packages for this experiment.

Example:

```python

api_experiment.get_installed_packages() ['absl-py==0.8.1', 'adal==1.2.2', 'alabaster==0.7.12', ...] ```

API.get_experiment_metadata¶

python get_experiment_metadata(self, experiment_key)

Returns the JSON metadata for an experiment

Returns:

python { 'archived': False, 'durationMillis': 7, 'endTimeMillis': 1586174765277, 'experimentKey': 'EXPERIMENT-KEY', 'experimentName': None, 'fileName': None, 'filePath': None, 'optimizationId': None, 'projectId': 'PROJECT-ID', 'projectName': 'PROJECT-NAME', 'running': False, 'startTimeMillis': 1586174757596, 'throttle': False, 'workspaceName': 'WORKSPACE-NAME', }

API.get_experiment_metric¶

python get_experiment_metric(self, experiment_key, metric)

Return an experiment metric values.

Args:

experiment_key: (string), key of experiment
metric: (string), name of a metric

Examples:

```python

from comet_ml.api import API api = API() api.get_experiment_metric(key, 'val_loss') ```

API.get_experiment_metrics_summaries¶

python get_experiment_metrics_summaries(self, experiment_key)

Return the experiment metrics summary.

Args:

experiment_key: (string), key of experiment

Examples:

```python

from comet_ml.api import API api = API() api.get_experiment_metrics_summaries(key) [{'name': 'val_loss', 'valueMax': '0.24951280827820302', 'valueMin': '0.13101346811652184', 'valueCurrent': '0.13101346811652184', 'timestampMax': 1558962367938, 'timestampMin': 1558962367938, 'timestampCurrent': 1558962376383, 'stepMax': 500, 'stepMin': 1500, 'stepCurrent': 1500}, ...] ```

API.get_experiment_model_asset_list¶

python get_experiment_model_asset_list(self, experiment_key, model_name)

Get an experiment model's asset list by model name.

Args:

model_name: str, the name of the model

Returns: a list of asset dictionaries with these fields:

fileName
fileSize
runContext
step
link
createdAt
dir
canView
audio
histogram
image
type
metadata
assetId

Example:

```python

from comet_ml import API api = API() api.get_experiment_model_asset_list(key, "Model Name") [ { "assetId": 74374637463476, "audio": False, "canView": False, "createdAt": 7337347634, "dir": "trained-models", "fileName": "model.h5", "fileSize": 254654, "histogram": False, "image": False, "link": "https://link-to-download-asset-file", "metadata": None, "remote": False, "runContext": "train", "step": 54, "type": "asset", } ] ```

API.get_experiment_model_graph¶

python get_experiment_model_graph(self, experiment_key)

Get the associated graph/model description for this experiment.

Example:

```python

api.get_experiment_model_graph(key) {"class_name": "Sequential", ...} ```

API.get_experiment_models¶

python get_experiment_models(self, experiment_key)

Given an experiment id, return a list of model data associated with an experiment.

Args:

experiment_id: the experiment's key

Returns [{'experimentModelId': 'MODEL-ID' 'experimentKey': 'EXPERIMENT-KEY', 'modelName': 'MODEL-NAME'}, ...]

API.get_experiment_name¶

python get_experiment_name(self, experiment_key, max_length=None)

Get the experiment name for this experiment key.

API.get_experiment_others_summaries¶

python get_experiment_others_summaries(self, experiment_key)

Get the other items logged in summary form.

Examples:

```python

from comet_ml.api import API api = API() api.get_experiment_others_summaries(key) [{'name': 'trainable_params', 'valueMax': '712723', 'valueMin': '712723', 'valueCurrent': '712723', 'timestampMax': 1558962363411, 'timestampMin': 1558962363411, 'timestampCurrent': 1558962363411}, ...] ```

API.get_experiment_output¶

python get_experiment_output(self, experiment_key)

Get the associated standard output for this experiment.

Example:

```python

api.get_experiment_output(key) "Experiment is live" ```

API.get_experiment_parameters_summaries¶

python get_experiment_parameters_summaries(self, experiment_key)

Return the experiment parameters summary.

Examples:

```python

from comet_ml.api import API api = API() api.get_experiment_parameters_summaries() [{'name': 'batch_size', 'valueMax': '120', 'valueMin': '120', 'valueCurrent': '120', 'timestampMax': 1558962363411, 'timestampMin': 1558962363411, 'timestampCurrent': 1558962363411}, ...] ```

API.get_experiment_tags¶

python get_experiment_tags(self, experiment_key)

Get the associated tags for this experiment.

Example:

```python

api.get_experiment_tags(key) ["best"] ```

API.get_experiment_url¶

python get_experiment_url(self, experiment_key)

Get the url of the experiment.

Example:

```python

api.get_experiment_url() "https://www.comet-ml.com/username/34637643746374637463476" ```

API.get_experiments¶

python get_experiments(self, workspace, project_name=None, pattern=None)

Get APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.

API.get_metrics_for_chart¶

python get_metrics_for_chart(self, experiments, metrics=[], parameters=[], independent_metrics=True, fetch_full=False)

Get multiple metrics and parameters from a set of experiments. This method is designed to make custom charting easier.

Args:

experiments: a list of experiments or experiment keys
metrics: an optional list of metric names (e.g., "loss"); OR
parameters: an optional list of parameter names (e.g., "learning-rate")
independent: Bool, get independent results?
full: Bool, fetch the full result?

Note: you should pass in a list of metric names, or a list of

parameter names, or both.

Returns: a dictionary of experiment keys with the following

structure. {EXPERIMENT_KEY: {'params'} will be None if there are no parameters passed in.

```python

from comet_ml.api import API api = API() api.get_metrics_for_chart([experiment_key1, experiment_key2, ...], ["loss"], ["hidden_layer_size"]) {EXPERIMENT_KEY: { 'experiment_key': EXPERIMENT_KEY, 'steps': STEPS, 'epochs': None, 'metrics': [ {'metricName': 'loss', 'values': [VALUE, ...], 'steps': [STEP, ...], 'epochs': [EPOCH, ...], 'timestamps': [TIMESTAMP, ...], 'durations': [DURATION, ...], }], 'params': {'hidden_layer_size': VALUE, ...}, }, ...} ```

API.get_model_graph¶

python get_model_graph(self, experiment_key)

Get the associated graph/model description for this experiment.

Example:

```python

api_experiment.get_model_graph() {"class_name": "Sequential", ...} ```

API.get_panel_experiment_keys¶

python get_panel_experiment_keys(self)

Get the current panel's experiment keys.

Example:

```python from comet_ml import API

api = API() experiment_keys = api.get_panel_experiment_keys() ```

This will return a list containing all of the current experiment keys. Specifically:

If on a Project View, the experiment keys from the visible experiment table page
If a filter is set, all of the matching experiments
If a Query is set for this Panel, the matching experiments

API.get_panel_experiments¶

python get_panel_experiments(self)

Get the current panel's experiments.

Example:

```python from comet_ml import API

api = API() experiment = api.get_panel_experiments() ```

Like the API.get_experiment_keys() method, but returns a list of APIExperiments rather than just their keys.

API.get_panel_metrics_names¶

python get_panel_metrics_names(self, include_sys=False)

Get the current panel's experiments' metrics' names.

Args:

include_sys (bool): if True, include the system metrics

Example:

```python from comet_ml import API, ui

api = API() metrics_names = api.get_panel_metrics_names() metric_name = ui.dropdown("Select a metric name:", metrics_names) ```

As shown, the metrics_names could be useful in creating a dropdown selection for plotting.

Note that, by default, this list does not includes all metrics. Use include_sys=True to include system metrics as well.

API.get_panel_options¶

python get_panel_options(self)

Get the current panel's options as a Python dictionary from the Option tab for this Panel instance.

Example:

```python from comet_ml import API

api = API() options = api.get_panel_options() ```

The Panel Options can be edited when in the Options tab of the Panel Code editor, and set when you are adding a Panel instance to a View.

API.get_panel_project_id¶

python get_panel_project_id(self)

Get the current panel's ID.

Example:

```python from comet_ml import API

api = API() project_id = api.get_panel_project_id() ```

The project_id can be useful to retrieve other Project-level information.

API.get_panel_project_name¶

python get_panel_project_name(self)

Get the current panel's name.

Example:

```python from comet_ml import API

api = API() project_name = api.get_panel_project_name() ```

The Project name could be useful in creating report-like Panels.

API.get_panel_report¶

python get_panel_report(self)

Get a string detailing the number of hits the panel has made to the cloud.

API.get_panel_request_count¶

python get_panel_request_count(self)

Get the number of requests made to the Comet cloud server.

API.get_panel_workspace¶

python get_panel_workspace(self)

Get a list of workspace names.

Example:

```python from comet_ml import API

api = API() workspace = api.get_panel_workspace() ```

The Workspace name could be useful in creating report-like Panels.

API.get_project¶

python get_project(self, workspace=None, project_name=None, project_id=None)

Return the details of a project in a workspace.

Args:

workspace: String, the name of the workspace
project_name: String, the name of the project

Returns a dict of project details if the workspace/project exists, otherwise None.

Examples:

```python

api.get_project("workspace", "project-name") {'projectId': 'project-id', 'projectName': 'project-name', 'ownerUserName': 'user name', 'projectDescription': 'my description', 'workspaceName': 'workspace', 'numberOfExperiments': 14, 'lastUpdated': 1571747775420, 'public': False}

api.get_project("workspace", "non-existent-project-name") None ```

API.get_project_columns¶

python get_project_columns(self, workspace=None, project_name=None, project_id=None)

Given a workspace and project_name return the column names, types, etc.

API.get_project_experiments¶

python get_project_experiments(self, workspace=None, project_name=None)

Return a dict of experiment_keys to metadata JSONs of the experiments in a project.

API.get_project_notes¶

python get_project_notes(self, project_id)

Get the notes of a project.

Args:

project_id: String, the ID of the project

Returns: a string

Example:

```python

api.get_experiment_project_notes("my-workspace", "my-project") "These are my project-level notes" ```

API.get_projects¶

python get_projects(self, workspace)

Return the details of the projects in a workspace.

Args:

workspace: String, the name of the workspace

Returns a list of project details in workspace.

API.get_query_variables¶

python get_query_variables(self, workspace=None, project_name=None, project_id=None)

Return the query variables of a project in a workspace. Used with API.query().

Args:

workspace: String, the name of the workspace
project_name: String, the name of the project

Returns objects used in forming queries, like:

python [Metadata("user_name"), Metadata("start_server_timestamp"), Tag("my_tag"), ...]

API.get_workspaces¶

python get_workspaces(self)

Return a list of names of the workspaces for this user.

API.query¶

python query(self, workspace=None, project_name=None, query=None, archived=False)

Perform a query on a workspace/project to find matching APIExperiment. Queries are composed of

Args:

workspace: String, the name of the workspace
project_name: String, the name of the project
query: a query expression (see below)
archived: (optional boolean), query the archived experiments if True

```python ((QUERY-VARIABLE OPERATOR VALUE) & ...)

or:¶

(QUERY-VARIABLE.METHOD(VALUE) & ...) ```

where:

QUERY-VARIABLE is Environment(NAME), Metric(NAME), Parameter(NAME), Other(NAME), Metadata(NAME), or Tag(VALUE).

OPERATOR is any of the standard mathematical operators ==, <=, >=, !=, <, >.

METHOD is between(), contains(), startswith(), or endswith().

You may also place the bitwise ~ not operator in front of an expression which means to invert the expression. Use & to combine additional criteria. Currently, | (bitwise or) is not supported.

VALUE can be any query type, includeing string, boolean, double, datetime, or timenumber (number of seconds). None and "" are special values that mean NULL and EMPTY, respectively. Use API.get_query_variables(WORKSPACE, PROJECT_NAME) to see query variables and types for a project.

When using datetime, be aware that the backend is using UTC datetimes. If you do not receive the correct experiments via a datetime query, please check with the web UI query builder to verify timezone of the server.

query() returns a list of matching APIExperiments().

Examples:

```python

Find all experiments that have an acc metric value > .98:¶

api.query("workspace", "project", Metric("acc") > .98) [APIExperiment(), ...]

Find all experiments that have a loss metric < .1 and¶

a learning_rate parameter value >= 0.3:¶

loss = Metric("loss") lr = Parameter("learning_rate") query = ((loss < .1) & (lr >= 0.3)) api.query("workspace", "project", query) [APIExperiment(), ...]

Find all of the experiments tagged "My simple tag":¶

tagged = Tag("My simple tag") api.query("workspace", "project", tagged) [APIExperiment(), ...]

Find all experiments started before Sept 24, 2019 at 5:00am:¶

q = Metadata("start_server_timestamp") < datetime(2019, 9, 24, 5) api.query("workspace", "project", q) [APIExperiment(), ...]

Find all experiments lasting more that 2 minutes (in seconds):¶

q = Metadata("duration") > (2 * 60) api.query("workspace", "project", q) [APIExperiment(), ...] ```

Notes:

Use ~ for not on any expression
Use ~QUERY-VARIABLE.between(2,3) for values not between 2 and 3
Use (QUERY-VARIABLE == True) for truth
Use (QUERY-VARIABLE == False) for not true
Use (QUERY-VARIABLE == None) for testing null
Use (QUERY-VARIABLE != None) or ~(QUERY-VARIABLE == None) for testing not null
Use (QUERY-VARIABLE == "") for testing empty
Use (QUERY-VARIABLE != "") or ~(QUERY-VARIABLE == "") for testing not empty
Use Pythons datetime(YEAR, MONTH, DAY, HOUR, MINUTE, SECONDS) for comparing datetimes, likeMetadata("start_server_timestamp")orMetadata("end_server_timestamp")`
Use seconds for comparing timenumbers, like Metadata("duration")
Use API.get_query_variables(WORKSPACE, PROJECT_NAME) to see query variables and types.

Do not use and, or, not, is, or in. These are logical operators and you must use mathematical operators for queries. For example, always use == where you might usually use is.

API.query_project¶

python query_project(self, workspace, project_name, predicates, archived=False)

Given a workspace, project_name, and predicates, return matching experiments.