Comet Python SDK

The following is a history of released comet_ml versions. It doesn't list everything that was changed in a release, but does mention the highlights and all public-facing additions, changes, and deprecations.

Installation instructions

You can install any version of comet_ml using pip:

pip install comet_ml

or using conda:

conda install -c anaconda -c conda-forge -c comet_ml comet_ml

For pip-based installations you can optionally install comet_ml[cpu_logging] to activate the CPU logging feature (and possibly additional utilities in the future). You can install that with the command:

pip install "comet_ml[cpu_logging]"

Releases

You can install any one of the following released version numbers. You can find the full list at the Python Package Index.

For installation of the Comet Python SDK on air-gapped computers, please see Offline Installation.

Note that items that refer to Experiment also apply to ExistingExperiment and OfflineExperiment.

Release 3.9.0

Release date: April 15, 2021

Release 3.8.1

Release date: April 8, 2021

  • Added support for the NO_PROXY environment variable; see HTTP Proxy for more information
  • Added additional information on the required use of Experiment.end() when in Jupyter Notebook-based environments

Release 3.8.0

Release date: April 6, 2021

Release 3.7.0

Release date: March 26, 2021

Release 3.6.0

Release date: March 18, 2021

Release 3.5.0

Release date: March 12, 2021

  • Removed netifaces from comet_ml Python SDK requirements
  • Added auto-logging for Facebook Prophet; see the Prophet Tutorial for more information
  • Expose ExistingOfflineExperiment and get_global_experiment from toplevel comet_ml imports (e.g., from comet_ml import ExistingOfflineExperiment)

Release 3.4.0

Release date: March 4, 2021

Release 3.3.5

Release date: February 26, 2021

  • Fixed a repeating warning when logging an empty dict of parameters
  • Fixed a command-line argument parse bug

Release 3.3.4

Release date: February 18, 2021

Release 3.3.3

Release date: February 11, 2021

  • Increased asset upload timeouts in the Python SDK to better handle large files and slow connections
  • Enhanced Experiment.log_remote_asset() remote file name; see Experiment.log_remote_asset()
  • Enhanced Experiment.log_table() documentation; see Experiment.log_table()
  • Fixed API.get_experiment_by_id(ID) to return None when ID does not exist; see API.get_experiment_by_id()
  • Refinements to comet offline; see Command-line API
  • Added Python 3.9 to our extensive internal test matrix

Release 3.3.2

Release date: February 4, 2021

  • Fixed an issue in rendering logged figures, including SHAP figures, in the UI
  • Additional refinements for better logging Jupyter Notebook source code

Release 3.3.1

Release date: February 2, 2021

  • Fixed a bug in experiment.end(), a required call in Jupyter notebooks, and optional in other scenarios. Calling Experiment.end() fails with version 3.3.0. See Experiment.end() for more information.

Release 3.3.0

Release date: February 1, 2021

  • This version of the Comet Python SDK uses a new Comet Optimizer that has many bug fixes and enhancements. See Optimizer for more details on using the Optimizer
  • Added remote asset logging. Useful for tracking remote or local files that you may not wish to upload but want to track. See Experiment.log_remote_asset()
  • Log multiple source files. Useful for when your experiment code is spread across multiple files, or the top-level Python script is not where Experiment() is called. The Experiment.log_code() function is also enhanced to log source code that was missed in prior versions. See Experiment.log_code()
  • Added auto-logging support for TensorFlow Estimators. Now logs model definition, .pbtxt model file, and hyperparameters in addition to metrics. See TensorFlow Estimator Integration
  • Added support for huuuge asset file uploads. See Experiment.log_asset() and related functions

Release 3.2.12

Release date: January 21, 2021

  • Use "simple" for auto_output_logging rather than "default" or "native" on MacOS. This is a temporary solution to address an issue with Python 3.8 and greater on Darwin. See Experiment() for more details on auto_output_logging.

Release 3.2.11

Release date: January 14, 2021

  • Added support for logging 3D point cloud and bounding boxes; see Experiment.log_points_3d()
  • Altered logging configuration to improve performance

Release 3.2.10

Release date: January 7, 2021

Release 3.2.9

Release date: December 18, 2020

  • New Experiment.log_code() for logging additional source code files and folders
  • Allow downloading latest version of model without specifying stage; see API.download_registry_model()
  • Expanded auto-logging for sklearn >=0.22
  • Updated internal check for live threads for latest Python 3.9
  • Updates for pytorch_lightning code logging

Release 3.2.8

Release date: December 8, 2020

Release 3.2.7

Release date: December 2, 2020

  • Added lightgbm auto-logger
  • Fixed an XGBoost callback issue

Release 3.2.6

Release date: November 23, 2020

Release 3.2.5

Release date: October 26, 2020

Release 3.2.4

Release date: October 12, 2020

  • Added Hydra config logging (thanks to Vozf)
  • Added environment query variable
  • Matplotlib's save doesn't flush stream (thanks to jliu)

Release 3.2.3

Release date: October 5, 2020

Release 3.2.2

Release date: September 22, 2020

  • Expanded Confusion Matrix to work with integer-based labeled y_true, rather than requiring onehot vectors: see Experiment.log_confusion_matrix() for more information.
  • Allow Torch tensors attached to a gradient to work throughout comet_ml (including confusion matrix, histogram logging, etc)

Release 3.2.1

Release date: September 18, 2020

  • Fixed a confusion matrix issue with non-integer indices (thanks ironcadiz for reporting)
  • Refactored and expanded xgboost auto-logging
  • Added output refinements when running experiments (reduced duplicated messages, better messages, etc.)
  • Fallback to tf.keras if keras is not installed (thanks DN6 for reporting)
  • Replaced comet-git-pure with dulwich

Release 3.2.0

Release date: August 28, 2020

Release 3.1.17

Release date: August 20, 2020

This release will be the last to support Python 3.4.

  • Fixed issue to allow COMET_AUTO_LOG_DISABLE to disable all comet_ml actions. For more information, see Experiment Configuration Parameters.
  • Fixed issues using comet_ml on Google Colab that required restarting Colab engine:
    • Can now use Colab's default version of jsonschema
    • comet upload failed without restart

Release 3.1.16

Release date: August 6, 2020

  • Refined Experiment(auto_weight_logging=True) for Keras:
    • creates separate 3D histograms for weight matrices and bias arrays
    • uses layer names for Histogram names
  • Added Histogram and image processing speed ups (up to 10x faster)
  • Added more flexible logging of parameters
  • Added more flexible source code logging for Experiment(log_code=True)

Release 3.1.15

Release date: August 3, 2020

Release 3.1.14

Release date: July 21, 2020

Release 3.1.13

Release date: July 17, 2020

Release 3.1.12

Release date: June 25, 2020

  • Add partial support for using the Optimizer with the Pytorch Lightning Comet Logger. See here for an example.
  • Add experiment.display_project for all kinds of experiments (online, offline and api). Behavior is similar to expriment.display.
  • Add file download timeout configuration, COMET_TIMEOUT_FILE_DOWNLOAD with a default value of 600 seconds. This is used only for registry downloading at the moment.

Release 3.1.11

Release date: June 17, 2020

  • Better logging of all kind of summary data when using tensorflow.summary.
  • Added better error catching in Keras callbacks.

Release 3.1.10

Release date: June 9, 2020

  • Added comet models download and comet models list commands, see comet models
  • Added comet init command to generate example scripts from cookiecutter recipe, see comet init
  • Removed dependency on typing_extensions to allow %pip install comet_ml to work with Google Colab without a kernel restart

Release 3.1.9

Release date: June 6, 2020

Release 3.1.8

Release date: May 26, 2020

  • Improvements to work better on Google Colab
  • Added API.move_experiments()
  • Restored ability to from comet_ml import get_global_experiment

Release 3.1.7

Release date: May 18, 2020

  • Refined APIExperiment.display()
  • Made HTTP timeouts configurable. See COMET_TIMEOUT_HTTP
  • Display a log message when an experiment has been throttled
  • Fixed support for Python 2.7 in conda package
  • Added new auto-logger for XGBoost
  • Added ability to fetch parameters with APIExperiment.get_metrics_for_chart()
  • Better invalid model name message: shows possible valid names

Release 3.1.6

Release date: April 20, 2020

  • Fixed documentation for ExistingExperiment() default parameters
  • Fixed console logging that was cut off
  • Added auto-logged cloud metadata (currently AWS, Azure and Google Cloud Platform are supported)
  • Added error checking for API.get_metrics_for_chart()
  • Added Histogram.from_json() and default args for histogram.display()

Release 3.1.5

Release date: April 14, 2020

  • Fixed empty Experiment Summary section headings from showing
  • Updated conda's comet_ml dependencies to match pip's comet_ml dependencies
  • Fixed reading non-utf8 Unix package descriptions
  • Fixed Experiment's disabled parameter docstrings
  • Fixed APIExperiment.set_os() docstrings

Release 3.1.4

Release date: April 7, 2020

Release 3.1.3

Release date: March 23, 2020

Release 3.1.2

Release date: March 12, 2020

  • Deprecated COMET_REST_API_KEY in Python API; just use COMET_API_KEY now
  • Noted the correct minimum psutil version (5.6.3) in setup.py
  • Fixed issue with auto output logging in OfflineExperiments from Jupyter
  • Fixed Keras and Tensorflow model graphs to produce diff-able representations

Release 3.1.1

Release date: February 27, 2020

Release 3.1.0

Release date: February 17, 2020

Release 3.0.3

Release date: February 10, 2020

Release 3.0.2

Release date: December 19, 2019

  • Added optional metadata to all Experiment.log_asset_data() methods
  • Added 'text' to Experiment.display(tab='text')
  • Updated automatic logging docstrings (thanks to lemairecarl for suggestion)
  • Added Experiment.log_text()
  • Added ability to stop a running experiment from Python API
  • Added license information

Release 3.0.1

Release date: December 10, 2019

  • Added Experiment.log_confusion_matrix()
  • Added Experiment.display(TABNAME) to open browser to Comet tab
  • Activated Keras and tensorflow.keras progress bars in console

Release 3.0.0

Release date: November 3, 2019

  • Released REST v2 read and write endpoints
  • Released updated Python API and updated APIExperiment
    • Dramatically increased speed of Python API by using REST v2
    • Added many additional Python API methods (update projects, delete projects, restore experiments, etc)
    • Standardized Python API JSON field names to camelCaseNaming
    • Added much more documentation and examples for Python API
    • Brought APIExperiment inline with Experiment, OfflineExperiment, and ExistingExperiment. Examples:
      • Create an experiment: APIExperiment(workspace=WORKSPACE", project_name=PROJECT_NAME)
      • Use an existing experiment: APIExperiment(previous_experiment=EXPERIMENT_ID | NAME)
    • Added ability for all aspects of an experiment to be downloaded and changed from the Python API
    • New and updated API methods:
      • API.clear_cache()
      • API.create_project(WORKSPACE, PROJECT_NAME, [PROJECT_DESCRIPTION, PUBLIC])
      • API.delete_project(workspace=WORKSPACE, project_name=PROJECT_NAME, delete_experiments=False)
      • API.delete_project(workspace=WORKSPACE, project_id=PROJECT_ID, delete_experiments=False)
      • API.get_project(WORKSPACE, PROJECT_NAME)
      • API.get_project_by_id(PROJECT_ID)
      • API.restore_experiment(EXPERIMENT_ID)
      • API.update_project(WORKSPACE, PROJECT_NAME, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
      • API.update_project_by_id(PROJECT_ID, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
      • API.use_cache(True | False)
    • New and updated APIExperiment methods:
      • New and updated read methods:
        • APIExperiment.get_additional_system_info()
        • APIExperiment.get_command()
        • APIExperiment.get_executable()
        • APIExperiment.get_gpu_static_info()
        • APIExperiment.get_hostname()
        • APIExperiment.get_ip()
        • APIExperiment.get_max_memory()
        • APIExperiment.get_network_interface_ips()
        • APIExperiment.get_os()
        • APIExperiment.get_os_type()
        • APIExperiment.get_pid()
        • APIExperiment.get_python_version()
        • APIExperiment.get_python_version_verbose()
        • APIExperiment.get_system_details()
        • APIExperiment.get_system_metric_names()
        • APIExperiment.get_tags()
        • APIExperiment.get_total_memory()
        • APIExperiment.get_user()
      • New and updated write methods:
        • APIExperiment.log_additional_system_info(KEY, VALUE)
        • APIExperiment.log_asset(FILENAME, [STEP, OVERWRITE, CONTEXT, FTYPE, METADATA])
        • APIExperiment.log_cpu_metrics(CPU_METRICS, [CONTEXT, STEP, EPOCH, TIMESTAMP])
        • APIExperiment.log_gpu_metrics(GPU_METRICS)
        • APIExperiment.log_html(HTML, [CLEAR, TIMESTAMP])
        • APIExperiment.log_image(FILENAME, [IMAGE_NAME, STEP, OVERWRITE, CONTEXT])
        • APIExperiment.log_load_metrics(LOAD_AVG, [CONTEXT, STEP, EPOCH, TIMESTAMP])
        • APIExperiment.log_metric(METRIC, VALUE, [STEP, TIMESTAMP])
        • APIExperiment.log_other(KEY, VALUE, [TIMESTAMP])
        • APIExperiment.log_output(LINES, [CONTEXT, STDERR, TIMESTAMP])
        • APIExperiment.log_parameter(PARAMETER, VALUE, [STEP, TIMESTAMP])
        • APIExperiment.log_ram_metrics(TOTAL_RAM, USED_RAM, [CONTEXT, STEP, EPOCH, TIMESTAMP])
        • APIExperiment.set_code(CODE)
        • APIExperiment.set_command(COMMAND_ARGS_LIST)
        • APIExperiment.set_end_time(TIME_MILLISECONDS)
        • APIExperiment.set_executable(EXECUTABLE)
        • APIExperiment.set_git_metadata(USER, ROOT, BRANCH, PARENT, ORIGIN)
        • APIExperiment.set_git_patch(FILE_DATA)
        • APIExperiment.set_gpu_static_info(GPU_STATIC_INFO)
        • APIExperiment.set_hostname(HOSTNAME)
        • APIExperiment.set_installed_packages(INSTALLED_PACKAGES)
        • APIExperiment.set_ip(IP)
        • APIExperiment.set_network_interface_ips(IPS)
        • APIExperiment.set_os(OS)
        • APIExperiment.set_os_type(OS_TYPE)
        • APIExperiment.set_pid(PID)
        • APIExperiment.set_python_version(PYTHON_VERSION)
        • APIExperiment.set_python_version_verbose(PYTHON_VERSION_VERBOSE)
        • APIExperiment.set_start_time(TIME_MILLISECONDS)
        • APIExperiment.set_user(USER)

For more information on the Python API, please see:

Release 2.0.18

Release date: November 22, 2019

  • Fixed issue when creating multiple experiments sequentially
  • Added ability for Experiment.log_asset_folder() to log files recursively with file names
  • Better handling of logging files and file-like objects
  • Added ability for users to control auto logging details

Release 2.0.17

Release date: November 7, 2019

  • Fixed a file-descriptor leak with native console logging
  • Fixed Experiment.log_figure() to more accurately check for empty figures
  • Added psutil to conda dependencies

Release 2.0.16

Release date: October 29, 2019

  • Corrected the offline file upload to have the right file extension
  • Fixed logging an empty CPU load avg metric
  • Fixed optional dependency declaration for the cpu logging feature. You can now use pip install comet_ml[cpu_logging] to install psutil.
  • Fixed a bug in OfflineExperiment that removed a file from disk when passing copy_to_tmp=False.
  • Improved file extension handling when uploading file with OfflineExperiment.

Release 2.0.15

Release date: October 16, 2019

  • Added optional timestamp (in seconds) to Python APIExperiment:
    • APIExperiment.log_other(..., timestamp=SECONDS)
    • APIExperiment.log_metric(..., timestamp=SECONDS)
    • APIExperiment.log_parameter(..., timestamp=SECONDS)
    • APIExperiment.log_html(..., timestamp=SECONDS)
    • APIExperiment.log_output(..., timestamp=SECONDS)
  • Sped up Experiment.end() and OfflineExperiment.end()
  • Pinned jsonschema to a Python 3.4 compatible version

Release 2.0.14

Release date: October 2, 2019

  • Refactored Python API
  • Added Query API
  • Added CPU logging
    • requires psutil (see above)
  • Deprecated passing experiment keyword arguments to Optimizer(); now pass using Optimizer.get_experiments(**kwargs)

Release 2.0.13

Release date: September 16, 2019

  • Fixed repeated keras model unable-to-log warnings
  • Fixed wrong context in keras callback Callback.on_test_*() methods; was test now validate
  • Experiment.log_histogram_3d(values, step) gives required-step error earlier

Release 2.0.12

Release date: September 4, 2019

  • Added support for TensorFlow v2 for tensorflow>=1.14, and tensorflow==2.0
  • Fixed a bug that was using auto_param_logging instead of auto_metric_logging to control tensorboard metric logging.
  • Added Experiment.log_histogram_3d() for logging time-series histograms.

Release 2.0.11

Release date: August 29, 2019

  • Redefined valid experiment key (key must be alphanumeric, 32 to 50 characters)

Release 2.0.10

Release date: August 26, 2019

  • Added support for all versions of TensorFlow v1 (1.11 - 2.0)
  • Added Experiment.get_callback("tf-keras")
  • Fixed spelling errors in messages, comments, and code
  • Removed reporting when websocket connection was closed on normal shutdown

Release 2.0.9

Release date: August 21, 2019

  • Fixed GPU schema to report usage better
  • Added API.add_tags(experiment, tags)
  • Fixed offline image upload issue
  • Fixed sklearn to respect auto_param_logging flag
  • Fixed SOCK HTTP Proxy which required an int port number

Release 2.0.8

Release date: August 14, 2019

  • Added {user} to configurable logging file name patterns
  • Added comet offline to explore offline experiment ZIP files
  • Added API.get_experiments() and API.get_experiment()
  • Added -m flag to comet python
  • Added COMET_EXPERIMENT_KEY for new experiments

Release 2.0.7

Release date: August 13, 2019

  • Added COMET_WORKSPACE to the config
  • Added COMET_CONSOLE to set the console log level and display tracebacks
  • Added COMET_LOGGING_FILE name patterns, like "comet-{project}.log"
  • Added Experiment.set_epoch(NUMBER)
  • Added Experiment.log_others({...})
  • Fixed summary metric count
  • Deprecated Experiment.get_keras_callback(); use Experiment.get_callback("keras")

Release 2.0.6

Release date: August 5, 2019

  • Added step parameter to all asset logging methods
  • Added Experiment(disable_summary=True) to disable display summary
  • Added a log message when an optimizer search has completed
  • Added comet python CLI to automatically import Comet first
  • Fixed send_notification compatibility with older backend
  • Added conda packaging support

Release 2.0.5

Release date: July 18, 2019

  • Added step parameter to Experiment.log_audio()
  • Added Experiment.send_notification()

Release 2.0.4

Release date: July 15, 2019

  • Added base64-encoded metadata in the log_audio call

Release 2.0.3

Release date: July 15, 2019

  • Added support for Experiment.log_audio()
  • Added comet_ml.config.save()
  • File upload now sends file extension

Release 2.0.2

Release date: July 2, 2019

  • Added support for logging binary assets with Experiment.log_asset_data()
  • Added more hook-points for tensorboard auto logging

Release 2.0.1

Release date: June 20, 2019

  • Fixed tensorflow context logging for Tensorflow >1.13.1

Release 2.0.0

Release date: June 18, 2019

  • Added a log message when an experiment was created offline
  • Added a log message when an experiment was stopped
  • Improved stop message
  • Removed deprecated methods, this makes this release not backward compatible
  • Added a new Optimizer which breaks the older API
  • Added new tensorflow hooks to set the context properly

Release 1.0.56

Release date: June 5, 2019

  • Updated OfflineExperiment() to accept the same API arguments as Experiment()