Comet Python SDK

The following is a history of released comet_ml versions. It doesn't list everything that was changed in a release, but does mention the highlights and all public-facing additions, changes, and deprecations.

Installation instructions (click to show/hide)

You can install any version of comet_ml using pip:

pip install comet_ml

or using conda:

conda install -c anaconda -c conda-forge -c comet_ml comet_ml

For pip-based installations you can optionally install psutil (greater than or equal to version 5.6.3) to activate the CPU logging feature. You can install that with the command:

pip install "comet_ml[cpu_logging]"

You can install any one of the following released version numbers. You can find the full list at the Python Package Index.

For installation of the Comet Python SDK on air-gapped computers, please see Offline Installation.

Note that items that refer to Experiment also apply to ExistingExperiment and OfflineExperiment.

Release 3.2.12

Release date: January 21, 2021

• Use "simple" for auto_output_logging rather than "default" or "native" on MacOS. This is a temporary solution to address an issue with Python 3.8 and greater on Darwin. See Experiment() for more details on auto_output_logging.

Release 3.2.11

Release date: January 14, 2021

• Added support for logging 3D point cloud and bounding boxes; see Experiment.log_points_3d()
• Altered logging configuration to improve performance

Release 3.2.10

Release date: January 7, 2021

• Adding tags will retry in case of connection errors; see Experiment.add_tag() and Experiment.add_tags()
• Fixed uploading of non-UTF-8 data issue

Release 3.2.9

Release date: December 18, 2020

• New Experiment.log_code() for logging additional source code files and folders
• Allow downloading latest version of model without specifying stage; see API.download_registry_model()
• Expanded autologging for sklearn >=0.22
• Updated internal check for live threads for latest Python 3.9
• Updates for pytorch_lightning code logging

Release 3.2.8

Release date: December 8, 2020

• Added SHAP auto-logging

Release 3.2.7

Release date: December 2, 2020

• Added lightgbm auto-logger
• Fixed an XGBoost callback issue

Release 3.2.6

Release date: November 23, 2020

• Added ExistingOfflineExperiment for continuing an experiment in offline mode
• Added Experiment.log_dataframe_profile()
• Extended Experiment.log_table() to be able to log a Pandas DataFrame
• Added ability for the comet_ml Keras auto-logger to log gradient and activation histograms in addition to weight histograms
• Added ability for comet_ml to run if the pynvml package is deleted or doesn't install properly (thanks to the Ludwig team for tracking down issue)

Release 3.2.5

Release date: October 26, 2020

• Added support for auto-logging TensorFlow Model Analysis results

Release 3.2.4

Release date: October 12, 2020

• Added Hydra config logging (thanks to Vozf)
• Added environment query variable
• Matplotlib's save doesn't flush stream (thanks to jliu)

Release 3.2.3

Release date: October 5, 2020

• Fixed comet models with Python 3
• Accept unknown from backend query variables
• Added additional Confusion Matrix error logging
• Enhanced index_to_example for Confusion Matrix creation
• Enhanced Pytorch AMP Support
• Allow logging epochs to be floating point values; used by HuggingFace Transformers

Release 3.2.2

Release date: September 22, 2020

• Expanded Confusion Matrix to work with integer-based labeled y_true, rather than requiring onehot vectors: see Experiment.log_confusion_matrix() for more information.
• Allow Torch tensors attached to a gradient to work throughout comet_ml (including confusion matrix, histogram logging, etc)

Release 3.2.1

Release date: September 18, 2020

• Fixed a confusion matrix issue with non-integer indices (thanks ironcadiz for reporting)
• Refactored and expanded xgboost auto-logging
• Added output refinements when running experiments (reduced duplicated messages, better messages, etc.)
• Fallback to tf.keras if keras is not installed (thanks DN6 for reporting)
• Replaced comet-git-pure with dulwich

Release 3.2.0

Release date: August 28, 2020

• Added major speed-up for logging histograms
  • About 10 times faster when used with numpy
• Added refinements for logging metrics with xgboost
• Added refinements for the Predictive Early Stopping
• Added refinements for logging parameters, others, and metrics (thanks for reporting @Idodox!)
  • Added better representations of dictionaries and other objects
  • Added warning if key or value is truncated
• Dropped support for Python 3.4

Release 3.1.17

Release date: August 20, 2020

This release will be the last to support Python 3.4.

• Fixed issue to allow COMET_AUTO_LOG_DISABLE to disable all comet_ml actions. For more information, see Experiment Configuration Parameters.
• Fixed issues using comet_ml on Google Colab that required restarting Colab engine:
  • Can now use Colab's default version of jsonschema
  • comet upload failed without restart

Release 3.1.16

Release date: August 6, 2020

• Refined Experiment(auto_weight_logging=True) for Keras:
  • creates separate 3D histograms for weight matrices and bias arrays
  • uses layer names for Histogram names
• Added Histogram and image processing speed ups (up to 10x faster)
• Added more flexible logging of parameters
• Added more flexible source code logging for Experiment(log_code=True)

Release 3.1.15

Release date: August 3, 2020

• Configurable Experiment arguments. You can now control all of the auto-logged Experiment parameters from your Comet config settings (environment, code, or config file)
• Better logging of model graph representation (can record JSON, yaml, and string representations). Occurs automatically with Experiment(log_code=True) or with COMET_AUTO_LOG_CODE.
• Better logging of source code (can better find source code when using ludwig or transformers)
• Added additional APIExperiment methods: APIExperiment.log_parameters(), and APIExperiment.log_metrics()
• Updated comet-git-pure conda recipe
• Fixed Comet Optimizer document examples
• Confusion matrix optimization: if Experiment.log_confusion(..., selected=[...]) and index_to_example_function given, only make those examples selected
• Added username to experiment metadata. See APIExperiment.get_metadata()

Release 3.1.14

Release date: July 21, 2020

• Added support for keras automatic weight and bias logging as a 3D histogram, see Experiment(auto_weight_logging=True)

Release 3.1.13

Release date: July 17, 2020

• Added support for pytorch apex, online-pytorch-lightning-apex-example.py
• Documented metadata for many Experiment methods
• Added new methods APIExperiment.get_curve(), and APIExperiment.get_curves() with step
• Added API.get_project_share_keys(), API.create_project_share_key(), and API.delete_project_share_key() for programmatically sharing projects
• Limited backend handshake message to only show once
• Changed some conversion warnings to debug level
• Added new file download timeout config for downloading assets, COMET_TIMEOUT_FILE_DOWNLOAD

Release 3.1.12

Release date: June 25, 2020

• Add partial support for using the Optimizer with the Pytorch Lightning Comet Logger. See here for an example.
• Add experiment.display_project for all kinds of experiments (online, offline and api). Behavior is similar to expriment.display.
• Add file download timeout configuration, COMET_TIMEOUT_FILE_DOWNLOAD with a default value of 600 seconds. This is used only for registry downloading at the moment.

Release 3.1.11

Release date: June 17, 2020

• Better logging of all kind of summary data when using tensorflow.summary.
• Added better error catching in Keras callbacks.

Release 3.1.10

Release date: June 9, 2020

• Added comet models download and comet models list commands, see comet models
• Added comet init command to generate example scripts from cookiecutter recipe, see comet init
• Removed dependency on typing_extensions to allow %pip install comet_ml to work with Google Colab without a kernel restart

Release 3.1.9

Release date: June 6, 2020

• Added Experiment.log_embedding()
• Added Experiment.log_table()
• IPython environments now log code, output, and Code.ipynb
• Added comet check --debug command, see comet check
• Added custom context manager, with experiment.context_manager(CONTEXT)
• Added support for downloading a model based on its stage, see API.download_registry_model()
• Increased file upload timeout to 15 minutes

Release 3.1.8

Release date: May 26, 2020

• Improvements to work better on Google Colab
• Added API.move_experiments()
• Restored ability to from comet_ml import get_global_experiment

Release 3.1.7

Release date: May 18, 2020

• Refined APIExperiment.display()
• Made HTTP timeouts configurable. See COMET_TIMEOUT_HTTP
• Display a log message when an experiment has been throttled
• Fixed support for Python 2.7 in conda package
• Added new auto-logger for XGBoost
• Added ability to fetch parameters with APIExperiment.get_metrics_for_chart()
• Better invalid model name message: shows possible valid names

Release 3.1.6

Release date: April 20, 2020

• Fixed documentation for ExistingExperiment() default parameters
• Fixed console logging that was cut off
• Added auto-logged cloud metadata (currently AWS, Azure and Google Cloud Platform are supported)
• Added error checking for API.get_metrics_for_chart()
• Added Histogram.from_json() and default args for histogram.display()

Release 3.1.5

Release date: April 14, 2020

• Fixed empty Experiment Summary section headings from showing
• Updated conda's comet_ml dependencies to match pip's comet_ml dependencies
• Fixed reading non-utf8 Unix package descriptions
• Fixed Experiment's disabled parameter docstrings
• Fixed APIExperiment.set_os() docstrings

Release 3.1.4

Release date: April 7, 2020

• Added support for deleting assets from the Python API:
  • APIExperiment.delete_asset(asset_id)
• Added support for setting code from a filename:
  • APIExperiment.set_code(filename="script.py")
  • Experiment.set_code(filename="script.py")
• Enhanced Experiment Summary:
  • Now shows parameters logged
  • Control with display_summary_level (display_summary is now deprecated):
    • Experiment(display_summary_level=LEVEL)
    • display_summary_level=0: display nothing
    • display_summary_level=1: display metrics, parameters, and upload counts and sizes
    • display_summary_level=2: everything at level 1, plus system metrics
• Added new method APIExperiment.archive()
• Added checks to prevent logging empty 3D histograms

Release 3.1.3

Release date: March 23, 2020

• Ability to control the git directories; see COMET_GIT_DIRECTORY
• Added Experiment.log_curve() and APIExperiment.log_curve()
• Added ability to assign resources per parallel instance when running comet optimizer; see COMET_OPTIMIZER_PROCESS_ID
• Added Experiment.get_name() and APIExperiment.get_name()
• Many enhancements and speedups

Release 3.1.2

Release date: March 12, 2020

• Deprecated COMET_REST_API_KEY in Python API; just use COMET_API_KEY now
• Noted the correct minimum psutil version (5.6.3) in setup.py
• Fixed issue with auto output logging in OfflineExperiments from Jupyter
• Fixed Keras and Tensorflow model graphs to produce diff-able representations

Release 3.1.1

Release date: February 27, 2020

• Added experiment model and registry model support to API, see User Interface/Models
• Added ability to properly set project notes to unicode in Python 2, see API.set_project_notes()

Release 3.1.0

Release date: February 17, 2020

• Added MLFlow auto-logging
• Released a Python universal wheel (available via pip) to improve Python 2 installation speed
• Fixed some logging methods that were not respecting Experiment.disabled when True

Release 3.0.3

Release date: February 10, 2020

• Added Experiment.log_asset_folder() folder name
• Added API.get_account_details()
• Added logs errors to OfflineExperiment
• Added Read/write project notes to Python API
• Added API.server_url property
• Fixed Comet CLI for Python 2; added comet --version
• Added Experiment.log_model(), APIExperiment.download_model() and APIExperiment.get_model_names()
• Added logging for optimizer_objective; use proper JSON for optimizer parameters logging
• Fixed convert_to_scalar to handle numpy type
• Ensure comet optimize exits with a non-zero exit code if a child exit with an error
• Stopped sending sample_rate as metadata when logging an audio file

Release 3.0.2

Release date: December 19, 2019

• Added optional metadata to all Experiment.log_asset_data() methods
• Added 'text' to Experiment.display(tab='text')
• Updated automatic logging docstrings (thanks to lemairecarl for suggestion)
• Added Experiment.log_text()
• Added ability to stop a running experiment from Python API
• Added license information

Release 3.0.1

Release date: December 10, 2019

• Added Experiment.log_confusion_matrix()
• Added Experiment.display(TABNAME) to open browser to Comet tab
• Activated Keras and tensorflow.keras progress bars in console

Release 3.0.0

Release date: November 3, 2019

• Released REST v2 read and write endpoints
• Released updated Python API and updated APIExperiment
  • Dramatically increased speed of Python API by using REST v2
  • Added many additional Python API methods (update projects, delete projects, restore experiments, etc)
  • Standardized Python API JSON field names to camelCaseNaming
  • Added much more documentation and examples for Python API
  • Brought APIExperiment inline with Experiment, OfflineExperiment, and ExistingExperiment. Examples:
    • Create an experiment: APIExperiment(workspace=WORKSPACE", project_name=PROJECT_NAME)
    • Use an existing experiment: APIExperiment(previous_experiment=EXPERIMENT_ID | NAME)
  • Added ability for all aspects of an experiment to be downloaded and changed from the Python API
  • New and updated API methods:
    • API.clear_cache()
    • API.create_project(WORKSPACE, PROJECT_NAME, [PROJECT_DESCRIPTION, PUBLIC])
    • API.delete_project(workspace=WORKSPACE, project_name=PROJECT_NAME, delete_experiments=False)
    • API.delete_project(workspace=WORKSPACE, project_id=PROJECT_ID, delete_experiments=False)
    • API.get_project(WORKSPACE, PROJECT_NAME)
    • API.get_project_by_id(PROJECT_ID)
    • API.restore_experiment(EXPERIMENT_ID)
    • API.update_project(WORKSPACE, PROJECT_NAME, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
    • API.update_project_by_id(PROJECT_ID, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
    • API.use_cache(True | False)
  • New and updated APIExperiment methods:
    • New and updated read methods:
      • APIExperiment.get_additional_system_info()
      • APIExperiment.get_command()
      • APIExperiment.get_executable()
      • APIExperiment.get_gpu_static_info()
      • APIExperiment.get_hostname()
      • APIExperiment.get_ip()
      • APIExperiment.get_max_memory()
      • APIExperiment.get_network_interface_ips()
      • APIExperiment.get_os()
      • APIExperiment.get_os_type()
      • APIExperiment.get_pid()
      • APIExperiment.get_python_version()
      • APIExperiment.get_python_version_verbose()
      • APIExperiment.get_system_details()
      • APIExperiment.get_system_metric_names()
      • APIExperiment.get_tags()
      • APIExperiment.get_total_memory()
      • APIExperiment.get_user()
    • New and updated write methods:
      • APIExperiment.log_additional_system_info(KEY, VALUE)
      • APIExperiment.log_asset(FILENAME, [STEP, OVERWRITE, CONTEXT, FTYPE, METADATA])
      • APIExperiment.log_cpu_metrics(CPU_METRICS, [CONTEXT, STEP, EPOCH, TIMESTAMP])
      • APIExperiment.log_gpu_metrics(GPU_METRICS)
      • APIExperiment.log_html(HTML, [CLEAR, TIMESTAMP])
      • APIExperiment.log_image(FILENAME, [IMAGE_NAME, STEP, OVERWRITE, CONTEXT])
      • APIExperiment.log_load_metrics(LOAD_AVG, [CONTEXT, STEP, EPOCH, TIMESTAMP])
      • APIExperiment.log_metric(METRIC, VALUE, [STEP, TIMESTAMP])
      • APIExperiment.log_other(KEY, VALUE, [TIMESTAMP])
      • APIExperiment.log_output(LINES, [CONTEXT, STDERR, TIMESTAMP])
      • APIExperiment.log_parameter(PARAMETER, VALUE, [STEP, TIMESTAMP])
      • APIExperiment.log_ram_metrics(TOTAL_RAM, USED_RAM, [CONTEXT, STEP, EPOCH, TIMESTAMP])
      • APIExperiment.set_code(CODE)
      • APIExperiment.set_command(COMMAND_ARGS_LIST)
      • APIExperiment.set_end_time(TIME_MILLISECONDS)
      • APIExperiment.set_executable(EXECUTABLE)
      • APIExperiment.set_git_metadata(USER, ROOT, BRANCH, PARENT, ORIGIN)
      • APIExperiment.set_git_patch(FILE_DATA)
      • APIExperiment.set_gpu_static_info(GPU_STATIC_INFO)
      • APIExperiment.set_hostname(HOSTNAME)
      • APIExperiment.set_installed_packages(INSTALLED_PACKAGES)
      • APIExperiment.set_ip(IP)
      • APIExperiment.set_network_interface_ips(IPS)
      • APIExperiment.set_os(OS)
      • APIExperiment.set_os_type(OS_TYPE)
      • APIExperiment.set_pid(PID)
      • APIExperiment.set_python_version(PYTHON_VERSION)
      • APIExperiment.set_python_version_verbose(PYTHON_VERSION_VERBOSE)
      • APIExperiment.set_start_time(TIME_MILLISECONDS)
      • APIExperiment.set_user(USER)

For more information on the Python API, please see:

• Python API Tutorial
• Python API
• APIExperiment

Release 2.0.18

Release date: November 22, 2019

• Fixed issue when creating multiple experiments sequentially
• Added ability for Experiment.log_asset_folder() to log files recursively with file names
• Better handling of logging files and file-like objects
• Added ability for users to control auto logging details

Release 2.0.17

Release date: November 7, 2019

• Fixed a file-descriptor leak with native console logging
• Fixed Experiment.log_figure() to more accurately check for empty figures
• Added psutil to conda dependencies

Release 2.0.16

Release date: October 29, 2019

• Corrected the offline file upload to have the right file extension
• Fixed logging an empty CPU load avg metric
• Fixed optional dependency declaration for the cpu logging feature. You can now use pip install comet_ml[cpu_logging] to install psutil.
• Fixed a bug in OfflineExperiment that removed a file from disk when passing copy_to_tmp=False.
• Improved file extension handling when uploading file with OfflineExperiment.

Release 2.0.15

Release date: October 16, 2019

• Added optional timestamp (in seconds) to Python APIExperiment:
  • APIExperiment.log_other(..., timestamp=SECONDS)
  • APIExperiment.log_metric(..., timestamp=SECONDS)
  • APIExperiment.log_parameter(..., timestamp=SECONDS)
  • APIExperiment.log_html(..., timestamp=SECONDS)
  • APIExperiment.log_output(..., timestamp=SECONDS)
• Sped up Experiment.end() and OfflineExperiment.end()
• Pinned jsonschema to a Python 3.4 compatible version

Release 2.0.14

Release date: October 2, 2019

• Refactored Python API
• Added Query API
• Added CPU logging
  • requires psutil (see above)
• Deprecated passing experiment keyword arguments to Optimizer(); now pass using Optimizer.get_experiments(**kwargs)

Release 2.0.13

Release date: September 16, 2019

• Fixed repeated keras model unable-to-log warnings
• Fixed wrong context in keras callback Callback.on_test_*() methods; was test now validate
• Experiment.log_histogram_3d(values, step) gives required-step error earlier

Release 2.0.12

Release date: September 4, 2019

• Added support for TensorFlow v2 for tensorflow>=1.14, and tensorflow==2.0
• Fixed a bug that was using auto_param_logging instead of auto_metric_logging to control tensorboard metric logging.
• Added Experiment.log_histogram_3d() for logging time-series histograms.

Release 2.0.11

Release date: August 29, 2019

• Redefined valid experiment key (key must be alphanumeric, 32 to 50 characters)

Release 2.0.10

Release date: August 26, 2019

• Added support for all versions of TensorFlow v1 (1.11 - 2.0)
• Added Experiment.get_callback("tf-keras")
• Fixed spelling errors in messages, comments, and code
• Removed reporting when websocket connection was closed on normal shutdown

Release 2.0.9

Release date: August 21, 2019

• Fixed GPU schema to report usage better
• Added API.add_tags(experiment, tags)
• Fixed offline image upload issue
• Fixed sklearn to respect auto_param_logging flag
• Fixed SOCK HTTP Proxy which required an int port number

Release 2.0.8

Release date: August 14, 2019

• Added {user} to configurable logging file name patterns
• Added comet offline to explore offline experiment ZIP files
• Added API.get_experiments() and API.get_experiment()
• Added -m flag to comet python
• Added COMET_EXPERIMENT_KEY for new experiments

Release 2.0.7

Release date: August 13, 2019

• Added COMET_WORKSPACE to the config
• Added COMET_CONSOLE to set the console log level and display tracebacks
• Added COMET_LOGGING_FILE name patterns, like "comet-{project}.log"
• Added Experiment.set_epoch(NUMBER)
• Added Experiment.log_others({...})
• Fixed summary metric count
• Deprecated Experiment.get_keras_callback(); use Experiment.get_callback("keras")

Release 2.0.6

Release date: August 5, 2019

• Added step parameter to all asset logging methods
• Added Experiment(disable_summary=True) to disable display summary
• Added a log message when an optimizer search has completed
• Added comet python CLI to automatically import Comet first
• Fixed send_notification compatibility with older backend
• Added conda packaging support

Release 2.0.5

Release date: July 18, 2019

• Added step parameter to Experiment.log_audio()
• Added Experiment.send_notification()

Release 2.0.4

Release date: July 15, 2019

• Added base64-encoded metadata in the log_audio call

Release 2.0.3

Release date: July 15, 2019

• Added support for Experiment.log_audio()
• Added comet_ml.config.save()
• File upload now sends file extension

Release 2.0.2

Release date: July 2, 2019

• Added support for logging binary assets with Experiment.log_asset_data()
• Added more hook-points for tensorboard auto logging

Release 2.0.1

Release date: June 20, 2019

• Fixed tensorflow context logging for Tensorflow >1.13.1

Release 2.0.0

Release date: June 18, 2019

• Added a log message when an experiment was created offline
• Added a log message when an experiment was stopped
• Improved stop message
• Removed deprecated methods, this makes this release not backward compatible
• Added a new Optimizer which breaks the older API
• Added new tensorflow hooks to set the context properly

Release 1.0.56

Release date: June 5, 2019

• Updated OfflineExperiment() to accept the same API arguments as Experiment()