Comet Python SDK¶
The following is a history of released comet_ml
versions. It doesn't
list everything that was changed in a release, but does mention the
highlights and all public-facing additions, changes, and deprecations.
Installation instructions (click to show/hide)
You can install any version of comet_ml using pip:pip install comet_ml
conda install -c anaconda -c conda-forge -c comet_ml comet_ml
pip install "comet_ml[cpu_logging]"
You can install any one of the following released version numbers. You can find the full list at the Python Package Index.
For installation of the Comet Python SDK on air-gapped computers, please see Offline Installation.
Note that items that refer to Experiment also apply to ExistingExperiment and OfflineExperiment.
Release 3.2.12¶
Release date: January 21, 2021
- Use "simple" for
auto_output_logging
rather than "default" or "native" on MacOS. This is a temporary solution to address an issue with Python 3.8 and greater on Darwin. See Experiment() for more details onauto_output_logging
.
Release 3.2.11¶
Release date: January 14, 2021
- Added support for logging 3D point cloud and bounding boxes; see Experiment.log_points_3d()
- Altered logging configuration to improve performance
Release 3.2.10¶
Release date: January 7, 2021
- Adding tags will retry in case of connection errors; see Experiment.add_tag() and Experiment.add_tags()
- Fixed uploading of non-UTF-8 data issue
Release 3.2.9¶
Release date: December 18, 2020
- New Experiment.log_code() for logging additional source code files and folders
- Allow downloading latest version of model without specifying stage; see API.download_registry_model()
- Expanded autologging for sklearn >=0.22
- Updated internal check for live threads for latest Python 3.9
- Updates for pytorch_lightning code logging
Release 3.2.8¶
Release date: December 8, 2020
- Added SHAP auto-logging
Release 3.2.7¶
Release date: December 2, 2020
- Added lightgbm auto-logger
- Fixed an XGBoost callback issue
Release 3.2.6¶
Release date: November 23, 2020
- Added ExistingOfflineExperiment for continuing an experiment in offline mode
- Added Experiment.log_dataframe_profile()
- Extended Experiment.log_table() to be able to log a Pandas DataFrame
- Added ability for the
comet_ml
Keras auto-logger to log gradient and activation histograms in addition to weight histograms - Added ability for comet_ml to run if the
pynvml
package is deleted or doesn't install properly (thanks to the Ludwig team for tracking down issue)
Release 3.2.5¶
Release date: October 26, 2020
- Added support for auto-logging TensorFlow Model Analysis results
Release 3.2.4¶
Release date: October 12, 2020
- Added Hydra config logging (thanks to Vozf)
- Added environment query variable
- Matplotlib's save doesn't flush stream (thanks to jliu)
Release 3.2.3¶
Release date: October 5, 2020
- Fixed
comet models
with Python 3 - Accept unknown from backend query variables
- Added additional Confusion Matrix error logging
- Enhanced index_to_example for Confusion Matrix creation
- Enhanced Pytorch AMP Support
- Allow logging epochs to be floating point values; used by HuggingFace Transformers
Release 3.2.2¶
Release date: September 22, 2020
- Expanded Confusion Matrix to work with integer-based labeled y_true, rather than requiring onehot vectors: see Experiment.log_confusion_matrix() for more information.
- Allow Torch tensors attached to a gradient to work throughout comet_ml (including confusion matrix, histogram logging, etc)
Release 3.2.1¶
Release date: September 18, 2020
- Fixed a confusion matrix issue with non-integer indices (thanks ironcadiz for reporting)
- Refactored and expanded xgboost auto-logging
- Added output refinements when running experiments (reduced duplicated messages, better messages, etc.)
- Fallback to
tf.keras
ifkeras
is not installed (thanks DN6 for reporting) - Replaced comet-git-pure with dulwich
Release 3.2.0¶
Release date: August 28, 2020
- Added major speed-up for logging histograms
- About 10 times faster when used with numpy
- Added refinements for logging metrics with xgboost
- Added refinements for the Predictive Early Stopping
- Added refinements for logging parameters, others, and metrics (thanks for reporting @Idodox!)
- Added better representations of dictionaries and other objects
- Added warning if key or value is truncated
- Dropped support for Python 3.4
Release 3.1.17¶
Release date: August 20, 2020
This release will be the last to support Python 3.4.
- Fixed issue to allow
COMET_AUTO_LOG_DISABLE
to disable allcomet_ml
actions. For more information, see Experiment Configuration Parameters. - Fixed issues using
comet_ml
on Google Colab that required restarting Colab engine:- Can now use Colab's default version of
jsonschema
comet upload
failed without restart
- Can now use Colab's default version of
Release 3.1.16¶
Release date: August 6, 2020
- Refined Experiment(auto_weight_logging=True) for Keras:
- creates separate 3D histograms for weight matrices and bias arrays
- uses layer names for Histogram names
- Added Histogram and image processing speed ups (up to 10x faster)
- Added more flexible logging of parameters
- Added more flexible source code logging for Experiment(log_code=True)
Release 3.1.15¶
Release date: August 3, 2020
- Configurable Experiment arguments. You can now control all of the auto-logged Experiment parameters from your Comet config settings (environment, code, or config file)
- Better logging of model graph representation (can record JSON, yaml, and string representations). Occurs automatically with
Experiment(log_code=True)
or withCOMET_AUTO_LOG_CODE
. - Better logging of source code (can better find source code when using
ludwig
ortransformers
) - Added additional APIExperiment methods: APIExperiment.log_parameters(), and APIExperiment.log_metrics()
- Updated comet-git-pure conda recipe
- Fixed Comet Optimizer document examples
- Confusion matrix optimization: if Experiment.log_confusion(..., selected=[...]) and
index_to_example_function
given, only make those examples selected - Added username to experiment metadata. See APIExperiment.get_metadata()
Release 3.1.14¶
Release date: July 21, 2020
- Added support for keras automatic weight and bias logging as a 3D histogram, see Experiment(auto_weight_logging=True)
Release 3.1.13¶
Release date: July 17, 2020
- Added support for pytorch apex, online-pytorch-lightning-apex-example.py
- Documented metadata for many Experiment methods
- Added new methods APIExperiment.get_curve(), and APIExperiment.get_curves() with step
- Added API.get_project_share_keys(), API.create_project_share_key(), and API.delete_project_share_key() for programmatically sharing projects
- Limited backend handshake message to only show once
- Changed some conversion warnings to debug level
- Added new file download timeout config for downloading assets,
COMET_TIMEOUT_FILE_DOWNLOAD
Release 3.1.12¶
Release date: June 25, 2020
- Add partial support for using the Optimizer with the Pytorch Lightning Comet Logger. See here for an example.
- Add
experiment.display_project
for all kinds of experiments (online, offline and api). Behavior is similar toexpriment.display
. - Add file download timeout configuration,
COMET_TIMEOUT_FILE_DOWNLOAD
with a default value of 600 seconds. This is used only for registry downloading at the moment.
Release 3.1.11¶
Release date: June 17, 2020
- Better logging of all kind of summary data when using
tensorflow.summary
. - Added better error catching in Keras callbacks.
Release 3.1.10¶
Release date: June 9, 2020
- Added
comet models download
andcomet models list
commands, seecomet models
- Added
comet init
command to generate example scripts from cookiecutter recipe, seecomet init
- Removed dependency on
typing_extensions
to allow%pip install comet_ml
to work with Google Colab without a kernel restart
Release 3.1.9¶
Release date: June 6, 2020
- Added Experiment.log_embedding()
- Added Experiment.log_table()
- IPython environments now log code, output, and Code.ipynb
- Added
comet check --debug
command, seecomet check
- Added custom context manager,
with experiment.context_manager(CONTEXT)
- Added support for downloading a model based on its stage, see API.download_registry_model()
- Increased file upload timeout to 15 minutes
Release 3.1.8¶
Release date: May 26, 2020
- Improvements to work better on Google Colab
- Added API.move_experiments()
- Restored ability to
from comet_ml import get_global_experiment
Release 3.1.7¶
Release date: May 18, 2020
- Refined APIExperiment.display()
- Made HTTP timeouts configurable. See
COMET_TIMEOUT_HTTP
- Display a log message when an experiment has been throttled
- Fixed support for Python 2.7 in conda package
- Added new auto-logger for XGBoost
- Added ability to fetch parameters with APIExperiment.get_metrics_for_chart()
- Better invalid model name message: shows possible valid names
Release 3.1.6¶
Release date: April 20, 2020
- Fixed documentation for
ExistingExperiment()
default parameters - Fixed console logging that was cut off
- Added auto-logged cloud metadata (currently AWS, Azure and Google Cloud Platform are supported)
- Added error checking for
API.get_metrics_for_chart()
- Added
Histogram.from_json()
and default args forhistogram.display()
Release 3.1.5¶
Release date: April 14, 2020
- Fixed empty Experiment Summary section headings from showing
- Updated conda's comet_ml dependencies to match pip's comet_ml dependencies
- Fixed reading non-utf8 Unix package descriptions
- Fixed Experiment's
disabled
parameter docstrings - Fixed APIExperiment.set_os() docstrings
Release 3.1.4¶
Release date: April 7, 2020
- Added support for deleting assets from the Python API:
- Added support for setting code from a filename:
- Enhanced Experiment Summary:
- Now shows parameters logged
- Control with
display_summary_level
(display_summary
is now deprecated):- Experiment(display_summary_level=LEVEL)
display_summary_level=0
: display nothingdisplay_summary_level=1
: display metrics, parameters, and upload counts and sizesdisplay_summary_level=2
: everything at level 1, plus system metrics
- Added new method APIExperiment.archive()
- Added checks to prevent logging empty 3D histograms
Release 3.1.3¶
Release date: March 23, 2020
- Ability to control the git directories; see COMET_GIT_DIRECTORY
- Added Experiment.log_curve() and APIExperiment.log_curve()
- Added ability to assign resources per parallel instance when running
comet optimizer
; see COMET_OPTIMIZER_PROCESS_ID - Added Experiment.get_name() and APIExperiment.get_name()
- Many enhancements and speedups
Release 3.1.2¶
Release date: March 12, 2020
- Deprecated
COMET_REST_API_KEY
in Python API; just useCOMET_API_KEY
now - Noted the correct minimum psutil version (5.6.3) in setup.py
- Fixed issue with auto output logging in
OfflineExperiments
from Jupyter - Fixed Keras and Tensorflow model graphs to produce diff-able representations
Release 3.1.1¶
Release date: February 27, 2020
- Added experiment model and registry model support to API, see User Interface/Models
- Added ability to properly set project notes to unicode in Python 2, see API.set_project_notes()
Release 3.1.0¶
Release date: February 17, 2020
- Added MLFlow auto-logging
- Released a Python universal wheel (available via pip) to improve Python 2 installation speed
- Fixed some logging methods that were not respecting Experiment.disabled when True
Release 3.0.3¶
Release date: February 10, 2020
- Added Experiment.log_asset_folder() folder name
- Added API.get_account_details()
- Added logs errors to
OfflineExperiment
- Added Read/write project notes to Python API
- Added
API.server_url
property - Fixed Comet CLI for Python 2; added comet --version
- Added
Experiment.log_model()
, APIExperiment.download_model() and APIExperiment.get_model_names() - Added logging for
optimizer_objective
; use proper JSON for optimizer parameters logging - Fixed convert_to_scalar to handle numpy type
- Ensure
comet optimize
exits with a non-zero exit code if a child exit with an error - Stopped sending sample_rate as metadata when logging an audio file
Release 3.0.2¶
Release date: December 19, 2019
- Added optional metadata to all
Experiment.log_asset_data()
methods - Added 'text' to
Experiment.display(tab='text')
- Updated automatic logging docstrings (thanks to lemairecarl for suggestion)
- Added
Experiment.log_text()
- Added ability to stop a running experiment from Python API
- Added license information
Release 3.0.1¶
Release date: December 10, 2019
- Added Experiment.log_confusion_matrix()
- Added
Experiment.display(TABNAME)
to open browser to Comet tab - Activated Keras and tensorflow.keras progress bars in console
Release 3.0.0¶
Release date: November 3, 2019
- Released REST v2 read and write endpoints
- Released updated Python API and updated APIExperiment
- Dramatically increased speed of Python API by using REST v2
- Added many additional Python API methods (update projects, delete projects, restore experiments, etc)
- Standardized Python API JSON field names to camelCaseNaming
- Added much more documentation and examples for Python API
- Brought APIExperiment inline with
Experiment
,OfflineExperiment
, andExistingExperiment
. Examples:- Create an experiment:
APIExperiment(workspace=WORKSPACE", project_name=PROJECT_NAME)
- Use an existing experiment:
APIExperiment(previous_experiment=EXPERIMENT_ID | NAME)
- Create an experiment:
- Added ability for all aspects of an experiment to be downloaded and changed from the Python API
- New and updated
API
methods:- API.clear_cache()
- API.create_project(WORKSPACE, PROJECT_NAME, [PROJECT_DESCRIPTION, PUBLIC])
- API.delete_project(workspace=WORKSPACE, project_name=PROJECT_NAME, delete_experiments=False)
- API.delete_project(workspace=WORKSPACE, project_id=PROJECT_ID, delete_experiments=False)
- API.get_project(WORKSPACE, PROJECT_NAME)
- API.get_project_by_id(PROJECT_ID)
- API.restore_experiment(EXPERIMENT_ID)
- API.update_project(WORKSPACE, PROJECT_NAME, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
- API.update_project_by_id(PROJECT_ID, [NEW_PROJECT_NAME, DESCRIPTION, PUBLIC])
- API.use_cache(True | False)
- New and updated
APIExperiment
methods:- New and updated read methods:
- APIExperiment.get_additional_system_info()
- APIExperiment.get_command()
- APIExperiment.get_executable()
- APIExperiment.get_gpu_static_info()
- APIExperiment.get_hostname()
- APIExperiment.get_ip()
- APIExperiment.get_max_memory()
- APIExperiment.get_network_interface_ips()
- APIExperiment.get_os()
- APIExperiment.get_os_type()
- APIExperiment.get_pid()
- APIExperiment.get_python_version()
- APIExperiment.get_python_version_verbose()
- APIExperiment.get_system_details()
- APIExperiment.get_system_metric_names()
- APIExperiment.get_tags()
- APIExperiment.get_total_memory()
- APIExperiment.get_user()
- New and updated write methods:
- APIExperiment.log_additional_system_info(KEY, VALUE)
- APIExperiment.log_asset(FILENAME, [STEP, OVERWRITE, CONTEXT, FTYPE, METADATA])
- APIExperiment.log_cpu_metrics(CPU_METRICS, [CONTEXT, STEP, EPOCH, TIMESTAMP])
- APIExperiment.log_gpu_metrics(GPU_METRICS)
- APIExperiment.log_html(HTML, [CLEAR, TIMESTAMP])
- APIExperiment.log_image(FILENAME, [IMAGE_NAME, STEP, OVERWRITE, CONTEXT])
- APIExperiment.log_load_metrics(LOAD_AVG, [CONTEXT, STEP, EPOCH, TIMESTAMP])
- APIExperiment.log_metric(METRIC, VALUE, [STEP, TIMESTAMP])
- APIExperiment.log_other(KEY, VALUE, [TIMESTAMP])
- APIExperiment.log_output(LINES, [CONTEXT, STDERR, TIMESTAMP])
- APIExperiment.log_parameter(PARAMETER, VALUE, [STEP, TIMESTAMP])
- APIExperiment.log_ram_metrics(TOTAL_RAM, USED_RAM, [CONTEXT, STEP, EPOCH, TIMESTAMP])
- APIExperiment.set_code(CODE)
- APIExperiment.set_command(COMMAND_ARGS_LIST)
- APIExperiment.set_end_time(TIME_MILLISECONDS)
- APIExperiment.set_executable(EXECUTABLE)
- APIExperiment.set_git_metadata(USER, ROOT, BRANCH, PARENT, ORIGIN)
- APIExperiment.set_git_patch(FILE_DATA)
- APIExperiment.set_gpu_static_info(GPU_STATIC_INFO)
- APIExperiment.set_hostname(HOSTNAME)
- APIExperiment.set_installed_packages(INSTALLED_PACKAGES)
- APIExperiment.set_ip(IP)
- APIExperiment.set_network_interface_ips(IPS)
- APIExperiment.set_os(OS)
- APIExperiment.set_os_type(OS_TYPE)
- APIExperiment.set_pid(PID)
- APIExperiment.set_python_version(PYTHON_VERSION)
- APIExperiment.set_python_version_verbose(PYTHON_VERSION_VERBOSE)
- APIExperiment.set_start_time(TIME_MILLISECONDS)
- APIExperiment.set_user(USER)
- New and updated read methods:
For more information on the Python API, please see:
Release 2.0.18¶
Release date: November 22, 2019
- Fixed issue when creating multiple experiments sequentially
- Added ability for
Experiment.log_asset_folder()
to log files recursively with file names - Better handling of logging files and file-like objects
- Added ability for users to control auto logging details
Release 2.0.17¶
Release date: November 7, 2019
- Fixed a file-descriptor leak with native console logging
- Fixed
Experiment.log_figure()
to more accurately check for empty figures - Added
psutil
to conda dependencies
Release 2.0.16¶
Release date: October 29, 2019
- Corrected the offline file upload to have the right file extension
- Fixed logging an empty CPU load avg metric
- Fixed optional dependency declaration for the cpu logging feature. You can now use
pip install comet_ml[cpu_logging]
to install psutil. - Fixed a bug in
OfflineExperiment
that removed a file from disk when passingcopy_to_tmp=False
. - Improved file extension handling when uploading file with
OfflineExperiment
.
Release 2.0.15¶
Release date: October 16, 2019
- Added optional timestamp (in seconds) to Python
APIExperiment
:APIExperiment.log_other(..., timestamp=SECONDS)
APIExperiment.log_metric(..., timestamp=SECONDS)
APIExperiment.log_parameter(..., timestamp=SECONDS)
APIExperiment.log_html(..., timestamp=SECONDS)
APIExperiment.log_output(..., timestamp=SECONDS)
- Sped up
Experiment.end()
andOfflineExperiment.end()
- Pinned
jsonschema
to a Python 3.4 compatible version
Release 2.0.14¶
Release date: October 2, 2019
- Refactored Python API
- Added Query API
- Added CPU logging
- requires
psutil
(see above)
- requires
- Deprecated passing experiment keyword arguments to
Optimizer()
; now pass usingOptimizer.get_experiments(**kwargs)
Release 2.0.13¶
Release date: September 16, 2019
- Fixed repeated keras model unable-to-log warnings
- Fixed wrong context in keras callback
Callback.on_test_*()
methods; wastest
nowvalidate
Experiment.log_histogram_3d(values, step)
gives required-step error earlier
Release 2.0.12¶
Release date: September 4, 2019
- Added support for TensorFlow v2 for tensorflow>=1.14, and tensorflow==2.0
- Fixed a bug that was using
auto_param_logging
instead ofauto_metric_logging
to control tensorboard metric logging. - Added
Experiment.log_histogram_3d()
for logging time-series histograms.
Release 2.0.11¶
Release date: August 29, 2019
- Redefined valid experiment key (key must be alphanumeric, 32 to 50 characters)
Release 2.0.10¶
Release date: August 26, 2019
- Added support for all versions of TensorFlow v1 (1.11 - 2.0)
- Added
Experiment.get_callback("tf-keras")
- Fixed spelling errors in messages, comments, and code
- Removed reporting when websocket connection was closed on normal shutdown
Release 2.0.9¶
Release date: August 21, 2019
- Fixed GPU schema to report usage better
- Added
API.add_tags(experiment, tags)
- Fixed offline image upload issue
- Fixed sklearn to respect auto_param_logging flag
- Fixed SOCK HTTP Proxy which required an int port number
Release 2.0.8¶
Release date: August 14, 2019
- Added
{user}
to configurable logging file name patterns - Added
comet offline
to explore offline experiment ZIP files - Added
API.get_experiments()
andAPI.get_experiment()
- Added -m flag to
comet python
- Added COMET_EXPERIMENT_KEY for new experiments
Release 2.0.7¶
Release date: August 13, 2019
- Added
COMET_WORKSPACE
to the config - Added
COMET_CONSOLE
to set the console log level and display tracebacks - Added
COMET_LOGGING_FILE
name patterns, like "comet-{project}.log" - Added
Experiment.set_epoch(NUMBER)
- Added
Experiment.log_others({...})
- Fixed summary metric count
- Deprecated
Experiment.get_keras_callback()
; useExperiment.get_callback("keras")
Release 2.0.6¶
Release date: August 5, 2019
- Added step parameter to all asset logging methods
- Added
Experiment(disable_summary=True)
to disable display summary - Added a log message when an optimizer search has completed
- Added
comet python
CLI to automatically import Comet first - Fixed send_notification compatibility with older backend
- Added conda packaging support
Release 2.0.5¶
Release date: July 18, 2019
- Added step parameter to
Experiment.log_audio()
- Added
Experiment.send_notification()
Release 2.0.4¶
Release date: July 15, 2019
- Added base64-encoded metadata in the log_audio call
Release 2.0.3¶
Release date: July 15, 2019
- Added support for
Experiment.log_audio()
- Added
comet_ml.config.save()
- File upload now sends file extension
Release 2.0.2¶
Release date: July 2, 2019
- Added support for logging binary assets with
Experiment.log_asset_data()
- Added more hook-points for tensorboard auto logging
Release 2.0.1¶
Release date: June 20, 2019
- Fixed tensorflow context logging for Tensorflow >1.13.1
Release 2.0.0¶
Release date: June 18, 2019
- Added a log message when an experiment was created offline
- Added a log message when an experiment was stopped
- Improved stop message
- Removed deprecated methods, this makes this release not backward compatible
- Added a new Optimizer which breaks the older API
- Added new tensorflow hooks to set the context properly
Release 1.0.56¶
Release date: June 5, 2019
- Updated
OfflineExperiment()
to accept the same API arguments asExperiment()