# Release Note
## 2022-03-30
Graviti Data Platform Improvements:
* Added Records on the Activity page to list the new datasets that were forked or created by Filter.
* Optimized users’ login process.
## 2022-03-10
Graviti Data Platform New Feature:
* Added compute management function. The compute consumed by Offline Filter, Cross-version comparison, Action, Sextant Model Evaluation and Squash\&Merge will be automatically calculated and the results will be displayed on the dashboard page and Your Asset page for viewing and management.
Graviti Data Platform Improvements:
* Optimized the Version Control function. Simplify users’ operation process by putting all the functions of Version Control on a single management page.
## 2022-02-15
Graviti Data Platform Improvements:
* Fixed bugs on data capacity statistics.
* Optimized the login UI of Graviti data platform.
* Added Crontab drop-down box component in the scheduled trigger. Optimized the input box to support special characters “?” “\”.
## 2022-01-24
Graviti Data Platform New Features:
* Added scheduled trigger in Action. Users can make workflows run on schedule.
* Added Activity Registration on the Website Home Page. Users can enter the Event Detail page for more information, registration and documents downloading.
Graviti Data Platform Improvements:
* Optimized features of the Data Dashboard. Users can view monthly Used Data Volume on the Details page.
* Optimized the copywriting of Sextant, changing Evaluation to Benchmark.
* Added Select All option on the Delete Data page to help users quickly delete the data.
* Added the ISO9001 certification icon and modified relative copywriting on the Data Hosting page.
* Optimized the User Interface of GroundTruth on the Select Dataset page. Added details of the Commit, including Type, Amount, Creator and Updated Time, to help users quickly choose datasets. Added Paging function to improve the user experience.
Python SDK New Features:
* Add open dataset loaders `VGGFace2`
Python SDK Bug Fix:
* Fix reading cached file failed in a multiprocess environment
* Fix the J`ob.started_at` will not be updated
* Fix the `ValueError` when init `PointList2D` and `RLE` with Numpy array
* Fix the `NoneType` error in `Job.update`
Python SDK Improvements:
* Remove the multiprocess lock and add PID to cached file path
## 2021-12-30
Graviti Data Platform New Feature:
* Optimized the cross-version comparison function. Graviti Data Platform now supports cross-version comparison by selecting the corresponding commit by branch and tag.
Python SDK New Features:
* Implement the async job framework
* Implement `JobMixin._create_job`
* Implement `JobMixin._get_job`
* Implement `Job.updat`
* Implement `JobMixin.delete_job`
* Implement `Job.retry` and `Job.abort`
* Implement `JobMixin._list_job`
* Implement SquashAndMerge feature by async job framework
* Implement `SquashAndMerge.create_job`
* Implement `SquashAndMerge.delete_job`
* Implement `SquashAndMerge.get_job`
* Implement `SquashAndMerge.list_jo`
* Implement `SquashAndMergeJob.result`
* Add the following open dataset loader:
* `CityscapesGTFine`
* `CityscapesGTCoarse`
* `RarePlanesRea`
* `RarePlanesSynthetic`
* `UrbanObjectDetection`
Python SDK Improvements:
* Getting mask URLs from OpenAPI `getDataDetails` to reduce the number of requests
Python SDK Documentation:
* Update the docs about "Squash And Merge" in "Features"
* Refine the docs
* Refine the docstring
Python SDK Documentation:
* Change the Sphinx Html theme to "furo"
* Change the Sphinx Html theme to "furo"
## 2021-12-17
Python SDK Bug Fix:
* Fix `RemoteFileMixin._urlopen()` failed when URL is expired
## 2021-12-16
Graviti Data Platform New Feature:
* Added secrets in Action. Users can configure their secrets in the settings of Action. The secrets that users configure are available to use in Action Yaml files. (only available for administrators)
Python SDK New Features:
* Add interfaces to communicate with Sextant Apps
* Implement `Sextant.list_benchmarks`
* Implement `Sextant.list_evaluations`
* Implement `Sextant.create_evaluations`
* Implement `Evaluation.get_result`
* Add `SemanticMask` for `BDD100K_MOTS2020` dataset
* Add `author`, `updated_at` and `parent_commit_id` for `Draft`
* Add `--show-drafts` for CLI `gas log` to support displaying open drafts
Python SDK Improvements:
* Remove the redundant attributes in `BDD100K` dataset
* Update `VersionControlClient` to `VersionControlMixin`
* Use `stem` to represent the filename without extension in `opendataset` module
Python SDK Bug Fixes:
* Add the missing `categoryDelimiter` to the `OxfordIIITPet` catalog
* Fix `NoneType` not subscriptable error in `listMaskUrls`
* Fix the `AttributeError` when calling the `mocker` class methods
* Correct the wrong attribute names in `nuImages` catalog
* Add "PaddlePaddle" instructions in `INTEGRATIONS` chapter
* Add documentation for `Sextant` in `Applications` chapter
* Reconstruct `Storage Config` in `Advanced Features` chapter
* Support deleting automatically generated rst files via `make clean`
* Refine the docstring
## 2021-12-07
Python SDK Bug Fixes:
* Fix the request timeout issue when reading data in multiprocess environment
* Fix the `TypeError` caused by positional arguments mismatch when using `Profile`
## 2021-12-03
Graviti Data Platform New Features:
* Add a data dashboard to enable users to view the data storage capacity of each workspace and used data volume
* Add an email subscription portal for users to get our latest updates
Graviti Data Platform Improvements:
* Optimize the creation steps of datasets and user interactive experience
* Optimize the functions of Action. Add a manual option, YAML code templates, and a stop workflow button (this function is only available to the team admin)
Python SDK New Features:
* Add `--sort` option for CLI `gas branch` and `gas tag` to support sorting branches and tags
* Support recording statistical info for `Data.open().read()` in `Profile` class
* Add `StorageConfig` class to store storage config info
* Add `SegmentClient.get_data` to get data info by its `remote_path`
* Support showing a piece of data via `remote_path` in CLI `gas ls`
Python SDK Improvements:
* add `URL` class for getting and updating the file URL
* Add `train`, `val` and `test` in segment name in `DAVIS2017`
Python SDK Bug Fixes:
* Fix the wrong graph in CLI `gas log` when merging branches
* Add the missing `categoryDelimiter` in `COCO2017` catalog
* Fix the `FileExistsError` when using cache with multiprocess
Python SDK Documentation:
* Use `autosummary` to generate `rst` files in `opendataset` module under "API" chapter
* Refine the docs
* Refine the docstring
## 2021-12-01
Python SDK Improvement:
* Make `timeout` setting in `tensorbay.client.config` take effect for `urlopen`
## 2021-11-17
Graviti Data Platform Improvements:
* Optimize the interface and interactivity of the Apps market
* Optimize the display of the data dashboard
Python SDK New Features:
* Add checking and warning for free storage space when enabling cache
* Add the following open dataset loaders:
* `SCUT_FBP5500`
* `DAVIS2017`
Python SDK Improvements:
* Add argument `encoding='utf-8'` to open() to make it compatible for Windows
* Use `xmltodict` to parse xml files instead of `xml.ElementTree`
* Add file `tensorbay/py.typed` to comply with PEP-561
Python SDK Bug Fix:
* Fix `Data.open` catch exception failed when url is expired
Python SDK Documentation:
* Reconstruct "Version Control" chapter
* Fix the typo in the docs
* Refine the docs
* Refine the docstring
Python SDK Deprecation:
* Remove the deprecated method `GAS.create_auth_dataset`
## 2021-11-02
TensorBay Bug Fix:
* Fix some bugs to optimize the user experience
Sextant Improvement:
* Add the Retry button to allow a dataset to be evaluated again after failing
Python SDK New Features:
* Support setting `is_public` in `GAS.create_dataset`
* Support importing authorized cloud files to fusion dataset
* Support cache when opening remote data
* Add `DatasetClientBase.get_total_size` to get the total data size in a dataset commit
* Add `VersionControlClient.squash_and_merge` to merge two different branches
Python SDK Improvements:
* Use `os.path.expanduser` to handle the CLI config file path in different systems
* Replace relative import with absolute import for all python files in SDK
* Remove the useless checks about commits and drafts in `ls.py`
* Add `X-Request-Id` to all the TensorBay OpenAPI request headers
* Add a detailed resuming message when uploading process got interrupted in CLI `gas cp`
Python SDK Bug Fixes:
* Add a workaround for the `IndexError` when loading the data with panoptic mask
* Remove the wrong error message in `gas dataset` when `tbrn` is absent
Python SDK Documentation:
* Add docs about the data cache
* Add docs for using `pharos` on remote server
* Refine the docs
## 2021-10-18
Python SDK New Features:
* Display elapsed time of the request in the debug log
* Use `response.elapsed` to get the cost time of a request in `profile`
* Add `DatasetClientBase.get_label_statistics` to get the label statistics info
* Support updating file urls when the urls are expired
* Add open dataset loaders `SegTrack2`
Python SDK Improvements:
* Consolidate all config-related functions in `ContextInfo`
Python SDK Performances:
* Adapt OpenAPI `getDataDetails` to increase the speed of reading labels and URLs
* Upload auth cloud storage data labels on the cloud-callback OpenAPI to increase the uploading speed
Python SDK Documentation:
* Add "Update Dataset Notes" section in "Update Dataset" chapter
* Add docs about cloud file in-place importing in "Cloud Storage"
* Add docs about getting the label statistics info in "Get Label Statistics"
* Refactor the "Version Control" chapter
* Refine the docs
## 2021-09-28
TensorBay Enhanced Features：
* Add discussion function on the public dataset details page in the mobile terminal
* Optimize the interaction of data list search
* Divide Apps into different types according to their attributes on the Apps Market page
Python SDK New Features:
* Support filtering drafts by `status` and `branch_name` in`VersionControlClient.list_drafts`
* Add class `Profile` to record the statistical information about the HTTP requests in SDK
* Support using `Profile` in the multi-process environment
* Support saving `Profile` summary to `csv`, `txt` or `json` file
* Add the following `BDD100K` related open dataset loaders
* `BDD100K`
* `BDD100K-10K`
* `BDD100K-MOT`
* `BDD100K-MOTS`
* Add the following open dataset loaders:
* `SegTrack`
* `COCO2017`
* Unify the format of prompt messages in CLI
Python SDK Bug fixes:
* Fix the dataset loader `LIP` and `CIHP` cannot be imported from `opendataset` module
* Correct the wrong visible status in `LIP` dataset loader
Python SDK Improvements:
* Unify the variable names about `directory` and `tbrn`
* Use `ValueError`, `StatusError` to replace `OperationError`
Python SDK Documentation:
* Add examples about `SemanticMask` and `InstanceMask`
* Add catalog structure for different label types
* Add docs about how to use `Profile`
* Fix the typos in docs
* Refine the docs
Python SDK Deprecations:
* Remove the deprecated class `ResponseSystemError`
## 2021-09-14
Sextant New features:
* Support customized metrics by uploading Github Repo Url into Sextant
* Add more data types that can be evaluated
TensorBay Enhanced Feature：
* Support creating new datasets by merging datasets via Python SDK
Python SDK New Features:
* Support creating a dataset with different cloud storage configs in`GAS.create_dataset`
* Add the following methods to create customized cloud storage configs for different cloud services:
* `GAS.create_oss_storage_config`
* `GAS.create_s3_storage_config`
* `GAS.create_azure_storage_config`
* Add `GAS.delete_storage_config` to delete customized cloud storage config
* Enable the method `SegmentClient.delete_data` to delete data
* Add `FusionSegmentClient.delete_frame` to delete frame
* Support getting and updating dataset `isPublic` flag
* Add the following open dataset loaders:
* `CIHP`
* `VOC2012Segmentation`
* `PASCALContext`
* `LIP`
* `OxfordIIITPet`
* `nuImages`
* Print more friendly error messages in CLI
Python SDK Interface Adjustments:
* Set current revision as the default in `VersionControlClient.list_commits`
* Disable method `FusionSegmentClient.delete_data` to stop deleting data in Fusion dataset
Python SDK Improvements:
* Prohibit closing current draft in `VersionControlClient.close_draft`
* Avoid getting `done_frames` when `skip_upload_files` is `False` in `FusionDatasetClient.upload_segment`
Python SDK Documentation:
* Add docs about "Update Dataset Meta"
* Add docs about "Delete Frame"
* Fix the typos in docs
* Refine the docs
Python SDK Deprecations:
* Remove the deprecated code before v1.10.0
* Deprecate `GAS.create_auth_dataset` and use `GAS.create_dataset` instead
## 2021-08-30
TeraGood New Feature：
* Support editing labels of the fusion dataset in TeraGood
TensorBay Enhanced Feature:
The data list is optimized from a one-dimensional list to a two-dimensional list. Users can view more details of the data list via subtables. Support users to switch versions, view information, search, view catalog, show diff and do other operations on the data list page.
Python SDK New Feature:
* Support new label types: **SemanticMask**, **InstanceMask** and **PanopticMask**
Python SDK Interface Adjustments:
* Remove the `loads` method for `DataBase`, `Data` and `AuthData`
* Add `get_callback_body` method to replace `dumps` in `Data`
* Add `from_response_body` method to replace `loads` in `RemoteData` & `Frame`
Python SDK Bug Fixes:
* Add a workaround for `MalformedPOSTRequest` error when posting files to Aliyun OSS
Python SDK Improvements:
* Add `FileMixin` and `RemoteFileMixin` for file-related methods
* Support setting custom loader and dumper in `AttrsMixin`
Python SDK Documentation:
* Add docs about `SemanticMask`, `InstanceMask` and `PanopticMask`
## 2021-08-17
New Features:
* Data storage configuration supports developers to choose different regions and team members to choose different cloud platforms or regions as the default storage configuration.
* Add data usage statistics function. Users can view the used data volume on the GAS homepage and the assets page.
* Support data synchronization within GroundTruth Tools. GroundTruth Tools support the 2d/3d fusion labeling tools and can synchronize the labeling data to Tensorbay.
* Support booking demo on the GAS home page.
Enhanced Feature:
* Support to select, modify server configuration and record parameters in Action.
Python SDK New Features:
* Support new label types **Polygon** and **Polyline2D**, add five label formats:
* `Polygon` + `MultiPolygon` + `RLE`
* `Polyline2D` + `MultiPolyline2D`
* Add the following open dataset loaders:
* `VOC2012Detection`
* `VOC2012ActionClassification`
* `CCPD` & `CCPDGreen`
* Add name conflict check for `SortedNameList`
* Support setting network request config (`is_internal`, `max_retries`, and `timeout`) in CLI `gas config`
* Support to validate the AccessKey and display the user info in CLI `gas auth`
* Add `--status` option for CLI `gas auth` to display the user and auth info
* Support displaying error messages for CLI `gas auth`
* Add `-l` option for CLI `gas ls` to show the total number of resources
* Add `--all` option for CLI `gas log` to show the commits in all branches
* Add `--graph` option for CLI `gas log` to show the graphical commits
* Set `-h` as an abbreviation of `--help` to show CLI help messages
Python SDK Interface Adjustments:
* Remove the `path` argument in `GAS.create_auth_dataset` because the `path` is bound to the cloud storage config
Python SDK Bug Fixes:
* Add a workaround for the issue Chinese characters display garbled in the Windows system
Python SDK Improvements:
* Use list comprehension in `PointList2D.__init__` instead of for-loop
* Set `None` as the default value of the argument `alias` in `GAS.update_dataset`
* Correct the illegal remote paths for `RP2K` dataset loader
* Encapsulate the moving segment logic into `DatasetClientBase._move_segment`
* Move the deprecation related classes into `deprecated.py`
* Stop showing the redundant field `name` in `AttributeInfo .__repr__`
* Cleanup the following useless codes:
* Remove the useless class variable `_label_attrs` from label related classes
* Remove the useless class `SubcatalogTypeRegister`
* Remove the useless class `LabelType`
* Remove the useless type `Subcatalogs`
Python SDK Documentation:
* Add docs about "Shell Completion" in the "CLI" section
* Add docs about new label formats **Polygon** and **Polyline2D**
* Add docs about setting network request config in CLI `gas config`
* Add docs about `--all` and `--graph` options in CLI `gas log`
* Fix the typo in docs
* Refine the docs
## 2021-08-04
New Features:
* Add model evaluation tools Sextant. Sextant supports quick evaluation of data and models, enabling you to gain insight into data quality and model accuracy, and providing references for data quality improvement.
* Add advanced search. Users can upload Github URL to get the corresponding file and set the file as a filter.
* Support modifying and conducting quality assurance of labels of 2D Box and 2D Polygon in TeraGood and GroundTruth.
Enhanced Feature:
* Optimize data search. Data search is performed offline now. Users can view the search progress and results and save search results for creating new datasets and so on.
Python SDK New Features:
* Support `override` and `skip` strategies in `SegmentClient.move_data`
* Add `GAS.get_user` to get the current user info
* Support manipulating dataset alias:
* Support getting the dataset alias in `GAS.get_dataset`
* Support setting the dataset alias in `GAS.create_dataset`
* Add `GAS.update_dataset` to update the dataset alias
* Support setting and showing `description` in CLI `gas draft`
* Add `--edit` option in CLI `gas draft` to edit a draft
* Add `--close` option in CLI `gas draft` to close a draft
* Add draft description to the pop-up editor of `gas commit` as default message
* Display the corresponding message after deleting data or a segment in `gas rm`
Python SDK Interface Adjustments:
* Change the signature of `Transform3D.set_rotation` and `Sensor.set_rotation`
Python SDK Improvements:
* Unify the `description` display logic in `gas draft` and `gas commit`
* Set the file name as `""` when posting files to cloud storage
* Add utility function `chunked` to break an iterable into length n tuples
* Set the minimum version of `urllib3` back to v1.15
Python SDK Performance:
* Speed up file uploads for fusion dataset in `GAS.upload_dataset` by adapting OpenAPI `multiCallback`
* Speed up file uploads by sending file size to TensorBay
* Speed up file uploads by enlarging the batch size of `multiCallback`
Python SDK Documentation:
* Add the docs about `--edit` and `--close` option in CLI `gas draft`
* Add the docs about CLI "Profile"
* Refine the docs
* Fix the typo in docstrings
Python SDK Deprecations:
* Remove the default value of the argument `title` in `VersionControlClient.create_draft`
* Remove the deprecated message about setting AccessKey in `gas config`
## 2021-07-16
Python SDK bug fixes:
* Fix CLI `gas log --oneline` that displays all commits in oneline
Python SDK Documentation:
* Replace `-t` with `-m` in the docs of CLI `gas draft`
* Add details about `strategy` in the docs of copy and move operations
* Fix the `Synopsis` section display error in CLI `gas commit --help`
* Use batch move in the example code of moving data instead of one by one
## 2021-07-14
New Features:
* Diff to compare data of different versions, view the difference between draft and parent commit, or between commit and parent commit, and visualize the corresponding data and tag distribution.
* Add or edit multiple tags for each data version, making it easy and fast to retrieve and manage.
Enhanced Feature:
* Teragood supports modifying existing annotated data and performing quality assurance on the data.
Python SDK New Features:
* Add `Data.get_url` to get the `file://` url for a local file
* Support more operations on draft:
* Add `status` and `description` to the `Draft` class
* Add `VersionControlClient.update_draft` to update `title` and `description` of a draft
* Add `VersionControlClient.close_draft` to close a draft
* Support setting `description` in `VesionControlClient.create_draft`
* Add `--message` option in CLI `gas draft` to set `title` and `description` of a draft
* Add `SegmentClient.list_urls` and `FusionSegmentClient.list_urls` to list the file urls
* Add the following open dataset loaders:
* `UAVDT`
* `CACD`
* `AADB`
* `COVID_CT`
Python SDK Bug fixes:
* Fix the `ImportError` when `import tensorbay` in python 3.6
Python SDK Improvements:
* Add `shorten` function to get the short commit ID
* Merge the `CLASSIFICATION` label into `BOX2D` label in the `CompCars` open dataset
* Remove useless `DatasetClient.import_all_files` interface
* Correct the illegal segment names in all the open dataset loaders
* Move customized `click` classes from `cli/cli.py` to `cli/custom.py`
* Stop showing the `description` in `Commit.__repr__`
* Exclude useless `docs` and `tests` modules from the `tensorbay` package
Python SDK Performance:
* Speed up file uploads in `GAS.upload_dataset` by adapting OpenAPI `multiCallback`
* Enhance the performance of `RemoteData.open` by batch requesting file urls
Python SDK Documentation:
* Add `Synopsis` section to CLI `--help` message
* Refine the CLI docs
Python SDK Deprecations:
* Deprecate the `--title` option in CLI `gas draft`
* Remove the deprecated CLI `gas create` and `gas delete`
* Remove the deprecated exception `CommitStatusError`
## 2021-06-22
New Features:
* Version control of datasets based on branches.
* Manage and edit datasets collaboratively with draft.
* Add operation guide on the visualization widgets Pharos.
* Developers edition users can now manage members and share "Viewer" permission for private datasets.
Enhanced Features:
* Optimized the performance and operation of dataset version control.
* Optimized the interface and UI elements of the Dashboard User Guide.
## 2021-06-03
Python SDK Bug Fixes:
* Fix the `AttributeError` when using `DatasetBase.keys()` to list segment names
## 2021-05-31
New Feature:
* Pharos visualization widgets can display JSON text files.
Enhanced Feature:
* Optimized the multi-level data interface in the list mode of the label distribution visualization widgets, and now you can directly view the distribution of sub-level data in the overall data.
Python SDK New Features:
* Redesign and implement the `gas` CLI based on TensorBay version control :
* Support `draft number` and `revision` in `TBRN`
* Add `gas dataset` command to create, list and delete dataset
* Support `gas ls` command to list segments and data in specific draft or revision
* Add `gas draft` command to create and list drafts
* Add `gas commit` command to commit draft
* Add `gas cp` command to upload files to draft
* Add `gas rm` command to remove segment and data in draft
* Add detailed resuming message when uploading process got interrupted
* Add `ModuleImportError` to print detailed install instruction when the optional requirement package is not installed
* Implement `DatasetBase.__delitem__` to support delete segment from dataset by `del`
* Add open dataset loader `LISA Traffic Sign`
Python SDK Bug fixes:
* Fix the `ValueError` when passing the last item to `UserSequence.index`
Python SDK Improvements:
* Remove the redundant class variable `description` in `KeypointsInfo`
* Implement python attr framework `AttrsMixin`
* Move `TBRN` from `utility` module to `cli` module
* Inherit `EqMixin` to replace `__eq__` in following intrinsic classes :
* `CameraMatrix`
* `DistortionCoefficients`
* `CameraIntrinsics`
Python SDK Documentation:
* Refactor CLI related docs
* Add `Integrations` chapter for `PyTorch` and `TensorFlow` integration
* Refine the docs
* Fix the typo in docs
Python SDK Deprecations:
* Deprecate CLI `gas create` and `gas delete`
* Deprecate legacy fusion dataset `TBRN`
* Remove the following deprecated interfaces :
* Following exceptions：
* `GASDatasetError`
* `GASDatasetTypeError`
* `GASException`
* `GASPathError`
* `GASResponseError`
* `GASSegmentError`
* The `start` and `stop` keyword arguments in following methods:
* `GAS.list_dataset_names`
* `DatasetClientBase.list_drafts`
* `DatasetClientBase.list_commits`
* `DatasetClientBase.list_tags`
* `DatasetClientBase.list_branches`
* `DatasetClientBase.list_segment_names`
* `SegmentClient.list_data_paths`
* `SegmentClient.list_data`
* `FusionSegmentClient.list_frames`
* Following methods：
* `DatasetClientBase.list_draft_titles_and_numbers`
## 2021-05-17
New Features:
* You can use developer tools to access the Pharos visualization widget to visualize local data. For details, please refer to the document.
* Launched the dataset discussion function. Discussions can be posted for all the open datasets with @ and Like functions being added. You can @ dataset publishers or other relevant users to invite them to join the discussion and give your Likes to excellent content.
* Released a new version of the dashboard and added the user guide to help users quickly understand the features of TensorBay and the way to access them.
* Actions: You can monitor changes in datasets by creating a workflow, and automatically trigger a pre-defined data processing program to quickly build an automated data processing procedure (in internal testing).
Enhanced Features:
* Optimized the UI design and interface of the label distribution visualization widgets.
* Optimized the interaction of the Pharos visualization widgets on the data filters page, and you can directly view the annotation types in the filters by default.
Python SDK New Features:
* The local visualization plugin `pharos` is released, it can be installed by `pip3 install pharos`:
* Pypi:
* Docs:
Python SDK Bug fixes:
* Fix the `JSONDecodeError` occurred when uploading dataset
* Fix the `AttributeError` when using `CADC` dataloader in python3.6
* Fix the `AttributeError` when resuming upload of fusion dataset
Python SDK Documentation:
* Add docs for local visualization plugin `pharos`
## 2021-05-15
Python SDK New Features:
* Make `PagingList` mutable, and follow `MutableSequence` protocol
* Support reading remote data lazily in `Dataset` and `Segment`:
* Lazy evaluation in `Segment`
* Lazy evaluation in `Dataset`
* Support getting segment by name in `DatasetBase.__getitem__`
* Use `segment = dataset["test"]` to get segment by name instead of `segment = dataset.get_segment_by_name("test")`.
* Use `segment_names = dataset.keys()` to get all segment names in a dataset.
* Add the following methods to convert between `category` and `index` for writing training code easier
* `CategoriesMixin.get_category_to_index`
* `CategoriesMixin.get_index_to_category`
* Add the following exceptions as subclasses of `ResponseError`
* `AccessDeniedError`
* `InvalidParamsError`
* `NameConflictError`
* `RequestParamsMissingError`
* `ResourceNotExistError`
* `ResponseSystemError`
* `UnauthorizedError`
* Support `skip_uploaded_files` flag in `GAS.upload_dataset` for fusion dataset
* Add open dataset loader `COVID-chestxray` and `nuScenes`
Python SDK Improvements:
* Refactor CLI related code to a new module
* Stop checking the `commit_id` in `DatasetClientBase.__init__` to avoid sending a redundant request
* Fix the possibly unbound variable warning in `CompCars`
Python SDK Documentation:
* Add `Update Dataset`, `Update Label` and `Update Data` chapter
* Add docs for specific response exceptions
* Add `continuity` and `tracking` in glossary
* Update docs for reading segments from lazy evaluation `Dataset` rather than `DatasetClient`
* Refine the example in docs
Python SDK Deprecations:
* Deprecate `DatasetBase.get_segment_by_name`
## 2021-05-12
Python SDK Interface Adjustment:
* Rename the following exceptions
* `TensorBayClientException` -> `ClientError`
* `TensorBayOpendatasetException` -> `OpenDatasetError`
## 2021-05-06
New Features:
* Filter by task types on the open datasets list page, so that you can quickly find the required dataset.
* You can modify the dataset name, URL and alias.
* A guide to help new users fill in basic information.
* Dataset discussion tab. You can post discussions on datasets (in internal testing, and currently available for MINST, BDD100K, nuScenes, CityScapes, KITTI-Object, CIFAR-10, and COCO).
* Fork records of the open datasets.
Enhanced Features:
* Optimized the UI design and interface of the data list page, and added filter logic for annotation information.
* Optimized the UI design and interface details of the visualization widgets.
Python SDK New Features:
* Implement `PagingList` to lazy access elements from paging requests, which follows `Sequence` protocol
* Use `PagingList` as the return value instead of `iterator` for the following methods:
* `GAS.list_dataaset_names`
* `DatasetClientBase.list_drafts`
* `DatasetClientBase.list_commits`
* `DatasetClientBase.list_tags`
* `DatasetClientBase.list_branches`
* `DatasetClientBase.list_segment_names`
* `SegmentClient.list_data_paths`
* `SegmentClient.list_data`
* `FusionSegmentClient.list_frames`
* Support creating a dataset with auth cloud storage:
* Add `GAS.get_auth_storage_config` and `GAS.list_auth_storage_configs`
* Add `GAS.create_auth_dataset`
* Add `client.config` to adjust the request configuration
* Support adjusting the config of the request retry strategy: `config.timeout`, `config.max_retry` etc.
* Support uploading/reading data from cloud server internal endpoint
* Build TensorBay exception system:
* Add `TensorBayException` as TensorBay base exception
* Add `NoFileError` and `FileStructureError` for `opendataset` module
* Add `CommitStatusError`, `ResponseError`, `DatasetTypeError` and `FrameError` for `client` module
* Add `TBRNError` for `CLI`
* Add `bin_point_cloud_fields` to dataset `Notes` to support `.bin` format point cloud with different fields
* Display the upload progress by process bar in `GAS.upload_dataset` and `(Fusion)DatasetClient.upload_segment`
* Add open dataset loader `opendataset.HalpeFullBody` and `opendataset.BioIDFace`
Python SDK Improvements:
* Update open dataset name according to TensorBay latest naming rule
* Display `commit_id` in `Commit.__repr__`
* Refine the error message of unexpected status code
* Delete duplicated loads `lexicon` code on `SentenceSubcatalog`
Python SDK Documentation:
* Add docs for "Cloud Storage"
* Add docs for "Request Configuration"
* Add docs for "Use Internal Endpoint"
* Add docs for TensorBay exception system
* Add docs for `bin_point_cloud_fields`
* Add `tag` to `glossary` docs
* Refine the docstring and docs
* Fix typos in docstring and docs
Python SDK Deprecations:
* Deprecate the `start` and `stop` arguments for methods using `PagingList` as a return value
* Deprecate the following legacy exceptions in `client` module :
* `GASDataTypeError`
* `GASLabelsetError`
* `GASLabelsetTypeError`
* `GASFrameError`
* `GASResponseError`
## 2021-04-21
Python SDK Bug Fixes:
* Fix `UnboundLocalError` when calling `DatasetClientBase.create_tag` with `revision`
* Fix `KeyError` when getting a `FusionSegment` instance
## 2021-04-19
New Features:
* You can view multiple annotation types simultaneously with the visualization widgets.
* You can apply for or share the dataset permissions. The dataset Admin can give users the corresponding permissions of the dataset.
* Dataset permissions and team management log.
* Automatically generate PythonSDK code and display API on the dataset details page.
Enhanced Feature:
* Optimized the UI design of the product pricing page.
Python SDK New Features:
* Add basic methods for dataset Version Control
* Add `DatasetClientBase.create_tag`
* Add `DatasetClientBase.list_tags`
* Add `DatasetClientBase.get_tag`
* Add `DatasetClientBase.delete_tag`
* Add `DatasetClientBase.list_branches`
* Add `DatasetClientBase.list_commits`
* Add `DatasetClientBase.get_draft`
* Support dataset with continuous data
* Add `Notes` to store dataset basic information
* Add `DatasetClientBase.update_notes`
* Add `DatasetClientBase.get_notes`
* Support "Fusion Dataset"
* Make request session safe in multiprocesses environment
* Add SDK version info to "Event Tracking" and support "Event Tracking" for CLI
* Add loader for CADC opendataset
* Add method `create_segment()` for dataset client
* Add `__eq__` for `Label`
Python SDK Improvements:
* Modify Tensorbay gateway to "gatewayv2"
* Refine the NotImplemented logic in binary operators
* Check whether the url starts with "https\://" in `Client`
* Implement decorator `Deprecated` for deprecated functions
Python SDK Packaging:
* Add minimum versions for dependencies
Python SDK Documentation:
* Add docs for dataset Version Control
* Add docs for "Fusion Dataset"
* Add docs for dataset with continuous data and `Notes`
* Update urls in docs
* Refine the docstring and docs
* Fix typos in docs
* Add examples to docstring
Deprecations:
* Deprecate `DatasetClientBase.list_draft_titles_and_numbers`, use `DatasetClientBase.list_draft` instead
## 2021-03-25
Python SDK New Features
* Support uploading data to Azure
Python SDK Fix Bugs
* Correct dataset names in data loaders according to Graviti Open Dataset
Python SDK Documentation
* Fix typos in docs
* Remove the `url` related content in the doc of CLI
* Refine the content of docs and docstring
* update the URLs in docs and "README.md"
* Fix typos in docs
## 2021-03-24
New Features:
* Visibility Settings. You can set the visibility of a dataset as public or private.
* Team Collaboration. You can create teams and invite members to their teams.
* Individual & Team Profiles
* Data visualization is available after data list filtering.
Enhanced Features:
* Upgraded UI design
.
* Optimized the process of creating new datasets and unified the interface of creating datasets (filter, merge, fork, etc.).
2021-02-19
New Feature:
* 2D Keypoints type for data visualization widgets.
2021-02-01
New Features:
* Sensor display for data visualization. The corresponding sensor name can be clearly viewed when displaying multi-sensor data.
* Light-color mode for data visualization. Users can switch between Light and Dark Mode.
Enhanced Features:
* Optimized the UI, interaction and loading speed of data visualization.
* Optimized the UI and interaction of the distribution visualization.
2021-01-25
New Features:
* Users can select the region where the Bucket is stored when uploading data on TensorBay.
* Version Control for TensorBay cloud authenticated data and AWS authentication.
* You can merge the annotations of the same type and the same label table.
* "Closed" status in Feedback.
Enhanced Feature:
* Optimized the display of text and images in the Feedback.
2021-01-05
New Features:
* Users can merge data from different datasets on TensorBay.
* Version Control for datasets after the annotations are synced.
* Designated reply in the Feedback Module enables users to reply to a specific message.
* "Disable Reminders" of the reply pop-up window in the Feedback Module.
Enhanced Features:
* Optimized the display of the Feedback Module on the mobile side.
* Optimized the statistical mode of the attribute data of the label distribution visualization widgets.
2020-12-21
New Features:
* Graviti Open API, please refer to Graviti Open API for details.
* Full-screen function for visualizing label distribution.
Enhanced Features:
* Optimized the UI design of the label distribution visualization widgets.
* Optimized the user experience of adding feedback.
2020-12-02
New Features:
* Users can create a new dataset by filtering data.
* Dataset types and custom labels are available when creating a new dataset.
* Audio data visualization is available for MP3 and WAV files. Added two annotation types: transcription and classification.
* Text data visualization is available for txt files and their annotation classification.
* You can visualize attribute statistics of data annotation.
Enhanced Features:
* Optimized the logic of deleting data.
* Removed the corresponding annotations when a file is deleted.
* Raw data that has already been cited will not be deleted.
* Versions can be deleted.
2020-11-18
New Features:
* Access TensorBay via cloud storage authentication.
* Graviti Developer Tools menu and documentation are added to introduce the use of SDK and other tools.
* Code samples for the use of API.
* Feedback board allows you to post feedback and suggestions, communicate in real time, and send messages through WeChat official accounts.
* Pie chart mode to visualize data distribution.
Enhanced Features:
* Optimized the Help Doc and highlighted the developer documentation.
* Optimized the UI design and interface of the official website.
2020-11-09
New Features:
* Version Control for Datasets & Annotation sets. Raw data and annotations are included when the dataset version is released.
* Open datasets can be forked and accessed on GAS where users can release versions, upload data and perform other operations.
* Project 3D point cloud annotation info onto the corresponding 2D image, visualize Gzip compressed point cloud and select the annotation box of 3D view to automatically turn the view angle.
* View sub-category with the data distribution visualization widgets.
* The order management system where you can purchase team seats and expand TensorBay capacity.
Enhanced Feature:
* Optimized account types to "Developers" and "Teams". Visit the link for details.
2020-10-19
New Features:
* On TensorBay My Datasets, users can manage raw data versioning, create new versions, and continue to edit and modify the data of the dataset after creating a version.
* The file list on the left side of the visualization widgets helps users quickly locate and switch the displayed files.
* 2D Classification, 3D type, Tracking type data and its annotations.
* You can view multiple annotations in the open datasets with the visualization widgets.
Enhanced Features:
* Optimized the return function of visualization widgets when sharing to off-site.
* Optimized the release page UI of GroundTruth My Annotation Order.
2020-09-14
New Features:
* You can filter data with segments and view data list on My Datasets.
* You can view the annotation details of open datasets, filter list and label, and visualize annotations and label distribution.
* You can now download final performance review on GroundTruth.
* You can download the JSON file of the project output in advance during the final review or approval phase on GroundTruth.
2020-08-24
New Features:
* Access to open datasets. On the GAS platform, users can acquire open datasets to view, download and visualize.
* Data visualization widgets: 2D rectangular boxes, 2D polygons, and 2D classification. The widgets can check the annotation quality and can be shared off-site.
* Data distribution visualization widgets. Users can view the label distribution of a dataset.
2020-07-21
New Features:
* AccessKey: When using PythonSDK, users need to obtain their AccessKey for identity and permission authentication.
* PythonSDK: Users can upload datasets to GAS through PythonSDK.
* Label filtering for annotation sets. You can filter an annotation set by labels on the annotation details page.
* Users can download the JSON files of a single project in the approval records.
2020-07-06
New Features:
* Help Center: a detailed introduction to GAS features.
* Enterprise users can post data requests on TensorBay to obtain or provide data.
Enhanced Features:
* Optimized the UI and interaction of the platform.
* Optimized the layout of the home page and the left navigation bar, highlighting TensorBay and GroundTruth.
* Renamed the "Developers Edition" of GAS platform to "Team Edition", with no change in functions.
2020-06-17
New Features:
* Message notification type.
* Individual users invited to My Annotation project will receive an invitation message. After processing, the user will be notified with an accepted/rejected reply.
* Enterprise users invited to join the team will receive an invitation message. After processing, the user will be notified with an accepted/rejected reply.
Enhanced Feature:
* Optimized the interaction of label selection, added English name content, and allowed users to customize label colors.
2020-05-28
New Features:
* Switch platform language between Chinese & English.
* The annotation committee establishes the standards for each annotation, determines annotation quality, ensures quality control, and reviews the annotations.
* Three types of WeChat official account notification
s.
* When all my annotation tasks are completed, the project creator will receive a push notification.
* When annotation orders are awaiting confirmation, the project creator will receive a push notification.
* When annotation orders are awaiting approval, the project creator will receive a push notification.
* Annotation completion report. You can download the report from the order details page after the order is completed. The report includes the total number of labels, accuracy rate, approval records, capacity report, etc.
* You can upload datasets in local zip file and the file will be automatically decompressed and saved after uploading to GAS.
2020-05-13
New Features:
* Instructions on role-based access control for Enterprise users.
* Enterprise users can invite members to join the enterprise on the enterprise member management page.
* Enterprise Admin can manage and modify the roles of the members on the enterprise member management page.
* Notification Center. The platform pushes messages such as member invitations, permission applications and project progress reminders.
* Sandbox model training (in internal testing). Please refer to the sandbox training section for details.
Enhanced Features:
* Optimized the dataset viewing. After clicking View Data on the data platform, users will be automatically directed to the annotation platform to view the data.
* Optimized the dataset selection interface when creating a new project, and sorted the dataset list by creation time.
2020-04-20
New Feature:
* Distinguish between Individual and Enterprise users on the home page.
Enhanced Feature:
* Optimized the order performance data processing algorithm. Now the number of boxes is used as the statistical unit.
2020-04-16
New Features:
* Users can modify their names in the personal center.
* Note for non-Chrome users to use Chrome to improve user experience.
Enhanced Feature:
* Optimized the UI of the sign-in interface.
2020-04-06
New Feature:
* View sample annotations: click view sample annotations when the order is confirmed.
Enhanced Feature:
* Optimized the guidance for enterprise user authentication to facilitate mailbox authentication.
2020-03-30
Enhanced Feature:
* Optimized the copywriting of SMS verification code.
2020-03-13
New Features:
* User self-registration. Users can register their own accounts on the Graviti official website as an Developers edition user. For enterprise edition users, please contact Graviti customer service for registration.
* User authentication. Developers edition users can complete email authentication in the personal center. After authentication, they can become authenticated users and enjoy the rights of authenticated users.
* My Annotation. You can create annotation projects and invite up to four members to join.
* Enterprise member role management. Enterprise Admin can manage the roles of enterprise members on the enterprise member management page.
Enhanced Feature:
* Optimized the order release process. Users can now directly select datasets to release orders.
2020-02-29
New Features:
* Order progress, capacity, and total amount.
* Rejected records. Users can view the records of accepted or rejected projects.
* Demand analysis feedback records. Users can view all the demand analysis feedback of the orders and view the sample annotations.
* The annotation service process is introduced when creating an order.
Enhanced Features:
* Optimized the description of the order status.
* Original order status: order in parsing, pending confirmation, customer feedback, order in progress, completed order.
* New order status: parsing, annotating and completed.
* Parsing refers to the orders that need the parsing confirmation. For the orders to be confirmed, the label "awaiting confirmation" is displayed in the upper right corner of the order.
* Annotating refers to the confirmed orders that have not been fully approved. For the orders to be approved, the label "awaiting approval" is displayed in the upper right corner of the order. Annotation progress calculation: approved data volume/total data volume.
* Completed refers to the orders that have completed all the processes and acquired approval.
2020-02-18
New Feature:
* Approval records. Users can view the approval records.
2020-01-14
New Features:
* Notification for binding the WeChat official account. When users follow the Graviti official account and bind it with their GAS accounts, they can receive reminder messages of the project progress.
Enhanced Features:
* Optimized the export format of annotations, converted the relative coordinates of 3D annotations to absolute coordinates, and unified the output format of annotations.
2020-01-06
New Features:
* Orders on the annotation platform are directly linked to datasets on the data platform, and users can directly select the datasets when placing orders.
Enhanced Features:
* Optimized item numbering from the original 32-digit code to a shorter and easier-to-use code.
* Optimized the display of project performance: separately display the annotation progress and the quality inspection progress to avoid confusion.
* Optimized the display of annotation type: the original icon is replaced by text.
2019-12-23
Enhanced Features:
* Optimized the UI of the sign-in interface.
* Optimized the order of the project list: sorting by creation time instead of project number.