Release Note

To help you keep track of the ongoing updates and improvements to Graviti products, we publish release notes that describe recent changes.

2022-03-30

Graviti Data Platform Improvements:

• Added Records on the Activity page to list the new datasets that were forked or created by Filter.

• Optimized users’ login process.

2022-03-10

Graviti Data Platform New Feature:

• Added compute management function. The compute consumed by Offline Filter, Cross-version comparison, Action, Sextant Model Evaluation and Squash&Merge will be automatically calculated and the results will be displayed on the dashboard page and Your Asset page for viewing and management.

Graviti Data Platform Improvements:

• Optimized the Version Control function. Simplify users’ operation process by putting all the functions of Version Control on a single management page.

2022-02-15

Graviti Data Platform Improvements:

• Fixed bugs on data capacity statistics.

• Optimized the login UI of Graviti data platform.

• Added Crontab drop-down box component in the scheduled trigger. Optimized the input box to support special characters “?” “\”.

2022-01-24

Graviti Data Platform New Features:

• Added scheduled trigger in Action. Users can make workflows run on schedule.

• Added Activity Registration on the Website Home Page. Users can enter the Event Detail page for more information, registration and documents downloading.

Graviti Data Platform Improvements:

• Optimized features of the Data Dashboard. Users can view monthly Used Data Volume on the Details page.

• Optimized the copywriting of Sextant, changing Evaluation to Benchmark.

• Added Select All option on the Delete Data page to help users quickly delete the data.

• Added the ISO9001 certification icon and modified relative copywriting on the Data Hosting page.

• Optimized the User Interface of GroundTruth on the Select Dataset page. Added details of the Commit, including Type, Amount, Creator and Updated Time, to help users quickly choose datasets. Added Paging function to improve the user experience.

Python SDK New Features:

• Add open dataset loaders VGGFace2

Python SDK Bug Fix:

• Fix reading cached file failed in a multiprocess environment

• Fix the Job.started_at will not be updated

• Fix the ValueError when init PointList2D and RLE with Numpy array

• Fix the NoneType error in Job.update

Python SDK Improvements:

• Remove the multiprocess lock and add PID to cached file path

2021-12-30

Graviti Data Platform New Feature:

• Optimized the cross-version comparison function. Graviti Data Platform now supports cross-version comparison by selecting the corresponding commit by branch and tag.

Python SDK New Features:

• Implement the async job framework

  • Implement JobMixin._create_job

  • Implement JobMixin._get_job

  • Implement Job.updat

  • Implement JobMixin.delete_job

  • Implement Job.retry and Job.abort

  • Implement JobMixin._list_job

• Implement SquashAndMerge feature by async job framework

  • Implement SquashAndMerge.create_job

  • Implement SquashAndMerge.delete_job

  • Implement SquashAndMerge.get_job

  • Implement SquashAndMerge.list_jo

  • Implement SquashAndMergeJob.result

• Add the following open dataset loader:

  • CityscapesGTFine

  • CityscapesGTCoarse

  • RarePlanesRea

  • RarePlanesSynthetic

  • UrbanObjectDetection

Python SDK Improvements:

• Getting mask URLs from OpenAPI getDataDetails to reduce the number of requests

Python SDK Documentation:

• Update the docs about "Squash And Merge" in "Features"

• Refine the docs

• Refine the docstring

Python SDK Documentation:

• Change the Sphinx Html theme to "furo"

• Change the Sphinx Html theme to "furo"

2021-12-17

Python SDK Bug Fix:

• Fix RemoteFileMixin._urlopen() failed when URL is expired

2021-12-16

Graviti Data Platform New Feature:

• Added secrets in Action. Users can configure their secrets in the settings of Action. The secrets that users configure are available to use in Action Yaml files. (only available for administrators)

Python SDK New Features:

• Add interfaces to communicate with Sextant Apps

  • Implement Sextant.list_benchmarks

  • Implement Sextant.list_evaluations

  • Implement Sextant.create_evaluations

  • Implement Evaluation.get_result

• Add SemanticMask for BDD100K_MOTS2020 dataset

• Add author, updated_at and parent_commit_id for Draft

• Add --show-drafts for CLI gas log to support displaying open drafts

Python SDK Improvements:

• Remove the redundant attributes in BDD100K dataset

• Update VersionControlClient to VersionControlMixin

• Use stem to represent the filename without extension in opendataset module

Python SDK Bug Fixes:

• Add the missing categoryDelimiter to the OxfordIIITPet catalog

• Fix NoneType not subscriptable error in listMaskUrls

• Fix the AttributeError when calling the mocker class methods

• Correct the wrong attribute names in nuImages catalog

• Add "PaddlePaddle" instructions in INTEGRATIONS chapter

• Add documentation for Sextant in Applications chapter

• Reconstruct Storage Config in Advanced Features chapter

• Support deleting automatically generated rst files via make clean

• Refine the docstring

2021-12-07

Python SDK Bug Fixes:

• Fix the request timeout issue when reading data in multiprocess environment

• Fix the TypeError caused by positional arguments mismatch when using Profile

2021-12-03

Graviti Data Platform New Features:

• Add a data dashboard to enable users to view the data storage capacity of each workspace and used data volume

• Add an email subscription portal for users to get our latest updates

Graviti Data Platform Improvements:

• Optimize the creation steps of datasets and user interactive experience

• Optimize the functions of Action. Add a manual option, YAML code templates, and a stop workflow button (this function is only available to the team admin)

Python SDK New Features:

• Add --sort option for CLI gas branch and gas tag to support sorting branches and tags

• Support recording statistical info for Data.open().read() in Profile class

• Add StorageConfig class to store storage config info

• Add SegmentClient.get_data to get data info by its remote_path

• Support showing a piece of data via remote_path in CLI gas ls

Python SDK Improvements:

• add URL class for getting and updating the file URL

• Add train, val and test in segment name in DAVIS2017

Python SDK Bug Fixes:

• Fix the wrong graph in CLI gas log when merging branches

• Add the missing categoryDelimiter in COCO2017 catalog

• Fix the FileExistsError when using cache with multiprocess

Python SDK Documentation:

• Use autosummary to generate rst files in opendataset module under "API" chapter

• Refine the docs

• Refine the docstring

2021-12-01

Python SDK Improvement:

• Make timeout setting in tensorbay.client.config take effect for urlopen

2021-11-17

Graviti Data Platform Improvements:

• Optimize the interface and interactivity of the Apps market

• Optimize the display of the data dashboard

Python SDK New Features:

• Add checking and warning for free storage space when enabling cache

• Add the following open dataset loaders:

  • SCUT_FBP5500

  • DAVIS2017

Python SDK Improvements:

• Add argument encoding='utf-8' to open() to make it compatible for Windows

• Use xmltodict to parse xml files instead of xml.ElementTree

• Add file tensorbay/py.typed to comply with PEP-561

Python SDK Bug Fix:

• Fix Data.open catch exception failed when url is expired

Python SDK Documentation:

• Reconstruct "Version Control" chapter

• Fix the typo in the docs

• Refine the docs

• Refine the docstring

Python SDK Deprecation:

• Remove the deprecated method GAS.create_auth_dataset

2021-11-02

TensorBay Bug Fix:

• Fix some bugs to optimize the user experience

Sextant Improvement:

• Add the Retry button to allow a dataset to be evaluated again after failing

Python SDK New Features:

• Support setting is_public in GAS.create_dataset

• Support importing authorized cloud files to fusion dataset

• Support cache when opening remote data

• Add DatasetClientBase.get_total_size to get the total data size in a dataset commit

• Add VersionControlClient.squash_and_merge to merge two different branches

Python SDK Improvements:

• Use os.path.expanduser to handle the CLI config file path in different systems

• Replace relative import with absolute import for all python files in SDK

• Remove the useless checks about commits and drafts in ls.py

• Add X-Request-Id to all the TensorBay OpenAPI request headers

• Add a detailed resuming message when uploading process got interrupted in CLI gas cp

Python SDK Bug Fixes:

• Add a workaround for the IndexError when loading the data with panoptic mask

• Remove the wrong error message in gas dataset when tbrn is absent

Python SDK Documentation:

• Add docs about the data cache

• Add docs for using pharos on remote server

• Refine the docs

2021-10-18

Python SDK New Features:

• Display elapsed time of the request in the debug log

• Use response.elapsed to get the cost time of a request in profile

• Add DatasetClientBase.get_label_statistics to get the label statistics info

• Support updating file urls when the urls are expired

• Add open dataset loaders SegTrack2

Python SDK Improvements:

• Consolidate all config-related functions in ContextInfo

Python SDK Performances:

• Adapt OpenAPI getDataDetails to increase the speed of reading labels and URLs

• Upload auth cloud storage data labels on the cloud-callback OpenAPI to increase the uploading speed

Python SDK Documentation:

• Add "Update Dataset Notes" section in "Update Dataset" chapter

• Add docs about cloud file in-place importing in "Cloud Storage"

• Add docs about getting the label statistics info in "Get Label Statistics"

• Refactor the "Version Control" chapter

• Refine the docs

2021-09-28

TensorBay Enhanced Features：

• Add discussion function on the public dataset details page in the mobile terminal

• Optimize the interaction of data list search

• Divide Apps into different types according to their attributes on the Apps Market page

Python SDK New Features:

• Support filtering drafts by status and branch_name inVersionControlClient.list_drafts

• Add class Profile to record the statistical information about the HTTP requests in SDK

  • Support using Profile in the multi-process environment

  • Support saving Profile summary to csv, txt or json file

• Add the following BDD100K related open dataset loaders

  • BDD100K

  • BDD100K-10K

  • BDD100K-MOT

  • BDD100K-MOTS

• Add the following open dataset loaders:

  • SegTrack

  • COCO2017

• Unify the format of prompt messages in CLI

Python SDK Bug fixes:

• Fix the dataset loader LIP and CIHP cannot be imported from opendataset module

• Correct the wrong visible status in LIP dataset loader

Python SDK Improvements:

• Unify the variable names about directory and tbrn

• Use ValueError, StatusError to replace OperationError

Python SDK Documentation:

• Add examples about SemanticMask and InstanceMask

• Add catalog structure for different label types

• Add docs about how to use Profile

• Fix the typos in docs

• Refine the docs

Python SDK Deprecations:

• Remove the deprecated class ResponseSystemError

2021-09-14

Sextant New features:

• Support customized metrics by uploading Github Repo Url into Sextant

• Add more data types that can be evaluated

TensorBay Enhanced Feature：

• Support creating new datasets by merging datasets via Python SDK

Python SDK New Features:

• Support creating a dataset with different cloud storage configs inGAS.create_dataset

• Add the following methods to create customized cloud storage configs for different cloud services:

  • GAS.create_oss_storage_config

  • GAS.create_s3_storage_config

  • GAS.create_azure_storage_config

• Add GAS.delete_storage_config to delete customized cloud storage config

• Enable the method SegmentClient.delete_data to delete data

• Add FusionSegmentClient.delete_frame to delete frame

• Support getting and updating dataset isPublic flag

• Add the following open dataset loaders:

  • CIHP

  • VOC2012Segmentation

  • PASCALContext

  • LIP

  • OxfordIIITPet

  • nuImages

• Print more friendly error messages in CLI

Python SDK Interface Adjustments:

• Set current revision as the default in VersionControlClient.list_commits

• Disable method FusionSegmentClient.delete_data to stop deleting data in Fusion dataset

Python SDK Improvements:

• Prohibit closing current draft in VersionControlClient.close_draft

• Avoid getting done_frames when skip_upload_files is False in FusionDatasetClient.upload_segment

Python SDK Documentation:

• Add docs about "Update Dataset Meta"

• Add docs about "Delete Frame"

• Fix the typos in docs

• Refine the docs

Python SDK Deprecations:

• Remove the deprecated code before v1.10.0

• Deprecate GAS.create_auth_dataset and use GAS.create_dataset instead

2021-08-30

TeraGood New Feature：

• Support editing labels of the fusion dataset in TeraGood

TensorBay Enhanced Feature:

The data list is optimized from a one-dimensional list to a two-dimensional list. Users can view more details of the data list via subtables. Support users to switch versions, view information, search, view catalog, show diff and do other operations on the data list page.

Python SDK New Feature:

• Support new label types: SemanticMask, InstanceMask and PanopticMask

Python SDK Interface Adjustments:

• Remove the loads method for DataBase, Data and AuthData

• Add get_callback_body method to replace dumps in Data

• Add from_response_body method to replace loads in RemoteData & Frame

Python SDK Bug Fixes:

• Add a workaround for MalformedPOSTRequest error when posting files to Aliyun OSS

Python SDK Improvements:

• Add FileMixin and RemoteFileMixin for file-related methods

• Support setting custom loader and dumper in AttrsMixin

Python SDK Documentation:

• Add docs about SemanticMask, InstanceMask and PanopticMask

2021-08-17

New Features:

• Data storage configuration supports developers to choose different regions and team members to choose different cloud platforms or regions as the default storage configuration.

• Add data usage statistics function. Users can view the used data volume on the GAS homepage and the assets page.

• Support data synchronization within GroundTruth Tools. GroundTruth Tools support the 2d/3d fusion labeling tools and can synchronize the labeling data to Tensorbay.

• Support booking demo on the GAS home page.

Enhanced Feature:

• Support to select, modify server configuration and record parameters in Action.

Python SDK New Features:

• Support new label types Polygon and Polyline2D, add five label formats:

  • Polygon + MultiPolygon + RLE

  • Polyline2D + MultiPolyline2D

• Add the following open dataset loaders:

  • VOC2012Detection

  • VOC2012ActionClassification

  • CCPD & CCPDGreen

• Add name conflict check for SortedNameList

• Support setting network request config (is_internal, max_retries, and timeout) in CLI gas config

• Support to validate the AccessKey and display the user info in CLI gas auth

• Add --status option for CLI gas auth to display the user and auth info

• Support displaying error messages for CLI gas auth

• Add -l option for CLI gas ls to show the total number of resources

• Add --all option for CLI gas log to show the commits in all branches

• Add --graph option for CLI gas log to show the graphical commits

• Set -h as an abbreviation of --help to show CLI help messages

Python SDK Interface Adjustments:

• Remove the path argument in GAS.create_auth_dataset because the path is bound to the cloud storage config

Python SDK Bug Fixes:

• Add a workaround for the issue Chinese characters display garbled in the Windows system

Python SDK Improvements:

• Use list comprehension in PointList2D.__init__ instead of for-loop

• Set None as the default value of the argument alias in GAS.update_dataset

• Correct the illegal remote paths for RP2K dataset loader

• Encapsulate the moving segment logic into DatasetClientBase._move_segment

• Move the deprecation related classes into deprecated.py

• Stop showing the redundant field name in AttributeInfo .__repr__

• Cleanup the following useless codes:

  • Remove the useless class variable _label_attrs from label related classes

  • Remove the useless class SubcatalogTypeRegister

  • Remove the useless class LabelType

  • Remove the useless type Subcatalogs

Python SDK Documentation:

• Add docs about "Shell Completion" in the "CLI" section

• Add docs about new label formats Polygon and Polyline2D

• Add docs about setting network request config in CLI gas config

• Add docs about --all and --graph options in CLI gas log

• Fix the typo in docs

• Refine the docs

2021-08-04

New Features:

• Add model evaluation tools Sextant. Sextant supports quick evaluation of data and models, enabling you to gain insight into data quality and model accuracy, and providing references for data quality improvement.

• Add advanced search. Users can upload Github URL to get the corresponding file and set the file as a filter.

• Support modifying and conducting quality assurance of labels of 2D Box and 2D Polygon in TeraGood and GroundTruth.

Enhanced Feature:

• Optimize data search. Data search is performed offline now. Users can view the search progress and results and save search results for creating new datasets and so on.

Python SDK New Features:

• Support override and skip strategies in SegmentClient.move_data

• Add GAS.get_user to get the current user info

• Support manipulating dataset alias:

  • Support getting the dataset alias in GAS.get_dataset

  • Support setting the dataset alias in GAS.create_dataset

  • Add GAS.update_dataset to update the dataset alias

• Support setting and showing description in CLI gas draft

• Add --edit option in CLI gas draft to edit a draft

• Add --close option in CLI gas draft to close a draft

• Add draft description to the pop-up editor of gas commit as default message

• Display the corresponding message after deleting data or a segment in gas rm

Python SDK Interface Adjustments:

• Change the signature of Transform3D.set_rotation and Sensor.set_rotation

Python SDK Improvements:

• Unify the description display logic in gas draft and gas commit

• Set the file name as "" when posting files to cloud storage

• Add utility function chunked to break an iterable into length n tuples

• Set the minimum version of urllib3 back to v1.15

Python SDK Performance:

• Speed up file uploads for fusion dataset in GAS.upload_dataset by adapting OpenAPI multiCallback

• Speed up file uploads by sending file size to TensorBay

• Speed up file uploads by enlarging the batch size of multiCallback

Python SDK Documentation:

• Add the docs about --edit and --close option in CLI gas draft

• Add the docs about CLI "Profile"

• Refine the docs

• Fix the typo in docstrings

Python SDK Deprecations:

• Remove the default value of the argument title in VersionControlClient.create_draft

• Remove the deprecated message about setting AccessKey in gas config

2021-07-16

Python SDK bug fixes:

• Fix CLI gas log --oneline that displays all commits in oneline

Python SDK Documentation:

• Replace -t with -m in the docs of CLI gas draft

• Add details about strategy in the docs of copy and move operations

• Fix the Synopsis section display error in CLI gas commit --help

• Use batch move in the example code of moving data instead of one by one

2021-07-14

New Features:

• Diff to compare data of different versions, view the difference between draft and parent commit, or between commit and parent commit, and visualize the corresponding data and tag distribution.

• Add or edit multiple tags for each data version, making it easy and fast to retrieve and manage.

Enhanced Feature:

• Teragood supports modifying existing annotated data and performing quality assurance on the data.

Python SDK New Features:

• Add Data.get_url to get the file:// url for a local file

• Support more operations on draft:

  • Add status and description to the Draft class

  • Add VersionControlClient.update_draft to update title and description of a draft

  • Add VersionControlClient.close_draft to close a draft

  • Support setting description in VesionControlClient.create_draft

• Add --message option in CLI gas draft to set title and description of a draft

• Add SegmentClient.list_urls and FusionSegmentClient.list_urls to list the file urls

• Add the following open dataset loaders:

  • UAVDT

  • CACD

  • AADB

  • COVID_CT

Python SDK Bug fixes:

• Fix the ImportError when import tensorbay in python 3.6

Python SDK Improvements:

• Add shorten function to get the short commit ID

• Merge the CLASSIFICATION label into BOX2D label in the CompCars open dataset

• Remove useless DatasetClient.import_all_files interface

• Correct the illegal segment names in all the open dataset loaders

• Move customized click classes from cli/cli.py to cli/custom.py

• Stop showing the description in Commit.__repr__

• Exclude useless docs and tests modules from the tensorbay package

Python SDK Performance:

• Speed up file uploads in GAS.upload_dataset by adapting OpenAPI multiCallback

• Enhance the performance of RemoteData.open by batch requesting file urls

Python SDK Documentation:

• Add Synopsis section to CLI --help message

• Refine the CLI docs

Python SDK Deprecations:

• Deprecate the --title option in CLI gas draft

• Remove the deprecated CLI gas create and gas delete

• Remove the deprecated exception CommitStatusError

2021-06-22

New Features:

• Version control of datasets based on branches.

• Manage and edit datasets collaboratively with draft.

• Add operation guide on the visualization widgets Pharos.

• Developers edition users can now manage members and share "Viewer" permission for private datasets.

Enhanced Features:

• Optimized the performance and operation of dataset version control.

• Optimized the interface and UI elements of the Dashboard User Guide.

2021-06-03

Python SDK Bug Fixes:

• Fix the AttributeError when using DatasetBase.keys() to list segment names

2021-05-31

New Feature:

• Pharos visualization widgets can display JSON text files.

Enhanced Feature:

• Optimized the multi-level data interface in the list mode of the label distribution visualization widgets, and now you can directly view the distribution of sub-level data in the overall data.

Python SDK New Features:

• Redesign and implement the gas CLI based on TensorBay version control :

  • Support draft number and revision in TBRN

  • Add gas dataset command to create, list and delete dataset

  • Support gas ls command to list segments and data in specific draft or revision

  • Add gas draft command to create and list drafts

  • Add gas commit command to commit draft

  • Add gas cp command to upload files to draft

  • Add gas rm command to remove segment and data in draft

• Add detailed resuming message when uploading process got interrupted

• Add ModuleImportError to print detailed install instruction when the optional requirement package is not installed

• Implement DatasetBase.__delitem__ to support delete segment from dataset by del

• Add open dataset loader LISA Traffic Sign

Python SDK Bug fixes:

• Fix the ValueError when passing the last item to UserSequence.index

Python SDK Improvements:

• Remove the redundant class variable description in KeypointsInfo

• Implement python attr framework AttrsMixin

• Move TBRN from utility module to cli module

• Inherit EqMixin to replace __eq__ in following intrinsic classes :

  • CameraMatrix

  • DistortionCoefficients

  • CameraIntrinsics

Python SDK Documentation:

• Refactor CLI related docs

• Add Integrations chapter for PyTorch and TensorFlow integration

• Refine the docs

• Fix the typo in docs

Python SDK Deprecations:

• Deprecate CLI gas create and gas delete

• Deprecate legacy fusion dataset TBRN

• Remove the following deprecated interfaces :

  • Following exceptions：

    • GASDatasetError

    • GASDatasetTypeError

    • GASException

    • GASPathError

    • GASResponseError

    • GASSegmentError

  • The start and stop keyword arguments in following methods:

    • GAS.list_dataset_names

    • DatasetClientBase.list_drafts

    • DatasetClientBase.list_commits

    • DatasetClientBase.list_tags

    • DatasetClientBase.list_branches

    • DatasetClientBase.list_segment_names

    • SegmentClient.list_data_paths

    • SegmentClient.list_data

    • FusionSegmentClient.list_frames

  • Following methods：

    • DatasetClientBase.list_draft_titles_and_numbers

2021-05-17

New Features:

• You can use developer tools to access the Pharos visualization widget to visualize local data. For details, please refer to the document.

• Launched the dataset discussion function. Discussions can be posted for all the open datasets with @ and Like functions being added. You can @ dataset publishers or other relevant users to invite them to join the discussion and give your Likes to excellent content.

• Released a new version of the dashboard and added the user guide to help users quickly understand the features of TensorBay and the way to access them.

• Actions: You can monitor changes in datasets by creating a workflow, and automatically trigger a pre-defined data processing program to quickly build an automated data processing procedure (in internal testing).

Enhanced Features:

• Optimized the UI design and interface of the label distribution visualization widgets.

• Optimized the interaction of the Pharos visualization widgets on the data filters page, and you can directly view the annotation types in the filters by default.

Python SDK New Features:

• The local visualization plugin pharos is released, it can be installed by pip3 install pharos:

  • Pypi: https://pypi.org/project/pharos/
  • Docs: https://tensorbay-python-sdk.graviti.com/en/v1.4.1/features/visualization.html

Python SDK Bug fixes:

• Fix the JSONDecodeError occurred when uploading dataset

• Fix the AttributeError when using CADC dataloader in python3.6

• Fix the AttributeError when resuming upload of fusion dataset

Python SDK Documentation:

• Add docs for local visualization plugin pharos

2021-05-15

Python SDK New Features:

• Make PagingList mutable, and follow MutableSequence protocol

• Support reading remote data lazily in Dataset and Segment:

  • Lazy evaluation in Segment

  • Lazy evaluation in Dataset

• Support getting segment by name in DatasetBase.__getitem__

  • Use segment = dataset["test"] to get segment by name instead of segment = dataset.get_segment_by_name("test").

  • Use segment_names = dataset.keys() to get all segment names in a dataset.

• Add the following methods to convert between category and index for writing training code easier

  • CategoriesMixin.get_category_to_index

  • CategoriesMixin.get_index_to_category

• Add the following exceptions as subclasses of ResponseError

  • AccessDeniedError

  • InvalidParamsError

  • NameConflictError

  • RequestParamsMissingError

  • ResourceNotExistError

  • ResponseSystemError

  • UnauthorizedError

• Support skip_uploaded_files flag in GAS.upload_dataset for fusion dataset

• Add open dataset loader COVID-chestxray and nuScenes

Python SDK Improvements:

• Refactor CLI related code to a new module

• Stop checking the commit_id in DatasetClientBase.__init__ to avoid sending a redundant request

• Fix the possibly unbound variable warning in CompCars

Python SDK Documentation:

• Add Update Dataset, Update Label and Update Data chapter

• Add docs for specific response exceptions

• Add continuity and tracking in glossary

• Update docs for reading segments from lazy evaluation Dataset rather than DatasetClient

• Refine the example in docs

Python SDK Deprecations:

• Deprecate DatasetBase.get_segment_by_name

2021-05-12

Python SDK Interface Adjustment:

• Rename the following exceptions

  • TensorBayClientException -> ClientError

  • TensorBayOpendatasetException -> OpenDatasetError

2021-05-06

New Features:

• Filter by task types on the open datasets list page, so that you can quickly find the required dataset.

• You can modify the dataset name, URL and alias.

• A guide to help new users fill in basic information.

• Dataset discussion tab. You can post discussions on datasets (in internal testing, and currently available for MINST, BDD100K, nuScenes, CityScapes, KITTI-Object, CIFAR-10, and COCO).

• Fork records of the open datasets.

Enhanced Features:

• Optimized the UI design and interface of the data list page, and added filter logic for annotation information.

• Optimized the UI design and interface details of the visualization widgets.

Python SDK New Features:

• Implement PagingList to lazy access elements from paging requests, which follows Sequence protocol

• Use PagingList as the return value instead of iterator for the following methods:

  • GAS.list_dataaset_names

  • DatasetClientBase.list_drafts

  • DatasetClientBase.list_commits

  • DatasetClientBase.list_tags

  • DatasetClientBase.list_branches

  • DatasetClientBase.list_segment_names

  • SegmentClient.list_data_paths

  • SegmentClient.list_data

  • FusionSegmentClient.list_frames

• Support creating a dataset with auth cloud storage:

  • Add GAS.get_auth_storage_config and GAS.list_auth_storage_configs

  • Add GAS.create_auth_dataset

• Add client.config to adjust the request configuration

  • Support adjusting the config of the request retry strategy: config.timeout, config.max_retry etc.

  • Support uploading/reading data from cloud server internal endpoint

• Build TensorBay exception system:

  • Add TensorBayException as TensorBay base exception

  • Add NoFileError and FileStructureError for opendataset module

  • Add CommitStatusError, ResponseError, DatasetTypeError and FrameError for client module

  • Add TBRNError for CLI

• Add bin_point_cloud_fields to dataset Notes to support .bin format point cloud with different fields

• Display the upload progress by process bar in GAS.upload_dataset and (Fusion)DatasetClient.upload_segment

• Add open dataset loader opendataset.HalpeFullBody and opendataset.BioIDFace

Python SDK Improvements:

• Update open dataset name according to TensorBay latest naming rule

• Display commit_id in Commit.__repr__

• Refine the error message of unexpected status code

• Delete duplicated loads lexicon code on SentenceSubcatalog

Python SDK Documentation:

• Add docs for "Cloud Storage"

• Add docs for "Request Configuration"

• Add docs for "Use Internal Endpoint"

• Add docs for TensorBay exception system

• Add docs for bin_point_cloud_fields

• Add tag to glossary docs

• Refine the docstring and docs

• Fix typos in docstring and docs

Python SDK Deprecations:

• Deprecate the start and stop arguments for methods using PagingList as a return value

• Deprecate the following legacy exceptions in client module :

  • GASDataTypeError

  • GASLabelsetError

  • GASLabelsetTypeError

  • GASFrameError

  • GASResponseError

2021-04-21

Python SDK Bug Fixes:

• Fix UnboundLocalError when calling DatasetClientBase.create_tag with revision

• Fix KeyError when getting a FusionSegment instance

2021-04-19

New Features:

• You can view multiple annotation types simultaneously with the visualization widgets.

• You can apply for or share the dataset permissions. The dataset Admin can give users the corresponding permissions of the dataset.

• Dataset permissions and team management log.

• Automatically generate PythonSDK code and display API on the dataset details page.

Enhanced Feature:

• Optimized the UI design of the product pricing page.

Python SDK New Features:

• Add basic methods for dataset Version Control

  • Add DatasetClientBase.create_tag

  • Add DatasetClientBase.list_tags

  • Add DatasetClientBase.get_tag

  • Add DatasetClientBase.delete_tag

  • Add DatasetClientBase.list_branches

  • Add DatasetClientBase.list_commits

  • Add DatasetClientBase.get_draft

• Support dataset with continuous data

  • Add Notes to store dataset basic information

  • Add DatasetClientBase.update_notes

  • Add DatasetClientBase.get_notes

• Support "Fusion Dataset"

• Make request session safe in multiprocesses environment

• Add SDK version info to "Event Tracking" and support "Event Tracking" for CLI

• Add loader for CADC opendataset

• Add method create_segment() for dataset client

• Add __eq__ for Label

Python SDK Improvements:

• Modify Tensorbay gateway to "gatewayv2"

• Refine the NotImplemented logic in binary operators

• Check whether the url starts with "https://" in Client

• Implement decorator Deprecated for deprecated functions

Python SDK Packaging:

• Add minimum versions for dependencies

Python SDK Documentation:

• Add docs for dataset Version Control

• Add docs for "Fusion Dataset"

• Add docs for dataset with continuous data and Notes

• Update urls in docs

• Refine the docstring and docs

• Fix typos in docs

• Add examples to docstring

Deprecations:

• Deprecate DatasetClientBase.list_draft_titles_and_numbers, use DatasetClientBase.list_draft instead

2021-03-25

Python SDK New Features

• Support uploading data to Azure

Python SDK Fix Bugs

• Correct dataset names in data loaders according to Graviti Open Dataset

Python SDK Documentation

• Fix typos in docs

• Remove the url related content in the doc of CLI

• Refine the content of docs and docstring

• update the URLs in docs and "README.md"

• Fix typos in docs

2021-03-24

New Features:

• Visibility Settings. You can set the visibility of a dataset as public or private.

• Team Collaboration. You can create teams and invite members to their teams.

• Individual & Team Profiles

• Data visualization is available after data list filtering.

Enhanced Features:

• Upgraded UI design .

• Optimized the process of creating new datasets and unified the interface of creating datasets (filter, merge, fork, etc.).

2021-02-19

New Feature:

• 2D Keypoints type for data visualization widgets.

2021-02-01

New Features:

• Sensor display for data visualization. The corresponding sensor name can be clearly viewed when displaying multi-sensor data.

• Light-color mode for data visualization. Users can switch between Light and Dark Mode.

Enhanced Features:

• Optimized the UI, interaction and loading speed of data visualization.

• Optimized the UI and interaction of the distribution visualization.

2021-01-25

New Features:

• Users can select the region where the Bucket is stored when uploading data on TensorBay.

• Version Control for TensorBay cloud authenticated data and AWS authentication.

• You can merge the annotations of the same type and the same label table.

• "Closed" status in Feedback.

Enhanced Feature:

• Optimized the display of text and images in the Feedback.

2021-01-05

New Features:

• Users can merge data from different datasets on TensorBay.

• Version Control for datasets after the annotations are synced.

• Designated reply in the Feedback Module enables users to reply to a specific message.

• "Disable Reminders" of the reply pop-up window in the Feedback Module.

Enhanced Features:

• Optimized the display of the Feedback Module on the mobile side.

• Optimized the statistical mode of the attribute data of the label distribution visualization widgets.

2020-12-21

New Features:

• Graviti Open API, please refer to Graviti Open API for details.

• Full-screen function for visualizing label distribution.

Enhanced Features:

• Optimized the UI design of the label distribution visualization widgets.

• Optimized the user experience of adding feedback.

2020-12-02

New Features:

• Users can create a new dataset by filtering data.

• Dataset types and custom labels are available when creating a new dataset.

• Audio data visualization is available for MP3 and WAV files. Added two annotation types: transcription and classification.

• Text data visualization is available for txt files and their annotation classification.

• You can visualize attribute statistics of data annotation.

Enhanced Features:

• Optimized the logic of deleting data.

• Removed the corresponding annotations when a file is deleted.

• Raw data that has already been cited will not be deleted.

• Versions can be deleted.

2020-11-18

New Features:

• Access TensorBay via cloud storage authentication.

• Graviti Developer Tools menu and documentation are added to introduce the use of SDK and other tools.

• Code samples for the use of API.

• Feedback board allows you to post feedback and suggestions, communicate in real time, and send messages through WeChat official accounts.

• Pie chart mode to visualize data distribution.

Enhanced Features:

• Optimized the Help Doc and highlighted the developer documentation.

• Optimized the UI design and interface of the official website.

2020-11-09

New Features:

• Version Control for Datasets & Annotation sets. Raw data and annotations are included when the dataset version is released.

• Open datasets can be forked and accessed on GAS where users can release versions, upload data and perform other operations.

• Project 3D point cloud annotation info onto the corresponding 2D image, visualize Gzip compressed point cloud and select the annotation box of 3D view to automatically turn the view angle.

• View sub-category with the data distribution visualization widgets.

• The order management system where you can purchase team seats and expand TensorBay capacity.

Enhanced Feature:

• Optimized account types to "Developers" and "Teams". Visit the link for details.

2020-10-19

New Features:

• On TensorBay My Datasets, users can manage raw data versioning, create new versions, and continue to edit and modify the data of the dataset after creating a version.

• The file list on the left side of the visualization widgets helps users quickly locate and switch the displayed files.

• 2D Classification, 3D type, Tracking type data and its annotations.

• You can view multiple annotations in the open datasets with the visualization widgets.

Enhanced Features:

• Optimized the return function of visualization widgets when sharing to off-site.

• Optimized the release page UI of GroundTruth My Annotation Order.

2020-09-14

New Features:

• You can filter data with segments and view data list on My Datasets.

• You can view the annotation details of open datasets, filter list and label, and visualize annotations and label distribution.

• You can now download final performance review on GroundTruth.

• You can download the JSON file of the project output in advance during the final review or approval phase on GroundTruth.

2020-08-24

New Features:

• Access to open datasets. On the GAS platform, users can acquire open datasets to view, download and visualize.

• Data visualization widgets: 2D rectangular boxes, 2D polygons, and 2D classification. The widgets can check the annotation quality and can be shared off-site.

• Data distribution visualization widgets. Users can view the label distribution of a dataset.

2020-07-21

New Features:

• AccessKey: When using PythonSDK, users need to obtain their AccessKey for identity and permission authentication.

• PythonSDK: Users can upload datasets to GAS through PythonSDK.

• Label filtering for annotation sets. You can filter an annotation set by labels on the annotation details page.

• Users can download the JSON files of a single project in the approval records.

2020-07-06

New Features:

• Help Center: a detailed introduction to GAS features.

• Enterprise users can post data requests on TensorBay to obtain or provide data.

Enhanced Features:

• Optimized the UI and interaction of the platform.

• Optimized the layout of the home page and the left navigation bar, highlighting TensorBay and GroundTruth.

• Renamed the "Developers Edition" of GAS platform to "Team Edition", with no change in functions.

2020-06-17

New Features:

• Message notification type.

• Individual users invited to My Annotation project will receive an invitation message. After processing, the user will be notified with an accepted/rejected reply.

• Enterprise users invited to join the team will receive an invitation message. After processing, the user will be notified with an accepted/rejected reply.

Enhanced Feature:

• Optimized the interaction of label selection, added English name content, and allowed users to customize label colors.

2020-05-28

New Features:

• Switch platform language between Chinese & English.

• The annotation committee establishes the standards for each annotation, determines annotation quality, ensures quality control, and reviews the annotations.

• Three types of WeChat official account notification s.

• When all my annotation tasks are completed, the project creator will receive a push notification.

• When annotation orders are awaiting confirmation, the project creator will receive a push notification.

• When annotation orders are awaiting approval, the project creator will receive a push notification.

• Annotation completion report. You can download the report from the order details page after the order is completed. The report includes the total number of labels, accuracy rate, approval records, capacity report, etc.

• You can upload datasets in local zip file and the file will be automatically decompressed and saved after uploading to GAS.

2020-05-13

New Features:

• Instructions on role-based access control for Enterprise users.

• Enterprise users can invite members to join the enterprise on the enterprise member management page.

• Enterprise Admin can manage and modify the roles of the members on the enterprise member management page.

• Notification Center. The platform pushes messages such as member invitations, permission applications and project progress reminders.

• Sandbox model training (in internal testing). Please refer to the sandbox training section for details.

Enhanced Features:

• Optimized the dataset viewing. After clicking View Data on the data platform, users will be automatically directed to the annotation platform to view the data.

• Optimized the dataset selection interface when creating a new project, and sorted the dataset list by creation time.

2020-04-20

New Feature:

• Distinguish between Individual and Enterprise users on the home page.

Enhanced Feature:

• Optimized the order performance data processing algorithm. Now the number of boxes is used as the statistical unit.

2020-04-16

New Features:

• Users can modify their names in the personal center.

• Note for non-Chrome users to use Chrome to improve user experience.

Enhanced Feature:

• Optimized the UI of the sign-in interface.

2020-04-06

New Feature:

• View sample annotations: click view sample annotations when the order is confirmed.

Enhanced Feature:

• Optimized the guidance for enterprise user authentication to facilitate mailbox authentication.

2020-03-30

Enhanced Feature:

• Optimized the copywriting of SMS verification code.

2020-03-13

New Features:

• User self-registration. Users can register their own accounts on the Graviti official website as an Developers edition user. For enterprise edition users, please contact Graviti customer service for registration.

• User authentication. Developers edition users can complete email authentication in the personal center. After authentication, they can become authenticated users and enjoy the rights of authenticated users.

• My Annotation. You can create annotation projects and invite up to four members to join.

• Enterprise member role management. Enterprise Admin can manage the roles of enterprise members on the enterprise member management page.

Enhanced Feature:

• Optimized the order release process. Users can now directly select datasets to release orders.

2020-02-29

New Features:

• Order progress, capacity, and total amount.

• Rejected records. Users can view the records of accepted or rejected projects.

• Demand analysis feedback records. Users can view all the demand analysis feedback of the orders and view the sample annotations.

• The annotation service process is introduced when creating an order.

Enhanced Features:

• Optimized the description of the order status.

• Original order status: order in parsing, pending confirmation, customer feedback, order in progress, completed order.

• New order status: parsing, annotating and completed.

• Parsing refers to the orders that need the parsing confirmation. For the orders to be confirmed, the label "awaiting confirmation" is displayed in the upper right corner of the order.

• Annotating refers to the confirmed orders that have not been fully approved. For the orders to be approved, the label "awaiting approval" is displayed in the upper right corner of the order. Annotation progress calculation: approved data volume/total data volume.

• Completed refers to the orders that have completed all the processes and acquired approval.

2020-02-18

New Feature:

• Approval records. Users can view the approval records.

2020-01-14

New Features:

• Notification for binding the WeChat official account. When users follow the Graviti official account and bind it with their GAS accounts, they can receive reminder messages of the project progress.

Enhanced Features:

• Optimized the export format of annotations, converted the relative coordinates of 3D annotations to absolute coordinates, and unified the output format of annotations.

2020-01-06

New Features:

• Orders on the annotation platform are directly linked to datasets on the data platform, and users can directly select the datasets when placing orders.

Enhanced Features:

• Optimized item numbering from the original 32-digit code to a shorter and easier-to-use code.

• Optimized the display of project performance: separately display the annotation progress and the quality inspection progress to avoid confusion.

• Optimized the display of annotation type: the original icon is replaced by text.

2019-12-23

Enhanced Features:

• Optimized the UI of the sign-in interface.

• Optimized the order of the project list: sorting by creation time instead of project number.