How to Create a New Dataset
Last updated
Was this helpful?
Last updated
Was this helpful?
You can select Graviti Storage to upload data to your private storage space on TensorBay, or select the Authorized Storage to authorise TensorBay to manage your data stored on other cloud servers.
Click TensorBay on the top navigation bar to enter My Datasets, and select Create a New Dataset in the new dataset drop-down box to create a dataset. By clicking TensorBay, an individual developer user will enter the My Datasets page by default, whereas a team user will enter the Team Datasets page.
After entering the dataset name and its alias, the URL including the dataset name will be generated automatically. Both the dataset name and alias can be changed.
Select or create your data storage method and upload the data to the corresponding storage space.
Note: this will take up your Graviti storage or Authorized storage space
If the storage configuration in the drop-down box does not meet your needs, you can click Add Data Storage Configuration to add a suitable storage method.
Developers are only supported to select Graviti Storage to upload data to your private storage space on TensorBay (note: this will take up your Graviti storage)
In addition to Graviti Storage, team members can choose Authorized Storage. TensorBay will read and manage your data on your other cloud storage platforms after being authorized, which will not occupy your Graviti storage space but your authorized storage space.
After filling in the corresponding configuration information, you can use your data on Alibaba Cloud OSS, Amazon S3 and Azure Blob by authorization. For details, please refer to Manage Data on the Cloud Storage Platforms.
Public indicates that the data in your dataset will be open to the public. Anyone can search, view, retrieve, and browse the data of this public dataset. Graviti supports your contribution to the AI community and will provide unlimited free storage capacity for your open datasets.
Private indicates that this is your private dataset, and no one has access to it without permission. If you create a private team dataset, by default, your team members can search for and view basic information about the dataset, and you can control who can view, use, edit, or manage the data in the team.
After creating a dataset, you will enter the Upload Data page belonging to the default draft. The draft is based on the Main Branch.
You can upload your data to a target Segment (You should create a segment before uploading). Tensorbay supports users to choose Upload Files from Local or Upload Files from Developers Tools to upload their data.
You can also choose Skip to enter Commit Draft page
Select Upload Files from Local. You can drag the local folders into the page, or select multiple data files to upload in the local directory. After adding the data, click Upload, and TensorBay will read the data from your local and transfer it to your private cloud storage space. Please do not leave the upload interface before the process is finished, otherwise, the transfer of the remaining data will fail.
With Upload with Developer Tools, you can upload data files and annotate data with Python SDK or Open API. This method supports the uploading, editing, and managing of fusion datasets, continuous data, and annotations. For details, see How to Use Developer Tools.
After completing the data upload, click Confirm to Commit draft page and enter Commit Title, Description(Optional) and Tag(Optional) as the description of the version, click Commit to publish the new version, and complete the Commit action. The version cannot be modified after the release.
Only committed datasets can be used for labeling, merging, filtering, training, and so on.
If you do not want to publish the draft, you can click Skip, and the dataset will remain a draft. Return to the Dataset Details page and click Manage Data, you can enter the Data List page to continue editing your data. You can also click Enter to Draft and enter the Draft List page to choose a draft and modify the dataset details.
Click Manage Data to enter the Data List page and continue to edit your data. You can view you dataset, conduct Search, viewing Catalog, creating new draft to edit data and creating a new dataset.
