Graviti Help Center
  • Graviti Help Center
  • Developer Documentation
    • Developer Tools
      • Graviti Python SDK
      • Graviti CLI
      • Graviti Open API
        • DataSet Operation
        • Data Operation
        • Examples
          • Model training
          • Data mining
  • Product Documentation
    • TensorBay
      • TensorBay: For All Stages of Algorithm Research
      • Quick Start for Developers
      • Quick Start for Team
      • How to Quickly Prepare a Dataset
        • How to Create a New Dataset
        • Manage Data in Cloud Storage
        • Create a Dataset by Filtering
        • Create a Dataset by Merging
        • Quick Use of Open Datasets by Forking
      • Version Control
        • Manage Data Files
        • Manage Annotations
        • Manage Dataset Information
        • Manage Versions
        • Manage Dataset Branches
        • Dataset Activities
        • Dataset Settings
      • Pharos Online Data Visualization
        • Explore Pharos
        • Pharos Visualization Widgets Instruction
      • Collaboration
        • Create a New Team
        • Invite Team Members
        • Team Management
        • Dataset Management
        • View Activity Log
      • How to Integrate TensorBay into Your Pipeline
      • Action
        • Create a Workflow
        • Run Workflows
        • Manage Workflows
        • YAML Syntax Description
        • Crontab syntax Description
        • Automatic Configurations
    • Open Datasets
      • Basic Concepts
      • Search for Datasets
      • Preview Data and Label Distribution Online
      • Use and Manage Datasets Online
      • Download Open Datasets
      • Didn't find the dataset you want?
  • APPs
    • GroundTruth Tools
      • Annotate pictures
    • Sextant
      • Create an Evaluation
      • Custom Metrics
      • Start to Evaluate
      • View Results
  • Release Note
  • Graviti Official Website
Powered by GitBook
On this page
  • Datasets
  • Fusion Datasets

Was this helpful?

  1. Product Documentation
  2. Open Datasets

Basic Concepts

PreviousOpen DatasetsNextSearch for Datasets

Last updated 2 years ago

Was this helpful?

When you use TensorBay, you'll come across some concepts related to dataset types, mainly Datasets and Fusion Datasets. Here we will give explanations for related concepts.

Datasets

Dataset represents the most common datasets such as MNIST and THUCNews. Datasets are the most commonly used data in algorithm research. Data files have no special correlation with one another. They may come from the same sensor.

  • Datasets: Common Datasets

A dataset contains a certain number of files, annotations, and relevant information. There are a certain number of segments in one dataset.

  • Catalog & SubCatalog

There could be one or several SubCatalogs under one Catalog, each of which only stores annotation metadata of one type.

Catalog stores the annotation metadata. It collects all the annotations that occurred in all data samples of a dataset. Each SubCatalog with a single annotation type contains only one type of metadata.

  • Segment

Segment refers to a data level secondary to Dataset in the TensorBay dataset structure. Each Segment contains a certain number of data. For example, all training samples can be organized into one segment.

  • Data: Data Required for Algorithm Research

Data is the most basic level in the TensorBay dataset structure. A data object contains one dataset sample, annotations, and other information such as a timestamp.

The structure of a dataset looks like this:

Fusion Datasets

A fusion dataset is composed of complex structured data or data collected from multiple sensors, usually referring to autonomous driving datasets, such as the nuScenes and KITTI-Tracking.

  • FusionDataset

A fusion dataset contains data collected from multiple sensors, including information on frames, labels, sensors, and more. A fusion dataset may contain multiple fusion segments.

  • FusionSegment

FusionSegment is a data level only secondary to a fusion dataset. Each FusionSegment contains a certain amount of Frames to store information from different sensors.

  • Sensor

A Sensor refers to a device that collects the data in the FusionSegment. Currently, we support the following four types of sensors, each of which stores sensor-related information, including sensor names, extrinsic parameters, and intrinsic parameters (only for camera-type sensors).

Sensor

Corresponding Data Category

Camera

Image

Fisheye Camera

Image

Lidar

Point Cloud

Radar

Point Cloud

  • Frame

Frame is a component of a FusionSegment. A Frame consists of multiple types of data collected at the same time from different sensors.

  • Data in a Fusion Dataset

Each of the data in a Frame corresponds to a data file collected by a Sensor. The data of fusion datasets is defined the same as the data of datasets.