Data Operation

UploadData
Upload data to a dataset
Users upload data to a dataset
1.Acquire data upload credentials
2.Transfer data directly to S3
3.Put callback
GetPolicy
Acquire upload credentials
Request path
GET /v1/datasets/{id}/policies
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Query
Name
Type
Descriptions
Value
​
draftNumber
int
Yes
Draft number
​
segmentName
string
No
Dataset segmentation
​
expired
int
No
The token expiration time measured in seconds cannot be negative. The default is 60 seconds and any time greater than 300 seconds will be treated as 300
​
Request instance
curl --location --request GET '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/policies?expired=60&draftNumber=1' \
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json' 
Output
# Response status
HttpStatus 200
# Response result
{
    "result": {
        "policy": "eyJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJmYXQtZGF0YXBsYXRmb3JtLXZlcnNpb24ifSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjNhZTBiYzI5M2Q3N2FiYjQwODZhMzc5ZjAyNjkwNmNlL2YwODk3NDUwLWNlZjEtNGQ3Zi05NDI3LWNiZGRhZjcxYzc1YS8iXSxbInN0YXJ0cy13aXRoIiwiJHN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyIsIiJdLFsic3RhcnRzLXdpdGgiLCIkQ29udGVudC1UeXBlIiwiIl0seyJ4LWFtei1jcmVkZW50aWFsIjoiQUtJQVFIVFdDQTNKTTZESTdVQ0YvMjAyMTAzMjQvdXMtd2VzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LHsieC1hbXotYWxnb3JpdGhtIjoiQVdTNC1ITUFDLVNIQTI1NiJ9LHsieC1hbXotZGF0ZSI6IjIwMjEwMzI0VDA1NDIwMFoifV0sImV4cGlyYXRpb24iOiIyMDIxLTAzLTI0VDA1OjQ0OjAwWiJ9",
        "success_action_status": "200",
        "x-amz-algorithm": "AWS4-HMAC-SHA256",
        "x-amz-credential": "AKIAQHTWCA3JM6DI7UCF/20210324/us-west-1/s3/aws4_request",
        "x-amz-date": "20210324T054200Z",
        "x-amz-signature": "8dbc309fa36caad5791907c385632e74a47362ca6f1517b748e150f1306851a0"
    },
    "extra": {
        "backendType": "s3",
        "host": "https://dataplatform-version.s3.us-west-1.amazonaws.com",
    },
    "expireAt": 1616564580
}
Front-end direct transmission based on the acquired credentials
The obtained result can be deserialized directly and transferred directly to S3. extra host is the access address
Request path
POST The value of "host" within the "extra" array from the output above
Request parameter
body
Name
Type
Required
Descriptions
x-amz-credential
string
Yes
Consistent with the returned x-amz-credential of the previously obtained credentials
success_action_status
string
Yes
Consistent with the return value success_action_status of the previously obtained credentials
x-amz-algorithm
string
Yes
Consistent with the returned x-amz-algorithm of the previously obtained credentials
x-amz-signature
string
Yes
Consistent with the return value x-amz-signature of the previously obtained credentials
policy
string
Yes
Consistent with the returned policy of the previously obtained credentials
x-amz-date
string
Yes
Consistent with the return value x-amz-date of the previously obtained credentials
key
string
Yes
Calculate the checksum of a file with SHA-1 cryptographic hash
file
file
Yes
File
Request instance
curl --location --request POST 'https://dataplatform-version.s3.us-west-1.amazonaws.com/' \
--form 'x-amz-credential=AKIAQHTWCA3JM6DI7UCF/20210323/us-west-1/s3/aws4_request' \
--form 'x-amz-algorithm=AWS4-HMAC-SHA256' \
--form 'x-amz-date=20210323T120601Z' \
--form 'x-amz-signature=42d7455003455a76796d67087b6a314b6c28445a14b80c9145029aeaa0091ec4' \
--form 'policy=eyJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJmYXQtZGF0YXBsYXRmb3JtLXZlcnNpb24ifSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjNhZTBiYzI5M2Q3N2FiYjQwODZhMzc5ZjAyNjkwNmNlL2YwODk3NDUwLWNlZjEtNGQ3Zi05NDI3LWNiZGRhZjcxYzc1YS8iXSxbInN0YXJ0cy13aXRoIiwiJHN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyIsIiJdLFsic3RhcnRzLXdpdGgiLCIkQ29udGVudC1UeXBlIiwiIl0seyJ4LWFtei1jcmVkZW50aWFsIjoiQUtJQVFIVFdDQTNKTTZESTdVQ0YvMjAyMTAzMjMvdXMtd2VzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LHsieC1hbXotYWxnb3JpdGhtIjoiQVdTNC1ITUFDLVNIQTI1NiJ9LHsieC1hbXotZGF0ZSI6IjIwMjEwMzIzVDEyMDYwMVoifV0sImV4cGlyYXRpb24iOiIyMDIxLTAzLTIzVDEyOjA4OjAxWiJ9' \
--form 'success_action_status=200' \
--form 'key=b9676b075dff6d3333bbebb7cdd154f3631bb926' \
--form 'file=@/{localFile}'
Output
# Response status
HttpStatus 204
# Response result
{}
PutCallback
Upload complete callback
Request path
PUT /v1/datasets/{id}/callback
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
body
Name
Type
Required
Descriptions
draftNumber
int
Yes
Draft number
checksum
string
Yes
The checksum of an uploaded file. Calculate the checksum of a file with SHA-1 cryptographic hash
remotePath
string
Yes
The remote file path, such as a.jpg
segmentName
string
Yes
Segment name
sensorName
string
No
Must be passed for the upload of fusion data; sensor name
frameId
string
No
Must be provided for the upload of fusion data. The frameID should be in the format of ulid. Please refer to https://github.com/oklog/ulid​
timestamp
float
No
Not required to be provided for Fusion data upload
label
map
No
For the annotation labels of the file, please refer to uploadLabel
Request instance
curl --location --request PUT '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/callback' \
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json'\
--data-raw '{
  "draftNumber": "1",
  "checksum": "b9676b075dff6d3333bbebb7cdd154f3631bb926",
  "remotePath": "car1.jpg",
  "frameId": "01D78XYFJ1PRM1WPBCBT3VHMNV",
  "segmentName": "car",
  "sensorName": "camera_car",
  "timestamp": 1532402934
}'
Output
# Response status
HttpStatus 200
# Response result
{}
MultiPutCallback
Batch callback
Request path
PUT /v1/datasets/{id}/multi/callback
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
body
Name
Type
Required
Descriptions
​
draftNumber
int
Yes
Draft number
​
segmentName
string
Yes
Segment name
​
objects
[]object
Yes
The batch callback objects
​
  ├─checksum
string
Yes
The checksum of an uploaded file. Calculate the checksum of a file with SHA-1 cryptographic hash
​
  ├─remotePath
string
Yes
The remote file path, such as a.jpg
​
  ├─sensorName
string
No
Must be passed for the upload of fusion data; sensor name
​
  ├─frameId
string
No
Must be provided for the upload of fusion data. The frameID should be in the format of ulid. Please refer to https://github.com/oklog/ulid​
​
  ├─timestamp
float
No
Not required to be provided for Fusion data upload
​
  ├─label
map
No
For the annotation labels of the file, please refer to uploadLabel
​
Request instance
curl --location --request PUT '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/callback' \
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json'\
--data-raw '{
  "draftNumber": "1",
  "segmentName": "car",
  "objects": [
        {
        "remotePath": "car1.jpg",
        "checksum": "b9676b075dff6d3333bbebb7cdd154f3631bb926",
        "frameId": "01D78XYFJ1PRM1WPBCBT3VHMNV",
        "sensorName": "camera_car",
        "timestamp": 1532402934,
        "label":{}
        }
  ]
}'
Output
# Response status
HttpStatus 200
# Response result
{}
createSegment
Create a segment
POST /v1/datasets/{id}/segments
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Body
Name
Type
Required
Descriptions
draftNumber
int
Yes
Draft number
name
string
Yes
Segment name. This cannot be null and should not include ".",""
description
string
No
Segment description
Request instance
curl --location --request POST '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/segments'\
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json' \
--data-raw '{
        "draftNumber":1,
        "name": "graviti",
        "description": "this is a segment"
    }'
Output
# Response status
HttpStatus 201
# Response result
{}
DeleteSegment
Delete segment
DELETE /v1/datasets/{id}/segments
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Body
Name
Type
Required
Descriptions
draftNumber
int
Yes
Draft number
segmentName
string
Yes
segmentName
Request instance
curl --location --request DELETE '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/segments' \
--header 'x-token: {your_accesskey}'\
--data-raw '{
    "draftNumber":1,
    "segmentName": "car",
}'
Output
# Response status
HttpStatus 200
# Response result
{}
ListSegment
Acquire dataset segment
GET /v1/datasets/{id}/segments
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Query
Name
Type
Required
Descriptions
draftNumber
int
No
Draft number. Only one of draft and commit should exist, but they should not exist at the same time
commit
string
No
Commit ID, commit tag, or branch name
limit
int
No
The default is set to 128. The maximum value for a single time is 128
offset
int
No
The default is 0
sortBy
string
No
Currently sorting is based or created_at and updated_at while entering other content will return InvalidParamsValue.
orderBy
string
No
The default is desc (descending order). The valid content is asc (ascending order) and desc
Request instance
curl --location --request GET '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/segments?commit=v1.0&offset=0&limit=2'\
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json' 
Output 
Normal dataset
# Response status
HttpStatus 200
{
  "segments": [
    {
        "name": "car",
        "description": "this is car"
    },
    {
        "name": "graviti",
        "description": "this is a segment"
    }
  ],
  "offset": 0,
  "recordSize": 2,
  "totalCount": 10
}
UploadLabelCatalog
Upload the labelCatalog
PUT /v1/datasets/{id}/labels/catalogs
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Body
Name
Type
Required
Descriptions
draftNumber
int
Yes
Draft number
catalog
map
yes
Annotation Type.
key:
CLASSIFICATION: annotation classification
BOX2D: 2D box annotation
BOX3D: 3D box annotation
POLYGON2D: 2D polygon annotation
POLYLINE2D: 2D polyline annotation
KEYPOINTS2D: 2D keypoints annotation
CUBOID2D: 2D cuboid annotation
SENTENCE: audio transcription annotation
 For the annotation type above, each one of them has its own structure. You can refer to the request instances for detailed information. Please follow the format of the input value for each annotation type
├─description
string
false
Descriptions
├─categoryDelimiter
string
false
The delimiter used to separate the parent-child relationship of category. The default is not to support the parent-child relationship. The recommended delimiter is "."
├─categories
object[]
false
Item type: object and category list, which includes all category information
├─isTracking
bool
No
Tracking or not
       ├─name
string
yes
Category name
       ├─description
string
No
Category description
├─attributes
object[]
No
Item type. It includes attribute-related information
      ├─name
string
Yes
Attribute name
      ├─type
string/string[]
false
attribute type:string, boolean, integer, number, array, null,such as"interger", ["integer", "null"]
      ├─minimum
float
No
The minimum value of the attribute. When the attribute is numerical, this field defines the minimum value (closed interval)
      ├─maximum
float
No
The maximum value of the attribute. When the attribute is numerical, this field defines the maximum value (closed interval)
      ├─description
string
No
Attribute description
      ├─enum
string[]
No
The options of the value of an attribute. This represents all the possible options this attribute may have.
      ├─parentCategories
string[]
No
Relational categories represent the categories to which the attribute belongs to. If it is empty, then this is a global attribute
      ├─items
object[]
No
Item type that defines multiple attributes. (When the "type" field is "array", it indicates multiple attributes)
            ├─type
string/string[]
No
type of an attribute value with multiple options:string, boolean, integer, number, array, null,such as "interger", ["integer", "null"]
            ├─minimum
float
No
The minimum value (closed interval) of the attribute value with multiple options
            ├─maximum
float
No
The maximum value (closed interval) of the multiple attribute
            ├─enum
string[]
No
The enumeration of the value of an attribute with multiple options. This represents all the possible options this attribute may have.
Request instance
curl --location --request PUT '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/labels/catalogs'\
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json'\
--data-raw '{
  "draftNumber": 1,
  "BOX2D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false
  },
  "BOX3D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false
  },
  "CLASSIFICATION": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog"
  },
  "CUBOID2D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false
  },
  "KEYPOINTS2D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false,
    "keypoints": [
      {
        "description": "this is key point",
        "names": [
          "ear",
          "leg",
          "mouse"
        ],
        "number": 3,
        "parentCategories": [],
        "skeleton": [
          [
            0,
            1
          ],
          [
            1,
            2
          ]
        ],
        "visible": "1"
      }
    ]
  },
  "POLYGON2D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false
  },
  "POLYLINE2D": {
    "attributes": [
      {
        "description": "attribute 'traffic_light_color' is a multiple choice attribute",
        "items": {
          "enum": [
            "unknown",
            "red",
            "yellow",
            "green"
          ]
        },
        "name": "traffic_light_color",
        "parentCategories": [
          "traffic_light"
        ],
        "type": "array"
      },
      {
        "description": "attribute 'visibility' is a float value between 0 to 1",
        "maximum": 1,
        "minimum": 0,
        "name": "visibility",
        "type": "number"
      },
      {
        "description": "this is color",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "name": "color",
        "parentCategories": [
          "car"
        ],
        "type": "string"
      }
    ],
    "categories": [
      {
        "description": "this is a car",
        "name": "car"
      }
    ],
    "categoryDelimiter": ".",
    "description": "this is a description of label catalog",
    "isTracking": false
  },
  "SENTENCE": {
    "attributes": [
      {
        "description": "this is a description of attribute",
        "enum": [
          "red",
          "blue",
          "green"
        ],
        "items": {
          "enum": [
            "red",
            "blue",
            "green"
          ],
          "maximum": 10,
          "minimum": 0,
          "type": {}
        },
        "maximum": 10,
        "minimum": 0,
        "name": "color",
        "parentCategories": [],
        "type": {}
      }
    ],
    "description": "this is a description of label catalog",
    "isSample": false,
    "lexicon": "[[\"啊\", \"aa\", \"a1\"], [\"阿婆\": \"aa\", \"a1\", \"p\", \"o2\"], [\"爱\", \"aa\", \"ai4\"], [\"爱祖国\",\"aa\", \"ai4\", \"z\", \"u3\", \"g\", \"uo2\"]]",
    "sampleRate": 32
  }
}'
Output
HttpStatus 200
{}
GetLabelCatalog
Acquire dataset label
GET /v1/datasets/{id}/labels/catalogs
Request parameter
Path
Name
Descriptions
Value
id
Yes
Dataset ID
Query
Name
Type
Required
Descriptions
draftNumber
int
No
Draft number. Only one of draft and commit should exist, but they should not exist at the same time
commit
string
false
Commit ID, commit tag, or branch name
Request instance
curl --location --request GET '{service}/v1/datasets/154e35bae8954f09969ef8c9445efd2c/labels/catalogs?commit=v1.0'\
--header 'x-token: {your_accesskey}'\
--header 'Content-Type: application/json' 
Output
# Response status
HttpStatus 200
# Response result
{
  "catalog": {
    "BOX2D": {
      "attributes": [
        {
          "name": "color"
        }
      ],
      "categories": [
        {
          "description": "hello",
          "name": "car"
        }
      ]
    }
  }
}
UploadS
Name	Type	Descriptions	Value
draftNumber	int	Yes	Draft number
segmentName	string	No	Dataset segmentation
expired	int	No	The token expiration time measured in seconds cannot be negative. The default is 60 seconds and any time greater than 300 seconds will be treated as 300
Name	Type	Required	Descriptions
x-amz-credential	string	Yes	Consistent with the returned x-amz-credential of the previously obtained credentials
success_action_status	string	Yes	Consistent with the return value success_action_status of the previously obtained credentials
x-amz-algorithm	string	Yes	Consistent with the returned x-amz-algorithm of the previously obtained credentials
x-amz-signature	string	Yes	Consistent with the return value x-amz-signature of the previously obtained credentials
policy	string	Yes	Consistent with the returned policy of the previously obtained credentials
x-amz-date	string	Yes	Consistent with the return value x-amz-date of the previously obtained credentials
key	string	Yes	Calculate the checksum of a file with SHA-1 cryptographic hash
file	file	Yes	File