How Do I Upload Data?

1. Click on project - Once you have made a project, click on the project you would like to upload data to
2. Select Assets - Can be found on the sidebar
3. Upload Images - Under Assets, you can drag and drop your files into the Upload Images rectangle or click on the box and select a folder in your local file explorer containing your images. Each image should be less than 10MB.
4. Add Assets to Group (Optional) - You can add uploaded assets to an assigned group, which acts like a foldering mechanism that allows you to filter for your uploaded data more easily. Learn more here.

Data Storage

📘

If you wish to store your data in a region that is currently not available, please contact us to see how we can provide support.

By default, Datature stores your data in a few regions (multi-region setup). Professional tier and above users are able to select specific regions such that the data is isolated within a particular region to comply with any regional privacy and security laws. You can select the data storage region during project creation, under the Data Localization dropdown.

The currently supported regions include:

Alternatives to Manually Uploading Your Dataset

If you have an external storage such as Amazon S3 or Azure Blob Storage, you can add assets through the Connection Manager. You can add a connection to your connection manager through our new features: Amazon S3 Bucket Integration and Azure Blob Storage Integration.

Image Requirements

Upload

Nexus supports .PNG, .JPG, and .JPEG. If you have other file types, such as DICOM, you will have to split into supported file types in order for the data to be usable. If you upload files of the same name, the most recently uploaded file will replace the old file, so please make sure to have separate names for different data. Each image should be less than 10 MB in size.

External Storage Sync

When syncing video assets from an external storage such as Amazon S3 or Azure Blob Storage, Synced images must now satisfy the following new criteria because we cannot strip EXIF tags from synced assets:

• The asset must have no EXIF orientation tag, or the image must have an EXIF orientation of 1 (i.e., it is already in the upright position)

Video Requirements

Upload

We support uploading of any MP4 files. To improve performance on our annotator, the video will be resized accordingly such that the longest dimension is 1024 pixels, and some lossy compression will be applied to the individual video frames. We also remove any audio tracks if present in the video, so there will be no sound when playing the video on our platform.

External Storage Sync

When syncing video assets from an external storage such as Amazon S3 or Azure Blob Storage, any MP4 files are supported, but with the following restrictions:

• major_brand or compatible_brand must include at least one of the supported brands: isom, iso2, mp41, mp42
• Pixel format (pix_fmt): yuv420p
• Video stream metadata should not contain any mentions of 4:4:4 or 4:2:2
• Number of frames (nb_frames): Should be similar to r_frame_rate * time_base * duration_ts
• [If Present] sample_aspect_ratio and SAR should either be 1:1 or any corresponding 1-to-1 ratio.

To check the above metadata in your videos, you can use ffprobe to retrieve the information:

# For Ubuntu-based systems
sudo apt-get install -y ffprobe
ffprobe <YOUR_VIDEO>.mp4

These restrictions ensure that the videos can play in all supported browsers in the annotator. Typical MP4 files should be able to meet these requirements.

Since we do not hold your video data on our platform, externally-synced video assets will not be modified in any way. This means that they will retain their original dimensions, quality, and audio, if any.

Medical Imaging Requirements

Upload

We support uploading of NIfTI and DICOM files, which are two commonly used medical image formats, but only through our Python SDK. As shown in the above link, DICOM files are integrated onto the platform as they are because they come in 2D slices, but NIfTI files allow for multiple angles for slicing the 3D imaging, which can be selected as an option.

Custom Asset Metadata

📘

The metadata size that can be attached depends on your price plan. Please reference here for more information!

You can store custom asset metadata such as timestamps and geospatial information together with your asset upload. This can be done using our Python SDK as shown in the code snippet below:

import datature

datature.secret_key = "5aa41e8ba........"

upload_session = datature.Asset.upload_session()
metadata = {
  "capturedAt": 1698402314,
  "latitude": 1.2796709,
  "longitude": 103.8564199,
  ...
}
upload_session.add("image.png", custom_metadata=metadata)
upload_session.start()

🚧

Only single-layered JSON objects are allowed to be uploaded as metadata. Examples of supported metadata types include strings, integers, floats. and boolean values. Nested lists and dictionaries are currently not supported.

Common Questions

What are the options for adding assets?

You can add images manually through our Uploading Data process. If you have assets stored in an S3 bucket, be sure to check out how to Connect Amazon S3 Bucket. Check Plans and Pricing to see if this option is available at your account tier.

I have some updated image/video data, how do I add them to the platform?

If you have additional or updated data you would like to add to your training after the initial data onboarding, you can upload them via the Assets page. Note that a recently uploaded file will replace the old file if they have the same file name.

Why are my images/videos not uploading?

This issue is likely due to insufficient quota for the current month. There will be a limit on the number of images that you can upload per month, based on your current plan. For videos, the quota is calculated based on the number of frames within the video. Your video may be rejected if the total number of frames exceed your remaining quota. Do check out your Usage Quota to monitor your monthly usage. If you would like to increase your quota, do consider upgrading your plan.

Can I upload new, additional images to the project after the onboarding process?

Yes, if you are still collecting your data or do not have access to the full dataset at the time of project creation, you can always upload your new, additional images later. Simply go to Assets and upload your images by dragging your files or folders into the box.

How do I delete images that I have uploaded on Nexus?

You can go to the Assets page and select the more options button at the bottom right of the image, and then select the Delete Image option.

What if my data is stored on Amazon S3?

We have recently released a new Connect Amazon S3 Bucket feature. Our users can now connect their AmazonS3 Bucket to Datature seamlessly. To see if this feature is a part of your plan tier, go to Plans and Pricing.

What happens if I upload duplicate files or different files with the same file name?

The most recently uploaded file will replace the old file, so please make sure to have separate names for different data.

What file formats do you support?

Nexus supports .PNG, .JPG, and .JPEG for images, .mp4 for videos, as well as medical scan file types like DICOM and NIfTI but only through our Python SDK.

Is there an image file size restriction?

Yes, an image should be less than 10 MB.

Can other collaborators upload data as well?

Yes, your collaborators can upload data on your projects. Go to Managing Project Collaborators to see how that can be managed.

What if I only have images but not annotation data?

While image data is the minimum requirement for using the Datature platform to build your computer vision pipeline, annotation data is not. You can always annotate your images using our Annotator. However, we do not have any support for models with purely unannotated images, so be sure to annotate before you run any workflows.