Job Management Functions

The functions listed on this page help with managing the creation and execution of inference batch jobs.

Create Job

datature batch jobs create

Creates a new batch job that runs inference on a specfied dataset and sends the prediction results to a specified endpoint URL.

Sample Output

$ datature batch jobs create

? Enter a name for the BatchJob: my-job
? Select a timezone: UTC
? Enter when you want the BatchJob to start [yyyy-mm-ddThh:mm:ss]: 
Start Time: August 19, 2024 02:03 PM UTC (Now)
? Enter when you want the BatchJob to stop [yyyy-mm-ddThh:mm:ss]: 
Stop Time: After all Dataset items are predicted
? Select how you want the BatchJob to deliver prediction results: Select an existing Webhook
? Which entry do you want to select? my-webhook
? Select how you want to specify the Dataset for prediction by this BatchJob: Select an existing Dataset
? Which entry do you want to select? my-dataset-08-19  [Expires in 26 days]
? Select how you want the BatchJob to perform prediction on Dataset items: Select an existing Deployment
? Which deployment do you want to select? batch-job-test

✓ Job created successfully.

ID:                  batchjob_e69a2dfd-beac-4c06-b982-42333d10443c
Name:                my-job
Start Time:          2024-08-19 22:03:37
Stop Time:           2024-08-22 22:03:37
Dataset ID:          dataset_df94c9d7-d31c-44af-90da-8b49f3c4fe08

Endpoint:
  - Webhook:         webhook_f92af240-2600-485a-a36f-96cf86bf5a33

Status:              Waiting
Message:             Waiting
Reason:              None

Items Processed           Count      Description
  - Total:                50         Total item count in the Dataset.        
  - Gathered:             0          Items successfully retrieved from the Dataset.
  - FailedGather:         0          Items failed to be retrieved from the Dataset.
  - Preprocessed:         0          Items successfully preprocessed.
  - FailedPreprocess:     0          Items failed to be preprocessed.
  - Predicted:            0          Items successfully predicted.
  - FailedPredict:        0          Items failed to be predicted.
  - Delivered:            0          Prediction results successfully delivered to the ResultDestination.
  - FailedDeliver:        0          Prediction results failed to be delivered to the ResultDestination.

List Jobs

datature batch jobs list

Lists all available jobs in the project.

Sample Output

$ datature batch jobs list

ID                                            Name                 Status              
batchjob_61edb503-c72f-4953-8fcf-068830a51e00 my-job-2024-08-07... Reached             
batchjob_3c89aa6d-de1d-416a-b35b-79bb47bc73e2 my-job-2024-08-07... Failed              
batchjob_3d561cfa-9a53-4eb3-81c1-4193ae27f4fc my-job-2024-08-08... Reached

Get Job

datature batch jobs get [JOB_ID]

Gets a batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.

Sample Output

$ datature batch jobs get

? Which entry do you want to select? 
❯ my-job-2024-08-07
  my-job-2024-08-08
  
? Which entry do you want to select? my-job-2024-08-07

ID:             batchjob_c99fb953-abc0-485e-af53-b502629d85e3
Name:           my-job-2024-08-07
Start Time:     1723089755714
Stop Time:      1723096954989
Dataset ID:     dataset_a9242c7b-b0f6-4e4a-b40a-716ab9042f5c

Endpoint:
  - Webhook ID: webhook_8a98ecec-7cc7-49d0-b6cc-4510e3ec4aea

Status:         Reached
Message:        Finished
Reason:         None

Items Processed           Count      Description
  - Total:                50         Total item count in the Dataset.        
  - Gathered:             50         Items successfully retrieved from the Dataset.
  - FailedGather:         0          Items failed to be retrieved from the Dataset.
  - Preprocessed:         50         Items successfully preprocessed.
  - FailedPreprocess:     0          Items failed to be preprocessed.
  - Predicted:            50         Items successfully predicted.
  - FailedPredict:        0          Items failed to be predicted.
  - Delivered:            50         Prediction results successfully delivered to the ResultDestination.
  - FailedDeliver:        0          Prediction results failed to be delivered to the ResultDestination.

Cancel Job

datature batch jobs cancel [JOB_ID]

Cancels a scheduled or running batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.

Sample Output

$ datature batch jobs cancel

? Which entry do you want to select? 
❯ my-dataset-2024-08-07
  my-dataset-2024-08-08
  
? Which entry do you want to select? my-dataset-2024-08-07
✓ Job cancelled successfully.

Wait Until Done

datature batch jobs wait-until-done [JOB_ID]

Waits for a batch job to be completed. Shows an interactive progress bar to indicate batch job progress.

Sample Output

$ datature batch jobs wait-until-done

? Which entry do you want to select? batchjob-2024-08-27-16-23-49
Processing Batch Job |████████████████████| 50/50 [100%] in 2:43.3 (0.22/s) 

✓ Job completed successfully.

ID:                       batchjob_899bd27a-f4de-4186-b84c-89592e4cd85a
Name:                     batchjob-2024-08-27-16-23-49
Start Time:               2024-08-27 16:24:15
Cutoff Time:              2024-08-30 16:24:15
Dataset ID:               dataset_9767ee76-1ee9-4422-bda9-b94c1f6dedba

Endpoint:
  - Webhook:              webhook_2e9bf419-28dd-4def-ae18-62e99d66ba4c

Status:                   Waiting
Message:                  Waiting
Reason:                   None

Items Processed           Count      Description
  - Total:                50         Total item count in the Dataset.        
  - Gathered:             50         Items successfully retrieved from the Dataset.
  - FailedGather:         0          Items failed to be retrieved from the Dataset.
  - Preprocessed:         50         Items successfully preprocessed.
  - FailedPreprocess:     0          Items failed to be preprocessed.
  - Predicted:            50         Items successfully predicted.
  - FailedPredict:        0          Items failed to be predicted.
  - Delivered:            50         Prediction results successfully delivered to the ResultDestination.
  - FailedDeliver:        0          Prediction results failed to be delivered to the ResultDestination.

Get Job Logs

datature batch jobs logs [JOB_ID]
    [--output OUTPUT_FILE_PATH]
    [--since TIMESTAMP_MS]
    [--until TIMESTAMP_MS]
    [--max_entries MAX_ENTRIES]
    [--level LEVEL]

Retrieves item-level logs for a specified batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.

--output OUTPUT_FILE_PATH: File path to dump the logs to, prints to terminal if not provided. Printing to terminal is capped at a maximum of 1000 entries.
--since TIMESTAMP_MS: Print only logs occurring after a specified timestamp (in milliseconds).
--until TIMESTAMP_MS: Print only logs occurring before a specified timestamp (in milliseconds).
--max_entries MAX_ENTRIES: Print only the N latest entries. Same functionality as datature batch jobs logs [JOB_ID] | tail -n MAX_ENTRIES.
--level: The logging level to filter logs output, either Info or Error.

Sample Output

$ datature batch jobs logs
? Which entry do you want to select? my-batch-job-08-16

TIME            LEVEL  ITEM                                        STATUS          DURATION (ms)   MESSAGE             
1723792158069   Info   https://my-website.url/blood_cell/50.jpg    Delivered       7920                                
1723792157971   Info   https://my-website.url/blood_cell/49.jpg    Delivered       7822                                
1723792157804   Info   https://my-website.url/blood_cell/48.jpg    Delivered       7655

Delete Job

datature batch jobs delete [JOB_ID]

Deletes a batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.

$ datature batch jobs delete

? Which entry do you want to select? 
❯ my-dataset-2024-08-07
  my-dataset-2024-08-08
  
? Which entry do you want to select? my-dataset-2024-08-07
✓ Job deleted successfully.