Job Management
Job Management Functions
The functions listed on this page help with managing the creation and execution of inference batch jobs.
Create Job
datature batch jobs create
Creates a new batch job that runs inference on a specfied dataset and sends the prediction results to a specified endpoint URL.
Sample Output
$ datature batch jobs create
? Enter a name for the BatchJob: my-job
? Select a timezone: UTC
? Enter when you want the BatchJob to start [yyyy-mm-ddThh:mm:ss]:
Start Time: August 19, 2024 02:03 PM UTC (Now)
? Enter when you want the BatchJob to stop [yyyy-mm-ddThh:mm:ss]:
Stop Time: After all Dataset items are predicted
? Select how you want the BatchJob to deliver prediction results: Select an existing Webhook
? Which entry do you want to select? my-webhook
? Select how you want to specify the Dataset for prediction by this BatchJob: Select an existing Dataset
? Which entry do you want to select? my-dataset-08-19 [Expires in 26 days]
? Select how you want the BatchJob to perform prediction on Dataset items: Select an existing Deployment
? Which deployment do you want to select? batch-job-test
✓ Job created successfully.
ID: batchjob_e69a2dfd-beac-4c06-b982-42333d10443c
Name: my-job
Start Time: 2024-08-19 22:03:37
Stop Time: 2024-08-22 22:03:37
Dataset ID: dataset_df94c9d7-d31c-44af-90da-8b49f3c4fe08
Endpoint:
- Webhook: webhook_f92af240-2600-485a-a36f-96cf86bf5a33
Status: Waiting
Message: Waiting
Reason: None
Items Processed Count Description
- Total: 50 Total item count in the Dataset.
- Gathered: 0 Items successfully retrieved from the Dataset.
- FailedGather: 0 Items failed to be retrieved from the Dataset.
- Preprocessed: 0 Items successfully preprocessed.
- FailedPreprocess: 0 Items failed to be preprocessed.
- Predicted: 0 Items successfully predicted.
- FailedPredict: 0 Items failed to be predicted.
- Delivered: 0 Prediction results successfully delivered to the ResultDestination.
- FailedDeliver: 0 Prediction results failed to be delivered to the ResultDestination.
List Jobs
datature batch jobs list
Lists all available jobs in the project.
Sample Output
$ datature batch jobs list
ID Name Status
batchjob_61edb503-c72f-4953-8fcf-068830a51e00 my-job-2024-08-07... Reached
batchjob_3c89aa6d-de1d-416a-b35b-79bb47bc73e2 my-job-2024-08-07... Failed
batchjob_3d561cfa-9a53-4eb3-81c1-4193ae27f4fc my-job-2024-08-08... Reached
Get Job
datature batch jobs get [JOB_ID]
Gets a batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.
Sample Output
$ datature batch jobs get
? Which entry do you want to select?
❯ my-job-2024-08-07
my-job-2024-08-08
? Which entry do you want to select? my-job-2024-08-07
ID: batchjob_c99fb953-abc0-485e-af53-b502629d85e3
Name: my-job-2024-08-07
Start Time: 1723089755714
Stop Time: 1723096954989
Dataset ID: dataset_a9242c7b-b0f6-4e4a-b40a-716ab9042f5c
Endpoint:
- Webhook ID: webhook_8a98ecec-7cc7-49d0-b6cc-4510e3ec4aea
Status: Reached
Message: Finished
Reason: None
Items Processed Count Description
- Total: 50 Total item count in the Dataset.
- Gathered: 50 Items successfully retrieved from the Dataset.
- FailedGather: 0 Items failed to be retrieved from the Dataset.
- Preprocessed: 50 Items successfully preprocessed.
- FailedPreprocess: 0 Items failed to be preprocessed.
- Predicted: 50 Items successfully predicted.
- FailedPredict: 0 Items failed to be predicted.
- Delivered: 50 Prediction results successfully delivered to the ResultDestination.
- FailedDeliver: 0 Prediction results failed to be delivered to the ResultDestination.
Cancel Job
datature batch jobs cancel [JOB_ID]
Cancels a scheduled or running batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.
Sample Output
$ datature batch jobs cancel
? Which entry do you want to select?
❯ my-dataset-2024-08-07
my-dataset-2024-08-08
? Which entry do you want to select? my-dataset-2024-08-07
✓ Job cancelled successfully.
Wait Until Done
datature batch jobs wait-until-done [JOB_ID]
Waits for a batch job to be completed. Shows an interactive progress bar to indicate batch job progress.
Sample Output
$ datature batch jobs wait-until-done
? Which entry do you want to select? batchjob-2024-08-27-16-23-49
Processing Batch Job |████████████████████| 50/50 [100%] in 2:43.3 (0.22/s)
✓ Job completed successfully.
ID: batchjob_899bd27a-f4de-4186-b84c-89592e4cd85a
Name: batchjob-2024-08-27-16-23-49
Start Time: 2024-08-27 16:24:15
Cutoff Time: 2024-08-30 16:24:15
Dataset ID: dataset_9767ee76-1ee9-4422-bda9-b94c1f6dedba
Endpoint:
- Webhook: webhook_2e9bf419-28dd-4def-ae18-62e99d66ba4c
Status: Waiting
Message: Waiting
Reason: None
Items Processed Count Description
- Total: 50 Total item count in the Dataset.
- Gathered: 50 Items successfully retrieved from the Dataset.
- FailedGather: 0 Items failed to be retrieved from the Dataset.
- Preprocessed: 50 Items successfully preprocessed.
- FailedPreprocess: 0 Items failed to be preprocessed.
- Predicted: 50 Items successfully predicted.
- FailedPredict: 0 Items failed to be predicted.
- Delivered: 50 Prediction results successfully delivered to the ResultDestination.
- FailedDeliver: 0 Prediction results failed to be delivered to the ResultDestination.
Get Job Logs
datature batch jobs logs [JOB_ID]
[--output OUTPUT_FILE_PATH]
[--since TIMESTAMP_MS]
[--until TIMESTAMP_MS]
[--max_entries MAX_ENTRIES]
[--level LEVEL]
Retrieves item-level logs for a specified batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.
--output OUTPUT_FILE_PATH
: File path to dump the logs to, prints to terminal if not provided. Printing to terminal is capped at a maximum of 1000 entries.--since TIMESTAMP_MS
: Print only logs occurring after a specified timestamp (in milliseconds).--until TIMESTAMP_MS
: Print only logs occurring before a specified timestamp (in milliseconds).--max_entries MAX_ENTRIES
: Print only the N latest entries. Same functionality asdatature batch jobs logs [JOB_ID] | tail -n MAX_ENTRIES
.--level
: The logging level to filter logs output, eitherInfo
orError
.
Sample Output
$ datature batch jobs logs
? Which entry do you want to select? my-batch-job-08-16
TIME LEVEL ITEM STATUS DURATION (ms) MESSAGE
1723792158069 Info https://my-website.url/blood_cell/50.jpg Delivered 7920
1723792157971 Info https://my-website.url/blood_cell/49.jpg Delivered 7822
1723792157804 Info https://my-website.url/blood_cell/48.jpg Delivered 7655
Delete Job
datature batch jobs delete [JOB_ID]
Deletes a batch job by ID, if no ID is provided, a dropdown list will be displayed for the user to choose the batch job.
$ datature batch jobs delete
? Which entry do you want to select?
❯ my-dataset-2024-08-07
my-dataset-2024-08-08
? Which entry do you want to select? my-dataset-2024-08-07
✓ Job deleted successfully.
Updated about 2 months ago