astronomer.providers.amazon.aws.triggers.batch

Module Contents

Classes

BatchOperatorTrigger

Checks for the state of a previously submitted job to AWS Batch.
class astronomer.providers.amazon.aws.triggers.batch.BatchOperatorTrigger(job_id, waiters, max_retries, region_name, aws_conn_id='aws_default')[source]

Bases: airflow.triggers.base.BaseTrigger

Checks for the state of a previously submitted job to AWS Batch. BatchOperatorTrigger is fired as deferred class with params to poll the job state in Triggerer

This class is deprecated and will be removed in 2.0.0. Use :class: ~airflow.providers.amazon.aws.triggers.batch.BatchOperatorTrigger instead

Parameters:

  • job_id (str | None) – the job ID, usually unknown (None) until the submit_job operation gets the jobId defined by AWS Batch

  • waiters (Any) – a BatchWaiters object (see note below); if None, polling is used with max_retries and status_retries.

  • max_retries (int) – exponential back-off retries, 4200 = 48 hours; polling is only used when waiters is None

  • aws_conn_id (str | None) – connection id of AWS credentials / region name. If None, credential boto3 strategy will be used.

  • region_name (str | None) – AWS region name to use . Override the region_name in connection (if provided)

serialize()[source]

Serializes BatchOperatorTrigger arguments and classpath.

async run()[source]

Make async connection using aiobotocore library to AWS Batch, periodically poll for the job status on the Triggerer

The status that indicates job completion are: ‘SUCCEEDED’|’FAILED’.

So the status options that this will poll for are the transitions from: ‘SUBMITTED’>’PENDING’>’RUNNABLE’>’STARTING’>’RUNNING’>’SUCCEEDED’|’FAILED’