:py:mod:`astronomer.providers.amazon.aws.operators.batch` ========================================================= .. py:module:: astronomer.providers.amazon.aws.operators.batch .. autoapi-nested-parse:: A Deferrable Airflow operator for AWS Batch services .. seealso:: - `Configuration `_ - `Batch `_ - `Welcome `_ Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: astronomer.providers.amazon.aws.operators.batch.BatchOperatorAsync .. py:class:: BatchOperatorAsync(*, job_name, job_definition, job_queue, overrides, array_properties = None, parameters = None, job_id = None, waiters = None, max_retries = None, status_retries = None, aws_conn_id = None, region_name = None, tags = None, wait_for_completion = True, **kwargs) Bases: :py:obj:`airflow.providers.amazon.aws.operators.batch.BatchOperator` Execute a job asynchronously on AWS Batch .. see also:: For more information on how to use this operator, take a look at the guide: :ref:`howto/operator:BatchOperator` :param job_name: the name for the job that will run on AWS Batch (templated) :param job_definition: the job definition name on AWS Batch :param job_queue: the queue name on AWS Batch :param overrides: the `containerOverrides` parameter for boto3 (templated) :param array_properties: the `arrayProperties` parameter for boto3 :param parameters: the `parameters` for boto3 (templated) :param job_id: the job ID, usually unknown (None) until the submit_job operation gets the jobId defined by AWS Batch :param waiters: an :class:`.BatchWaiters` object (see note below); if None, polling is used with max_retries and status_retries. :param max_retries: exponential back-off retries, 4200 = 48 hours; polling is only used when waiters is None :param status_retries: number of HTTP retries to get job status, 10; polling is only used when waiters is None :param aws_conn_id: connection id of AWS credentials / region name. If None, credential boto3 strategy will be used. :param region_name: region name to use in AWS Hook. Override the region_name in connection (if provided) :param tags: collection of tags to apply to the AWS Batch job submission if None, no tags are submitted .. note:: Any custom waiters must return a waiter for these calls: | ``waiter = waiters.get_waiter("JobExists")`` | ``waiter = waiters.get_waiter("JobRunning")`` | ``waiter = waiters.get_waiter("JobComplete")`` .. py:method:: execute(context) Airflow runs this method on the worker and defers using the trigger. Submit the job and get the job_id using which we defer and poll in trigger .. py:method:: execute_complete(context, event) Callback for when the trigger fires - returns immediately. Relies on trigger to throw an exception, otherwise it assumes execution was successful.