:py:mod:`astronomer.providers.amazon.aws.triggers.emr`
======================================================

.. py:module:: astronomer.providers.amazon.aws.triggers.emr


Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   astronomer.providers.amazon.aws.triggers.emr.EmrContainerSensorTrigger
   astronomer.providers.amazon.aws.triggers.emr.EmrContainerOperatorTrigger
   astronomer.providers.amazon.aws.triggers.emr.EmrStepSensorTrigger
   astronomer.providers.amazon.aws.triggers.emr.EmrJobFlowSensorTrigger




.. py:class:: EmrContainerSensorTrigger(virtual_cluster_id, job_id, max_retries = None, aws_conn_id = 'aws_default', poll_interval = 10)

   Bases: :py:obj:`airflow.triggers.base.BaseTrigger`

   The EmrContainerSensorTrigger is triggered when EMR container is created, it polls for the AWS EMR EKS Virtual
   Cluster Job status. It is fired as deferred class with params to run the task in trigger worker

   :param virtual_cluster_id: Reference Emr cluster id
   :param job_id:  job_id to check the state
   :param max_retries: maximum retry for poll for the status
   :param aws_conn_id: Reference to AWS connection id
   :param poll_interval: polling period in seconds to check for the status

   .. py:method:: serialize(self)

      Serializes EmrContainerSensorTrigger arguments and classpath.


   .. py:method:: run(self)
      :async:

      Make async connection to EMR container, polls for the job state



.. py:class:: EmrContainerOperatorTrigger(virtual_cluster_id, name, job_id, aws_conn_id = 'aws_default', poll_interval = 30, max_tries = None)

   Bases: :py:obj:`airflow.triggers.base.BaseTrigger`

   The EmrContainerSensorTrigger is triggered when EMR container is created, it polls for the AWS EMR EKS Virtual
   Cluster Job status. It is fired as deferred class with params to run the task in trigger worker

   :param virtual_cluster_id: The EMR on EKS virtual cluster ID.
   :param name: The name of the job run.
   :param execution_role_arn: The IAM role ARN associated with the job run.
   :param release_label: The Amazon EMR release version to use for the job run.
   :param job_driver: Job configuration details, e.g. the Spark job parameters.
   :param configuration_overrides: The configuration overrides for the job run,
           specifically either application configuration or monitoring configuration.
   :param client_request_token: The client idempotency token of the job run request.
   :param aws_conn_id: Reference to AWS connection id.
   :param poll_interval: polling period in seconds to check for the status.
   :param max_retries: maximum retry for poll for the status.

   .. py:attribute:: INTERMEDIATE_STATES
      :annotation: :List[str] = ['PENDING', 'SUBMITTED', 'RUNNING']

      

   .. py:attribute:: FAILURE_STATES
      :annotation: :List[str] = ['FAILED', 'CANCELLED', 'CANCEL_PENDING']

      

   .. py:attribute:: SUCCESS_STATES
      :annotation: :List[str] = ['COMPLETED']

      

   .. py:attribute:: TERMINAL_STATES
      :annotation: :List[str] = ['COMPLETED', 'FAILED', 'CANCELLED', 'CANCEL_PENDING']

      

   .. py:method:: serialize(self)

      Serializes EmrContainerOperatorTrigger arguments and classpath.


   .. py:method:: run(self)
      :async:

      Run until EMR container reaches the desire state



.. py:class:: EmrStepSensorTrigger(job_flow_id, step_id, aws_conn_id, poke_interval, target_states = None, failed_states = None)

   Bases: :py:obj:`airflow.triggers.base.BaseTrigger`

   A trigger that fires once AWS EMR cluster step reaches either target or failed state

   :param job_flow_id: job_flow_id which contains the step check the state of
   :param step_id: step to check the state of
   :param aws_conn_id: aws connection to use, defaults to 'aws_default'
   :param poke_interval: Time in seconds to wait between two consecutive call to
       check emr cluster step state
   :param target_states: the target states, sensor waits until
       step reaches any of these states
   :param failed_states: the failure states, sensor fails when
       step reaches any of these states

   .. py:method:: serialize(self)

      Serializes EmrStepSensorTrigger arguments and classpath.


   .. py:method:: run(self)
      :async:

      Run until AWS EMR cluster step reach target or failed state



.. py:class:: EmrJobFlowSensorTrigger(job_flow_id, aws_conn_id, poll_interval, target_states = None, failed_states = None)

   Bases: :py:obj:`airflow.triggers.base.BaseTrigger`

   EmrJobFlowSensorTrigger is fired as deferred class with params to run the task in trigger worker, when
   EMR JobFlow is created

   :param job_flow_id: job_flow_id to check the state of
   :param target_states: the target states, sensor waits until
       job flow reaches any of these states
   :param failed_states: the failure states, sensor fails when
       job flow reaches any of these states
   :param poll_interval:  polling period in seconds to check for the status

   .. py:method:: serialize(self)

      Serializes EmrJobFlowSensorTrigger arguments and classpath.


   .. py:method:: run(self)
      :async:

      Make async connection to EMR container, polls for the target job state