astronomer.providers.microsoft.azure.triggers.data_factory

Module Contents

Classes

ADFPipelineRunStatusSensorTrigger

ADFPipelineRunStatusSensorTrigger is fired as deferred class with params to run the task in trigger worker, when

AzureDataFactoryTrigger

AzureDataFactoryTrigger is triggered when Azure data factory pipeline job succeeded or failed.
class astronomer.providers.microsoft.azure.triggers.data_factory.ADFPipelineRunStatusSensorTrigger(run_id, azure_data_factory_conn_id, poke_interval, resource_group_name=None, factory_name=None)[source]

Bases: airflow.triggers.base.BaseTrigger

ADFPipelineRunStatusSensorTrigger is fired as deferred class with params to run the task in trigger worker, when ADF Pipeline is running

Parameters:

  • run_id (str) – The pipeline run identifier.

  • azure_data_factory_conn_id (str) – The connection identifier for connecting to Azure Data Factory.

  • poke_interval (float) – polling period in seconds to check for the status

  • resource_group_name (Optional[str]) – The resource group name.

  • factory_name (Optional[str]) – The data factory name.

serialize()[source]

Serializes ADFPipelineRunStatusSensorTrigger arguments and classpath.

async run()[source]

Make async connection to Azure Data Factory, polls for the pipeline run status

class astronomer.providers.microsoft.azure.triggers.data_factory.AzureDataFactoryTrigger(run_id, azure_data_factory_conn_id, end_time, resource_group_name=None, factory_name=None, wait_for_termination=True, check_interval=60)[source]

Bases: airflow.triggers.base.BaseTrigger

AzureDataFactoryTrigger is triggered when Azure data factory pipeline job succeeded or failed. When wait_for_termination is set to False it triggered immediately with success status

Parameters:

  • run_id (str) – Run id of a Azure data pipeline run job.

  • azure_data_factory_conn_id (str) – The connection identifier for connecting to Azure Data Factory.

  • end_time (float) – Time in seconds when triggers will timeout.

  • resource_group_name (Optional[str]) – The resource group name.

  • factory_name (Optional[str]) – The data factory name.

  • wait_for_termination (bool) – Flag to wait on a pipeline run’s termination.

  • check_interval (int) – Time in seconds to check on a pipeline run’s status.

QUEUED = 'Queued'
IN_PROGRESS = 'InProgress'
SUCCEEDED = 'Succeeded'
FAILED = 'Failed'
CANCELING = 'Canceling'
CANCELLED = 'Cancelled'
INTERMEDIATE_STATES: List[str]
FAILURE_STATES: List[str]
SUCCESS_STATES: List[str]
TERMINAL_STATUSES: List[str]
serialize()[source]

Serializes AzureDataFactoryTrigger arguments and classpath.

async run()[source]

Make async connection to Azure Data Factory, polls for the pipeline run status