astronomer.providers.microsoft.azure.triggers.wasb

Module Contents

Classes

WasbBlobSensorTrigger

WasbBlobSensorTrigger is fired as deferred class with params to run the task in trigger worker to check for

WasbPrefixSensorTrigger

WasbPrefixSensorTrigger is fired as a deferred class with params to run the task in trigger worker.
class astronomer.providers.microsoft.azure.triggers.wasb.WasbBlobSensorTrigger(container_name, blob_name, wasb_conn_id='wasb_default', public_read=False, poke_interval=5.0)[source]

Bases: airflow.triggers.base.BaseTrigger

WasbBlobSensorTrigger is fired as deferred class with params to run the task in trigger worker to check for existence of the given blob in the provided container.

Parameters:

  • container_name (str) – name of the container in which the blob should be searched for

  • blob_name (str) – name of the blob to check existence for

  • wasb_conn_id (str) – the connection identifier for connecting to Azure WASB

  • poke_interval (float) – polling period in seconds to check for the status

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

serialize()[source]

Serializes WasbBlobSensorTrigger arguments and classpath.

async run()[source]

Makes async connection to Azure WASB and polls for existence of the given blob name.

class astronomer.providers.microsoft.azure.triggers.wasb.WasbPrefixSensorTrigger(container_name, prefix, include=None, delimiter='/', wasb_conn_id='wasb_default', public_read=False, poke_interval=5.0)[source]

Bases: airflow.triggers.base.BaseTrigger

WasbPrefixSensorTrigger is fired as a deferred class with params to run the task in trigger worker. It checks for the existence of a blob with the given prefix in the provided container.

Parameters:

  • container_name (str) – name of the container in which the blob should be searched for

  • prefix (str) – prefix of the blob to check existence for

  • include (Optional[List[str]]) – specifies one or more additional datasets to include in the response. Options include: snapshots, metadata, uncommittedblobs, copy`, ``deleted

  • delimiter (Optional[str]) – filters objects based on the delimiter (for e.g ‘.csv’)

  • wasb_conn_id (str) – the connection identifier for connecting to Azure WASB

  • poke_interval (float) – polling period in seconds to check for the status

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

serialize()[source]

Serializes WasbPrefixSensorTrigger arguments and classpath.

async run()[source]

Makes async connection to Azure WASB and polls for existence of a blob with given prefix.