astronomer.providers.microsoft.azure.sensors.wasb

Module Contents

Classes

WasbBlobSensorAsync

Polls asynchronously for the existence of a blob in a WASB container.

WasbPrefixSensorAsync

Polls asynchronously for the existence of a blob having the given prefix in a WASB container.
class astronomer.providers.microsoft.azure.sensors.wasb.WasbBlobSensorAsync(*, container_name, blob_name, wasb_conn_id='wasb_default', public_read=False, poll_interval=5.0, **kwargs)[source]

Bases: airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor

Polls asynchronously for the existence of a blob in a WASB container.

Parameters:

  • container_name (str) – name of the container in which the blob should be searched for

  • blob_name (str) – name of the blob to check existence for

  • wasb_conn_id (str) – the connection identifier for connecting to Azure WASB

  • poll_interval (float) – polling period in seconds to check for the status

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

execute(context)[source]

Defers trigger class to poll for state of the job run until it reaches a failure state or success state

execute_complete(context, event)[source]

Callback for when the trigger fires - returns immediately. Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class astronomer.providers.microsoft.azure.sensors.wasb.WasbPrefixSensorAsync(*, container_name, prefix, include=None, delimiter='/', wasb_conn_id='wasb_default', public_read=False, poll_interval=5.0, **kwargs)[source]

Bases: airflow.providers.microsoft.azure.sensors.wasb.WasbPrefixSensor

Polls asynchronously for the existence of a blob having the given prefix in a WASB container.

Parameters:

  • container_name (str) – name of the container in which the blob should be searched for

  • blob_name – name of the blob to check existence for

  • include (Optional[List[str]]) – specifies one or more additional datasets to include in the response. Options include: snapshots, metadata, uncommittedblobs, copy`, ``deleted

  • delimiter (Optional[str]) – filters objects based on the delimiter (for e.g ‘.csv’)

  • wasb_conn_id (str) – the connection identifier for connecting to Azure WASB

  • poll_interval (float) – polling period in seconds to check for the status

  • public_read (bool) – whether an anonymous public read access should be used. Default is False

execute(context)[source]

Defers trigger class to poll for state of the job run until it reaches a failure state or success state

execute_complete(context, event)[source]

Callback for when the trigger fires - returns immediately. Relies on trigger to throw an exception, otherwise it assumes execution was successful.