astronomer.providers.amazon.aws.triggers.redshift_cluster

Module Contents

Classes

RedshiftClusterTrigger

RedshiftClusterTrigger is fired as deferred class with params to run the task in trigger worker

RedshiftClusterSensorTrigger

RedshiftClusterSensorTrigger is fired as deferred class with params to run the task in trigger worker
class astronomer.providers.amazon.aws.triggers.redshift_cluster.RedshiftClusterTrigger(task_id, aws_conn_id, cluster_identifier, operation_type, polling_period_seconds=5.0, skip_final_cluster_snapshot=True, final_cluster_snapshot_identifier=None)[source]

Bases: airflow.triggers.base.BaseTrigger

RedshiftClusterTrigger is fired as deferred class with params to run the task in trigger worker

Parameters:

  • task_id (str) – Reference to task id of the Dag

  • polling_period_seconds (float) – polling period in seconds to check for the status

  • aws_conn_id (str) – Reference to AWS connection id for redshift

  • cluster_identifier (str) – unique identifier of a cluster

  • operation_type (str) – Reference to the type of operation need to be performed eg: pause_cluster, resume_cluster, delete_cluster

  • skip_final_cluster_snapshot (bool) – determines cluster snapshot creation

  • final_cluster_snapshot_identifier (Optional[str]) – name of final cluster snapshot

serialize()[source]

Serializes RedshiftClusterTrigger arguments and classpath.

async run()[source]

Make async connection to redshift, based on the operation type call the RedshiftHookAsync functions if operation_type is ‘resume_cluster’ it will call the resume_cluster function in RedshiftHookAsync if operation_type is ‘pause_cluster it will call the pause_cluster function in RedshiftHookAsync

class astronomer.providers.amazon.aws.triggers.redshift_cluster.RedshiftClusterSensorTrigger(task_id, aws_conn_id, cluster_identifier, target_status, poke_interval)[source]

Bases: airflow.triggers.base.BaseTrigger

RedshiftClusterSensorTrigger is fired as deferred class with params to run the task in trigger worker

Parameters:

  • task_id (str) – Reference to task id of the Dag

  • aws_conn_id (str) – Reference to AWS connection id for redshift

  • cluster_identifier (str) – unique identifier of a cluster

  • target_status (str) – Reference to the status which needs to be checked

  • poke_interval (float) – polling period in seconds to check for the status

serialize()[source]

Serializes RedshiftClusterSensorTrigger arguments and classpath.

async run()[source]

Simple async function run until the cluster status match the target status.