astronomer.providers.sftp.triggers.sftp
¶
Module Contents¶
Classes¶
Trigger that fires when either the path on the SFTP server does not exist, |
- class astronomer.providers.sftp.triggers.sftp.SFTPTrigger(path, file_pattern='', sftp_conn_id='sftp_default', newer_than=None, poke_interval=5)[source]¶
Bases:
airflow.triggers.base.BaseTrigger
Trigger that fires when either the path on the SFTP server does not exist, or when there are no files matching the file pattern at the path
- Parameters:
path (str) – The path on the SFTP server to search for a file matching the file pattern. Authentication method used in the SFTP connection must have access to this path
file_pattern (str) – Pattern to be used for matching against the list of files at the path above. Uses the fnmatch module from std library to perform the matching.
sftp_conn_id (str) – SFTP connection ID to be used for connecting to SFTP server
poke_interval (float) – How often, in seconds, to check for the existence of the file on the SFTP server
- async run()[source]¶
Makes a series of asynchronous calls to sftp servers via async sftp hook. It yields a Trigger
If file matching file pattern exists at the specified path return it,
If file pattern was not provided, it looks directly into the specific path which was provided.
If newer then datetime was provided it looks for the file path last modified time and check whether the last modified time is greater, if true return file if false it polls again.