astronomer.providers.google.cloud.extractors.bigquery

Module Contents

Classes

SqlContext	Internal SQL context for holding query parser results
BigQueryAsyncExtractor	This extractor provides visibility on the metadata of a BigQuery Insert Job

class astronomer.providers.google.cloud.extractors.bigquery.SqlContext

Internal SQL context for holding query parser results

sql :str

inputs :Optional[str]

outputs :Optional[str]

parser_error :Optional[str]

class astronomer.providers.google.cloud.extractors.bigquery.BigQueryAsyncExtractor(operator)

Bases: openlineage.airflow.extractors.base.BaseExtractor

This extractor provides visibility on the metadata of a BigQuery Insert Job including billedBytes, rowCount, size, etc. submitted from a BigQueryInsertJobOperatorAsync operator.

classmethod get_operator_classnames()

Returns the list of operators this extractor works on.

extract()

Empty extract implementation for the abstractmethod of the BaseExtractor class.

extract_on_complete(task_instance)

Callback on task completion to fetch metadata extraction details that are to be pushed to the Lineage server.

Parameters:

task_instance (airflow.models.taskinstance.TaskInstance) – Instance of the Airflow task whose metadata needs to be extracted.

parse_sql_context()

Gets SQL from operator and transforms it to SqlContext object.