Bigquery Operator Async

To execute BigQuery job asynchronously BigQueryInsertJobOperatorAsync.

insert_query_job = BigQueryInsertJobOperatorAsync(
    task_id="insert_query_job",
    configuration={
        "query": {
            "query": INSERT_ROWS_QUERY,
            "useLegacySql": False,
        }
    },
    location=LOCATION,
    gcp_conn_id=GCP_CONN_ID,
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/google/cloud/example_dags/example_bigquery_queries.py

To performs checks against BigQuery result asynchronously BigQueryCheckOperatorAsync.

check_count = BigQueryCheckOperatorAsync(
    task_id="check_count",
    sql=f"SELECT COUNT(*) FROM {DATASET}.{TABLE_1}",
    use_legacy_sql=False,
    location=LOCATION,
    gcp_conn_id=GCP_CONN_ID,
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/google/cloud/example_dags/example_bigquery_queries.py

To fetches the data from a BigQuery table asynchronously BigQueryGetDataOperatorAsync.

get_data = BigQueryGetDataOperatorAsync(
    task_id="get_data",
    dataset_id=DATASET,
    table_id=TABLE_1,
    max_results=10,
    selected_fields="value,name",
    location=LOCATION,
    gcp_conn_id=GCP_CONN_ID,
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/google/cloud/example_dags/example_bigquery_queries.py

To checks that the values of metrics given as SQL expressions are within a certain tolerance BigQueryIntervalCheckOperatorAsync.

check_interval = BigQueryIntervalCheckOperatorAsync(
    task_id="check_interval",
    table=f"{DATASET}.{TABLE_1}",
    days_back=1,
    metrics_thresholds={"COUNT(*)": 1.5},
    use_legacy_sql=False,
    location=LOCATION,
    gcp_conn_id=GCP_CONN_ID,
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/google/cloud/example_dags/example_bigquery_queries.py

To performs a simple value check using sql code BigQueryValueCheckOperatorAsync.

check_value = BigQueryValueCheckOperatorAsync(
    task_id="check_value",
    sql=f"SELECT COUNT(*) FROM {DATASET}.{TABLE_1}",
    pass_value=2,
    use_legacy_sql=False,
    location=LOCATION,
    gcp_conn_id=GCP_CONN_ID,
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/google/cloud/example_dags/example_bigquery_queries.py