`astronomer.providers.google.cloud.hooks.bigquery`¶

Module Contents¶

`BigQueryHookAsync`	Big query async hook inherits from GoogleBaseHookAsync class and connects to the Google Big Query
`BigQueryTableHookAsync`	Class to get async hook for Bigquery Table Async

class astronomer.providers.google.cloud.hooks.bigquery.BigQueryHookAsync(**kwargs)[source]¶

Big query async hook inherits from GoogleBaseHookAsync class and connects to the Google Big Query

async get_job_instance(project_id, job_id, session)[source]¶

Get the specified job resource by job ID and project ID.

async get_job_status(job_id, project_id=None)[source]¶

Polls for job status asynchronously using gcloud-aio.

Note that an OSError is raised when Job results are still pending. Exception means that Job finished with errors

async get_job_output(job_id, project_id=None)[source]¶

Get the big query job output for the given job id asynchronously using gcloud-aio.

get_records(query_results)[source]¶

Given the output query response from gcloud aio bigquery, convert the response to records.

Parameters:: query_results (Dict[str, Any]) – the results from a SQL query

value_check(sql, pass_value, records, tolerance=None)[source]¶

Match a single query resulting row and tolerance with pass_value

interval_check(row1, row2, metrics_thresholds, ignore_zero, ratio_formula)[source]¶

Checks that the values of metrics given as SQL expressions are within a certain tolerance

Parameters:

row1 (Optional[str]) – first resulting row of a query execution job for first SQL query
row2 (Optional[str]) – first resulting row of a query execution job for second SQL query
metrics_thresholds (Dict[str, Any]) – a dictionary of ratios indexed by metrics, for example ‘COUNT(*)’: 1.5 would require a 50 percent or less difference between the current day, and the prior days_back.
ignore_zero (bool) – whether we should ignore zero metrics
ratio_formula (str) – which formula to use to compute the ratio between the two metrics. Assuming cur is the metric of today and ref is the metric to today - days_back. max_over_min: computes max(cur, ref) / min(cur, ref) relative_diff: computes abs(cur-ref) / ref

class astronomer.providers.google.cloud.hooks.bigquery.BigQueryTableHookAsync(**kwargs)[source]¶

Class to get async hook for Bigquery Table Async

async get_table_client(dataset, table_id, project_id, session)[source]¶

Returns a Google Big Query Table object.

Parameters:

dataset (str) – The name of the dataset in which to look for the table storage bucket.
table_id (str) – The name of the table to check the existence of.
project_id (str) – The Google cloud project in which to look for the table. The connection supplied to the hook must provide access to the specified project.
session (aiohttp.ClientSession) – aiohttp ClientSession