astronomer.providers.databricks.hooks.databricks

Module Contents

Classes

DatabricksHookAsync

Interact with Databricks.

Attributes

USER_AGENT_HEADER

astronomer.providers.databricks.hooks.databricks.USER_AGENT_HEADER
class astronomer.providers.databricks.hooks.databricks.DatabricksHookAsync(databricks_conn_id=BaseDatabricksHook.default_conn_name, timeout_seconds=180, retry_limit=3, retry_delay=1.0, retry_args=None, caller='DatabricksHook')[source]

Bases: airflow.providers.databricks.hooks.databricks.DatabricksHook

Interact with Databricks.

Parameters:

  • databricks_conn_id (str) – Reference to the Databricks connection.

  • timeout_seconds (int) – The amount of time in seconds the requests library will wait before timing-out.

  • retry_limit (int) – The number of times to retry the connection in case of service outages.

  • retry_delay (float) – The number of seconds to wait between retries (it might be a floating point number).

async get_run_state_async(run_id)[source]

Retrieves run state of the run using an asynchronous api call. :param run_id: id of the run :return: state of the run

async get_run_response(run_id)[source]

Makes Async API call to get the run state info.

Parameters:

run_id (str) – id of the run

async get_run_output_response(task_run_id)[source]

Retrieves run output of the run.

Parameters:

task_run_id (str) – id of the run