astronomer.providers.google.cloud.hooks.dataproc

Module Contents

Classes

DataprocHookAsync

Async Hook for Google Cloud Dataproc APIs.

Attributes

JobType

astronomer.providers.google.cloud.hooks.dataproc.JobType
class astronomer.providers.google.cloud.hooks.dataproc.DataprocHookAsync(gcp_conn_id='google_cloud_default', delegate_to=None, impersonation_chain=None)[source]

Bases: airflow.providers.google.cloud.hooks.dataproc.DataprocHook

Async Hook for Google Cloud Dataproc APIs.

All the methods in the hook where project_id is used must be called with keyword arguments rather than positional.

async get_job(self, job_id, project_id, region=None, location=None, retry=None, timeout=None, metadata=())[source]

Gets the resource representation for a job in a project.

Parameters

  • job_id (str) – Id of the Dataproc job

  • project_id (str) – Required. The ID of the Google Cloud project the cluster belongs to.

  • region (Optional[str]) – Required. The Cloud Dataproc region in which to handle the request.

  • location (Optional[str]) – (To be deprecated). The Cloud Dataproc region in which to handle the request.

  • retry (Optional[google.api_core.retry.Retry]) – A retry object used to retry requests. If None is specified, requests will not be retried.

  • timeout (Optional[float]) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.

  • metadata (Sequence[Tuple[str, str]]) – Additional metadata that is provided to the method.