:py:mod:`astronomer.providers.apache.livy.hooks.livy`
=====================================================

.. py:module:: astronomer.providers.apache.livy.hooks.livy

.. autoapi-nested-parse::

   This module contains the Apache Livy hook async.



Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   astronomer.providers.apache.livy.hooks.livy.LivyHookAsync




.. py:class:: LivyHookAsync(livy_conn_id = default_conn_name, extra_options = None, extra_headers = None)

   Bases: :py:obj:`astronomer.providers.http.hooks.http.HttpHookAsync`, :py:obj:`airflow.utils.log.logging_mixin.LoggingMixin`

   Hook for Apache Livy through the REST API using LivyHookAsync

   :param livy_conn_id: reference to a pre-defined Livy Connection.
   :param extra_options: Additional option can be passed when creating a request.
           For example, ``run(json=obj)`` is passed as ``aiohttp.ClientSession().get(json=obj)``
   :param extra_headers: A dictionary of headers passed to the HTTP request to livy.

   .. seealso::
       For more details refer to the Apache Livy API reference:
       `Apache Livy API reference <https://livy.apache.org/docs/latest/rest-api.html>`_

   .. py:attribute:: TERMINAL_STATES

      

   .. py:attribute:: conn_name_attr
      :value: 'livy_conn_id'

      

   .. py:attribute:: default_conn_name
      :value: 'livy_default'

      

   .. py:attribute:: conn_type
      :value: 'livy'

      

   .. py:attribute:: hook_name
      :value: 'Apache Livy'

      

   .. py:method:: run_method(endpoint, method = 'GET', data = None, headers = None)
      :async:

      Wrapper for HttpHookAsync, allows to change method on the same HttpHookAsync

      :param method: http method
      :param endpoint: endpoint
      :param data: request payload
      :param headers: headers
      :return: http response
      :rtype: requests.Response


   .. py:method:: get_batch_state(session_id)
      :async:

      Fetch the state of the specified batch asynchronously.

      :param session_id: identifier of the batch sessions
      :return: batch state
      :rtype: BatchState


   .. py:method:: get_batch_logs(session_id, log_start_position, log_batch_size)
      :async:

      Gets the session logs for a specified batch asynchronously.

      :param session_id: identifier of the batch sessions
      :param log_start_position: Position from where to pull the logs
      :param log_batch_size: Number of lines to pull in one batch

      :return: response body
      :rtype: dict


   .. py:method:: dump_batch_logs(session_id)
      :async:

      Dumps the session logs for a specified batch asynchronously

      :param session_id: identifier of the batch sessions
      :return: response body
      :rtype: dict


   .. py:method:: build_post_batch_body(file, args = None, class_name = None, jars = None, py_files = None, files = None, archives = None, name = None, driver_memory = None, driver_cores = None, executor_memory = None, executor_cores = None, num_executors = None, queue = None, proxy_user = None, conf = None)
      :staticmethod:

      Build the post batch request body.

      :param file: Path of the file containing the application to execute (required).
      :param proxy_user: User to impersonate when running the job.
      :param class_name: Application Java/Spark main class string.
      :param args: Command line arguments for the application s.
      :param jars: jars to be used in this sessions.
      :param py_files: Python files to be used in this session.
      :param files: files to be used in this session.
      :param driver_memory: Amount of memory to use for the driver process  string.
      :param driver_cores: Number of cores to use for the driver process int.
      :param executor_memory: Amount of memory to use per executor process  string.
      :param executor_cores: Number of cores to use for each executor  int.
      :param num_executors: Number of executors to launch for this session  int.
      :param archives: Archives to be used in this session.
      :param queue: The name of the YARN queue to which submitted string.
      :param name: The name of this session string.
      :param conf: Spark configuration properties.
      :return: request body
      :rtype: dict

      For more information about the format refer to
      .. seealso:: https://livy.apache.org/docs/latest/rest-api.html