hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-7926) long-lived daemons for query fragment execution, I/O and caching
Date Mon, 01 Sep 2014 05:23:21 GMT
Sergey Shelukhin created HIVE-7926:
--------------------------------------

             Summary: long-lived daemons for query fragment execution, I/O and caching
                 Key: HIVE-7926
                 URL: https://issues.apache.org/jira/browse/HIVE-7926
             Project: Hive
          Issue Type: New Feature
            Reporter: Sergey Shelukhin


We are proposing a new execution model for Hive that is a combination of existing process-based
tasks and long-lived daemons running on worker nodes. These nodes can take care of efficient
I/O, caching and query fragment execution, while heavy lifting like most joins, ordering,
etc. can be handled by tasks.
The proposed model is not a 2-system solution for small and large queries; neither it is a
separate execution engine like MR or Tez. It can be used by any Hive execution engine, if
support is added; in future even external products (e.g. Pig) can use it.

The document with high-level design we are proposing will be attached shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message