hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-2202) multi-thread hive io and execution
Date Sat, 30 Jul 2011 04:16:10 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ning Zhang reassigned HIVE-2202:
--------------------------------

    Assignee: Tim Armstrong

> multi-thread hive io and execution
> ----------------------------------
>
>                 Key: HIVE-2202
>                 URL: https://issues.apache.org/jira/browse/HIVE-2202
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Namit Jain
>            Assignee: Tim Armstrong
>
> Multi-threading hive's IO part and execution part. For Hive's RCFile, the reader can
be put in a separate thread, and can keep 2 blocks in memory. One block for current processing,
and the other for next batch. Potentially this can be more intelligent (like determine how
many threads to read depending on the execution speed), but this will depends on the HDFS
client's implementation. And the writer can also put into a separate thread, this way, the
execution will not need to block on writer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message