hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] [Created] (HIVE-2202) multi-thread hive io and execution
Date Tue, 07 Jun 2011 00:54:59 GMT
multi-thread hive io and execution

                 Key: HIVE-2202
             Project: Hive
          Issue Type: Improvement
            Reporter: Namit Jain

Multi-threading hive's IO part and execution part. For Hive's RCFile, the reader can be put
in a separate thread, and can keep 2 blocks in memory. One block for current processing, and
the other for next batch. Potentially this can be more intelligent (like determine how many
threads to read depending on the execution speed), but this will depends on the HDFS client's
implementation. And the writer can also put into a separate thread, this way, the execution
will not need to block on writer.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message