hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2051) getInputSummary() to call FileSystem.getContentSummary() in parallel
Date Mon, 21 Mar 2011 22:02:06 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009426#comment-13009426
] 

Joydeep Sen Sarma commented on HIVE-2051:
-----------------------------------------

committed - thanks Siying

> getInputSummary() to call FileSystem.getContentSummary() in parallel
> --------------------------------------------------------------------
>
>                 Key: HIVE-2051
>                 URL: https://issues.apache.org/jira/browse/HIVE-2051
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>            Priority: Minor
>         Attachments: HIVE-2051.1.patch, HIVE-2051.2.patch, HIVE-2051.3.patch, HIVE-2051.4.patch,
HIVE-2051.5.patch
>
>
> getInputSummary() now call FileSystem.getContentSummary() one by one, which can be extremely
slow when the number of input paths are huge. By calling those functions in parallel, we can
cut latency in most cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message