Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Sun, 20 Mar 2011 14:32:29 +0000 (UTC)
From: "Joydeep Sen Sarma (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: 
 <2141367347.14392.1300631549627.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1816002944.14574.1299881039536.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] Commented: (HIVE-2051) getInputSummary() to call
 FileSystem.getContentSummary() in parallel
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008903#comment-13008903 ] 

Joydeep Sen Sarma commented on HIVE-2051:
-----------------------------------------

based on: http://www.ibm.com/developerworks/java/library/j-jtp05236.html

it seems that the right thing to do here is to catch the interruptedexception and then call Thread.currentThread.interrupt() (grep for 'swallow interrupt' in this article).

we could also rethrow it - but the problem then will merely be punted to the higher layer (which probably will ignore it as well)

> getInputSummary() to call FileSystem.getContentSummary() in parallel
> --------------------------------------------------------------------
>
>                 Key: HIVE-2051
>                 URL: https://issues.apache.org/jira/browse/HIVE-2051
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>            Priority: Minor
>         Attachments: HIVE-2051.1.patch, HIVE-2051.2.patch, HIVE-2051.3.patch, HIVE-2051.4.patch
>
>
> getInputSummary() now call FileSystem.getContentSummary() one by one, which can be extremely slow when the number of input paths are huge. By calling those functions in parallel, we can cut latency in most cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira