hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10223) Consolidate several redundant FileSystem API calls.
Date Tue, 07 Apr 2015 00:23:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482275#comment-14482275
] 

Gunther Hagleitner commented on HIVE-10223:
-------------------------------------------

The smb test I've seen fail before. I don't think that's you. The other one seems to have
a small difference in rounding in some aggregates. That's most likely related to how the data
was split. The result isn't incorrect (all datatypes are doubles), and probably also not caused
by this patch. I'll run the tests locally too and see if it passes for me as well.

> Consolidate several redundant FileSystem API calls.
> ---------------------------------------------------
>
>                 Key: HIVE-10223
>                 URL: https://issues.apache.org/jira/browse/HIVE-10223
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HIVE-10223.1.patch
>
>
> This issue proposes to consolidate several Hive calls to the Hadoop Common {{FileSystem}}
API into a fewer number of calls that still accomplish the equivalent work.  {{FileSystem}}
API calls typically translate into RPCs to other services like the HDFS NameNode or alternative
file system implementations.  Consolidating RPCs will lower latency a bit for Hive code and
reduce some load on these external services.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message