Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Mon, 28 Apr 2014 03:52:15 +0000 (UTC)
From: "Prasanth J (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12710694.1398472272580.189951.1398657135290@arcas>
In-Reply-To: <JIRA.12710694.1398472272580@arcas>
References: <JIRA.12710694.1398472272580@arcas>
Subject: [jira] [Updated] (HIVE-6979) Hadoop-2 test failures related to
 quick stats not being populated correctly
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HIVE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prasanth J updated HIVE-6979:
-----------------------------

    Attachment: HIVE-6979.3.patch

union_remove_25 was failing as it wasn't able to find partition. So increased the limit count so that the specific partition value is always present. Updated stats_partscan_1_23 test as well. Other tests seems to pass in my local setup (Mac OS X).

> Hadoop-2 test failures related to quick stats not being populated correctly
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-6979
>                 URL: https://issues.apache.org/jira/browse/HIVE-6979
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0
>            Reporter: Prasanth J
>            Assignee: Prasanth J
>         Attachments: HIVE-6979.1.patch, HIVE-6979.2.patch, HIVE-6979.3.patch
>
>
> The test failures that are currently reported by Hive QA running on hadoop-2 (https://issues.apache.org/jira/browse/HIVE-6968?focusedCommentId=13980570&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13980570) are related to difference in the way hadoop FileSystem.globStatus() api behaves. For a directory structure like below
> {code}
> dir1/file1
> dir1/file2
> {code}
> Two level of path pattern like dir1/*/* will return both files in hadoop 1.x but will return empty result in hadoop 2.x (in fact it will say no such file or directory and return empty file status array). Hadoop 2.x seems to be compliant to linux behaviour (ls dir1/*/*) but hadoop 1.x is not.
> As a result of this, the fast statistics (NUM_FILES and TOTAL_SIZE) are populated wrongly causing diffs in qfile tests for hadoop-1 and hadoop-2.


--
This message was sent by Atlassian JIRA
(v6.2#6252)