Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B833710873 for ; Tue, 13 Aug 2013 00:53:49 +0000 (UTC) Received: (qmail 58263 invoked by uid 500); 13 Aug 2013 00:53:47 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 58191 invoked by uid 500); 13 Aug 2013 00:53:47 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 58132 invoked by uid 500); 13 Aug 2013 00:53:47 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 58118 invoked by uid 99); 13 Aug 2013 00:53:47 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2013 00:53:47 +0000 Date: Tue, 13 Aug 2013 00:53:47 +0000 (UTC) From: "shanyu zhao (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-5070) Need to implement listLocatedStatus() in ProxyFileSystem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated HIVE-5070: ------------------------------ Fix Version/s: 0.11.1 Status: Patch Available (was: Open) > Need to implement listLocatedStatus() in ProxyFileSystem > -------------------------------------------------------- > > Key: HIVE-5070 > URL: https://issues.apache.org/jira/browse/HIVE-5070 > Project: Hive > Issue Type: Bug > Components: CLI > Affects Versions: 0.11.0 > Reporter: shanyu zhao > Fix For: 0.11.1 > > > MAPREDUCE-1981 introduced a new API for FileSystem - listLocatedStatus. It is used in Hadoop's FileInputFormat.getSplits(). Hive's ProxyFileSystem class needs to implement this API in order to make Hive unit test work. > Otherwise, you'll see these exceptions when running TestCliDriver test case, e.g. results of running allcolref_in_udf.q: > [junit] Running org.apache.hadoop.hive.cli.TestCliDriver > [junit] Begin query: allcolref_in_udf.q > [junit] java.lang.IllegalArgumentException: Wrong FS: pfile:/GitHub/Monarch/project/hive-monarch/build/ql/test/data/warehouse/src, expected: file:/// > [junit] at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:642) > [junit] at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:69) > [junit] at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:375) > [junit] at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1482) > [junit] at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1522) > [junit] at org.apache.hadoop.fs.FileSystem$4.(FileSystem.java:1798) > [junit] at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1797) > [junit] at org.apache.hadoop.fs.ChecksumFileSystem.listLocatedStatus(ChecksumFileSystem.java:579) > [junit] at org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:235) > [junit] at org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:235) > [junit] at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:264) > [junit] at org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217) > [junit] at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:69) > [junit] at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:385) > [junit] at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:351) > [junit] at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:389) > [junit] at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:503) > [junit] at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:495) > [junit] at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:390) > [junit] at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) > [junit] at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) > [junit] at java.security.AccessController.doPrivileged(Native Method) > [junit] at javax.security.auth.Subject.doAs(Subject.java:396) > [junit] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1481) > [junit] at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) > [junit] at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557) > [junit] at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:552) > [junit] at java.security.AccessController.doPrivileged(Native Method) > [junit] at javax.security.auth.Subject.doAs(Subject.java:396) > [junit] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1481) > [junit] at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:552) > [junit] at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:543) > [junit] at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:448) > [junit] at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688) > [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > [junit] at java.lang.reflect.Method.invoke(Method.java:597) > [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:212) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira