Return-Path: 
 <dev-return-52113-apmail-hive-dev-archive=hive.apache.org@hive.apache.org>
X-Original-To: apmail-hive-dev-archive@www.apache.org
Delivered-To: apmail-hive-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 20D5DF6E5
	for <apmail-hive-dev-archive@www.apache.org>;
 Wed,  8 May 2013 03:11:19 +0000 (UTC)
Received: (qmail 35913 invoked by uid 500); 8 May 2013 03:11:18 -0000
Delivered-To: apmail-hive-dev-archive@hive.apache.org
Received: (qmail 35589 invoked by uid 500); 8 May 2013 03:11:16 -0000
Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@hive.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@hive.apache.org>
List-Post: <mailto:dev@hive.apache.org>
List-Id: <dev.hive.apache.org>
Reply-To: dev@hive.apache.org
Delivered-To: mailing list dev@hive.apache.org
Received: (qmail 35557 invoked by uid 500); 8 May 2013 03:11:16 -0000
Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org
Received: (qmail 35550 invoked by uid 99); 8 May 2013 03:11:15 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 May 2013 03:11:15 +0000
Date: Wed, 8 May 2013 03:11:15 +0000 (UTC)
From: "Yanhui Ma (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12646390.1367923467158.285775.1367982675894@arcas>
In-Reply-To: <JIRA.12646390.1367923467158@arcas>
References: <JIRA.12646390.1367923467158@arcas>
Subject: [jira] [Updated] (HIVE-4515) "select count(*) from table" query on
 hive-0.10.0, hbase-0.94.7 integration throws exceptions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/HIVE-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanhui Ma updated HIVE-4515:
----------------------------

    Priority: Critical  (was: Major)
    
> "select count(*) from table" query on hive-0.10.0, hbase-0.94.7 integration throws exceptions
> ---------------------------------------------------------------------------------------------
>
>                 Key: HIVE-4515
>                 URL: https://issues.apache.org/jira/browse/HIVE-4515
>             Project: Hive
>          Issue Type: Bug
>          Components: HBase Handler
>    Affects Versions: 0.10.0
>         Environment: hive-0.10.0
> hbase-0.94.7
> zookeeper-3.4.3
> hadoop-1.0.4
> centos-5.7
>            Reporter: Yanhui Ma
>            Priority: Critical
>
> After integration hive-0.10.0+hbase-0.94.7, these commands could be executed sucessfully:
> create table
> insert overwrite table
> select * from table
> However, when execute "select count(*) from table", throws exception:
> hive> select count(*) from test;         
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=<number>
> Starting Job = job_201305061042_0028, Tracking URL = http://master0:50030/jobdetails.jsp?jobid=job_201305061042_0028
> Kill Command = /opt/modules/hadoop/hadoop-1.0.4/libexec/../bin/hadoop job  -kill job_201305061042_0028
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
> 2013-05-07 18:41:42,649 Stage-1 map = 0%,  reduce = 0%
> 2013-05-07 18:42:14,789 Stage-1 map = 100%,  reduce = 100%
> Ended Job = job_201305061042_0028 with errors
> Error during job, obtaining debugging information...
> Job Tracking URL: http://master0:50030/jobdetails.jsp?jobid=job_201305061042_0028
> Examining task ID: task_201305061042_0028_m_000002 (and more) from job job_201305061042_0028
> Task with the most failures(4): 
> -----
> Task ID:
>   task_201305061042_0028_m_000000
> URL:
>   http://master0:50030/taskdetails.jsp?jobid=job_201305061042_0028&tipid=task_201305061042_0028_m_000000
> -----
> Diagnostic Messages for this Task:
> java.lang.NegativeArraySizeException: -1
> 	at org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:148)
> 	at org.apache.hadoop.hbase.mapreduce.TableSplit.readFields(TableSplit.java:133)
> 	at org.apache.hadoop.hive.hbase.HBaseSplit.readFields(HBaseSplit.java:53)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:150)
> 	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
> 	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
> 	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:396)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> MapReduce Jobs Launched: 
> Job 0: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
> ==================================================================
> The log of tasktracker:
> stderr logs
> 13/05/07 18:43:20 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 13/05/07 18:43:20 INFO mapred.TaskRunner: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/distcache/1073284782955556390_-1298160740_2123690974/master0/tmp/hive-hadoop/hive_2013-05-07_18-41-30_290_832140779606816147/-mr-10003/fd22448b-e923-498c-bc00-2164ca68447d <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/HIVE_PLANfd22448b-e923-498c-bc00-2164ca68447d
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/javolution <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/javolution
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/org <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/org
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/hive-exec-log4j.properties <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/hive-exec-log4j.properties
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/META-INF <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/META-INF
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/.job.jar.crc <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/.job.jar.crc
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/job.jar <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/job.jar
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/javax <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/javax
> 13/05/07 18:43:20 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/jars/javaewah <- /tmp/hadoop-hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201305061042_0028/attempt_201305061042_0028_m_000000_0/work/javaewah
> 13/05/07 18:43:20 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 13/05/07 18:43:21 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: MapTask metrics system started
> 13/05/07 18:43:21 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
> 13/05/07 18:43:21 WARN impl.MetricsSystemImpl: Source name ugi already exists!
> 13/05/07 18:43:21 INFO impl.MetricsSourceAdapter: MBean for source jvm registered.
> 13/05/07 18:43:21 INFO util.ProcessTree: setsid exited with exit code 0
> 13/05/07 18:43:21 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2d7aece8
> 13/05/07 18:43:21 INFO mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 13/05/07 18:43:21 INFO nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.
> 13/05/07 18:43:21 INFO nativeio.NativeIO: Got UserName hadoop for UID 1001 from the native implementation
> 13/05/07 18:43:21 WARN mapred.Child: Error running child
> java.lang.NegativeArraySizeException: -1
> 	at org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:148)
> 	at org.apache.hadoop.hbase.mapreduce.TableSplit.readFields(TableSplit.java:133)
> 	at org.apache.hadoop.hive.hbase.HBaseSplit.readFields(HBaseSplit.java:53)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:150)
> 	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
> 	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
> 	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:396)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 13/05/07 18:43:21 INFO mapred.Task: Runnning cleanup for the task
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: Stopping MapTask metrics system...
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: Stopping metrics source ugi(org.apache.hadoop.security.UgiInstrumentation)
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: Stopping metrics source jvm(org.apache.hadoop.metrics2.source.JvmMetricsSource)
> 13/05/07 18:43:21 INFO impl.MetricsSystemImpl: MapTask metrics system stopped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira