Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EF25C17D2C for ; Fri, 30 Jan 2015 17:45:34 +0000 (UTC) Received: (qmail 20753 invoked by uid 500); 30 Jan 2015 17:45:35 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 20683 invoked by uid 500); 30 Jan 2015 17:45:35 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 20670 invoked by uid 500); 30 Jan 2015 17:45:35 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 20667 invoked by uid 99); 30 Jan 2015 17:45:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Jan 2015 17:45:35 +0000 Date: Fri, 30 Jan 2015 17:45:35 +0000 (UTC) From: "Xuefu Zhang (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-9409) Avoid ser/de loggers as logging framework can be incompatible on driver and workers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-9409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-9409: ------------------------------ Fix Version/s: spark-branch > Avoid ser/de loggers as logging framework can be incompatible on driver and workers > ----------------------------------------------------------------------------------- > > Key: HIVE-9409 > URL: https://issues.apache.org/jira/browse/HIVE-9409 > Project: Hive > Issue Type: Sub-task > Components: Spark > Environment: CentOS6.5 > Java version: 1.7.0_67 > Reporter: Xin Hao > Assignee: Rui Li > Fix For: spark-branch, 0.15.0 > > Attachments: HIVE-9409.1.patch, HIVE-9409.1.patch, HIVE-9409.1.patch > > > When we use current [Spark Branch] to build hive package. deploy it on our cluster and execute hive queries (e.g. BigBench case Q10, Q18, Q19, Q27) by default mode (i.e. just Hive on MR, not HiveOnSpark), Error 'java.lang.ClassNotFoundException: org.apache.commons.logging.impl.SLF4JLocationAwareLog' will occurs. > For other released apache or CDH hive version(e.g. apache hive 0.14), there is no this issue. > By the way, if we use 'add jar /location/to/jcl-over-slf4j-1.7.5.jar' before hive query execution, the issue will be workaround. > The detail diagnostic messages are as below: > ========================================================== > Diagnostic Messages for this Task: > Error: java.lang.RuntimeException: Failed to load plan: hdfs://bhx1:8020/tmp/hive/root/4a4cbeb2-cf42-4eb7-a78a-7ecea6af2aff/hive_2015-01-17_10-45-51_360_5581900288096206774-1/-mr-10004/1c6c4667-8b81-41ed-a42e-fe099ae3379f/map.xml: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: org.apache.commons.logging.impl.SLF4JLocationAwareLog > Serialization trace: > LOG (org.apache.hadoop.hive.ql.exec.UDTFOperator) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) > at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:431) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:287) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:268) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:484) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:477) > at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:657) > at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find cl > Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: org.apache.commons.logging.impl.SLF4JLocationAwareLog > Serialization trace: > LOG (org.apache.hadoop.hive.ql.exec.UDTFOperator) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) > at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138) > at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112) > at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139) > at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694) > at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507) > at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:1050) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:941) > at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:955) > at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:396) > ... 13 more > Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.impl.SLF4JLocationAwareLog > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136) > ... 40 more > FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask > MapReduce Jobs Launched: > Stage-Stage-1: Map: 2 HDFS Read: 0 HDFS Write: 0 FAIL > Total MapReduce CPU Time Spent: 0 msec > WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. > WARN: Please see http://www.slf4j.org/codes.html#release for an explanation. > ========================================================== -- This message was sent by Atlassian JIRA (v6.3.4#6332)