Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F1AEA10907 for ; Wed, 20 May 2015 18:07:39 +0000 (UTC) Received: (qmail 45565 invoked by uid 500); 20 May 2015 18:07:32 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 45454 invoked by uid 500); 20 May 2015 18:07:32 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 45444 invoked by uid 99); 20 May 2015 18:07:32 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 May 2015 18:07:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 0114AC11B9 for ; Wed, 20 May 2015 18:07:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.15 X-Spam-Level: *** X-Spam-Status: No, score=3.15 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id WsSFJXf5hhup for ; Wed, 20 May 2015 18:07:23 +0000 (UTC) Received: from mail-ig0-f174.google.com (mail-ig0-f174.google.com [209.85.213.174]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 9989F2315C for ; Wed, 20 May 2015 18:07:22 +0000 (UTC) Received: by igcau1 with SMTP id au1so46166213igc.1 for ; Wed, 20 May 2015 11:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=nHUn9c7kIF9cLDHXZ57iw+4IUkiynWLjYDsl1S/DbJY=; b=x8brTK7GMD51gRAm8l3BcHd23YZPe0nBMZNR9WL2RjkU23dIE2bSlzdV1e+1VVS107 gXF/vVJbRqsONB26wMDYgLUOd4+v6A70CjiB5c+HBLdo9IokhHKtkYrXAHQhHx5cxIna 9hPqWJ0bdt8Uf011XF2amr5m7QokAQxWC6SsIXq2TwYnhsTuVseaAwfTr8NqPDTrILqh pjJC3mfXwY40tHvmhpctjdX3D28IibdM10raSxoZTTVDTWDYChN0ck0XQiQP4c8qqOlb 4d0UaWs01nVNdAnxE4pzv2J9GhanzEMnJt3GcuQtXZZI173DEZDChP/utBBEpxz3WNaX PgMw== MIME-Version: 1.0 X-Received: by 10.107.18.104 with SMTP id a101mr9101027ioj.51.1432145196420; Wed, 20 May 2015 11:06:36 -0700 (PDT) Received: by 10.107.36.65 with HTTP; Wed, 20 May 2015 11:06:36 -0700 (PDT) Date: Wed, 20 May 2015 14:06:36 -0400 Message-ID: Subject: Mapreduce Hcatalog upgrade issue From: Anubhav Agarwal To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a113f2d9eb317bb051687492e --001a113f2d9eb317bb051687492e Content-Type: text/plain; charset=UTF-8 Hi, My code was working with CDH 5.3 but after upgrading the code to 5.4 I am getting the following error while using Hcatalog:- Task Logs: 'attempt_201505131431_0013_m_000030_1' *stdout logs* ------------------------------ *stderr logs* SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/mapred/local/taskTracker/distcache/-8482588650269571027_-1142469741_1919960239/master.cloudera/tmp/temp-2016036914/tmp-1892709241/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] ------------------------------ *syslog logs* 2015-05-20 13:51:24,609 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2015-05-20 13:51:25,323 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /mapred/local/taskTracker/distcache/-3321617315377295081_1858235370_1919948628/master.cloudera/tmp/temp-2016036914/tmp-488467356 <- /mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/attempt_201505131431_0013_m_000030_1/work/pigrepl_scope-17_1531833640_1432144269081_1 2015-05-20 13:51:25,331 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/jars/job.jar <- /mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/attempt_201505131431_0013_m_000030_1/work/job.jar 2015-05-20 13:51:25,333 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/jars/.job.jar.crc <- /mapred/local/taskTracker/cloudera/jobcache/job_201505131431_0013/attempt_201505131431_0013_m_000030_1/work/.job.jar.crc 2015-05-20 13:51:25,413 INFO org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2015-05-20 13:51:25,414 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2015-05-20 13:51:26,328 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2015-05-20 13:51:26,332 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.NoSuchFieldError: CHAR at org.apache.hive.hcatalog.data.schema.HCatFieldSchema$Type.(HCatFieldSchema.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.lang.Class.getEnumConstantsShared(Class.java:3085) at java.lang.Class.enumConstantDirectory(Class.java:3106) at java.lang.Enum.valueOf(Enum.java:231) at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1741) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at java.util.ArrayList.readObject(ArrayList.java:771) at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at java.util.Hashtable.readObject(Hashtable.java:996) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at java.util.HashMap.readObject(HashMap.java:1180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:57) at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:192) at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFContext(MapRedUtil.java:173) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:229) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:275) at org.apache.hadoop.mapred.Task.initialize(Task.java:523) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:313) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.Child.main(Child.java:262) --001a113f2d9eb317bb051687492e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,
My code was working with CDH 5.3 but after upgradi= ng the code to 5.4 I am getting the following error while using Hcatalog:-<= /div>

Task Logs: 'attempt_201505131431_0013_m_000030_1'= ;



stdout logs


SLF4J: C=
lass path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.=
0.p0.27/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.cla=
ss]
SLF4J: Found binding in [jar:file:/mapred/local/taskTracker/cloudera/jobcac=
he/job_201505131431_0013/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.cl=
ass]
SLF4J: Found binding in [jar:file:/mapred/local/taskTracker/distcache/-8482=
588650269571027_-1142469741_1919960239/master.cloudera/tmp/temp-2016036914/=
tmp-1892709241/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See ht=
tp://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]


syslog logs
2015-05-20 13:51:24,609 WARN mapredu=
ce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use=
 org.apache.hadoop.mapreduce.TaskCounter instead
2015-05-20 13:51:25,323 INFO org.apache.hadoop.mapred.TaskRunner: Creating =
symlink: /mapred/local/taskTracker/distcache/-3321617315377295081_185823537=
0_1919948628/master.cloudera/tmp/temp-2016036914/tmp-488467356 <- /mapre=
d/local/taskTracker/cloudera/jobcache/job_201505131431_0013/attempt_2015051=
31431_0013_m_000030_1/work/pigrepl_scope-17_1531833640_1432144269081_1
2015-05-20 13:51:25,331 INFO org.apache.hadoop.filecache.TrackerDistributed=
CacheManager: Creating symlink: /mapred/local/taskTracker/cloudera/jobcache=
/job_201505131431_0013/jars/job.jar <- /mapred/local/taskTracker/clouder=
a/jobcache/job_201505131431_0013/attempt_201505131431_0013_m_000030_1/work/=
job.jar
2015-05-20 13:51:25,333 INFO org.apache.hadoop.filecache.TrackerDistributed=
CacheManager: Creating symlink: /mapred/local/taskTracker/cloudera/jobcache=
/job_201505131431_0013/jars/.job.jar.crc <- /mapred/local/taskTracker/cl=
oudera/jobcache/job_201505131431_0013/attempt_201505131431_0013_m_000030_1/=
work/.job.jar.crc
2015-05-20 13:51:25,413 INFO org.apache.hadoop.conf.Configuration.deprecati=
on: session.id is deprecated. Instead, us=
e dfs.metrics.session-id
2015-05-20 13:51:25,414 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Init=
ializing JVM Metrics with processName=3DMAP, sessionId=3D
2015-05-20 13:51:26,328 INFO org.apache.hadoop.mapred.TaskLogsTruncater: In=
itializing logs' truncater with mapRetainSize=3D-1 and reduceRetainSize=
=3D-1
2015-05-20 13:51:26,332 FATAL org.apache.hadoop.mapred.Child: Error running=
 child : java.lang.NoSuchFieldError: CHAR
	at org.apache.hive.hcatalog.data.schema.HCatFieldSchema$Type.<clinit>=
;(HCatFieldSchema.java:54)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja=
va:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.lang.Class.getEnumConstantsShared(Class.java:3085)
	at java.lang.Class.enumConstantDirectory(Class.java:3106)
	at java.lang.Enum.valueOf(Enum.java:231)
	at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1741)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990=
)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:179=
8)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at java.util.ArrayList.readObject(ArrayList.java:771)
	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:179=
8)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990=
)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:179=
8)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at java.util.Hashtable.readObject(Hashtable.java:996)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja=
va:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:179=
8)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at java.util.HashMap.readObject(HashMap.java:1180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja=
va:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:179=
8)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.=
java:57)
	at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:192)
	at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFC=
ontext(MapRedUtil.java:173)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputF=
ormat.setupUdfEnvAndStores(PigOutputFormat.java:229)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputF=
ormat.getOutputCommitter(PigOutputFormat.java:275)
	at org.apache.hadoop.mapred.Task.initialize(Task.java:523)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:313)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati=
on.java:1671)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
--001a113f2d9eb317bb051687492e--