pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tobias Schlottke (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-3231) Problems with pig (TRUNK, 0.11) after upgrading to CDH4.2(yarn) using avro input
Date Mon, 04 Mar 2013 07:07:13 GMT
Tobias Schlottke created PIG-3231:
-------------------------------------

             Summary: Problems with pig (TRUNK, 0.11) after upgrading to CDH4.2(yarn) using
avro input
                 Key: PIG-3231
                 URL: https://issues.apache.org/jira/browse/PIG-3231
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.11
         Environment: CDH4.2, yarn, avro
            Reporter: Tobias Schlottke


Hi there,

we've got a strange issue after switching to a new cluster with cdh4.2 (from cdh3):
Pig seems to create temporary avro files for its map reduce jobs, which it either deletes
or never creates.

Pig fails with the "no error returned by hadoop"-message, but in nn-logs I found something
interesting.
The actual exception from nn-log is:
a
{code}
2013-03-01 12:59:30,858 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.1.28:37814: error: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException:
No lease on /user/metrigo/event_logger/compact_log/2013/01/14/_temporary/1/_temporary/attempt_1362133122980_0017_m_000007_0/part-m-00007.avro
File does not exist. Holder DFSClient_attempt_1362133122980_0017_m_000007_0_1992466008_1 does
not have any open files.
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /user/metrigo/event_logger/compact_log/2013/01/14/_temporary/1/_temporary/attempt_1362133122980_0017_m_000007_0/part-m-00007.avro
File does not exist. Holder DFSClient_attempt_1362133122980_0017_m_000007_0_1992466008_1 does
not have any open files.
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2396)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2387)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2183)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:481)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:297)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44080)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
{code}


Please note that we're analyzing a bunch of files (~200 files, we're using glob matchers),
some of them are small.
We made it work once without the small files.
Any Idea on how to find the reason for this?

Best,

Tobias


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message