hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12563) Updated utility to create/modify token files
Date Fri, 22 Apr 2016 02:59:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253240#comment-15253240
] 

Brahma Reddy Battula commented on HADOOP-12563:
-----------------------------------------------

Following test fails after this in....Since jenkins did not run on YARN and MAPREDUCE projects,
these tests are missed..

   TestRMContainerAllocator.testRMContainerAllocatorResendsRequestsOnRMRestart:2535 » IllegalState
   TestContainerManagerRecovery.testApplicationRecovery:189->startContainer:511 » IllegalState
  TestContainerManagerRecovery.testContainerCleanupOnShutdown:412->startContainer:511 »
IllegalState
  TestContainerManagerRecovery.testContainerResizeRecovery:351->startContainer:511 » IllegalState

See https://builds.apache.org/job/Hadoop-Yarn-trunk/2051/

{noformat}

FAILED:  org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery.testApplicationRecovery

Error Message:
InputStream#read(byte[]) returned invalid result: 0 The InputStream implementation is buggy.

Stack Trace:
java.lang.IllegalStateException: InputStream#read(byte[]) returned invalid result: 0 The InputStream
implementation is buggy.
	at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:739)
	at com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
	at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
	at org.apache.hadoop.security.proto.SecurityProtos$CredentialsProto.<init>(SecurityProtos.java:1828)
	at org.apache.hadoop.security.proto.SecurityProtos$CredentialsProto.<init>(SecurityProtos.java:1792)
	at org.apache.hadoop.security.proto.SecurityProtos$CredentialsProto$1.parsePartialFrom(SecurityProtos.java:1892)
	at org.apache.hadoop.security.proto.SecurityProtos$CredentialsProto$1.parsePartialFrom(SecurityProtos.java:1887)
	at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
	at org.apache.hadoop.security.proto.SecurityProtos$CredentialsProto.parseFrom(SecurityProtos.java:2100)
	at org.apache.hadoop.security.Credentials.readProtos(Credentials.java:331)
	at org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:226)
	at org.apache.hadoop.yarn.server.utils.YarnServerSecurityUtils.parseCredentials(YarnServerSecurityUtils.java:131)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainerInternal(ContainerManagerImpl.java:924)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:815)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery$3.run(TestContainerManagerRecovery.java:514)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery$3.run(TestContainerManagerRecovery.java:511)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery.startContainer(TestContainerManagerRecovery.java:511)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery.testApplicationRecovery(TestContainerManagerRecovery.java:189)
{noformat}

> Updated utility to create/modify token files
> --------------------------------------------
>
>                 Key: HADOOP-12563
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12563
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Allen Wittenauer
>            Assignee: Matthew Paduano
>             Fix For: 3.0.0
>
>         Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch, HADOOP-12563.03.patch,
HADOOP-12563.04.patch, HADOOP-12563.05.patch, HADOOP-12563.06.patch, HADOOP-12563.07.patch,
HADOOP-12563.07.patch, HADOOP-12563.08.patch, HADOOP-12563.09.patch, HADOOP-12563.10.patch,
HADOOP-12563.11.patch, HADOOP-12563.12.patch, HADOOP-12563.13.patch, dtutil-test-out, example_dtutil_commands_and_output.txt,
generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost exclusively towards
HDFS operations.  Additionally, the token files that are created use Java serializations which
are hard/impossible to deal with in other languages. It should be replaced with a better utility
in common that can read/write protobuf-based token files, has enough flexibility to be used
with other services, and offers key functionality such as append and rename. The old version
file format should still be supported for backward compatibility, but will be effectively
deprecated.
> A follow-on JIRA will deprecrate fetchdt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message