hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1082) Command line UI for queues' information is broken with hierarchical queues.
Date Tue, 10 Nov 2009 09:34:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775354#action_12775354
] 

Vinod K V commented on MAPREDUCE-1082:
--------------------------------------

I've looked at the attached patch. It has some problems. The most important one is that for
converting {{QueueInfo}} obects to {{JobQueueInfo}} objects, this patch creates a new {{JobQueueInfo}}
objects everytime. This causes other tests like {{TestQueueMangerRefresh}} to fail. This has
also proved to be problematic in the past too while working with MAPREDUCE-893.

Going back and summarizing the main cause for this bug, the NPE occurs while trying to serialize
a hierarchy of {{JobQueueInfo}} objects. {{JobQueueInfo}} extends {{QueueInfo}} and so serialization
code is accordingly inherited. So, the problem is as follows:

Hierarchy before serialization:
{code}
JobQueueInfo-one
   |-- JobQueueInfo-child-one
   `-- JobQueueInfo-child-two
{code}

Deserialized hierarchy:
{code}
JobQueueInfo-one
   |-- QueueInfo-child-one
   `-- QueueInfo-child-two
{code}

This poses problem while traversing the deserializing hierarchy.

Two solutions possible:
 - From the deserialized hierarchy, construct a proper hierarchy of {{JobQueueInfo}}s and
use this from then on.
 - Or solve the central probem which is about fixing the inconsistent handling of relation
{{JobQueueInfo}} with {{QueueInfo}} objects.

Given that the central problem has already resulted in some issues before too, I propose that
once and for all we fix the problems with {{JobQueueInfo}} <-> {{QueueInfo}} inter-conversion
as part of this JIRA issue. Thoughts?

> Command line UI for queues' information is broken with hierarchical queues.
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1082
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1082
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, jobtracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: V.V.Chaitanya Krishna
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: MAPREDUCE-1082-1.txt
>
>
> When the command "./bin/mapred --config ~/tmp/conf/ queue -list" is run, it just hangs.
I can see the following in the JT logs:
> {code}
> 2009-10-08 13:19:26,762 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 55555
caught: java.lang.NullPointerException
>         at org.apache.hadoop.mapreduce.QueueInfo.write(QueueInfo.java:217)
>         at org.apache.hadoop.mapreduce.QueueInfo.write(QueueInfo.java:223)
>         at org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:159)
>         at org.apache.hadoop.io.ObjectWritable.writeObject(ObjectWritable.java:126)
>         at org.apache.hadoop.io.ObjectWritable.write(ObjectWritable.java:70)
>         at org.apache.hadoop.ipc.Server.setupResponse(Server.java:1074)
>         at org.apache.hadoop.ipc.Server.access$2400(Server.java:77)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:983)
> {code}
> Same is the case with "./bin/mapred --config ~/tmp/conf/ queue -info <any-container-queue>"

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message