eagle-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lingang Deng (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (EAGLE-565) Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match the SchedulerWrapper object
Date Sun, 22 Jan 2017 07:04:26 GMT

     [ https://issues.apache.org/jira/browse/EAGLE-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lingang Deng resolved EAGLE-565.
--------------------------------
    Resolution: Fixed

> Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match
the SchedulerWrapper object
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: EAGLE-565
>                 URL: https://issues.apache.org/jira/browse/EAGLE-565
>             Project: Eagle
>          Issue Type: Bug
>    Affects Versions: v0.4.0
>         Environment: hadoop2.6.0-cdh5.4.3
>            Reporter: Lingang Deng
>            Assignee: Lingang Deng
>            Priority: Critical
>
> I got the exception {code:java}Caused by: org.codehaus.jackson.map.JsonMappingException:
Can not deserialize instance of java.lang.String out of START_ARRAY token
>  at [Source: java.io.ByteArrayInputStream@1578809b; line: 1, column: 2350] (through reference
chain: org.apache.eagle.hadoop.queue.model.scheduler.SchedulerWrapper["scheduler"]->org.apache.eagle.hadoop.queue.model.scheduler.Scheduler["schedulerInfo"]->org.apache.eagle.hadoop.queue.model.scheduler.SchedulerInfo["queues"]->org.apache.eagle.hadoop.queue.model.scheduler.Queues["queue"]->org.apache.eagle.hadoop.queue.model.scheduler.Queue["queues"]->org.apache.eagle.hadoop.queue.model.scheduler.Queues["queue"]->org.apache.eagle.hadoop.queue.model.scheduler.Queue["type"])
> {code} The response of https://some.server.address:50030/ws/v1/cluster/scheduler?anonymous=true,
the "type" field maybe return Array, but the right is String, and the field is  declared as
String in class Queue. Then, I got the root cause, there is a YARN JIRA, [YARN-4785|https://issues.apache.org/jira/browse/YARN-4785].Let
me give a case:
> {code:xml}
> {
>     "scheduler": {
>         "schedulerInfo": {
>             "type": "capacityScheduler",
>             "capacity": 100,
>             "usedCapacity": 0,
>             "maxCapacity": 100,
>             "queueName": "root",
>             "queues": {
>                 "queue": [
>                     {
>                         "type": "capacitySchedulerLeafQueueInfo",
>                         "capacity": 35,
>                         "usedCapacity": 0,
>                         "maxCapacity": 100,
>                         "absoluteCapacity": 35,
>                         "absoluteMaxCapacity": 100,
>                         "absoluteUsedCapacity": 0,
>                         "numApplications": 0,
>                         "queueName": "A",
>                         "state": "RUNNING",
>                         "resourcesUsed": {
>                             "memory": 0,
>                             "vCores": 0
>                         },
>                         "hideReservationQueues": false,
>                         "nodeLabels": [
>                             "*"
>                         ],
>                         "numActiveApplications": 0,
>                         "numPendingApplications": 0,
>                         "numContainers": 0,
>                         "maxApplications": 20000,
>                         "maxApplicationsPerUser": 40000,
>                         "maxActiveApplications": 20,
>                         "maxActiveApplicationsPerUser": 14,
>                         "userLimit": 100,
>                         "users": null,
>                         "userLimitFactor": 2
>                     },
>                     {
>                         "capacity": 35,
>                         "usedCapacity": 0,
>                         "maxCapacity": 35,
>                         "absoluteCapacity": 35,
>                         "absoluteMaxCapacity": 35,
>                         "absoluteUsedCapacity": 0,
>                         "numApplications": 0,
>                         "queueName": "B",
>                         "state": "RUNNING",
>                         "queues": {
>                             "queue": [
>                                 {
>                                     "capacity": 37,
>                                     "usedCapacity": 0,
>                                     "maxCapacity": 100,
>                                     "absoluteCapacity": 12.95,
>                                     "absoluteMaxCapacity": 35,
>                                     "absoluteUsedCapacity": 0,
>                                     "numApplications": 0,
>                                     "queueName": "B1",
>                                     "state": "RUNNING",
>                                     "queues": {
>                                         "queue": [
>                                             {
>                                                 "type": "capacitySchedulerLeafQueueInfo",
>                                                 "capacity": 45,
>                                                 "usedCapacity": 0,
>                                                 "maxCapacity": 100,
>                                                 "absoluteCapacity": 5.8275,
>                                                 "absoluteMaxCapacity": 35,
>                                                 "absoluteUsedCapacity": 0,
>                                                 "numApplications": 0,
>                                                 "queueName": "B11",
>                                                 "state": "RUNNING",
>                                                 "resourcesUsed": {
>                                                     "memory": 0,
>                                                     "vCores": 0
>                                                 },
>                                                 "hideReservationQueues": false,
>                                                 "nodeLabels": [
>                                                     "*"
>                                                 ],
>                                                 "numActiveApplications": 0,
>                                                 "numPendingApplications": 0,
>                                                 "numContainers": 0,
>                                                 "maxApplications": 20000,
>                                                 "maxApplicationsPerUser": 200000,
>                                                 "maxActiveApplications": 7,
>                                                 "maxActiveApplicationsPerUser": 20,
>                                                 "userLimit": 100,
>                                                 "users": null,
>                                                 "userLimitFactor": 10
>                                             },
>                                             {
>                                                 "type": "capacitySchedulerLeafQueueInfo",
>                                                 "capacity": 55,
>                                                 "usedCapacity": 0,
>                                                 "maxCapacity": 100,
>                                                 "absoluteCapacity": 7.1225004,
>                                                 "absoluteMaxCapacity": 35,
>                                                 "absoluteUsedCapacity": 0,
>                                                 "numApplications": 0,
>                                                 "queueName": "B12",
>                                                 "state": "RUNNING",
>                                                 "resourcesUsed": {
>                                                     "memory": 0,
>                                                     "vCores": 0
>                                                 },
>                                                 "hideReservationQueues": false,
>                                                 "nodeLabels": [
>                                                     "*"
>                                                 ],
>                                                 "numActiveApplications": 0,
>                                                 "numPendingApplications": 0,
>                                                 "numContainers": 0,
>                                                 "maxApplications": 712,
>                                                 "maxApplicationsPerUser": 712,
>                                                 "maxActiveApplications": 2,
>                                                 "maxActiveApplicationsPerUser": 1,
>                                                 "userLimit": 100,
>                                                 "users": null,
>                                                 "userLimitFactor": 1
>                                             }
>                                         ]
>                                     },
>                                     "resourcesUsed": {
>                                         "memory": 0,
>                                         "vCores": 0
>                                     },
>                                     "hideReservationQueues": false,
>                                     "nodeLabels": [
>                                         "*"
>                                     ]
>                                 },
>                                 {
>                                     "type": [
>                                         "capacitySchedulerLeafQueueInfo"
>                                     ],
>                                     "capacity": 63,
>                                     "usedCapacity": 0,
>                                     "maxCapacity": 100,
>                                     "absoluteCapacity": 22.05,
>                                     "absoluteMaxCapacity": 35,
>                                     "absoluteUsedCapacity": 0,
>                                     "numApplications": 0,
>                                     "queueName": "B2",
>                                     "state": "RUNNING",
>                                     "resourcesUsed": {
>                                         "memory": 0,
>                                         "vCores": 0
>                                     },
>                                     "hideReservationQueues": false,
>                                     "nodeLabels": [
>                                         "*"
>                                     ],
>                                     "numActiveApplications": 0,
>                                     "numPendingApplications": 0,
>                                     "numContainers": 0,
>                                     "maxApplications": 20000,
>                                     "maxApplicationsPerUser": 30000,
>                                     "maxActiveApplications": 7,
>                                     "maxActiveApplicationsPerUser": 8,
>                                     "userLimit": 100,
>                                     "users": null,
>                                     "userLimitFactor": 1.5
>                                 }
>                             ]
>                         },
>                         "resourcesUsed": {
>                             "memory": 0,
>                             "vCores": 0
>                         },
>                         "hideReservationQueues": false,
>                         "nodeLabels": [
>                             "*"
>                         ]
>                     }
>                 ]
>             }
>         }
>     }
> }
> {code} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message