hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4210) HBase reader throws NPE if Get returns no rows
Date Tue, 29 Sep 2015 21:58:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935976#comment-14935976
] 

Li Lu commented on YARN-4210:
-----------------------------

Applied the patch and I can see the NPE problem is gone. Digging a little bit more I thing
two quick changes would make this a more comprehensive fix the the flowrun inquiry:
- I'm not sure why I always need to add the {{?userid=}} part to query for the flows launched
by myself? Are we missing anything there in parseUser? Why can't we directly use callerUGI
if there's no userid parameter?
- From my end to end test I can see there were exceptions converting Integer to Long in FlowRunEntityReader#parseEntity.
Specifically, I changed the code to firstly converting startTime and endTime to Number then
get their long values to get rid of the problem. Not sure why we're getting integers here.


cc/[~vrushalic][~sjlee0]. 

> HBase reader throws NPE if Get returns no rows
> ----------------------------------------------
>
>                 Key: YARN-4210
>                 URL: https://issues.apache.org/jira/browse/YARN-4210
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4210-YARN-2928.01.patch, YARN-4210-YARN-2928.02.patch
>
>
> If HBase Get does not fetch any rows for the query, we still try to parse the result
and read fields. This leads to NPE while reading metrics. We should not attempt to read anything
if no row is returned i.e. result is empty.
> Found during web UI poc testing. 
> {noformat}
> 2015-09-29 20:22:32,027 ERROR [95336304@qtp-1814206058-0] reader.TimelineReaderWebServices
(TimelineReaderWebServices.java:handleException(199)) - Error while processing REST request
> java.lang.NullPointerException
>         at org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnHelper.readResultsWithTimestamps(ColumnHelper.java:176)
>         at org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunColumnPrefix.readResultsWithTimestamps(FlowRunColumnPrefix.java:182)
>         at org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readMetrics(TimelineEntityReader.java:212)
>         at org.apache.hadoop.yarn.server.timelineservice.storage.FlowRunEntityReader.parseEntity(FlowRunEntityReader.java:136)
>         at org.apache.hadoop.yarn.server.timelineservice.storage.TimelineEntityReader.readEntity(TimelineEntityReader.java:137)
>         at org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl.getEntity(HBaseTimelineReaderImpl.java:72)
>         at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderManager.getEntity(TimelineReaderManager.java:93)
>         at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderWebServices.getFlowRun(TimelineReaderWebServices.java:403)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message