spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Assigned] (SPARK-17193) HadoopRDD NPE at DEBUG log level when getLocationInfo == null
Date Mon, 22 Aug 2016 20:42:20 GMT


Apache Spark reassigned SPARK-17193:

    Assignee: Apache Spark  (was: Sean Owen)

> HadoopRDD NPE at DEBUG log level when getLocationInfo == null
> -------------------------------------------------------------
>                 Key: SPARK-17193
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.0.0
>            Reporter: Sean Owen
>            Assignee: Apache Spark
>            Priority: Trivial
> When I set the log level to "DEBUG" in one of my apps that reads from Parquet, I notice
several NullPointerExceptions logged from HadoopRDD.getPreferredLocations. 
> It doesn't affect executions as it just results in "no preferred locations". It happens
when InputSplitWithLocationInfo.getLocationInfo produces null, which it may. The code just
dereferences it however. 
> It's cleaner to check this directly (and maybe tighten up the code slightly) and avoid
polluting the log, though, it's just at debug level. No big deal, but enough of an annoyance
when I was debugging something that it's probably worth zapping.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message