hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaka Jancar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8870) NullPointerException when glob doesn't return files
Date Tue, 02 Oct 2012 10:23:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jaka Jancar updated HADOOP-8870:
--------------------------------

    Description: 
Reading

{code}s3n://bucket/{a/,b/,c/}{code}

if one of the globs matches nothing, I get:

{code}
Exception in thread "main" java.lang.NullPointerException
	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:992)
	at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:177)
	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
	at spark.HadoopRDD.<init>(HadoopRDD.scala:51)
	at spark.SparkContext.hadoopFile(SparkContext.scala:186)
	at spark.SparkContext.textFile(SparkContext.scala:155)
	at com.celtra.analyzer.LogAnalyzer.analyzeSufficientS3Logs(LogAnalyzer.scala:52)
	at com.celtra.analyzer.App$.main(App.scala:164)
	at com.celtra.analyzer.App.main(App.scala)
{code}

I'm not sure whether this is specific to S3 or all filesystems.

This was occuring in 0.20.205 and I confirmed it's still present in 1.0.3.

  was:
Reading

{code}s3n://bucket/{a/*,b/*,c/*}{code}

if one of the globs matches nothing, I get:

{code}
Exception in thread "main" java.lang.NullPointerException
	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:992)
	at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:177)
	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
	at spark.HadoopRDD.<init>(HadoopRDD.scala:51)
	at spark.SparkContext.hadoopFile(SparkContext.scala:186)
	at spark.SparkContext.textFile(SparkContext.scala:155)
	at com.celtra.analyzer.LogAnalyzer.analyzeSufficientS3Logs(LogAnalyzer.scala:52)
	at com.celtra.analyzer.App$.main(App.scala:164)
	at com.celtra.analyzer.App.main(App.scala)
{code}

I'm not sure whether this is specific to S3 or all filesystems.

This was occuring in 0.20.205 and I confirmed it's still present in 1.0.3.

    
> NullPointerException when glob doesn't return files
> ---------------------------------------------------
>
>                 Key: HADOOP-8870
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8870
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, fs/s3
>    Affects Versions: 0.20.205.0, 1.0.3
>            Reporter: Jaka Jancar
>
> Reading
> {code}s3n://bucket/{a/,b/,c/}{code}
> if one of the globs matches nothing, I get:
> {code}
> Exception in thread "main" java.lang.NullPointerException
> 	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:992)
> 	at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:177)
> 	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
> 	at spark.HadoopRDD.<init>(HadoopRDD.scala:51)
> 	at spark.SparkContext.hadoopFile(SparkContext.scala:186)
> 	at spark.SparkContext.textFile(SparkContext.scala:155)
> 	at com.celtra.analyzer.LogAnalyzer.analyzeSufficientS3Logs(LogAnalyzer.scala:52)
> 	at com.celtra.analyzer.App$.main(App.scala:164)
> 	at com.celtra.analyzer.App.main(App.scala)
> {code}
> I'm not sure whether this is specific to S3 or all filesystems.
> This was occuring in 0.20.205 and I confirmed it's still present in 1.0.3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message