hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christoph Rupp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3533) NPE in DFSClient$DFSInputStream.openInfo
Date Thu, 17 Jan 2013 09:44:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13556016#comment-13556016
] 

Christoph Rupp commented on HDFS-3533:
--------------------------------------

I am running into the same issue. My current workaround is to retry opening the file, after
a retry it usually works. Sometimes i have to retry multiple times (up to 4 was the maximum
so far.)

{code:title=Pseudocode.java|borderStyle=solid}
void Open(Path path) {
  DFSDataInputStream in;
  try {
     in = (DFSClient.DFSDataInputStream)mFilesystem.open(path);
  }
  catch (NullPointerException e) {
    // wait for a while
    Thread.sleep(1000);
    // call recursively
    Open(path);
    return;
  }
  // ...
}
{code}

My application is very similar to the one described by Amareshwari; i have a producer and
a consumer, both are usually on different datanodes. Every 5 seconds the consumer tries to
continue reading the new data.

My question: is this a viable workaround? Do i need the sleep()? Is there a better/faster
way to refresh the node list?
                
> NPE in DFSClient$DFSInputStream.openInfo
> ----------------------------------------
>
>                 Key: HDFS-3533
>                 URL: https://issues.apache.org/jira/browse/HDFS-3533
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 0.20.203.0
>            Reporter: Amareshwari Sriramadasu
>         Attachments: cannot-seek-after-eof-dn-logs.txt, cannot-seek-after-eof-nn-logs.txt,
nn-logs-npe.txt, npe-dn-logs.txt
>
>
> I'm seeing the following NPE:
> java.lang.NullPointerException
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1633)
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1593)
>         at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:428)
>         at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
>         at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:456)
> Here, the file gets opened while it's being written.   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message