hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3063) NameNode should validate all coming file path
Date Fri, 09 Mar 2012 15:04:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226121#comment-13226121

Daryn Sharp commented on HDFS-3063:

I was posing it as a question.  I'm not a rpc expert, but quickly running through the call
code does indeed look impossible to hook in w/o artificially coupling the rpc layer to the
namenode protocol.  If so, definitely scratch that idea.  A rpc domain expert might provide

My only other suggestions would be to consider calling a method that encapsulates the if &
throw.  If we want to change the exception type or message, it's 1 instead of N-many locations
to change.  Since we use generic IOExceptions everywhere, the client often has to mince the
error string which makes enforced consistency especially important.

It may make sense to always perform the check as the first statement of the methods.  The
validity of the path is unrelated to safemode, so should I really have to wait for the NN
to be operational before knowing my paths are invalid?
> NameNode should validate all coming file path
> ---------------------------------------------
>                 Key: HDFS-3063
>                 URL: https://issues.apache.org/jira/browse/HDFS-3063
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions:
>            Reporter: Denny Ye
>            Priority: Minor
>              Labels: namenode
>         Attachments: HDFS-3063.patch
> NameNode provides RPC service for not only DFS client but also user defined program.
A common case we always met is that user transfers file path prefixed with HDFS protocol("hdfs://{namenode:{port}}/{folder}/{file}").
NameNode cannot map node meta-data with this path and always throw NPE. In user client, we
only see the NullPointerException, no other tips for which step it occurs. 
> Also, NameNode should validate all coming file path with regular format.
> One exception I met:
> Exception in thread "main" org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> 	at org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:334)
> 	at org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:329)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message