hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dimitrios Sarigiannis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10252) Is DataNode aware of the name of the file that it is going to store?
Date Sat, 02 Apr 2016 11:35:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dimitrios Sarigiannis updated HDFS-10252:
-----------------------------------------
    Description: 
I am going through the HDFS Namenode and Datanode code and I am trying to see if the DataNode
is aware of the names of the files that are stored in it (and other metadata as well).

Assuming that we have the most simple case: 
1 NameNode
1 DataNode
1 single machine running HDFS with replication factor 1. 

and considering the way HDFS works a use case could be: 
A client requests to write a file from local to HDFS (for example: "hdfs dfs -put file /file")
He first communicates with NameNode and gets where this file should be stored.
Then, after receiving an answer, he requests to the DataNode to store that file.

(At that point I am going to be a little more specific about the code)
The DataNode has a DataXceiverServer class which runs and waits for requests. When a request
comes, it starts a DataXceiver thread and try to serve that request. What I would like to
know is, if at that specific point the DataNode knows the name of the file that it is going
to store. I spent hours of debugging but I could not find it. Is it somewhere there, or only
the NameNode knows the name of that file?

  was:
I am going through the HDFS Namenode and Datanode code and I am trying to see if the DataNode
is aware of the names of the files that are stored in it (and other metadata as well).

Assuming that we have the most simple case: 
1 NameNode
1 DataNode
1 single machine running HDFS with replication factor 1. 

and considering the way HDFS works a use case could be: 
A client requests to write a file from local to HDFS
"hdfs dfs -put file /file"
He first communicates with NameNode and gets where this file should be stored.
Then, after receiving an answer, he requests to the DataNode to store that file.

(At that point I am going to be a little more specific about the code)
The DataNode has a DataXceiverServer class which runs and waits for requests. When a request
comes, it starts a DataXceiver thread and try to serve that request. What I would like to
know is, if at that specific point the DataNode knows the name of the file that it is going
to store. I spent hours of debugging but I could not find it. Is it somewhere there, or only
the NameNode knows the name of that file?


> Is DataNode aware of the name of the file that it is going to store?
> --------------------------------------------------------------------
>
>                 Key: HDFS-10252
>                 URL: https://issues.apache.org/jira/browse/HDFS-10252
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: datanode, namenode
>            Reporter: Dimitrios Sarigiannis
>            Priority: Minor
>
> I am going through the HDFS Namenode and Datanode code and I am trying to see if the
DataNode is aware of the names of the files that are stored in it (and other metadata as well).
> Assuming that we have the most simple case: 
> 1 NameNode
> 1 DataNode
> 1 single machine running HDFS with replication factor 1. 
> and considering the way HDFS works a use case could be: 
> A client requests to write a file from local to HDFS (for example: "hdfs dfs -put file
/file")
> He first communicates with NameNode and gets where this file should be stored.
> Then, after receiving an answer, he requests to the DataNode to store that file.
> (At that point I am going to be a little more specific about the code)
> The DataNode has a DataXceiverServer class which runs and waits for requests. When a
request comes, it starts a DataXceiver thread and try to serve that request. What I would
like to know is, if at that specific point the DataNode knows the name of the file that it
is going to store. I spent hours of debugging but I could not find it. Is it somewhere there,
or only the NameNode knows the name of that file?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message