hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eisuke Umeda (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10815) The state of the EC file is erroneously recognized when you restart the NameNode.
Date Tue, 30 Aug 2016 02:53:21 GMT
Eisuke Umeda created HDFS-10815:
-----------------------------------

             Summary: The state of the EC file is erroneously recognized when you restart
the NameNode.
                 Key: HDFS-10815
                 URL: https://issues.apache.org/jira/browse/HDFS-10815
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha1
         Environment: 2 NameNodes, 5 DataNodes, Erasured code policy is set as "RS-DEFAULT-3-2-64k"
            Reporter: Eisuke Umeda


After carrying out an examination in the following procedures, an EC files came to be recognized
as corrupt files.
These files were able to get in "hdfs dfs -get".
NameNode might be causing the false recognition.

NameNodes:
  namenode1(active)
  namenode2(standby)

The directory which there is "Under-erasure-coded block groups": /tmp/tpcds-generate/test
{code}
$ sudo -u hdfs hdfs erasurecode -getPolicy /tmp/tpcds-generate/test
ErasureCodingPolicy=[Name=RS-DEFAULT-3-2-64k, Schema=[ECSchema=[Codec=rs-default, numDataUnits=3,
numParityUnits=2]], CellSize=65536 ]
{code}

The following is the steps to reproduce:
1) hdfs dfs -cp /tmp/tpcds-generate/25/store_sales/* /tmp/tpcds-generate/test
2) datanode1: (in the middle of the copy) sudo pkill -9 -f datanode
3) start a process of datanode1 two minutes later
4) carry out hdfs fsck and confirm that Under-Replicated Blocks occurred
5) wait until Under-Replicated Blocks becomes 0
6) (namenode1) /etc/init.d/hadoop-hdfs-namenode restart
7) (namenode2) /etc/init.d/hadoop-hdfs-namenode restart



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message