hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject use setrep change number of file replicas,but not work
Date Thu, 10 Apr 2014 01:46:50 GMT
hi,maillist:
            i try modify replica number on some dir but it seems not work
,anyone know why?

# sudo -u hdfs hadoop fs -setrep -R 2 /user/hive/warehouse/mytest
Replication 2 set:
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000

the file still store 3 replica ,but the echo number changed
# hadoop fs -ls /user/hive/warehouse/mytest/dsp_request/2014-01-26
Found 1 items
-rw-r--r--   2 hdfs hdfs      17660 2014-01-26 18:34
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000

# sudo -u hdfs hdfs fsck
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 -files
-blocks -locations
Connecting to namenode via http://ch11:50070
FSCK started by hdfs (auth:SIMPLE) from /192.168.11.12 for path
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 at Thu Apr 10
09:39:51 CST 2014
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 17660 bytes,
1 block(s):  OK
0.
BP-1043055049-192.168.11.11-1382442676609:blk_-9219869107960013037_1976591
len=17660 repl=3 [192.168.11.13:50010, 192.168.11.10:50010,
192.168.11.14:50010]

i remove the file ,and upload new file ,as i understand ,the new file
should be stored in 2 replica,but it still store 3 replica ,why?
# sudo -u hdfs hadoop fs -rm -r -skipTrash
/user/hive/warehouse/mytest/dsp_request/2014-01-26/*
Deleted /user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000
# hadoop fs -put ./data_00000
/user/hive/warehouse/mytest/dsp_request/2014-01-26/
[root@ch12 ~]# hadoop fs -ls
/user/hive/warehouse/mytest/dsp_request/2014-01-26
Found 1 items
-rw-r--r--   3 root hdfs      17660 2014-04-10 09:40
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000
# sudo -u hdfs hdfs fsck
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 -files
-blocks -locations
Connecting to namenode via http://ch11:50070
FSCK started by hdfs (auth:SIMPLE) from /192.168.11.12 for path
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 at Thu Apr 10
09:41:12 CST 2014
/user/hive/warehouse/mytest/dsp_request/2014-01-26/data_00000 17660 bytes,
1 block(s):  OK
0.
BP-1043055049-192.168.11.11-1382442676609:blk_6517693524032437780_8889786
len=17660 repl=3 [192.168.11.12:50010, 192.168.11.15:50010,
192.168.11.13:50010]

Mime
View raw message