hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kiupbyun (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-2600) regarding HDFS-RAID
Date Tue, 29 Nov 2011 16:41:42 GMT
regarding HDFS-RAID
-------------------

                 Key: HDFS-2600
                 URL: https://issues.apache.org/jira/browse/HDFS-2600
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: contrib/raid
    Affects Versions: 0.21.0
            Reporter: kiupbyun
            Priority: Critical
             Fix For: 0.21.0


I've tried to do HDFS-RAID over hadoop-0.21.0 ...
 
But, It's failed to raid when I've tried to start raidnode (start-raidnode.sh)
 
What do I have to correct it?
 
This is the information. Would you tell me how to correct it?
 
Namenode : xxx.23.129.40:9006
Datanode: xxx.23.129.39, 38
Raidnode : xxx.23.129.40 ( I want to seperate raidnoe from namenode, how can I do?)
 
* hdfs-site.xml
  <name>hdfs.raid.locations</name>
    <value>hdfs://xxx.23.129.40:9006/raid</value>   
=>  (What's the meaning of this, which address do I have to write down?)
    <description>The location for parity files. If this is
      is not defined, then defaults to /raid.
    </description>
  </property>
 
* raid.xml
<configuration>
    <srcPath prefix="hdfs://xxx.23.129.40:9006/TEST">  
=>  (What's the meaning of this, which address do I have to write down?)
      <policy name = "TEST">          =>  (What's this? )
        <property>
          <name>srcReplication</name>
          <value>3</value>
          <description> pick files for RAID only if their replication factor is
                        greater than or equal to this value.
          </description>
        </property>
        <property>
          <name>targetReplication</name>
          <value>2</value>
          <description> after RAIDing, decrease the replication factor of a file to
                        this value.
          </description>
        </property>
        <property>
          <name>metaReplication</name>
          <value>2</value>
          <description> the replication factor of the RAID meta file
          </description>
        </property>
        <property>
          <name>modTimePeriod</name>
          <value>100</value>
          <description> time (milliseconds) after a file is modified to make it a
                        candidate for RAIDing
          </description>
        </property>
 
 
*log:  when start-raidnode.sh

/************************************************************
SHUTDOWN_MSG: Shutting down RaidNode at test40/xxx.23.129.40
************************************************************/
2011-11-25 23:59:13,417 INFO org.apache.hadoop.raid.RaidNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting RaidNode
STARTUP_MSG:   host = test40/xxx.23.129.40
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.21.0
************************************************************/
2011-11-25 23:59:13,518 ERROR org.apache.hadoop.raid.ConfigManager: Reloading config 

file /KTBHOME/hadoop/hadoop/conf/raid.xml
2011-11-25 23:59:13,611 INFO org.apache.hadoop.security.Groups: Group mapping 

impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
2011-11-25 23:59:13,647 WARN org.apache.hadoop.conf.Configuration: mapred.task.id is 

deprecated. Instead, use mapreduce.task.attempt.id
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager: 

TEST.srcReplication = 3
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager: 

TEST.targetReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager: 

TEST.metaReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager: 

TEST.modTimePeriod = 100
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager: 

k_table1.targetReplication = 1
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager: 

k_table1.metaReplication = 2
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager: 

k_table1.modTimePeriod = 100
2011-11-25 23:59:13,728 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2011-11-25 23:59:13,751 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing 

RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,810 INFO org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: 

Initializing RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,812 INFO org.apache.hadoop.raid.RaidNode: RaidNode up at: 

/127.0.0.1:60000
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: 

starting
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 

60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 

60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 

60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 

60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 

60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 

60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 

60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 

60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 

60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 

60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 

60000: starting
2011-11-25 23:59:13,818 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy 

Filter TEST hdfs://xxx.23.129.40:9006/TEST
2011-11-25 23:59:13,819 INFO org.apache.hadoop.raid.RaidNode: Started archive scan
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: No filtered paths for 

policy TEST
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy 
Filter k_table1 hdfs://xxx.23.129.40:9006
2011-11-25 23:59:13,829 INFO org.apache.hadoop.raid.RaidNode: No filtered paths for 
policy k_table1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message