hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-503) Implement erasure coding as a layer on HDFS
Date Mon, 13 Jun 2011 02:27:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048444#comment-13048444
] 

sri commented on HDFS-503:
--------------------------

Error in name node 
ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.dfs.DistributedRaidFileSystem
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1304)
	at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:65)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1328)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:109)
	at org.apache.hadoop.fs.Trash.<init>(Trash.java:62)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startTrashEmptier(NameNode.java:292)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:288)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:434)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1153)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1162)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.dfs.DistributedRaidFileSystem
	at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:334)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
	... 11 more

Can some body help me out..


> Implement erasure coding as a layer on HDFS
> -------------------------------------------
>
>                 Key: HDFS-503
>                 URL: https://issues.apache.org/jira/browse/HDFS-503
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: contrib/raid
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>             Fix For: 0.21.0
>
>         Attachments: raid1.txt, raid2.txt
>
>
> The goal of this JIRA is to discuss how the cost of raw storage for a HDFS file system
can be reduced. Keeping three copies of the same data is very costly, especially when the
size of storage is huge. One idea is to reduce the replication factor and do erasure coding
of a set of blocks so that the over probability of failure of a block remains the same as
before.
> Many forms of error-correcting codes are available, see http://en.wikipedia.org/wiki/Erasure_code.
Also, recent research from CMU has described DiskReduce https://opencirrus.org/system/files/Gibson-OpenCirrus-June9-09.ppt.
> My opinion is to discuss implementation strategies that are not part of base HDFS, but
is a layer on top of HDFS.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message