hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Any related paper on how to resolve hadoop SPOF issue?
Date Sat, 27 Aug 2011 09:07:49 GMT
Hi Alex,

> UC Santa Cruz contributed a ;login: article describing replacing HDFS with 
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of 
> Ceph over HDFS is that Ceph distributes its metadata service over multiple 
> metadata servers.

Were you able to maintain stable operation with Ceph at a high load? Did you try running HBase?

Best regards,


       - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Alex Nelson <ajnelson@cs.ucsc.edu>
> To: George Kousiouris <gkousiou@mail.ntua.gr>
> Cc: common-dev@hadoop.apache.org; common-user@hadoop.apache.org; Carlos Maltzahn <carlosm@soe.ucsc.edu>
> Sent: Friday, August 26, 2011 2:44 AM
> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
> 
> Hi George,
> 
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with 
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of 
> Ceph over HDFS is that Ceph distributes its metadata service over multiple 
> metadata servers.
> 
> I hope that helps.
> 
> --Alex
> 
> 
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
> 
>> 
>>  Hi,
>> 
>>  many thanks!!
>> 
>>  I also found this one, which has some related work also with other efforts, 
> in case it is helpful for someone else:
>> 
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>> 
>> 
>>  BR,
>>  George
>> 
>>  On 8/25/2011 12:46 PM, Nan Zhu wrote:
>>>  Hope it helps
>>> 
>>>  http://www.springerlink.com/content/h17r882710314147/
>>> 
>>>  Best,
>>> 
>>>  Nan
>>> 
>>>  On Thu, Aug 25, 2011 at 5:43 PM, George 
> Kousiouris<gkousiou@mail.ntua.gr>wrote:
>>> 
>>>>  Hi guys,
>>>> 
>>>>  We are currently in the process of writing a paper regarding hadoop 
> and we
>>>>  would like to reference any attempt to remove the single point of 
> failure of
>>>>  the Namenode. We have found in various presentations some efforts 
> (like
>>>>  dividing the namespace between more than one namenodes if i 
> remember
>>>>  correctly) but the search for a concrete paper on the issue came to 
> nothing.
>>>> 
>>>>  Is anyone aware or has participated in such an effort?
>>>> 
>>>>  Thanks,
>>>>  George
>>>> 
>>>>  --
>>>> 
>>>>  ---------------------------
>>>> 
>>>>  George Kousiouris
>>>>  Electrical and Computer Engineer
>>>>  Division of Communications,
>>>>  Electronics and Information Engineering
>>>>  School of Electrical and Computer Engineering
>>>>  Tel: +30 210 772 2546
>>>>  Mobile: +30 6939354121
>>>>  Fax: +30 210 772 2569
>>>>  Email: gkousiou@mail.ntua.gr
>>>>  Site: http://users.ntua.gr/gkousiou/
>>>> 
>>>>  National Technical University of Athens
>>>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>> 
>>>> 
>>> 
>> 
>> 
>>  -- 
>> 
>>  ---------------------------
>> 
>>  George Kousiouris
>>  Electrical and Computer Engineer
>>  Division of Communications,
>>  Electronics and Information Engineering
>>  School of Electrical and Computer Engineering
>>  Tel: +30 210 772 2546
>>  Mobile: +30 6939354121
>>  Fax: +30 210 772 2569
>>  Email: gkousiou@mail.ntua.gr
>>  Site: http://users.ntua.gr/gkousiou/
>> 
>>  National Technical University of Athens
>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>> 
>

Mime
View raw message