hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan A. P. Pendleton" ...@geekdom.net>
Subject Re: Question about HDFS
Date Thu, 01 Feb 2007 18:57:41 GMT
HDFS should do this, but there are some caveats:

1) HDFS doesn't have complete POSIX semantics. No append, and files are
invisible until closed being the two primary disadvantages to know about.
2) You can use this for your application, where you need N fileservers, if
you keep the replication of your files set to N. The default is 3, so you'll
do fine in this situation.
3) Be aware that HDFS currently assumes somewhat "local" semantics. Mostly,
that means there's no authentication or security layer. You'll probably want
to set up an appropriate VPN between locations to secure your traffic, and
firewall the various ports that HDFS uses to machines not on your VPN.

On 2/1/07, Achim Stumpf <newgrp@gmx.de> wrote:
> Hi list,
> I need a solution where it is possible to have three fileservers in
> three different locations. All of them have the same data. This data
> should be replicated over all of those three fileserver nodes. It should
> work similar to drbd, but with more than two nodes. It must not work on
> blocklevel. Rsync is not the solution, it should work more like a
> filesystem or blockdevice.
> I have made some experince with coda, but i am wondering if HDFS could
> do a similiar job.
> Thanks,
> Achim

Bryan A. P. Pendleton
Ph: (877) geek-1-bp

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message