hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <ma...@cloudera.com>
Subject Re: HDFS architecture based on GFS?
Date Sun, 15 Feb 2009 20:07:30 GMT
Hi Amandeep,
Hadoop is definitely inspired by MapReduce/GFS and aims to provide those
capabilities as an open-source project. HDFS is similar to GFS (large
blocks, replication, etc); some notable things missing are read-write
support in the middle of a file (unlikely to be provided because few Hadoop
applications require it) and multiple appenders (the record append
operation). You can read about HDFS architecture at
http://hadoop.apache.org/core/docs/current/hdfs_design.html. The MapReduce
part of Hadoop interacts with HDFS in the same way that Google's MapReduce
interacts with GFS (shipping computation to the data), although Hadoop
MapReduce also supports running over other distributed filesystems.

Matei

On Sun, Feb 15, 2009 at 11:57 AM, Amandeep Khurana <amansk@gmail.com> wrote:

> Hi
>
> Is the HDFS architecture completely based on the Google Filesystem? If it
> isnt, what are the differences between the two?
>
> Secondly, is the coupling between Hadoop and HDFS same as how it is between
> the Google's version of Map Reduce and GFS?
>
> Amandeep
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message