hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: What about append in hadoop files ?
Date Fri, 14 Jul 2006 09:28:54 GMT
drwho wrote:
> If so, GFS, is also suitable only for large, offline, batch computations ?
> I wonder how Google is going to use GFS for writely or their online
> spreadsheet or their  BigTable (their gigantic relational DB).

Did I say anything about GFS?  I don't think so.  Also, I said, 
"currently" and "primarily", not "forever" and "exclusively".  I would 
love for DFS to be more suitable for online, incremental stuff, but 
we're a ways from that right now.  As I said, we're pursuing 
reliability, scalability and performance before features like append. 
If you'd like to try to implement append w/o disrupting work on 
reliability scalability and performance, we'd welcome your 
contributions.  The project direction is determined by contributors.

Note that BigTable is a complex layer on top of GFS that caches and 
batches i/o.  So, while GFS does implement some features that DFS still 
does not (like appends), GFS is probably not used directly by, e.g., 
writely.  Finally, BigTable is not relational.


> Doug Cutting <cutting@apache.org> wrote: <chopped>
> DFS is currently primarily used to support large, offline, batch 
> computations.  For example, a log of critical data with tight 
> transactional requirements is probably an inappropriate use of DFS at 
> this time.  Again, this may change, but that's where we are now.
> Doug
> Thanks much.
> -eric  

View raw message