hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: can i use hdfs in this scenery ?
Date Tue, 13 Sep 2011 05:01:26 GMT
Hi Kang Hua,

It sounds like the software you're building works just like HBase -
HBase is essentially a log structure database. We indeed call hflush()
on every edit (in addition to performing Group Commit for better
throughput)

You might check out the HBase source and look particularly at the HLog class.

-Todd

2011/9/12 kang hua <kanghua151@msn.com>:
> Hi Todd Lipcon:
>
> I am developing a log structure -like filesystem on hdfs . I hope data will
> not be lost when machine that client on hanppen to crash, so I hope to flush
> operation log to hdfs as soon as possible for roll forword when back from
> crash. However , operation log is small , So I flush it very frequently 。
>
> I use hflush methodfor above scenery ?  It is resonable?    and Can I
> use hdfsAvailable method(or others) to get file lenght that I hflush to but
> not close ?
>
> kanghua
>
>  Thansk you
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message