ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: How IGFS keep sync with HDFS?
Date Fri, 02 Dec 2016 11:48:35 GMT
Hi,

IGFS always propagate updates to HDFS immediately (or with slight delay in
case of writes in DUAL_ASYNC mode). It doesn't remove data from memory
after flushing it to HDFS. You can try configuring
*org.apache.ignite.cache.eviction.igfs.IgfsPerBlockLruEvictionPolicy
*to evict some data blocks from IGFS data cache.

As per expiry policy, it doesn't affect flushing logic anyhow. If eviction
happens after flush, then data will not be lost. Otherwise it can be lost.
Though, this is possible only in DUAL_ASYNC mode with very low TTL.

Vladimir.

On Tue, Nov 29, 2016 at 2:17 PM, Kaiming Wan <344277934@qq.com> wrote:

> If IGFS can't hold more input data, will it flush data to HDFS? How IGFS
> flush data to HDFS? Async or sync?
>
> If it a async mode, when IGFS can't hold more data, it will using cache
> expire policy. And if the data which is expired is not persisted to HDFS
> timely, the data will be lost.  Is is possible?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/How-IGFS-keep-sync-with-HDFS-tp9258p9262.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Mime
View raw message