ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferry Syafei Sapei <ferry.sa...@googlemail.com>
Subject Re: Continuous update Data Grid Cache
Date Mon, 30 Nov 2015 11:22:47 GMT
Hi Denis,

Using a database, we can import new data to new tables and then rename the table.

In ElasticSearch, there is an alias. The client executes queries against index alias. On the
background, the alias will point to the actual index. In my use case, I just need to create
a new index and then update the alias’ reference to the new index. The client does not realize
that the index has been changed, since it only communicates with the alias.

Are there these kind of operations in apache ignite?


fsapei


> Am 30.11.2015 um 11:57 schrieb Denis Magda <dmagda@gridgain.com>:
> 
> Hi Fsapei,
> 
> I don't think that it's feasible or at least easy to implement your approach without
a downtime or changing cache name in queries once the second cache is fully propagated.
> How would you implement your use case if instead of an Ignite cache you would be using
a data base? 
> 
> --
> Denis
> 
> On 11/28/2015 11:50 AM, Ferry Syafei Sapei wrote:
>> Thank you for the reply. 
>> 
>> The client uses JDBC Driver to get the data from Ignite. 
>> 
>> The swap between new cache and old cache should be hidden from the client.
>> 
>> Would it be possible to do this in Ignite?
>> 
>> My first idea is to create a new cache and fill it with new data data. Afterwards,
within a transaction, the old cache content will be deleted and all the content from new cache
will be inserted to the old cache. 
>> 
>> 
>>> Am 28.11.2015 um 03:35 schrieb Dmitriy Setrakyan <dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>:
>>> 
>>> Hi Fsapei,
>>> 
>>> Ignite supports dynamic caches, so you can create and destroy caches on the fly:
>>> https://apacheignite.readme.io/docs/data-grid#section-jcache <https://apacheignite.readme.io/docs/data-grid#section-jcache>
>>> 
>>> To support your use case, you can:
>>> 
>>> 1. create a new cache and start populating it
>>> 2. while the new cache is populated, the queries should go to the old cache.
>>> 3. when new cache is fully populated, route the queries to the new cache
>>> 4. delete the old cache
>>> 
>>> Will this work?
>>> 
>>> D.
>>> 
>>> On Sat, Nov 28, 2015 at 3:38 AM, Ferry Syafei Sapei < <mailto:ferry.sapei@googlemail.com>ferry.sapei@googlemail.com
<mailto:ferry.sapei@googlemail.com>> wrote:
>>> Hallo,
>>> 
>>> I would like to use Ignite as Data Grid. The cache will be initially loaded with
a lot of data (e.g. a CSV file with a size more than 10 GB).
>>> 
>>> When a new CSV file is available, the current Cache content must be replaced
with the new data from the new CSV. The old cache content must be completely deleted.
>>> 
>>> The switch from old cache content to the new one must happen without downtime.
When any SQL Clients execute query during the switch, they should still obtain any data from
the old cache content, since the switch has not finished yet.
>>> 
>>> Could somebody please advise me, how to continuously update the cache like the
description above?
>>> 
>>> Thank you.
>>> 
>>> 
>>> Best regards,
>>> fsapei
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
> 
> -- 
> Regards,
> Denis Magda
> Lead Professional Services Engineer, GridGain Systems
> http://www.gridgain.com/ <http://www.gridgain.com/>

Mime
View raw message