ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferry Syafei Sapei <ferry.sa...@googlemail.com>
Subject Re: Continuous update Data Grid Cache
Date Tue, 01 Dec 2015 09:08:25 GMT
Thank you for your idea. Your solution should work.

May it be possible to perform the following operation?
- Create a new cache and isolate it from being replicated with the old cache.
- Populate data on new cache
- Shutdown old cache and remove the new cache isolation.

fsapei


> Am 30.11.2015 um 12:51 schrieb Denis Magda <dmagda@gridgain.com>:
> 
> Fsapei,
> 
> Ignite neither supports a cache renaming nor has a notion of aliases for caches.
> 
> In general you can support aliases-like functionality by extending Ignite's JDBC Driver
implementation.
> The flow should look like this:
> - create a new cache and populate it;
> - redirect all the queries to the new cache. To achieve this the code has to parse a
SQL query string and exchange an aliase name with a real cache name;
> - delete the old cache. 
> 
> Will this work for you?
> 
> --
> Denis
> 
> On 11/30/2015 2:22 PM, Ferry Syafei Sapei wrote:
>> Hi Denis,
>> 
>> Using a database, we can import new data to new tables and then rename the table.
>> 
>> In ElasticSearch, there is an alias. The client executes queries against index alias.
On the background, the alias will point to the actual index. In my use case, I just need to
create a new index and then update the alias’ reference to the new index. The client does
not realize that the index has been changed, since it only communicates with the alias.
>> 
>> Are there these kind of operations in apache ignite?
>> 
>> 
>> fsapei
>> 
>> 
>>> Am 30.11.2015 um 11:57 schrieb Denis Magda <dmagda@gridgain.com <mailto:dmagda@gridgain.com>>:
>>> 
>>> Hi Fsapei,
>>> 
>>> I don't think that it's feasible or at least easy to implement your approach
without a downtime or changing cache name in queries once the second cache is fully propagated.
>>> How would you implement your use case if instead of an Ignite cache you would
be using a data base? 
>>> 
>>> --
>>> Denis
>>> 
>>> On 11/28/2015 11:50 AM, Ferry Syafei Sapei wrote:
>>>> Thank you for the reply. 
>>>> 
>>>> The client uses JDBC Driver to get the data from Ignite. 
>>>> 
>>>> The swap between new cache and old cache should be hidden from the client.
>>>> 
>>>> Would it be possible to do this in Ignite?
>>>> 
>>>> My first idea is to create a new cache and fill it with new data data. Afterwards,
within a transaction, the old cache content will be deleted and all the content from new cache
will be inserted to the old cache. 
>>>> 
>>>> 
>>>>> Am 28.11.2015 um 03:35 schrieb Dmitriy Setrakyan < <mailto:dsetrakyan@apache.org>dsetrakyan@apache.org
<mailto:dsetrakyan@apache.org>>:
>>>>> 
>>>>> Hi Fsapei,
>>>>> 
>>>>> Ignite supports dynamic caches, so you can create and destroy caches
on the fly:
>>>>> https://apacheignite.readme.io/docs/data-grid#section-jcache <https://apacheignite.readme.io/docs/data-grid#section-jcache>
>>>>> 
>>>>> To support your use case, you can:
>>>>> 
>>>>> 1. create a new cache and start populating it
>>>>> 2. while the new cache is populated, the queries should go to the old
cache.
>>>>> 3. when new cache is fully populated, route the queries to the new cache
>>>>> 4. delete the old cache
>>>>> 
>>>>> Will this work?
>>>>> 
>>>>> D.
>>>>> 
>>>>> On Sat, Nov 28, 2015 at 3:38 AM, Ferry Syafei Sapei < <mailto:ferry.sapei@googlemail.com>ferry.sapei@googlemail.com
<mailto:ferry.sapei@googlemail.com>> wrote:
>>>>> Hallo,
>>>>> 
>>>>> I would like to use Ignite as Data Grid. The cache will be initially
loaded with a lot of data (e.g. a CSV file with a size more than 10 GB).
>>>>> 
>>>>> When a new CSV file is available, the current Cache content must be replaced
with the new data from the new CSV. The old cache content must be completely deleted.
>>>>> 
>>>>> The switch from old cache content to the new one must happen without
downtime. When any SQL Clients execute query during the switch, they should still obtain any
data from the old cache content, since the switch has not finished yet.
>>>>> 
>>>>> Could somebody please advise me, how to continuously update the cache
like the description above?
>>>>> 
>>>>> Thank you.
>>>>> 
>>>>> 
>>>>> Best regards,
>>>>> fsapei
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> -- 
>>> Regards,
>>> Denis Magda
>>> Lead Professional Services Engineer, GridGain Systems
>>> http://www.gridgain.com/ <http://www.gridgain.com/>


Mime
View raw message