Mailing-List: contact user-help@ignite.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <1439317864212-912.post@n6.nabble.com>
References: <1439300564318-907.post@n6.nabble.com>
	<CABDss3hb7Bufzpk2Gmi0GH282R-g0ZVrCv+BEiomBxYXVXqeGg@mail.gmail.com>
	<1439317864212-912.post@n6.nabble.com>
Date: Tue, 11 Aug 2015 14:30:52 -0700
Message-ID: 
 <CABDss3gtzjmoeUjeNqttU+BFqS+eBH4TJkAmyPcVi3GJ-i+r8w@mail.gmail.com>
Subject: Re: Some question about the data loading
From: Alexey Goncharuk <alexey.goncharuk@gmail.com>
To: user@ignite.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a1135490a149cb0051d0fd1e4

--001a1135490a149cb0051d0fd1e4
Content-Type: text/plain; charset=UTF-8

1. When new nodes join a cluster, partition-to-node assignment changes.
Let's assume you have one backup. When you have just one node, it is
responsible for all partitions, so your first node will try to load all the
partitions. When second node joins grid, it will try to re-balance existing
data from the first node and at the same time it will try to load the same
set of partitions since we assumed you have one backup. When third node
joins, it again will rebalance existing data from two first nodes and will
try to load from the database 2/3s of partitions, and so on. In this
scenario all nodes but the last one will try to load more partitions than
necessary. The best approach here is to wait until your topology has enough
nodes (you can create an event listener for node join/leave event) and then
call loadCache().
2. It should not run on client node because it does not make sense, I will
be surprised if it is.
3. In the ticket you have created your XML configuration does not have a
name, but in the code you do have cache name. So in your cluster you end up
with two caches, one with cache store (defined in the XML) and one without
cache store, so it looks like you end up calling loadCache on the cache
without store.

2015-08-11 11:31 GMT-07:00 kcheng.mvp <kcheng.mvp@gmail.com>:

> In fact I have another question about the data loading.
>
> 1:
> Right now I am going to implement the data loading in the LifecycleBean
> call
> back method.
>
> @Override
>         public void onLifecycleEvent(LifecycleEventType evt) throws
> IgniteException
> {
>                 if(evt == LifecycleEventType.AFTER_NODE_START){
>                         // do data loading here
>                 }
>
>         }
>
>
> when I start the node one by one(I have four nodes), with the node starts
> then the node cluster topology will change accordingly.
>
> I guess the *partition ID* will be changing with the topology changes. In
> this case there are some data would not be loaded into cache, right?
>
> if the *LifecycleBean*is not the ideal place to preload the data, so what's
> the best practice to do pre data loading
>
>
>
>
> 2: there are two method to trigger data loading *IgniteCache.loadCache()*
> and *IgniteCache.localLoadCache()*
>
> suppose there are 5 nodes(1 client nodes + 4 server nodes), when call
> *IgniteCache.loadCache()*
> from server node, client node would not load the data right?
>
> 3:
> In fact I am running into issue when try to do data load, and I fired a bug
> https://issues.apache.org/jira/browse/IGNITE-1234
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Some-question-about-the-data-loading-tp907p912.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

--001a1135490a149cb0051d0fd1e4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">1. When new nodes join a cluster, partition-to-node assign=
ment changes. Let&#39;s assume you have one backup. When you have just one =
node, it is responsible for all partitions, so your first node will try to =
load all the partitions. When second node joins grid, it will try to re-bal=
ance existing data from the first node and at the same time it will try to =
load the same set of partitions since we assumed you have one backup. When =
third node joins, it again will rebalance existing data from two first node=
s and will try to load from the database 2/3s of partitions, and so on. In =
this scenario all nodes but the last one will try to load more partitions t=
han necessary. The best approach here is to wait until your topology has en=
ough nodes (you can create an event listener for node join/leave event) and=
 then call loadCache().<div>2. It should not run on client node because it =
does not make sense, I will be surprised if it is.</div><div>3. In the tick=
et you have created your XML configuration does not have a name, but in the=
 code you do have cache name. So in your cluster you end up with two caches=
, one with cache store (defined in the XML) and one without cache store, so=
 it looks like you end up calling loadCache on the cache without store.</di=
v></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">2015-08-1=
1 11:31 GMT-07:00 kcheng.mvp <span dir=3D"ltr">&lt;<a href=3D"mailto:kcheng=
.mvp@gmail.com" target=3D"_blank">kcheng.mvp@gmail.com</a>&gt;</span>:<br><=
blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
 #ccc solid;padding-left:1ex"><span class=3D"">In fact I have another quest=
ion about the data loading.<br>
<br>
1:<br>
Right now I am going to implement the data loading in the LifecycleBean cal=
l<br>
back method.<br>
<br>
@Override<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 public void onLifecycleEvent(LifecycleEventType=
 evt) throws IgniteException<br>
{<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if(evt =3D=3D Lifec=
ycleEventType.AFTER_NODE_START){<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 // do data loading here<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
<br>
<br>
when I start the node one by one(I have four nodes), with the node starts<b=
r>
then the node cluster topology will change accordingly.<br>
<br>
I guess the *partition ID* will be changing with the topology changes. In<b=
r>
this case there are some data would not be loaded into cache, right?<br>
<br>
if the *LifecycleBean*is not the ideal place to preload the data, so what&#=
39;s<br>
the best practice to do pre data loading<br>
<br>
<br>
<br>
<br>
2: there are two method to trigger data loading *IgniteCache.loadCache()*<b=
r>
and *IgniteCache.localLoadCache()*<br>
<br>
</span>suppose there are 5 nodes(1 client nodes + 4 server nodes), when cal=
l<br>
<span class=3D"">*IgniteCache.loadCache()*<br>
from server node, client node would not load the data right?<br>
<br>
3:<br>
In fact I am running into issue when try to do data load, and I fired a bug=
<br>
<a href=3D"https://issues.apache.org/jira/browse/IGNITE-1234" rel=3D"norefe=
rrer" target=3D"_blank">https://issues.apache.org/jira/browse/IGNITE-1234</=
a><br>
<br>
<br>
<br>
<br>
<br>
<br>
</span>--<br>
View this message in context: <a href=3D"http://apache-ignite-users.70518.x=
6.nabble.com/Some-question-about-the-data-loading-tp907p912.html" rel=3D"no=
referrer" target=3D"_blank">http://apache-ignite-users.70518.x6.nabble.com/=
Some-question-about-the-data-loading-tp907p912.html</a><br>
<div class=3D"HOEnZb"><div class=3D"h5">Sent from the Apache Ignite Users m=
ailing list archive at Nabble.com.<br>
</div></div></blockquote></div><br></div>

--001a1135490a149cb0051d0fd1e4--