ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Getmanov <mikh...@getmanov.name>
Subject Re: Cache size Ignite 2.0?
Date Tue, 06 Jun 2017 14:27:39 GMT
<div>Counting for previously saved values:</div><div>* execute scan queries
<a href="https://apacheignite.readme.io/docs/cache-queries">https://apacheignite.readme.io/docs/cache-queries</a></div><div>*
for every object calculate length <a href="http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-td5131.html">http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-td5131.html</a> </div><div><div>Counting
while saving values:</div><div>* create local listener <a href="https://apacheignite.readme.io/docs/continuous-queries#section-local-listener">https://apacheignite.readme.io/docs/continuous-queries#section-local-listener</a></div><div>*
calculate length <a href="http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-td5131.html">http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-td5131.html</a> </div><div> </div><div>But
this "extravaganza" is missed: indices, meta information, keys, etc.</div></div><div><br
/></div><div><br /></div><div style="text-align:left;"><span
style="color:#808080;">__________________</span></div><div style="text-align:left;"><span
style="color:#808080;"><em><strong><span style="font-family:Helvetica,Arial,sans-serif;font-size:12px;line-height:15.5938px;text-align:justify;">Best
Regards</span></strong></em></span></div><div style="text-align:left;"><span
style="color:#808080;"><em><strong><span style="font-family:Helvetica,Arial,sans-serif;font-size:12px;line-height:15.5938px;text-align:justify;">Mikhail
Getmanov</span></strong></em></span></div><div><br
/></div><div><br /></div><div><br /></div><div>06.06.2017,
18:46, "Алексей Рябов" &lt;ryabov.a@gmail.com&gt;:</div><blockquote
type="cite"><div dir="ltr">Hi Mikhail,<div><div><br /></div><div>What
do you mean? Calculate a size of each BinaryObject instance before putting it i cache and
sum all of them to get final result?</div><div>Should be sufficient for me, but
is there any methid for this calculation?</div></div><div><br /></div><div>Thanks.</div></div><div><br
/><div><span>2017-06-06 13</span>:59 GMT+03:00 Mikhail Getmanov <span
dir="ltr">&lt;<a href="mailto:mikhail@getmanov.name" target="_blank">mikhail@getmanov.name</a>&gt;</span>:<br
/><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>Hello,</div><div>This
functionality does not work for a long time. This is a bad solution, but you can bypass the
cache and calculate the size of the objects that you store.</div><div><br /></div><div><br
/></div><div style="text-align:left;"><span style="color:#808080;">__________________</span></div><div
style="text-align:left;"><span style="color:#808080;"><em><strong><span
style="font-family:Helvetica,Arial,sans-serif;font-size:12px;line-height:15.5938px;text-align:justify;">Best
Regards</span></strong></em></span></div><div style="text-align:left;"><span
style="color:#808080;"><em><strong><span style="font-family:Helvetica,Arial,sans-serif;font-size:12px;line-height:15.5938px;text-align:justify;">Mikhail
Getmanov</span></strong></em></span></div><div><br
/></div><div><br /></div><div><br /></div><div>06.06.2017,
14:15, "Алексей Рябов" &lt;<a href="mailto:ryabov.a@gmail.com" target="_blank">ryabov.a@gmail.com</a>&gt;:</div><blockquote
type="cite"><div dir="ltr">HI Alexey,<div>I did not build Ignite myself - i
got binary 2.0 release. Here is the full output log:</div><div><br /></div><div><blockquote
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">[10:12:26]
   __________  ________________ <br />[10:12:26]   /  _/ ___/ |/ /  _/_  __/
__/ <br />[10:12:26]  _/ // (7 7    // /  / / / _/   <br />[10:12:26] /___/\___/_/|_/___/
/_/ /___/  <br />[10:12:26] <br />[10:12:26] ver. 2.0.0#<span><span>20170430</span></span>-sha1:d4eef3c6<br
/>[10:12:26] 2017 Copyright(C) Apache Software Foundation<br />[10:12:26] <br
/>[10:12:26] Ignite documentation: <a href="http://ignite.apache.org/" target="_blank">http://ignite.apache.org</a><br
/>[10:12:26] <br />[10:12:26] Quiet mode.<br />[10:12:26]   ^-- Logging to
file 'C:\bigdata\apache-ignite-<wbr />fabric-2.0.0-bin\work\log\<wbr />ignite-cf1c6ec9.0.log'<br
/>[10:12:26]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to
ignite.{sh|bat}<br />[10:12:26] <br />[10:12:26] OS: Windows 7 6.1 amd64<br
/>[10:12:26] VM information: Java(TM) SE Runtime Environment 1.8.0_101-b13 Oracle Corporation
Java HotSpot(TM) 64-Bit Server VM 25.101-b13<br />[10:12:26] Initial heap size is 254MB
(should be no less than 512MB, use -Xms512m -Xmx512m).<br />[10:12:26] Configured plugins:<br
/>[10:12:26]   ^-- None<br />[10:12:26] <br />[10:12:26] Message queue limit
is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC
or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.<br />[10:12:26]
Security status [authentication=off, tls/ssl=off]<br />[10:12:28] Performance suggestions
for grid  (fix if possible)<br />[10:12:28] To disable, set -DIGNITE_PERFORMANCE_<wbr
/>SUGGESTIONS_DISABLED=true<br />[10:12:28]   ^-- Enable G1 Garbage Collector (add
'-XX:+UseG1GC' to JVM options)<br />[10:12:28]   ^-- Specify JVM heap max size (add
'-Xmx&lt;size&gt;[g|G|m|M|k|K]' to JVM options)<br />[10:12:28]   ^-- Set max
direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=&lt;<wbr
/>size&gt;[g|G|m|M|k|K]' to JVM options)<br />[10:12:28]   ^-- Disable processing
of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)<br />[10:12:28]
Refer to this page for more performance suggestions: <a href="https://apacheignite.readme.io/docs/jvm-and-system-tuning"
target="_blank">https://apacheignite.readme.<wbr />io/docs/jvm-and-system-tuning</a><br
/>[10:12:28] <br />[10:12:28] To start Console Management &amp; Monitoring run
ignitevisorcmd.{sh|bat}<br />[10:12:28] <br />[10:12:28] Ignite node started
OK (id=cf1c6ec9)<br />[10:12:28] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8,
heap=3.5GB]<br />statistics: true<br />0<br />[10:12:28] Ignite node stopped
OK [uptime=00:00:00:166]<br />Process finished with exit code 0</blockquote></div></div><div><br
/><div><span><span>2017-06-05 23</span></span>:11 GMT+03:00
Alexey Goncharuk <span dir="ltr">&lt;<a href="mailto:alexey.goncharuk@gmail.com"
target="_blank">alexey.goncharuk@gmail.com</a>&gt;</span>:<br /><blockquote
style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div dir="ltr">Alexey,<div><br
/></div><div>There is no CacheMemoryMode in Ignite 2.0 anymore since it has
been removed in favor of the new Ignite architecture. It seems that you've built Ignite from
one of the intermediate states between 1.9 and 2.0.</div><div><br /></div><div>Can
you try with the ignite-2.0 release?</div><div><br /></div><div>--AG</div></div><div><br
/><div><span><span>2017-05-30 17</span></span>:00 GMT+03:00
Алексей Рябов <span dir="ltr">&lt;<a href="mailto:ryabov.a@gmail.com"
target="_blank">ryabov.a@gmail.com</a>&gt;</span>:<br /><blockquote
style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div dir="ltr">Hello,<div><br
/></div><div>i am trying to get the offHeap size of my cache. This code:</div><div><br
/></div><div><pre style="color:rgb(0,0,0);font-family:&quot;Courier
New&quot;;"><span style="color:rgb(0,0,128);font-weight:bold;">try </span>(Ignite
ignite = Ignition.<span style="font-style:italic;">start</span>()){<br />
   CacheConfiguration&lt;String, BinaryObject&gt; cfg = <span style="color:rgb(0,0,128);font-weight:bold;">new
</span>CacheConfiguration&lt;&gt;();<br />    cfg.setName(<span style="color:rgb(0,128,0);font-weight:bold;">"sample"</span>);<br
/>    cfg.setStatisticsEnabled(<span style="color:rgb(0,0,128);font-weight:bold;">true</span>)<wbr
/>;<br />    cfg.setMemoryMode(CacheMemoryM<wbr />ode.<span style="color:rgb(102,14,122);font-weight:bold;font-style:italic;">OFFHEAP_TIERED</span>);<br
/><br />    IgniteCache&lt;String, BinaryObject&gt; cache = ignite.getOrCreateCache(cfg).w<wbr
/>ithKeepBinary();<br />    cache.clear();<br />    <span style="color:rgb(0,0,128);font-weight:bold;">for
</span>(<span style="color:rgb(0,0,128);font-weight:bold;">int </span>i=<span
style="color:rgb(0,0,255);">0</span>; i&lt;<span style="color:rgb(0,0,255);">100</span>;
i++) {<br />        BinaryObjectBuilder builder = ignite.binary().builder(<span style="color:rgb(0,128,0);font-weight:bold;">"sampl<wbr
/>e"</span>);<br />        builder.setField(<span style="color:rgb(0,128,0);font-weight:bold;">"field1"</span>,
i*<span style="color:rgb(0,0,255);">1000</span>);<br />        builder.setField(<span
style="color:rgb(0,128,0);font-weight:bold;">"field2"</span>, i);<br />   
    cache.put(Integer.<span style="font-style:italic;">toString</span>(i), builder.build());<br
/>    }<br />    System.<span style="color:rgb(102,14,122);font-weight:bold;font-style:italic;">out</span>.println(<span
style="color:rgb(0,128,0);font-weight:bold;">"statistics<wbr />: " </span>+
cache.metrics().isStatisticsEn<wbr />abled());<br />    System.<span style="color:rgb(102,14,122);font-weight:bold;font-style:italic;">out</span>.println(cache.metri<wbr
/>cs().getOffHeapAllocatedSize()<wbr />);<br />} <span style="color:rgb(0,0,128);font-weight:bold;">catch
</span>(Exception e) {<br />    e.printStackTrace();<br />}</pre><pre
style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;"><br /></pre><pre
style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">works fine in Ignite
version 1.9.0:</pre><pre style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">...</pre><pre><font
color="#000000" face="Courier New">[16:57:39] Topology snapshot [ver=1, servers=1, clients=0,
CPUs=8, heap=3.5GB]
statistics: true
11890
[16:57:39] Ignite node stopped OK [uptime=00:00:00:230]<br /></font></pre><pre
style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">...</pre><pre
style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">but in 2.0.0 (i commented
cout line cfg.setMemoryMode(CacheMemoryM<wbr />ode.OFFHEAP_TIERED)) methon getOffHeapAllocatedSize
returns zero:</pre><pre><font color="#000000" face="Courier New">...</font></pre><pre><font
color="#000000" face="Courier New">[16:59:13] Topology snapshot [ver=1, servers=1, clients=0,
CPUs=8, heap=3.5GB]
statistics: true
0
[16:59:22] Ignite node stopped OK [uptime=00:00:08:965]<br /></font></pre><pre><font
color="#000000" face="Courier New">...</font></pre><pre style="color:rgb(0,0,0);font-family:&quot;Courier
New&quot;;">Is there anything special that I should do in 2.0.0 to get the size of
my cache?</pre><pre style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;"><br
/></pre><pre style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">Thanks
a lot,</pre><pre style="color:rgb(0,0,0);font-family:&quot;Courier New&quot;;">Aleksey.</pre></div><div><br
/></div></div>
</blockquote></div><br /></div>
</blockquote></div><br /></div>
</blockquote></blockquote></div><br /></div>
</blockquote>
Mime
View raw message