cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <SEAN_R_DUR...@homedepot.com>
Subject RE: Cassandra 3.1 - Aggregation query failure
Date Wed, 23 Dec 2015 17:02:48 GMT
An aggregate only within a partition? That is rather useless and shouldn’t be called an aggregate.

I am hoping the functionality can be used to support at least “normal” types of aggregates
like count, sum, avg, etc.


Sean Durity – Lead Cassandra Admin

From: Jonathan Haddad [mailto:jon@jonhaddad.com]
Sent: Monday, December 21, 2015 2:50 PM
To: user@cassandra.apache.org; dinesh.shanbhag@isanasystems.com
Subject: Re: Cassandra 3.1 - Aggregation query failure

Even if you get this to work for now, I really recommend using a different tool, like Spark.
 Personally I wouldn't use UDAs outside of a single partition.

On Mon, Dec 21, 2015 at 1:50 AM Dinesh Shanbhag <dinesh.shanbhag@isanasystems.com<mailto:dinesh.shanbhag@isanasystems.com>>
wrote:

Thanks for the pointers!  I edited jvm.options in
$CASSANDRA_HOME/conf/jvm.options to increase -Xms and -Xmx to 1536M.
The result is the same.

And in $CASSANDRA_HOME/logs/system.log, grep GC system.log produces this
(when jvm.options had not been changed):

INFO  [Service Thread] 2015-12-18 15:26:31,668 GCInspector.java:284 -
ConcurrentMarkSweep GC in 296ms.  CMS Old Gen: 18133664 -> 15589256;
Code Cache: 5650880 -> 8122304; Compressed Class Space: 2530064 ->
3345624; Metaspace: 21314000 -> 28040984; Par Eden Space: 7019256 ->
164070848;
INFO  [Service Thread] 2015-12-18 15:48:39,736 GCInspector.java:284 -
ConcurrentMarkSweep GC in 379ms.  CMS Old Gen: 649257416 -> 84190176;
Code Cache: 20772224 -> 20726848; Par Eden Space: 2191408 -> 52356736;
Par Survivor Space: 2378448 -> 2346840
INFO  [Service Thread] 2015-12-18 15:58:35,118 GCInspector.java:284 -
ConcurrentMarkSweep GC in 406ms.  CMS Old Gen: 648847808 -> 86954856;
Code Cache: 21182080 -> 21188032; Par Eden Space: 1815696 -> 71525744;
Par Survivor Space: 2388648 -> 2364696
INFO  [Service Thread] 2015-12-18 16:13:45,821 GCInspector.java:284 -
ConcurrentMarkSweep GC in 211ms.  CMS Old Gen: 648343768 -> 73135720;
Par Eden Space: 3224880 -> 7957464; Par Survivor Space: 2379912 -> 2414520
INFO  [Service Thread] 2015-12-18 16:32:46,419 GCInspector.java:284 -
ConcurrentMarkSweep GC in 387ms.  CMS Old Gen: 648476072 -> 68888832;
Par Eden Space: 2006624 -> 64263360; Par Survivor Space: 2403792 -> 2387664
INFO  [Service Thread] 2015-12-18 16:42:38,648 GCInspector.java:284 -
ConcurrentMarkSweep GC in 365ms.  CMS Old Gen: 649126336 -> 137359384;
Code Cache: 22972224 -> 22979840; Metaspace: 41374464 -> 41375104; Par
Eden Space: 4286080 -> 154449480; Par Survivor Space: 1575440 -> 2310768
INFO  [Service Thread] 2015-12-18 16:51:57,538 GCInspector.java:284 -
ConcurrentMarkSweep GC in 322ms.  CMS Old Gen: 648338928 -> 79783856;
Par Eden Space: 2058968 -> 56931312; Par Survivor Space: 2342760 -> 2400336
INFO  [Service Thread] 2015-12-18 17:02:49,543 GCInspector.java:284 -
ConcurrentMarkSweep GC in 212ms.  CMS Old Gen: 648702008 -> 122954344;
Par Eden Space: 3269032 -> 61433328; Par Survivor Space: 2395824 -> 3448760
INFO  [Service Thread] 2015-12-18 17:11:54,090 GCInspector.java:284 -
ConcurrentMarkSweep GC in 306ms.  CMS Old Gen: 648748576 -> 70965096;
Par Eden Space: 2174840 -> 27074432; Par Survivor Space: 2365992 -> 2373984
INFO  [Service Thread] 2015-12-18 17:22:28,949 GCInspector.java:284 -
ConcurrentMarkSweep GC in 350ms.  CMS Old Gen: 648243024 -> 90897272;
Par Eden Space: 2150168 -> 43487192; Par Survivor Space: 2401872 -> 2410728


After modifying jvm.options to increase -Xms & -Xmx (to 1536M):

INFO  [Service Thread] 2015-12-21 11:39:24,918 GCInspector.java:284 -
ConcurrentMarkSweep GC in 342ms.  CMS Old Gen: 18579136 -> 16305144;
Code Cache: 8600128 -> 10898752; Compressed Class Space: 3431288 ->
3761496; Metaspace: 29551832 -> 33307352; Par Eden Space: 4822000 ->
94853272;
INFO  [Service Thread] 2015-12-21 11:39:30,710 GCInspector.java:284 -
ParNew GC in 206ms.  CMS Old Gen: 22932208 -> 41454520; Par Eden Space:
167772160 -> 0; Par Survivor Space: 13144872 -> 20971520
INFO  [Service Thread] 2015-12-21 13:08:14,922 GCInspector.java:284 -
ConcurrentMarkSweep GC in 468ms.  CMS Old Gen: 21418016 -> 16146528;
Code Cache: 11693888 -> 11744704; Compressed Class Space: 4331224 ->
4344192; Metaspace: 37191144 -> 37249960; Par Eden Space: 146089224 ->
148476848;
INFO  [Service Thread] 2015-12-21 13:08:53,068 GCInspector.java:284 -
ParNew GC in 216ms.  CMS Old Gen: 16146528 -> 26858568; Par Eden Space:
167772160 -> 0;


Earlier the node had OpenJDK 8.  For today's tests I installed and used
Oracle Java 8.

Do the above messages provide any clue? Or any debug logging I can
enable to progress further?
Thanks,
Dinesh.

On 12/18/2015 9:56 PM, Tyler Hobbs wrote:
>
> On Fri, Dec 18, 2015 at 9:17 AM, DuyHai Doan <doanduyhai@gmail.com<mailto:doanduyhai@gmail.com>
> <mailto:doanduyhai@gmail.com<mailto:doanduyhai@gmail.com>>> wrote:
>
>     Cassandra will perform a full table scan and fetch all the data in
>     memory to apply the aggregate function.
>
>
> Just to clarify for others on the list: when executing aggregation
> functions, Cassandra /will/ use paging internally, so at most one page
> worth of data will be held in memory at a time.  However, if your
> aggregation function retains a large amount of data, this may
> contribute to heap pressure.
>
>
> --
> Tyler Hobbs
> DataStax <http://datastax.com/>

________________________________

The information in this Internet Email is confidential and may be legally privileged. It is
intended solely for the addressee. Access to this Email by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying, distribution or any action taken
or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed
to our clients any opinions or advice contained in this Email are subject to the terms and
conditions expressed in any applicable governing The Home Depot terms of business or client
engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy
and content of this attachment and for any damages or losses arising from any inaccuracies,
errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature,
which may be contained in this attachment and shall not be liable for direct, indirect, consequential
or special damages in connection with this e-mail message or its attachment.
Mime
View raw message