ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Ortiz <konstt2...@gmail.com>
Subject Re: GroupBy with index is really slow.
Date Wed, 10 May 2017 15:25:54 GMT
I'm using 2.0.0.
Right now in my laptop with 2Mill objects.


*WITH INDICES*
******Result SELECT distinct(MONTH) FROM PERSONWITHINDEX
September,February,March,August,June,April,July,January,May,November,October,
Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms
[SELECT DISTINCT
    __Z0.MONTH AS __C0_0
FROM "PersonWithIndex".PERSONWITHINDEX __Z0
    /* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */]
[SELECT DISTINCT
    __C0_0 AS MONTH
FROM PUBLIC.__T0
    /* "PersonWithIndex"."merge_scan" */]


*WITHOUT INDICES*
******Result SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX
September,November,February,March,May,April,July,August,January,June,October,
Elapsed time SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX:1690ms
[SELECT DISTINCT
    __Z0.MONTH AS __C0_0
FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
    /* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
[SELECT DISTINCT
    __C0_0 AS MONTH
FROM PUBLIC.__T0
    /* "PersonWithoutIndex"."merge_scan" */]
***************************

public class PersonWithIndex implements Serializable {
    @QuerySqlField(index = true)
    public int id;
    @QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=1)})
    public int age;
    @QuerySqlField
    public String dni;
    @QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=0)})
    public String month;


2017-05-10 17:06 GMT+02:00 afedotov <alexander.fedotoff@gmail.com>:

> Hi,
>
> Which Ignite version do you use?
> How many Ignite nodes do you run?
> What is your cache configuration?
>
> As well, could you please try the following query and see the performance?
> SELECT DISTINCT age FROM PersonWithindex;
>
>
> Kind regards,
> Alex.
>
> On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite Users]
> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=12593&i=0>>
> wrote:
>
>> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
>> and it takes about 30sec.
>> I have tried to execute the query with the same collection with and
>> without indices with the same time results.
>>
>> This is a test before to run in the real cluster with 6 nodes of 512Gb
>> and 48cores each one. We tried to execute there with a real dataset(22M
>> rows, 7gb) but it takes so long that query is aborted.
>>
>>
>> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=12592&i=0>>:
>>
>>> Hi,
>>>
>>> Why do you think query is slow?
>>> What its execution time? What is expected time?
>>> How many nodes do you use? How much data does cache contains?
>>>
>>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=12592&i=1>> wrote:
>>>
>>>> I have a simple cache where I have Person object (id, age and so on..)
>>>>
>>>> I have created indeces by id and age to try some examples but queries
>>>> with "group by" go really slow.
>>>>
>>>> I'm trying this query:
>>>> SELECT age
>>>> FROM
>>>> PersonWithindex
>>>> group by age
>>>>
>>>>
>>>> SELECT
>>>>     AGE AS __C0
>>>> FROM "personCacheWithIndex".PERSONWITHINDEX
>>>>     /* "personCacheWithIndex"."age_idx" */
>>>> GROUP BY AGE
>>>> /* group sorted */
>>>>
>>>> SELECT
>>>>     __C0 AS AGE
>>>> FROM PUBLIC.__T0
>>>>     /* "personCacheWithIndex"."merge_scan" */
>>>> GROUP BY __C0
>>>>
>>>> Although it seems that it uses index, why is it going so slow? I think
>>>> that it should be pretty fast with an index.
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-
>> index-is-really-slow-tp12526p12592.html
>> To start a new topic under Apache Ignite Users, email [hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=12593&i=1>
>> To unsubscribe from Apache Ignite Users, click here.
>> NAML
>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: GroupBy with index is really slow.
> <http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-index-is-really-slow-tp12526p12593.html>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>

Mime
View raw message