ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexei Scherbakov <alexey.scherbak...@gmail.com>
Subject Re: Allow distributed SQL query execution over explicit set of partitions
Date Wed, 18 Jan 2017 22:04:57 GMT
I mean distributed joins.

2017-01-19 0:10 GMT+03:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com>:

> Guys,
>
> I've finished adding API changes and implemented proper nodes routing.
>
> Currently it doesn't work with distributed queries.But I think this
> feature should be compatible with it.
>
> Could anyone take a look at current branch state while I'm looking deeper
> into dsitributed queries code?
>
> Issue: https://issues.apache.org/jira/browse/IGNITE-4523
> PR: https://github.com/apache/ignite/pull/1418
>
>
>
> 2017-01-13 15:55 GMT+03:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com
> >:
>
>> OK, let's do it this way.
>>
>>
>>
>>
>>
>> 2017-01-13 13:27 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:
>>
>>> Internally we still use int[] when we send partitions (see
>>> GridH2QueryRequest.parts). It looks like we only do more work with
>>> PartitionSet.
>>>
>>> I like the idea of bitset for partitions, but
>>>
>>> 1. We have to change internals first to use it, otherwise the
>>> optimization
>>> makes no sense.
>>> 2. We will need to have a method SqlQuery.setPartitions(int... parts) for
>>> usability reasons anyways.
>>>
>>> Thus I suggest for now to go the straightforward way with int[] and
>>> create
>>> a separate ticket describing the optimization with bitset.
>>>
>>> Sergi
>>>
>>> 2017-01-13 13:06 GMT+03:00 Alexei Scherbakov <
>>> alexey.scherbakoff@gmail.com>:
>>>
>>> > PartitionSet hides internal implementation of int array.
>>> >
>>> > This allows as to efficiently represent contiguous range of partitions
>>> and
>>> > defines clear API for ordered iteration over partitions and containment
>>> > check.
>>> >
>>> > Even better to go with compressed bitmap, as I mentioned in ticket
>>> comment.
>>> > This will allow us to minimize heap footprint for this object.
>>> >
>>> > Moreover, it will be useful to create reusable compressed bitmap
>>> > implementation in Ignite and use it in other cases, on example, for
>>> > replacing H2's IntArray and Set<Integer>.
>>> >
>>> > Should I create a ticket for this ?
>>> >
>>> > .
>>> >
>>> > 2017-01-13 1:01 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
>>> >
>>> > > On Thu, Jan 12, 2017 at 6:12 AM, Sergi Vladykin <
>>> > sergi.vladykin@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > I looked at the code. The PartitionSet concept looks
>>> overengineered to
>>> > > me,
>>> > > > why wouldn't we just go with int[]?
>>> > > >
>>> > >
>>> > > Agree.
>>> > >
>>> > >
>>> > > >
>>> > > > Sergi
>>> > > >
>>> > > > 2017-01-12 15:18 GMT+03:00 Alexei Scherbakov <
>>> > > alexey.scherbakoff@gmail.com
>>> > > > >:
>>> > > >
>>> > > > > Done.
>>> > > > >
>>> > > > > 2017-01-11 20:39 GMT+03:00 Dmitriy Setrakyan <
>>> dsetrakyan@apache.org
>>> > >:
>>> > > > >
>>> > > > > > Alexey,
>>> > > > > >
>>> > > > > > I am not sure I am seeing the API changes documented
in the
>>> ticket.
>>> > > Can
>>> > > > > you
>>> > > > > > please either document them or add GIT links for the
new
>>> classes?
>>> > > > > >
>>> > > > > > D.
>>> > > > > >
>>> > > > > > On Wed, Jan 11, 2017 at 9:29 AM, Alexei Scherbakov <
>>> > > > > > alexey.scherbakoff@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Guys,
>>> > > > > > >
>>> > > > > > > I've just submitted a PR for
>>> > > > > > > https://issues.apache.org/jira/browse/IGNITE-4523.
>>> > > > > > >
>>> > > > > > > Please review API changes while waiting for TC
results.
>>> > > > > > >
>>> > > > > > > --
>>> > > > > > >
>>> > > > > > > Best regards,
>>> > > > > > > Alexei Scherbakov
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > >
>>> > > > > Best regards,
>>> > > > > Alexei Scherbakov
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> > Best regards,
>>> > Alexei Scherbakov
>>> >
>>>
>>
>>
>>
>> --
>>
>> Best regards,
>> Alexei Scherbakov
>>
>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>



-- 

Best regards,
Alexei Scherbakov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message