asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sattam Alsubaiee <salsuba...@gmail.com>
Subject Re: PrimaryIndexOperationTracker
Date Tue, 24 Oct 2017 03:12:51 GMT
There is a fundamental reason behind that (unless that has changed). All
partitions of a dataset (primary and secondary indexes) share the memory
budget for their in-memory components. So they use the same optracker to
synchronize their operations.


Sattam

On Oct 24, 2017 5:59 AM, "Wail Alkowaileet" <wael.y.k@gmail.com> wrote:

> Thanks Abdullah!
>
> On Mon, Oct 23, 2017 at 7:15 PM, abdullah alamoudi <bamousaa@gmail.com>
> wrote:
>
> > Hi Wail,
> > There is no fundamental reason why it is one. In fact, it has been on our
> > todo for a long time to make it one per partition.
> >
> > Cheers,
> > Abdullah.
> >
> > > On Oct 23, 2017, at 7:14 PM, Wail Alkowaileet <wael.y.k@gmail.com>
> > wrote:
> > >
> > > Dear devs,
> > >
> > > I have a question regarding the opTracker. Currently, we initialize one
> > > opTracker per dataset in every NC.
> > >
> > > My question is why it's per dataset not per partition ? Is there a
> > > transactional constraints for that ?
> > >
> > > From what I can see that the opTracker can create a lot of contention
> in
> > > case there're many IO devices. For instance, each insert will call
> > > *LSMHarness.getAndEnterComponents()* [1], which
> > *synchronize(opTracker). *That
> > > means (correct me if I'm wrong), insert is going to serialize the
> > > *enterComponent()* part among partitions.
> > >
> > > [1]
> > > https://github.com/apache/asterixdb/blob/master/hyracks-
> > fullstack/hyracks/hyracks-storage-am-lsm-common/src/
> > main/java/org/apache/hyracks/storage/am/lsm/common/impls/
> > LSMHarness.java#L86
> > >
> > > --
> > >
> > > *Regards,*
> > > Wail Alkowaileet
> >
> >
>
>
> --
>
> *Regards,*
> Wail Alkowaileet
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message