geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stolz <mst...@pivotal.io>
Subject Re: [DISCUSS] Addition of isValid API to Index interface
Date Mon, 11 Sep 2017 20:23:29 GMT
I think there was an intention of having CREATION of an index require a
higher privilege than DATA:WRITE, but it shouldn't affect applying the
index on either of put or get operations.

If we are requiring something like CLUSTER:MANAGE for put on an indexed
region, that is an incorrect requirement. Only DATA:WRITE should be
required to put an entry and have it be indexed if an index is present.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771

On Fri, Sep 8, 2017 at 6:04 PM, Anilkumar Gingade <agingade@pivotal.io>
wrote:

> Indexes are critical for querying; most of the databases doesn't allow
> insert/update if there is any failure with index maintenance...
>
> As Geode OQL supports two ways (sync and async) to maintain the indexes, we
> need be careful about the error handling in both cases...
>
> My take is:
> 1. For synchronous index maintenance:
> If there is any failure in updating any index (security/auth or logical
> error) on the region; throw an exception and rollback the cache update/op
> (index management id done under region.entry lock - we should be able to
> revert the op). If index or cache is left in bad state, then its a bug that
> needs to be addressed.
>
> Most of the time, If there is any logical error in index, it will be
> detected as soon as index is created (on existing data) or when first
> update is done to the cache.
>
> 2. For Asynchronous index maintenance:
> As this is async (assuming) user has good understanding of the risk
> involved with async, any error with index maintenance, the index should be
> invalidated...
>
>  About the security/auth, the user permission with region read/write needs
> to be applied for index updates, there should not be different permission
> on index.
>
> -Anil.
>
>
>
> On Fri, Sep 8, 2017 at 2:01 PM, Nabarun Nag <nnag@pivotal.io> wrote:
>
> > Hi Mike,
> >
> > Please do find our answers below:
> > *Question:* What if there were multiple indices that were in flight and
> > only the third
> > one errors out, will they all be marked invalid?
> >
> > *Answer:* Only the third will be marked invalid and only the third one
> will
> > not be used for query execution.
> >
> > *Question/Statement:* If anything goes wrong with the put it should
> > probably still throw back to
> > the caller. Silent invalidation of the index is probably not desirable.
> >
> > *Answer: *
> > In our current design this the flow of execution of a put operation:
> > entry put into region -> update index -> other wan related executions /
> > callbacks etc.
> >
> > If an exception happens while updating the index, the cache gets into a
> bad
> > state, and we may end up getting different results depending on the index
> > we are using. As the failure happens half way in a put operation, the
> > regions / cache are now in a bad state.
> > --------------------------
> > We are thinking that if index is created  over a method invocation in an
> > empty region and then we do puts, but method invocation is not allowed as
> > per security policies. The puts will now be successful but the index will
> > be rendered invalid. Previously the puts will fail with exception and put
> > the entire cache in a bad state.
> >
> >
> >
> > Regards
> > Nabarun
> >
> >
> >
> >
> >
> > On Fri, Sep 8, 2017 at 10:43 AM Michael Stolz <mstolz@pivotal.io> wrote:
> >
> > > Just to help me understand, the index is corrupted in a way beyond just
> > the
> > > field that errors out?
> > > What if there were multiple indices that were in flight and only the
> > third
> > > one errors out, will they all be marked invalid?
> > > If anything goes wrong with the put it should probably still throw back
> > to
> > > the caller. Silent invalidation of the index is probably not desirable.
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, GemFire Product Manager
> > > Mobile: +1-631-835-4771 <(631)%20835-4771>
> > >
> > > On Fri, Sep 8, 2017 at 12:34 PM, Dan Smith <dsmith@pivotal.io> wrote:
> > >
> > > > +1
> > > >
> > > > -Dan
> > > >
> > > > On Thu, Sep 7, 2017 at 9:14 PM, Nabarun Nag <nnag@apache.org> wrote:
> > > >
> > > > > *Proposal:*
> > > > > * Index interface will include an API - isValid() which will return
> > > true
> > > > if
> > > > > the index is still valid / uncorrupted, else will return false if
> it
> > > > > corrupted / invalid.
> > > > > * gfsh command "list index" will have one more column "Is Valid"
> > which
> > > > will
> > > > > state the status as "true" or "false".
> > > > > * Invalid indexes will not be used during query executions.
> > > > > * In case the index is found to be invalid, the user will be able
> to
> > > > > remove/destroy the index.
> > > > > * When a put operation corrupts an index, it will be logged.
> > > > >
> > > > > *Reasoning:*
> > > > > * Currently if a put operation raises an exception while updating
> the
> > > > > index, the put operation fails with an exception to the putter.
> > > > > * For example, if an index is created on an object method, and that
> > > > method
> > > > > causes an exception while updating the index as a part of a put
> > > > operation,
> > > > > then the put operation fails for that particular entry and the
> index
> > is
> > > > > left in a bad state.
> > > > > * This may occur also due to lack of permission to update index but
> > > have
> > > > > permission to do puts.
> > > > > * We are proposing that in the above mentioned scenarios, the put
> > > > succeeds
> > > > > in putting the entry in the region but the index which it was
> trying
> > to
> > > > > update will be marked invalid and will not be used for query
> > > executions.
> > > > > * This can be justified because the corrupted index will not
> > correctly
> > > > > represent the region entries.
> > > > >
> > > > > *Status Quo:*
> > > > > * Index creation will still fail if we are trying to create an
> index
> > > > over a
> > > > > region containing an entry/entries  which will cause an exception
> > while
> > > > > loading the entry into the index.
> > > > >
> > > > > Regards
> > > > > Nabarun Nag
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message