couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kyle Snavely <kjsnav...@gmail.com>
Subject Re: [DISCUSS] Reduce on FDB take 3
Date Fri, 24 Jul 2020 00:35:07 GMT
When in doubt, throw us a build at Cloudant with ebtree maps and we'll see
if it comes close to the crazy fast KV map queries.

Kyle

On Thu, Jul 23, 2020, 2:17 PM Robert Samuel Newson <rnewson@apache.org>
wrote:

> I (perhaps obviously) don't agree that I'm tying myself to old CouchDB or
> failing to embrace FDB. FDB is a toolkit and does not, to my mind, force us
> down any particular path.
>
> I haven't sat down to modify couch_views in the manner I've suggested
> (where ebtree is used as designed; being fed the output of the emit() calls
> and calculating reductions as it does so) but I think it's a worthwhile
> exercise. I'd be surprised if performance of map-only traversals would be
> disappointing but who knows? I also expect it would allow for significant
> simplification of the code, which is one of the higher virtues.
>
> Adam, can you describe in a little more detail how you picture "b+tree is
> only used for incremental aggregations,"? It's implied in your reply that
> it would preserve the "interesting property" of keeping user data out of
> FDB Keys (for casual readers: the new native database encryption, called
> "aegis", only encrypts the FDB value. It can't encrypt the key as this
> would change the order of keys, which the current code depends on). Did I
> misread you?
>
> B.
>
> > On 23 Jul 2020, at 20:11, Adam Kocoloski <kocolosk@apache.org> wrote:
> >
> > OK thanks for the clarification. As I said I wasn’t all that confident I
> understood the design :)
> >
> > I like the idea that the b+tree is only used for incremental
> aggregations, rather than storing the entire materialized view, for the
> same reasons that Garren stated.
> >
> > An index maintained entirely in ebtree has the interesting property that
> it does not leak any user data into FDB Keys, which could be attractive for
> security reasons.
> >
> > Adam
> >
> >> On Jul 23, 2020, at 1:54 PM, Garren Smith <garren@apache.org> wrote:
> >>
> >> On Thu, Jul 23, 2020 at 6:55 PM Paul Davis <paul.joseph.davis@gmail.com
> >
> >> wrote:
> >>
> >>>> I would like to keep ebtree to use just for the reduce index.
> >>>
> >>> Could you expand on your reasoning here, Garren? I haven't done any
> >>> experiments on my own to understand if I'm missing something
> >>> important. My initial reaction is to not diverge too far from the
> >>> previous shape of the implementation since we have a decent idea of
> >>> how that behaves already but perhaps you've seen or measured something
> >>> I'm not thinking of?
> >>>
> >>
> >> I think this must have been a misunderstanding on my part. I always
> thought
> >> of using ebtree to solve reduce and wasn't planning to use it for the
> map
> >> index.
> >> I don't like the idea that we have ordered distributed key/value store
> and
> >> then implementing a b-tree on top of that for indexing. Especially
> since we
> >> know that the current map index is fast,
> >> easy to use, and follows recommended practices in the FDB community on
> how
> >> to do indexing. ebtree makes sense for reduce and is a means to an end
> to
> >> give us CouchDB's reduce api, which is heavily reliant on a b-tree, with
> >> CouchDB on FDB. This feels like a step backward and I worry we are tying
> >> ourselves heavily to old CouchDB instead of using the fact that we
> moving
> >> to FDB which then allows us to design new api's and functionality.
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message