incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Carlson <lancecarl...@gmail.com>
Subject Re: Mass updates
Date Fri, 10 May 2013 16:53:47 GMT
So, since I'm dealing with this problem now.. I open sourced a small script
that helps at least get couch to Redis and a brain dump of my ideas are
included in the README. Feedback appreciated!

https://github.com/lancecarlson/couchout.go


On Thu, May 9, 2013 at 4:32 PM, svilen <az@svilendobrev.com> wrote:

> yeah i know. i've been through it, needing temporaly-versioned
> code stored in a db, apart of the data for that time-period.. i know
> what it is. Near to impossible - still, it's the only correct one.
>
> > Ideally, there would be some robust extension to couchdb to handle
> > schema changes for large online databases in a sane way.
> btw i read that something like virtual views is going to be
> made.. one that would help multi-pass queries (which now are made just
> via artificial temporary dbs). Maybe these multi-version schemas are
> kind of that..
>
> On Thu, 09 May 2013 13:07:50 -0700
> Wendall Cada <wendallc@apache.org> wrote:
>
> > On 05/09/2013 05:52 AM, svilen wrote:
> > >>>>>> The main concern I have is maintaining a consistent state
> > >>>>>> across code releases.  Presumably, our data model will
change
> > >>>>>> over the course of time, and when it does, we need to make
the
> > >>>>>> several million old documents conform to the new model.
> > > a question: do u need to keep the old variant/state too?
> > > (think bi-temporal stuff)
> > >
> > > that one, and if the once-off conversion process is going to take
> > > loooong time, it might be better to bite the bullet and allow for
> > > multiple (e.g. 2) versions of the "schema" to coexist - in both
> > > server/views and client code. Thus once u already have some doc in
> > > the new variant, use it. Else, fallback to runtime-conversion.
> > >
> > > i know it isn't easy to organise but otherwise u're fighting
> > > reality.
> > >
> > > svilen
> >
> > This may sound ideal, but in my experience, this can lead to crazy,
> > buggy, eventually un-maintainable code. Additionally, this approach
> > ends up quickly in a situation where you're not supporting just two
> > schemas, but multiple others as well, as you can never guarantee that
> > the schema was updated across the entire database.
> >
> > This has been discussed at my workplace quite extensively recently.
> > Ideally, there would be some robust extension to couchdb to handle
> > schema changes for large online databases in a sane way.
> >
> > The real issue here isn't updating the schema in the db docs, it's
> > updating the schema and the requisite view indexes. If the schema
> > changes break your ddoc code, there isn't any live db solution
> > available that can fix this issue. This part of any database
> > maintenance is really, really hard work. It's what DBAs do for a
> > living, and is just difficult in different ways depending on the
> > database.
> >
> > Wendall
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message