incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Alfke <j...@couchbase.com>
Subject Re: Refactoring a CouchDB.
Date Thu, 17 Jan 2013 18:48:19 GMT

On Jan 17, 2013, at 4:08 AM, Tim Hankins <timchankins@gmail.com> wrote:

> Two things have to be replicated from server to client.
>    1). Each user's subjective data for the past 14 days.
>    2). Each user's Impact Factor document for the current day.
...
> The problem is that this filter function takes too long. ( >10minutes)

Sounds like you might be bottlenecked on date parsing; this is surprisingly expensive. If
you’re storing dates as strings, the filter function to implement (1) is going to have to
parse each one and compare it against the current time. You might be able to speed up the
filter a lot by storing dates in the database as numeric timestamps, in which case the filter’s
test becomes a simple subtraction.

Or you could just skip the time-based filtering entirely. On the initial replication you'd
end up downloading all the user’s subjective data and Impact Factor documents, but thereafter
only new ones would be downloaded. This might be worthwhile if the surplus data isn’t too
large. (After the first sync your mobile app can purge — not delete — the older docs from
its local database to save room.)

—Jens
Mime
View raw message