incubator-allura-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Brondsema" <>
Subject [allura:tickets] #4397 Run timeline aggregations in the background with taskd
Date Mon, 06 Jan 2014 21:33:53 GMT
Looking pretty good.  I thought `needs_aggregation` might be doing a lot of work just to return
a bool, so I turned up timermiddleware to DEBUG level to see what kind of queries were going
on.  I noticed a number of queries containing `'node_id': {'$in': []}`  We should short-circuit
that either at the storage layer or higher.  Full capture from one `create_timelines` task:

Back on `needs_aggregation`, the `since` param should keep the query relatively small, but
you might as well add `limit=1` too, so it can quit after the first is found.


** [tickets:#4397] Run timeline aggregations in the background with taskd**

**Status:** in-progress
**Labels:** v2 activitystreams 
**Created:** Fri Jun 15, 2012 05:59 PM UTC by Tim Van Steenburgh
**Last Updated:** Mon Jan 06, 2014 05:49 PM UTC
**Owner:** Tim Van Steenburgh

Once the volume ramps up, we probably won't want to be doing timeline aggregations on demand
if we can help it. Find good spots to fire off aggregations in the background using taskd,
so that when an activitystream page is requested, the cached timeline can just be pulled from
mongo w/o doing an aggregation.

For users, a good spot to do this might be on login. For projects, not sure...needs more thought.

Do we need to worry about two aggregations for the same node running at the same time?


Sent from because is subscribed to

To unsubscribe from further messages, a project admin can change settings at
 Or, if this is a mailing list, you can unsubscribe from the mailing list.
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message