falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Seetharam Venkatesh <venkat...@innerzeal.com>
Subject Re: Architecture questions - Falcon's knowledge of feed instances
Date Thu, 23 Apr 2015 20:15:41 GMT
On Thu, Apr 23, 2015 at 12:42 PM, Scott Preece <preece@yahoo-inc.com.invalid
> wrote:

> Hi,
> Wanted to verify a few things and ask a couple of questions...
> Chasing through the code a bit, it looks like Falcon stores information
> about feed instances in a graph DB, using TinkerPop.

The source of truth for instances are in Oozie DB. Only succeeded instances
are stored in the graph db.

> From the configs, it looks like TinkerPop is on top of BerkeleyJE. Is that
> correct? Is it running as an in-memory database that gets loaded at startup
> or as a persistent store that is queried as information is needed?
The backing store for titan graph db is by default configured to be
berkeleydbje but is configurable to use HBase.

> Is information about instances stored anywhere else? That is, when you use
> the API to list he instances of a feed, does it get that information from
> the graph DB?
This is gotten from Oozie.

> What information does it store about an instance? scot, is the instance
> represented as an object with a pre-defined structure or is it a graph node
> to which properties are attached?

> Is any of this documented anywhere? Is this what the "Falcon config store
> / Graph" in the Architecture diagram refers to?
The config store is different from graph store. Config store stores the
user submitted entities. Graph store stores the lineage information and is

> thanks,scott


“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message