apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhupesh Chawda <bhupeshcha...@gmail.com>
Subject Re: What-if analysis with apex
Date Mon, 25 Jan 2016 12:12:22 GMT
Hi Amit,

Thanks for the details.

The polling I was referring to was to identify the user changes. Since you
mention that these changes will be from a UI, I think you can directly send
these changes to an operator. Rather the operator can listen to the changes
that happen through the UI.

Also, by the rule set, I mean the set of rules (change to Table X,Row Y,
Col Z implies change to Table D, Row E, Col F). I assume this is large
enough not to fit in memory. If this is the case, then we can have it in
another store which can be queried by an operator.

Here is the approach using Apex:

O1 -> O2 -> O3

   - O1 listens to UI changes and propagates them in form of a triple
   (X,Y,Z) to O2. Here X -> Table, Y -> Row, Z -> Column.
   - O2 receives the triple from O1 and queries a data store to identify
   the set of target changes.
   - Assume that the flattened rule graph (X, Y, Z) -> [(D, E, F, C)] is
      stored in a data store which can be queried. (D, E, F) corresponds to the
      Table, Row and Column, while C refers to the recalculation formula.
      - Once O2 receives the changes that need to be done [(D, E, F)], it
   can forward them to O3 in form of triples (D1, E1, F1), (D2, E2, F2), etc.
   O3 then takes in these records and makes the required changes to the
   appropriate target tables.


On Mon, Jan 25, 2016 at 4:56 PM, Amit Shah <amits.84@gmail.com> wrote:

> Thanks Bhupesh for the reply.
> Answering your questions first
> what is the data store that you are using
> We have Oracle as our data store. With the consideration of a big data
> processing platform like apache apex, I think we would have to consider
> looking for some distributed in-memory data stores like memsql
> how is the user change captured
> User changes would be captured through an UI interface.
>  how large is the rule set which you are considering for your use case?
> We expect the user input to affect a bunch of tables (~ 10-20).
> Following your suggestion on DAG, I think I would have to expand the graph
> to store row level dependencies
> for.e.g. table A, row X, column index 5 depends on table B, row M, column
> index 10 & table C, row P, column index 7 along with the formula that is
> used to re-calculate the values. So when user changes the value of table B,
> row M, column index 10, the value for table A, row X, column index 5 would
> be re-calculated based on the new value.
> If I store this graph somewhere (?) I wouldn't have to poll the table for
> changes, right? Once the user changes a particular table, I can query the
> graph and determine the cell values that need to be re-calculated. What do
> you think?
> If this maps to the approach you mentioned, does apache apex help in
> calculations?
> Thanks,
> Amit.
> On Mon, Jan 25, 2016 at 4:36 PM, Bhupesh Chawda <bhupeshchawda@gmail.com>
> wrote:
>> Hi Amit,
>> From what I understand, you can do something like the following:
>> You can create a DAG like:
>> O1 -> O2
>> where O1 is an input operator while O2 is an output operator.
>> O1 listens for changes in values in a table / multiple tables. I assume
>> that the user change will result in change of some data in one of the
>> tables. Listening for a change may include monitoring a set of tables for
>> any change. This may be achieved by continuously / intermittently querying
>> the table for any change. In case this becomes very IO intensive, you can
>> use something like database triggers which update a meta data table with
>> the change that has happened.
>> Once the change is detected, O1 sends the change to O2 which according to
>> a set of rules defined in it, can update the corresponding target stores.
>> The set of rules that you define is analogous to the excel macros that are
>> defined.
>> It would help if you also mention what is the data store that you are
>> using and how is the user change captured. Also, how large is the rule set
>> which you are considering for your use case?
>> -Bhupesh
>> On Mon, Jan 25, 2016 at 4:10 PM, Amit Shah <amits.84@gmail.com> wrote:
>>> Hello,
>>> I am trying to evaluate apache apex for building an application that
>>> supports what-if analysis support to users. This co-relates closed with
>>> excel kind of functionality where changing a value in one cell triggers
>>> changes in other cell values. In our case we would have multiple rows in
>>> various tables getting updated when the user changes a row value. The
>>> response needs to be in real-time or near real-time.
>>> Does Apex fit such an use-case? If so, what would be some of initial
>>> steps to evaluate it for this use case?
>>> Thanks!
>> --
>> Regards,
>> Bhupesh Chawda

Bhupesh Chawda

View raw message