apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandni Singh <chan...@datatorrent.com>
Subject Re: Renaming Idempotent Storage Manager
Date Tue, 17 Nov 2015 06:22:34 GMT
---
I like 1 and 3 but prefer 3 because we may not always be logging data but
saving meta-data.
--

We are saving data or meta-data somewhere for recovery. WindowDataManager
doesn't reflect that data or meta-data is persisted.

Chandni

On Mon, Nov 16, 2015 at 10:12 PM, Yogi Devendra <yogidevendra@apache.org>
wrote:

> +1 for WindowDataManager
>
> On 17 November 2015 at 10:58, Sandeep Deshmukh <sandeep@datatorrent.com>
> wrote:
>
> > +1 for 3.
> > On 17 Nov 2015 07:25, "Timothy Farkas" <tim@datatorrent.com> wrote:
> >
> > > +1 for 1
> > >
> > > On Mon, Nov 16, 2015 at 2:35 PM, Pramod Immaneni <
> pramod@datatorrent.com
> > >
> > > wrote:
> > >
> > > > I like 1 and 3 but prefer 3 because we may not always be logging data
> > but
> > > > saving meta-data.
> > > >
> > > > On Mon, Nov 16, 2015 at 2:30 PM, Chandni Singh <
> > chandni@datatorrent.com>
> > > > wrote:
> > > >
> > > > > Currently going by Java code, the IdempotentStorageManager extends
> > > > > StorageAgent which is why WindowStorageAgent came up.
> > > > > It does delete data till window that gets committed.
> > > > >
> > > > >
> > > > > These are some options which I have came across.
> > > > >
> > > > > 1. WindowDataLogger
> > > > > 2. WindowStorageAgent
> > > > > 3. WindowDataManager
> > > > >
> > > > > My vote is for option 1 "WindowDataLogger".
> > > > >
> > > > > Thanks,
> > > > > Chandni
> > > > >
> > > > > On Mon, Nov 16, 2015 at 2:12 PM, Pramod Immaneni <
> > > pramod@datatorrent.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > It would still to manage the log isn't it, like clearing out
data
> > > > > > corresponding to committed windows.
> > > > > >
> > > > > > On Mon, Nov 16, 2015 at 2:11 PM, Thomas Weise <
> > > thomas@datatorrent.com>
> > > > > > wrote:
> > > > > >
> > > > > > > The underlying abstraction is a log. Why shouldn't the
name
> > reflect
> > > > > that?
> > > > > > > Or is there anything added that makes it a "manager" or
> "agent"?
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Nov 16, 2015 at 2:01 PM, Chandni Singh <
> > > > > chandni@datatorrent.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > WindowStorageAgent was another suggestion as
> > > > IdempotentStorageManager
> > > > > > > > extends StorageAgent.
> > > > > > > > If no one has any objection will call it that.
> > > > > > > >
> > > > > > > > Chandni
> > > > > > > >
> > > > > > > > On Sun, Nov 15, 2015 at 8:07 PM, Chandni Singh <
> > > > > > chandni@datatorrent.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > @Sandeep Will deprecate old IdempotentStorageManager.
This
> > will
> > > > be
> > > > > a
> > > > > > > copy
> > > > > > > > > of the old one with a different name.
> > > > > > > > >
> > > > > > > > > @Sandesh
> > > > > > > > > A data store, in my opinion, is an entity which
store data.
> > > This
> > > > > > > doesn't
> > > > > > > > > store data. It is used to save state per window.
> > > > > > > > >
> > > > > > > > > @Pramod
> > > > > > > > > Do you like WindowStateLogger?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Chandni
> > > > > > > > >
> > > > > > > > > On Nov 15, 2015 6:59 PM, "Sandeep Deshmukh" <
> > > > > sandeep@datatorrent.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Will there be any issue in semantic versioning
because of
> > this
> > > > > > change?
> > > > > > > > >> On 16 Nov 2015 07:35, "Sandesh Hegde" <
> > > sandesh@datatorrent.com>
> > > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >> > As it is used to store & retrieve
data, just call it
> > > > DataStore.
> > > > > > > > >> >
> > > > > > > > >> > On Sun, Nov 15, 2015 at 3:40 PM Pramod
Immaneni <
> > > > > > > > pramod@datatorrent.com
> > > > > > > > >> >
> > > > > > > > >> > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Since this is mainly for storing
information
> pertaining
> > to
> > > > > data
> > > > > > or
> > > > > > > > the
> > > > > > > > >> > data
> > > > > > > > >> > > itself how about WindowDataManager
or something like
> > that.
> > > > > > > > >> > >
> > > > > > > > >> > > On Sun, Nov 15, 2015 at 3:16 PM,
Chandni Singh <
> > > > > > > > >> chandni@datatorrent.com>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > Hi,
> > > > > > > > >> > > >
> > > > > > > > >> > > > Received feedback from couple
of folks that
> > > > > > > > >> IdempotentStorageManager is
> > > > > > > > >> > > not
> > > > > > > > >> > > > a good name for that component.
It is used in some
> of
> > > our
> > > > > > > > >> input/output
> > > > > > > > >> > > > operator for saving per window
data.
> > > > > > > > >> > > >
> > > > > > > > >> > > > I was thinking of calling
it WindowLogger.
> > > > > > > > >> > > >
> > > > > > > > >> > > > Let me know if anyone has
a better suggestion.
> > > > > > > > >> > > >
> > > > > > > > >> > > > Chandni
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message