Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAKJfLDMYp5i8eYtdKmSmdEThrkE-4t=ONpAfjxDobFD+9iSE0w@mail.gmail.com>
References: 
 <CAKaBFBkbFGxAsXjCNvZUKiNtN=ZyM8P-rM4fsCgBCTzsNwnO-Q@mail.gmail.com>
	<706A1512-6CE5-482D-86A3-70AB3FC5CB18@datatorrent.com>
	<CAKaBFBmZBNgOE5MePMLqSiVQQVoS03N_ak6P2EARpsrNQrX5ew@mail.gmail.com>
	<CAKJfLDMspdrUa5DOOY1k0-vcmftJE72qzLUVTxVRY55bS_NtOQ@mail.gmail.com>
	<5859565A-659C-4C9E-8514-F8E1EC2347E8@datatorrent.com>
	<CADS+zCuNfaAXe5WhTOGPFKFHAxvrqs_XXti5V-Ehjg1K+D1WpQ@mail.gmail.com>
	<CAKaBFB=YfpzaiZAoZ2yXvAwWHxKDzLyRy9kvkYsQFt6Ex=-vZw@mail.gmail.com>
	<CAKaBFB=iEUqwAcaa56XOJxPhFWfzocnd5M6oHrV4n0TzK89Xaw@mail.gmail.com>
	<3D1508BA-1749-4FFD-A503-133C5C2904B1@datatorrent.com>
	<09546DAE-DD51-4463-8B36-B54E8981FBD4@datatorrent.com>
	<CAA7LveFv0HhHZ8V5rt4jO+aE6WuDn1xokjB3y9+LwHpMriXr+Q@mail.gmail.com>
	<CAJ484BiK1mAQ0dvZ1QqHCXL9C0Ok7jsQ9DiC8BPonae_1L60fA@mail.gmail.com>
	<CAKaBFBnYosJQGsYYP0j-JnD95wr0U74_aFabk76k8xvfX1q9VA@mail.gmail.com>
	<D27897AE.33B9%cnarsude@cisco.com>
	<CAKaBFBn2GmfPGY_5o9ngSAyUMnUx8hnhpp2MhFFzB9y4pNbPkw@mail.gmail.com>
	<CAA7LveFYDf6au=KDnC9d9+w_9ajZ8YaWj98rtsJaVQh2yNdjLw@mail.gmail.com>
	<CADS+zCsBhTrHf9qbs7Zj7Txc1-PFzihoJxc0seQbr6uHzCjjvg@mail.gmail.com>
	<CAA7LveGfrBXGfGyo7WxM7M_x7jU8yxwm6J3oW_7wZW9AL5j4bQ@mail.gmail.com>
	<CAKJfLDOorYoE6yvenEyjkyqLjAdNbnRjBLeTBH=iNE_eBgBScA@mail.gmail.com>
	<D27B8644.3593%cnarsude@cisco.com>
	<CAKJfLDOyO-H2fi9LvCAEc3aUpTQDmv6ydd2weTpr_XC1HPEiXg@mail.gmail.com>
	<D27B8BAD.35A1%cnarsude@cisco.com>
	<CAKJfLDMYp5i8eYtdKmSmdEThrkE-4t=ONpAfjxDobFD+9iSE0w@mail.gmail.com>
Date: Fri, 11 Dec 2015 09:36:29 -0800
Message-ID: 
 <CAKaBFBnXm9BzR4WNs1VuNgcF6YGk_R4O4Ybdkbw_vHh-RP5N-Q@mail.gmail.com>
Subject: Re: Why is Async checkpointing made default?
From: Chandni Singh <chandni@datatorrent.com>
To: dev@apex.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a114701ea75d1740526a2c3c8

--001a114701ea75d1740526a2c3c8
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,

There hasn't been a close bracket here (borrowing Ram's expression :-) ).
>From what I see the majority agrees with making the fix.

Do we need to start a vote for this?

On Fri, Nov 27, 2015 at 3:00 PM, Thomas Weise <thomas@datatorrent.com>
wrote:

> Chetan,
>
> Would like to understand how the checkpointed callback helps you with wha=
t
> you indicated. This may require a specific example. Let's take it offline=
.
>
> Thomas
>
> On Wed, Nov 25, 2015 at 4:07 PM, Chetan Narsude (cnarsude) <
> cnarsude@cisco.com> wrote:
>
> > Yes - a few but cannot share the details - protected under NDA - ping m=
e
> > in private and I can probably be able to give you more generic details =
on
> > similar cooked up examples.
> >
> >  The part that follows =E2=80=9Ce.g.=E2=80=9D below is an example that =
probably is
> > sufficient to infer the use case logically, I think. I shared that to
> > exemplify how changing the semantics will break semver.
> >
> > =E2=80=94
> > Chetan
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On 11/25/15, 3:51 PM, "Thomas Weise" <thomas@datatorrent.com> wrote:
> >
> > >Do you have a specific example?
> > >
> > >I see this happening in committed(), but not in checkpointed() where t=
he
> > >checkpoint remains intermediate, whether it was copied to HDFS or not.
> > >
> > >
> > >On Wed, Nov 25, 2015 at 3:42 PM, Chetan Narsude (cnarsude) <
> > >cnarsude@cisco.com> wrote:
> > >
> > >> >
> > >> >Until we have this, how about we restore the previous behavior
> > >> >temporarily?
> > >> >Calling checkpointed() immediately does not seem to pose any
> practical
> > >> >issue but ensures that the code that was written under this
> assumption
> > >>is
> > >> >not broken.
> > >>
> > >> We can=C2=B9t do it. It would be incorrect. It breaks all the other =
code
> that
> > >> (unassumingly) correctly complied to the semantics. e.g. an operator
> > >>which
> > >> informs interesting parties that the checkpointed data is available
> for
> > >> immediate consumption from storage.
> > >>
> > >> =E2=80=B9
> > >> Chetan
> > >>
> > >>
> >
> >
>

--001a114701ea75d1740526a2c3c8--