Mailing-List: contact dev-help@apex.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <D27B8BAD.35A1%cnarsude@cisco.com>
References: 
 <CAKaBFBkbFGxAsXjCNvZUKiNtN=ZyM8P-rM4fsCgBCTzsNwnO-Q@mail.gmail.com>
	<706A1512-6CE5-482D-86A3-70AB3FC5CB18@datatorrent.com>
	<CAKaBFBmZBNgOE5MePMLqSiVQQVoS03N_ak6P2EARpsrNQrX5ew@mail.gmail.com>
	<CAKJfLDMspdrUa5DOOY1k0-vcmftJE72qzLUVTxVRY55bS_NtOQ@mail.gmail.com>
	<5859565A-659C-4C9E-8514-F8E1EC2347E8@datatorrent.com>
	<CADS+zCuNfaAXe5WhTOGPFKFHAxvrqs_XXti5V-Ehjg1K+D1WpQ@mail.gmail.com>
	<CAKaBFB=YfpzaiZAoZ2yXvAwWHxKDzLyRy9kvkYsQFt6Ex=-vZw@mail.gmail.com>
	<CAKaBFB=iEUqwAcaa56XOJxPhFWfzocnd5M6oHrV4n0TzK89Xaw@mail.gmail.com>
	<3D1508BA-1749-4FFD-A503-133C5C2904B1@datatorrent.com>
	<09546DAE-DD51-4463-8B36-B54E8981FBD4@datatorrent.com>
	<CAA7LveFv0HhHZ8V5rt4jO+aE6WuDn1xokjB3y9+LwHpMriXr+Q@mail.gmail.com>
	<CAJ484BiK1mAQ0dvZ1QqHCXL9C0Ok7jsQ9DiC8BPonae_1L60fA@mail.gmail.com>
	<CAKaBFBnYosJQGsYYP0j-JnD95wr0U74_aFabk76k8xvfX1q9VA@mail.gmail.com>
	<D27897AE.33B9%cnarsude@cisco.com>
	<CAKaBFBn2GmfPGY_5o9ngSAyUMnUx8hnhpp2MhFFzB9y4pNbPkw@mail.gmail.com>
	<CAA7LveFYDf6au=KDnC9d9+w_9ajZ8YaWj98rtsJaVQh2yNdjLw@mail.gmail.com>
	<CADS+zCsBhTrHf9qbs7Zj7Txc1-PFzihoJxc0seQbr6uHzCjjvg@mail.gmail.com>
	<CAA7LveGfrBXGfGyo7WxM7M_x7jU8yxwm6J3oW_7wZW9AL5j4bQ@mail.gmail.com>
	<CAKJfLDOorYoE6yvenEyjkyqLjAdNbnRjBLeTBH=iNE_eBgBScA@mail.gmail.com>
	<D27B8644.3593%cnarsude@cisco.com>
	<CAKJfLDOyO-H2fi9LvCAEc3aUpTQDmv6ydd2weTpr_XC1HPEiXg@mail.gmail.com>
	<D27B8BAD.35A1%cnarsude@cisco.com>
Date: Fri, 27 Nov 2015 15:00:43 -0800
Message-ID: 
 <CAKJfLDMYp5i8eYtdKmSmdEThrkE-4t=ONpAfjxDobFD+9iSE0w@mail.gmail.com>
Subject: Re: Why is Async checkpointing made default?
From: Thomas Weise <thomas@datatorrent.com>
To: dev@apex.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a113db09842792005258da94c

--001a113db09842792005258da94c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Chetan,

Would like to understand how the checkpointed callback helps you with what
you indicated. This may require a specific example. Let's take it offline.

Thomas

On Wed, Nov 25, 2015 at 4:07 PM, Chetan Narsude (cnarsude) <
cnarsude@cisco.com> wrote:

> Yes - a few but cannot share the details - protected under NDA - ping me
> in private and I can probably be able to give you more generic details on
> similar cooked up examples.
>
>  The part that follows =E2=80=9Ce.g.=E2=80=9D below is an example that pr=
obably is
> sufficient to infer the use case logically, I think. I shared that to
> exemplify how changing the semantics will break semver.
>
> =E2=80=94
> Chetan
>
>
>
>
>
>
>
>
>
>
>
> On 11/25/15, 3:51 PM, "Thomas Weise" <thomas@datatorrent.com> wrote:
>
> >Do you have a specific example?
> >
> >I see this happening in committed(), but not in checkpointed() where the
> >checkpoint remains intermediate, whether it was copied to HDFS or not.
> >
> >
> >On Wed, Nov 25, 2015 at 3:42 PM, Chetan Narsude (cnarsude) <
> >cnarsude@cisco.com> wrote:
> >
> >> >
> >> >Until we have this, how about we restore the previous behavior
> >> >temporarily?
> >> >Calling checkpointed() immediately does not seem to pose any practica=
l
> >> >issue but ensures that the code that was written under this assumptio=
n
> >>is
> >> >not broken.
> >>
> >> We can=C2=B9t do it. It would be incorrect. It breaks all the other co=
de that
> >> (unassumingly) correctly complied to the semantics. e.g. an operator
> >>which
> >> informs interesting parties that the checkpointed data is available fo=
r
> >> immediate consumption from storage.
> >>
> >> =E2=80=B9
> >> Chetan
> >>
> >>
>
>

--001a113db09842792005258da94c--