Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAHM4Ztnh9i+W=orNNKU_vrq_3V3WP2Bi1o+bRyJcacyQ2VRsUA@mail.gmail.com>
References: <CAK2z7j7SttXV=jXsfcGfw5ftUrCzPuoedYJ-uD2_0nYYUTQB0A@mail.gmail.com>
 <CAAjCYUX8YwR5Ts7YmxkJgnQqzZP7p+YZTDhsfgYPEpFoF0Sbtg@mail.gmail.com> <CAHM4Ztnh9i+W=orNNKU_vrq_3V3WP2Bi1o+bRyJcacyQ2VRsUA@mail.gmail.com>
From: Vishnu Viswanath <vishnu.viswanath25@gmail.com>
Date: Mon, 6 Mar 2017 11:38:20 -0500
Message-ID: <CAK2z7j7MuW=Vhe4ed-rve4NzubKRmdEMxnok_qvG50duQpamUQ@mail.gmail.com>
Subject: Re: Incremental checkpoint branch
To: Dev <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c07b874286ad2054a1287aa
archived-at: Mon, 06 Mar 2017 17:46:50 -0000

--94eb2c07b874286ad2054a1287aa
Content-Type: text/plain; charset=UTF-8

Thanks everyone for the reply.

@Xiaogang design document looks very good, looking forward to check the
implementation and to try it out. I see that hdfs:// is mentioned as the
backend storage in the documentation, what about s3://. Is it something
that needed to be considered in the design, or is it handled by RocksDB.

Thanks,
Vishnu

On Sat, Mar 4, 2017 at 2:58 AM, SHI Xiaogang <shixiaogangg@gmail.com> wrote:

> Hi Vinshnu,
>
> We have obtained an initial design of incremental checkpointing [1] and
> will start working on incremental checkpointing the next week. You can
> watch the issue  FLINK-5053 [2] to get timely notification of the updates.
> All suggestions are welcome.
>
> [1]
> https://docs.google.com/document/d/1VvvPp09gGdVb9D2wHx6NX99yQK0jS
> UMWHQPrQ_mn520/edit#heading=h.2gshh42txc4z
> [2] https://issues.apache.org/jira/browse/FLINK-5053
>
> Regards,
> Xiaogang
>
>
>
> 2017-03-04 15:43 GMT+08:00 Shaoxuan Wang <wshaoxuan@gmail.com>:
>
> > Vinshnu,
> > You can find the latest design discussion for incremental checkpoint in
> > http://apache-flink-mailing-list-archive.1008284.n3.
> > nabble.com/DISCUSS-Support-Incremental-Checkpointing-in-
> Flink-td15931.html
> >  @Stefan Richter
> > <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=srichter>
> > and @Xiaogang
> > Shi
> > <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> name=xiaogang.shi>
> > might
> > be the right person to provide you its on-going status.
> >
> > cheers,
> > Shaoxuan
> >
> > On Sat, Mar 4, 2017 at 5:14 AM, Vishnu Viswanath <
> > vishnu.viswanath25@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Can someone point me to the branch where the ongoing work for
> incremental
> > > checkpoint is going on, I would like to try it out even if the work is
> > not
> > > complete.
> > >
> > > I have a use case where the state size increase about ~1gb every 5
> > minutes.
> > >
> > > Thanks,
> > > Vishnu
> > >
> >
>

--94eb2c07b874286ad2054a1287aa--