beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eduardo.morales@gmail.com <eduardo.mora...@gmail.com>
Subject Re: Go SDK: How are re-starts handled?
Date Fri, 22 Jun 2018 22:28:53 GMT


On 2018/06/22 21:35:29, Lukasz Cwik <lcwik@google.com> wrote: 
> There can only be one pipeline in Dataflow with the same job name so if you
> attempt to submit another job with the same job name you'll get back an
> identifier for the currently executing pipeline.

But beam.Run() only returns an error. How do I get the job name back?
My guess is that I have to use a different API (https://godoc.org/google.golang.org/api/dataflow/v1b3).
Is that the correct way to detect job name collisions?

Thanks again.

> On Fri, Jun 22, 2018 at 2:27 PM eduardo.morales@gmail.com <
> eduardo.morales@gmail.com> wrote:
> 
> > If I have a k8s process launching dataflow pipelines, what happens when
> > the process is restarted? Can Apache Beam detect a running pipeline and
> > join accordingly? or will the pipeline be duplicated?
> >
> > Thanks in advance.
> >
> 

Mime
View raw message