beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Bingham (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-2438) Need option to run pipeline without stable unique names
Date Sun, 11 Jun 2017 15:33:18 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Bingham updated BEAM-2438:
-----------------------------------
    Description: 
SUMMARY
When a PTransform does not have a stable unique name, the python DirectRunner raises a RuntimeError
that prevents pipeline execution. In the Java SDK, this is a warning by default, which allows
the pipeline to run.

OBSERVED
I ran a pipeline with a globally unique name that's generated only at runtime:

my_unique_name = runtime_value
pvalue | my_unique_name >> transform

RuntimeError: Transform XYZ does not have a stable unique label. This will prevent updating
of pipelines. To apply a transform with a specified label write pvalue | "label" >>
transform

EXPECTED
Documented way to change from RuntimeError to warning, or a documented workaround.

NOTES
This is a blocker for my project, since I'm dynamically constructing a pipeline graph at runtime.
I'm porting from Java, where it works by default with just a warning message, to Python, where
it raises a runtime error and can't proceed.



  was:
SUMMARY
When a PTransform does not have a stable unique name, the python DirectRunner raises a RuntimeError
that prevents pipeline execution. In the Java SDK, this is a warning by default, which allows
the pipeline to run.

OBSERVED
RuntimeError: Transform XYZ does not have a stable unique label. This will prevent updating
of pipelines. To apply a transform with a specified label write pvalue | "label" >>
transform

EXPECTED
Documented way to change from RuntimeError to warning, or a documented workaround.

NOTES
This is a blocker for my project, since I'm dynamically constructing a pipeline graph at runtime.
I'm porting from Java, where it works by default with just a warning message, to Python, where
it raises a runtime error and can't proceed.




> Need option to run pipeline without stable unique names
> -------------------------------------------------------
>
>                 Key: BEAM-2438
>                 URL: https://issues.apache.org/jira/browse/BEAM-2438
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model-runner-api
>            Reporter: Jonathan Bingham
>            Assignee: Kenneth Knowles
>            Priority: Critical
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> SUMMARY
> When a PTransform does not have a stable unique name, the python DirectRunner raises
a RuntimeError that prevents pipeline execution. In the Java SDK, this is a warning by default,
which allows the pipeline to run.
> OBSERVED
> I ran a pipeline with a globally unique name that's generated only at runtime:
> my_unique_name = runtime_value
> pvalue | my_unique_name >> transform
> RuntimeError: Transform XYZ does not have a stable unique label. This will prevent updating
of pipelines. To apply a transform with a specified label write pvalue | "label" >>
transform
> EXPECTED
> Documented way to change from RuntimeError to warning, or a documented workaround.
> NOTES
> This is a blocker for my project, since I'm dynamically constructing a pipeline graph
at runtime. I'm porting from Java, where it works by default with just a warning message,
to Python, where it raises a runtime error and can't proceed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message