beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
Date Tue, 25 Sep 2018 16:24:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-5288?focusedWorklogId=147642&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147642
]

ASF GitHub Bot logged work on BEAM-5288:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 25/Sep/18 16:23
            Start Date: 25/Sep/18 16:23
    Worklog Time Spent: 10m 
      Work Description: mxm commented on a change in pull request #6441: [BEAM-5288] Support
environment pipeline option in Java and Python.
URL: https://github.com/apache/beam/pull/6441#discussion_r220256754
 
 

 ##########
 File path: runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/LazyJobBundleFactory.java
 ##########
 @@ -53,92 +55,81 @@
 import org.apache.beam.sdk.fn.IdGenerator;
 import org.apache.beam.sdk.fn.IdGenerators;
 import org.apache.beam.sdk.fn.data.FnDataReceiver;
+import org.apache.beam.sdk.fn.function.ThrowingFunction;
 import org.apache.beam.sdk.fn.stream.OutboundObserverFactory;
 import org.apache.beam.sdk.util.WindowedValue;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 /**
- * A base for a {@link JobBundleFactory} for which the implementation can specify a custom
{@link
+ * A {@link JobBundleFactory} for which the implementation can specify a custom {@link
  * EnvironmentFactory} for environment management. Note that returned {@link StageBundleFactory
  * stage bundle factories} are not thread-safe. Instead, a new stage factory should be created
for
- * each client.
+ * each client. {@link LazyJobBundleFactory} initializes the Environment lazily when the
forStage is
+ * called for a stage. This factory is not capable of handling a mixed types of environment.
  */
 @ThreadSafe
-public abstract class JobBundleFactoryBase implements JobBundleFactory {
-  private static final Logger LOG = LoggerFactory.getLogger(JobBundleFactoryBase.class);
+public class LazyJobBundleFactory implements JobBundleFactory {
 
 Review comment:
   Would also keep the old name `JobBundleFactory` because it is the only one. The lazy attribute
is now the new default. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 147642)

> Modify Environment to support non-dockerized SDK harness deployments 
> ---------------------------------------------------------------------
>
>                 Key: BEAM-5288
>                 URL: https://issues.apache.org/jira/browse/BEAM-5288
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Maximilian Michels
>            Assignee: Ankur Goenka
>            Priority: Major
>          Time Spent: 11h
>  Remaining Estimate: 0h
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to extend the
Environment information. In addition to the Docker environment, the extended environment holds
deployment options for 1) a process-based environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
>  message Environment {
>    // (Required) The URN of the payload
>    string urn = 1;
>    // (Optional) The data specifying any parameters to the URN. If
>    // the URN does not require any arguments, this may be omitted.
>    bytes payload = 2;
>  }
>  message StandardEnvironments {
>    enum Environments {
>      DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
>      PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
>      EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>    }
>  }
>  // The payload of a Docker image
>  message DockerPayload {
>    string container_image = 1;  // implicitly linux_amd64.
>  }
>  message ProcessPayload {
>    string os = 1;  // "linux", "darwin", ..
>    string arch = 2;  // "amd64", ..
>    string command = 3; // process to execute
>    map<string, string> env = 4; // environment variables
>  }
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message