beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2270) Examples archetype bundles Hadoop 2.6 in its jar for ApexRunner; cannot run on Hadoop 2.7?
Date Thu, 11 May 2017 22:58:04 GMT
Kenneth Knowles created BEAM-2270:
-------------------------------------

             Summary: Examples archetype bundles Hadoop 2.6 in its jar for ApexRunner; cannot
run on Hadoop 2.7?
                 Key: BEAM-2270
                 URL: https://issues.apache.org/jira/browse/BEAM-2270
             Project: Beam
          Issue Type: Bug
          Components: examples-java, runner-apex, sdk-java-extensions
            Reporter: Kenneth Knowles
            Assignee: Thomas Weise


In an instantiated examples archetype, with {{-P apex-runner}}, Apex depends on Hadoop 2.6.0
and this is bundles into the examples jar.

In order to get this to run on Hadoop 2.7.3 I added this to the profile:

{code}
      <properties>
        <hadoop.version>2.7.3</hadoop.version>
      </properties>
 
      <dependencies>
        <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-yarn-client</artifactId>
          <version>${hadoop.version}</version>
        </dependency>
        <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-common</artifactId>
          <version>${hadoop.version}</version>
        </dependency>
      </dependencies>
{code}

It is not clear to me what the best path is, here. Clearly the way we bundle is brittle and
probably not the recommended best practice. But also perhaps the deps of the runner can be
modified to {{provided}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message