beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mairbek Khadikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2837) Writing To Spanner From Google Cloud DataFlow - Failure
Date Mon, 18 Sep 2017 21:40:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170773#comment-16170773
] 

Mairbek Khadikov commented on BEAM-2837:
----------------------------------------

As a workaround, you could exclude _grpc-google-common-protos_ library that comes with beam
and add an explicit dependency 

{code:java}
        <dependency>
            <groupId>com.google.api.grpc</groupId>
            <artifactId>grpc-google-common-protos</artifactId>
            <version>0.1.9</version>
        </dependency>
        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
            <version>${beam.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>com.google.api.grpc</groupId>
                    <artifactId>grpc-google-common-protos</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
{code}


> Writing To Spanner From Google Cloud DataFlow - Failure
> -------------------------------------------------------
>
>                 Key: BEAM-2837
>                 URL: https://issues.apache.org/jira/browse/BEAM-2837
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>    Affects Versions: 2.1.0
>         Environment: Google Cloud DataFlow
>            Reporter: Al Yaros
>            Assignee: Mairbek Khadikov
>
> Simple Java Program That reads from Pub\Sub and Writes to Spanner Fails with cryptic
error message.
> Simple Program to Demonstrate the Error:
> [https://github.com/alyaros/ExamplePubSubToSpannerViaDataFlow]
> {code:java}
> *Caused by: org.apache.beam.sdk.util.UserCodeException: java.lang.NoClassDefFoundError:
Could not initialize class com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor
>         org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
>         org.apache.beam.sdk.io.
> gcp.spanner.SpannerWriteGroupFn$DoFnInvoker.invokeSetup(Unknown Source)
>         com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:66)
>         com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:48)
>         com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:104)
>         com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:66)
>         com.google.cloud.dataflow.worker.MapTaskExecutorFactory.createParDoOperation(MapTaskExecutorFactory.java:360)
>         com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:271)
>         com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:253)
>         com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:55)
>         com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:43)
>         com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:78)
>         com.google.cloud.dataflow.worker.MapTaskExecutorFactory.create(MapTaskExecutorFactory.java:142)
>         com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:925)
>         com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$800(StreamingDataflowWorker.java:133)
>         com.google.cloud.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:771)
>         java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         java.lang.Thread.run(Thread.java:745)*
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message