beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-1209) GcsUtil can try to execute empty batches in some cases.
Date Thu, 22 Dec 2016 17:48:59 GMT
Daniel Halperin created BEAM-1209:
-------------------------------------

             Summary: GcsUtil can try to execute empty batches in some cases.
                 Key: BEAM-1209
                 URL: https://issues.apache.org/jira/browse/BEAM-1209
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-gcp
            Reporter: Daniel Halperin
            Assignee: Pei He


I think, but have not confirmed, that this happens because we try to remove an empty set of
files.

{code}
SEVERE: 2016-12-22T16:59:17.063Z: (e72c8b756ce69f1a): java.io.IOException: Error executing
batch GCS request
        at org.apache.beam.sdk.util.GcsUtil.executeBatches(GcsUtil.java:486)
        at org.apache.beam.sdk.util.GcsUtil.remove(GcsUtil.java:535)
        at org.apache.beam.sdk.util.GcsIOChannelFactory.remove(GcsIOChannelFactory.java:108)
        at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.removeTemporaryFiles(FileBasedSink.java:468)
        at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.finalize(FileBasedSink.java:382)
        at org.apache.beam.sdk.io.Write$Bound$2.processElement(Write.java:417)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: no
JSON input found
        at org.apache.beam.sdk.repackaged.com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
        at org.apache.beam.sdk.repackaged.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
        at org.apache.beam.sdk.repackaged.com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:79)
        at org.apache.beam.sdk.util.GcsUtil.executeBatches(GcsUtil.java:481)
        at org.apache.beam.sdk.util.GcsUtil.remove(GcsUtil.java:535)
        at org.apache.beam.sdk.util.GcsIOChannelFactory.remove(GcsIOChannelFactory.java:108)
        at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.removeTemporaryFiles(FileBasedSink.java:468)
        at org.apache.beam.sdk.io.FileBasedSink$FileBasedWriteOperation.finalize(FileBasedSink.java:382)
        at org.apache.beam.sdk.io.Write$Bound$2.processElement(Write.java:417)
        at org.apache.beam.sdk.io.Write$Bound$2$auxiliary$3HTfRVcv.invokeProcessElement(Unknown
Source)
        at org.apache.beam.sdk.transforms.DoFnAdapters$SimpleDoFnAdapter.processElement(DoFnAdapters.java:234)
        at org.apache.beam.runners.core.SimpleOldDoFnRunner.invokeProcessElement(SimpleOldDoFnRunner.java:118)
        at org.apache.beam.runners.core.SimpleOldDoFnRunner.processElement(SimpleOldDoFnRunner.java:104)
        at com.google.cloud.dataflow.worker.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:192)
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message