beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davor Bonaci <da...@google.com>
Subject Re: expected a valid 'gs://' path but was given '/tmp/tmpLocation'
Date Tue, 24 May 2016 11:52:12 GMT
Yes -- MinimalWordCount example currently defaults to the
DataflowPipelineRunner, which runs pipelines on the Google Cloud Dataflow
service. (We'll be changing this.) In general, Cloud-based runners don't
have access to your local machine, hence the exception you saw.

DirectPipelineRunner can execute pipelines locally, mainly for testing
purposes.

On Tue, May 24, 2016 at 3:48 AM, Robertson Williams <rwilliams.gr@gmail.com>
wrote:

> Just find out what goes wrong. Changing to use
>
>   org.apache.beam.sdk.options.DirectPipelineOptions
>   org.apache.beam.sdk.runners.DirectPipelineRunner
>
> fixing the problem.
>
> Thanks
>
>
> On Tue, May 24, 2016 at 6:24 PM, Robertson Williams <
> rwilliams.gr@gmail.com> wrote:
>
>> I try with the latest version 0.1.0-SNAPSHOT cloned from git, but when
>> testing with MinimalWordCount, it throws
>>
>>     expected a valid 'gs://' path but was given '/tmp/tmpLocation'
>>
>> Can I run MinimalWordCount example locally (by supplying tmp location at
>> local file system e.g. file://) or is it bound to gs only? The source[1][2]
>> seems to me it reads from gs only, but I maybe missing something I am not
>> aware.
>>
>> Which part can I change so MinimalWordCount can execute without such
>> error?
>>
>> Thanks
>>
>> [1]
>> https://github.com/apache/incubator-beam/blob/e3105c8e109535f801fd145b91b0c7aa93b86d1a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/DataflowPathValidator.java
>>
>> [2]
>> https://github.com/apache/incubator-beam/blob/96765f19b1bd8149240cd77eb7cf7fb636e477e4/sdks/java/core/src/main/java/org/apache/beam/sdk/util/gcsfs/GcsPath.java
>>
>
>

Mime
View raw message