beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Casters <mattcast...@gmail.com>
Subject [No Subject]
Date Thu, 29 Nov 2018 14:56:51 GMT
Hello Beam,

I've been taking great steps forward in having Kettle generate Beam
pipelines and they actually execute just find in unit testing in IntelliJ.
The problem starts when I collect all the libraries needed for Beam and the
Runners and throw them into the Kettle project as a plugin.

Caused by: java.lang.IllegalArgumentException: No filesystem found for
scheme gs
at
org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(FileSystems.java:456)
at org.apache.beam.sdk.io.FileSystems.matchNewResource(FileSystems.java:526)
at
org.apache.beam.sdk.io.FileBasedSink.convertToFileResourceIfPossible(FileBasedSink.java:213)
at org.apache.beam.sdk.io.TextIO$TypedWrite.to(TextIO.java:700)
at org.apache.beam.sdk.io.TextIO$Write.to(TextIO.java:1028)
at
org.kettle.beam.core.transform.BeamOutputTransform.expand(BeamOutputTransform.java:87)
... 32 more

This also happens for local file execution ("scheme file" in that case).

So the questions are: how is Beam bootstrapped? How does Beam determine
which libraries to use and what is the recommended way for packaging things
up properly?
The Beam plugin is running in a separate URLClassloader so I think
something is going awry there.

Thanks a lot for any answers or tips you might have!

Matt
---
Matt Casters <m <mcasters@pentaho.org>attcasters@gmail.com>
Senior Solution Architect, Kettle Project Founder

Mime
View raw message