beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akanksha Sharma B <>
Subject writing to maprfs?
Date Thu, 24 May 2018 07:24:25 GMT
Hi All,

I have a pipeline; it writes to hdfs, using "" package.

I was hoping that it will work with maprfs as well. However I have been debugging for some
days, with no success.

I do not provide hdfsConfiguration from command line, and instead use the configuration file
/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml (please find it attached with the mail)

However I get the stack trace ("No FileSystem for scheme: maprfs"). Please find the detailed
trace attached. I made sure that I have /opt/mapr/lib/maprfs-5.2.2-mapr.jar on classpath.
I use spark submit to run the pipeline.

If you have already tried writing to maprfs from beam, ccould you please help?



View raw message