bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantinos Tsakalozos <>
Subject Re: Spark with storage other than HDFS
Date Mon, 16 May 2016 07:28:18 GMT
Hi everyone,

Thank you for the replies.

@Cos: I am working on a Juju charm that will deploy Apache Bigtop Spark. It
seems reasonable that such charm would deploy only Spark by default.
When/if you relate the Spark charm to HDFS (or any other supported storage
eg Cassandra, HBase etc) it would reconfigure Spark accordingly.

@Roman: Indeed these hard coded URLs in the Spark module is a good starting
point. Thank you for pointing them out.


On Sun, May 15, 2016 at 3:10 AM, Roman Shaposhnik <>

> On Thu, May 12, 2016 at 3:21 AM, Konstantinos Tsakalozos
> <> wrote:
> > Hi everyone,
> >
> > Is there a way to deploy spark using the puppet scripts with a storage
> layer
> > other than HDFS?
> IIRC, the biggest issue with our Puppet code for spark could be hard-coded
> URLs for HDFS. E.g.: hdfs://....
> It'd be a good exercise to make sure those are tweakable (with the default
> still
> being HDFS).
> Thanks,
> Roman.

Konstantinos Tsakalozos

View raw message