bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantinos Tsakalozos <kos.tsakalo...@canonical.com>
Subject Re: Spark with storage other than HDFS
Date Mon, 16 May 2016 07:28:18 GMT
Hi everyone,

Thank you for the replies.

@Cos: I am working on a Juju charm that will deploy Apache Bigtop Spark. It
seems reasonable that such charm would deploy only Spark by default.
When/if you relate the Spark charm to HDFS (or any other supported storage
eg Cassandra, HBase etc) it would reconfigure Spark accordingly.

@Roman: Indeed these hard coded URLs in the Spark module is a good starting
point. Thank you for pointing them out.

Thanks,
Konstantinos

On Sun, May 15, 2016 at 3:10 AM, Roman Shaposhnik <roman@shaposhnik.org>
wrote:

> On Thu, May 12, 2016 at 3:21 AM, Konstantinos Tsakalozos
> <kos.tsakalozos@canonical.com> wrote:
> > Hi everyone,
> >
> > Is there a way to deploy spark using the puppet scripts with a storage
> layer
> > other than HDFS?
>
> IIRC, the biggest issue with our Puppet code for spark could be hard-coded
> URLs for HDFS. E.g.: hdfs://....
>
> It'd be a good exercise to make sure those are tweakable (with the default
> still
> being HDFS).
>
> Thanks,
> Roman.
>



-- 
Konstantinos Tsakalozos

Mime
View raw message