mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trevor Grant <>
Subject Re: Mahout and Spark 2.2 compatibility
Date Mon, 04 Dec 2017 17:25:22 GMT
Hi Marc,

Actually, it's not THAT hard to get Spark 2.2 compatibility for Mahout.

The code base is more or less compatible.  The issues were with respect to
Spark dependencies, specifically.

1. You have to build with Java 1.8
2. You have to have Hadoop 2.6+ (I think) support.

PR #335 outlines this pretty well.

This can be done from source, I'd have to double check the specific mvn
command line you would need.

As for supporting with a binary, maybe 0.13.2 (we're hoping to get spark
2.0/2.1 support in upcoming 0.13.1)

The hangups with releasing a Spark 2.2 binary is that we will have to
version bump java to 2.2 and hadoop to 2.6 or 2.7, and I'm not sure we were
ready to do that ATM.  Mainly bc it might alienate some users, and
secondarily Java 1.8 for a release is a real stickler on JavaDocs, and
there is a bit of work required to go back and clean them all up.

Let me know if building from source is an option at your company.  I'll
attempt to figure out how to build for SPark 2.2, and try to update the
website. I'll also post back here.


On Mon, Dec 4, 2017 at 11:18 AM, Marc Cardus Garcia <
> wrote:

> Hello all,
> First time I write into this mailing list, so if there is something wrong
> with my message please let me know.
> I work for a company using Mahout and Spark. We have recently started a
> project using Spark 2.2 and we would like to use Mahout but if I am not
> wrong, according to his issue MAHOUT-2000<https://issues.
>> there is still no compatibility
> between Spark 2.x and Mahout.  So regarding this issue I have to questions:
>   *   Is it planned to release this compatibility into the near future?
>   *
> If I want to add/help adding Spark 2.2.0 as supported binary release? How
> could I do that?
> Thank you,
> Marc.
> Marc Cardús Garcia
> Data Engineer | Data Science and Big Data Analytics     [Web Eurecat] <
>  +34 932 381 400   |  **  |<mailto:
>   Carrer Camí Antic de València 54-56, Edifici A - 08005 - Barcelona
> @Eurecat_news<>
> ________________________________
> DISCLAIMER: Aquest missatge pot contenir informació confidencial. Si vostè
> no n'és el destinatari, si us plau, esborri'l i faci'ns-ho saber
> immediatament a la següent adreça: Si el destinatari
> d'aquest missatge no consent la utilització del correu electrònic via
> Internet i la gravació de missatges, li preguem que ens ho comuniqui
> immediatament.
> DISCLAIMER: Este mensaje puede contener información confidencial. Si usted
> no es el destinatario del mensaje, por favor bórrelo y notifíquenoslo
> inmediatamente a la siguiente dirección: Si el
> destinatario de este mensaje no consintiera la utilización del correo
> electrónico vía Internet y la grabación de los mensajes, rogamos lo ponga
> en nuestro conocimiento de forma inmediata.
> DISCLAIMER: Privileged/Confidential Information may be contained in this
> message. If you are not the addressee indicated in this message you should
> destroy this message, and notify us immediately to the following address:
> If the addressee of this message does not consent to
> the use of Internet e-mail and message recording, please notify us
> immediately.
> ________________________________

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message