mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Malouf <malouf.g...@gmail.com>
Subject Re: Mesos + Hadoop 2.2 for protobuf 2.5 dependency
Date Wed, 26 Mar 2014 00:41:43 GMT
We have a file crush job that requires MRV1, otherwise we would be free
from it altogether.
On Mar 25, 2014 8:28 PM, "Jastrzebski, Lukasz (NSN - US/Mountain View)" <
lukasz.jastrzebski@nsn.com> wrote:

>  Hi Gary,
>
>  I'm testing right now Mesos 0.17 CDH5 beta2 and Spark 0.9.1,
>
>  So far what I see it is sufficient to run only HDFS, there is not need
> to run or install yarn or map reduce.
>
>  However if u are using MR1 or MR2 then u need to do that, I think
> running MR2 on mesos is in progress, please correct me if I am wrong.
>  ---
> Best regards
> Lukasz Jastrzebski
>
>
>
>
>  On Mar 25, 2014, at 5:14 PM, ext Gary Malouf <malouf.gary@gmail.com>
> wrote:
>
>  For various reasons, our team needs to keep all of our projects on the
> same protobuf version.  We've now hit a point where we need to upgrade
> protobuf from 2.4.1 to 2.5.0 across the board in our projects and dependent
> platforms.
>
>  *Current stack*: Mesos 0.15, Chronos, CDH 4.2.1-MRV1, Spark
> 0.9-pre-scala-2.10 build off master
>
>  *Ideal stack after upgrade*: Mesos 0.17, Chronos, CDH5 beta2, Spark
> 0.9.1 (hadoop 2.2 build)
>
>  From what we understand, we need a dependency on Hadoop 2.2 to get the
> necessary protobuf upgrade.  From reading Cloudera's documentation and
> multiple google searches, it is not clear to me how we can construct the
> stack to continue to work.
>
>  Has anyone else requested info on getting this combination to work?
>  From others we've spoken to, they've basically said that we'll be forced
> to use Yarn for Hadoop support in the very near future anyway so we should
> switch.  Since we colocate Spark with out HDFS nodes, it's hard to see how
> we would run both Yarn and Mesos on the same servers.
>
>
>

Mime
View raw message