spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Chen <tnac...@gmail.com>
Subject Re: HDFS as Shuffle Service
Date Tue, 26 Apr 2016 20:31:12 GMT
Are you suggesting to have shuffle service persist and fetch data with hdfs, or skip shuffle
service altogether and just write to hdfs?

Tim


> On Apr 26, 2016, at 11:20 AM, Michael Gummelt <mgummelt@mesosphere.io> wrote:
> 
> Has there been any thought or work on this (or any other networked file system)?  It
would be valuable to support dynamic allocation without depending on the shuffle service.
> 
> -- 
> Michael Gummelt
> Software Engineer
> Mesosphere

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message