spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Clarifying that spark-x.x.x-bin-hadoopx.x.tgz doesn't include Hadoop itself
Date Fri, 29 Jul 2016 20:09:52 GMT
Yeah, and the without hadoop was even more confusing... because if you
weren't using hdfs at all, you still needed to download one of the
hadoop-x packages in order to get hadoop io classes used by almost
everything.  :)

On Fri, Jul 29, 2016 at 3:06 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:
> Why do you say Hadoop is not included?
>
> The Hadoop jars are there in the tarball, and match the advertised
> version. There is (or at least there was in 1.x) a version called
> "without-hadoop" which did not include any Hadoop jars.
>
> On Fri, Jul 29, 2016 at 12:56 PM, Nicholas Chammas
> <nicholas.chammas@gmail.com> wrote:
>> I had an interaction on my project today that suggested some people may be
>> confused about what the packages available on the downloads page are
>> actually for.
>>
>> Specifically, the various -hadoopx.x.tgz packages suggest that Hadoop itself
>> is actually included in the package. I’m not 100% sure myself honestly, but
>> as I explained in my comment linked above, I believe the -hadoopx.x.tgz just
>> indicates the version of Hadoop that Spark was built against.
>>
>> Does it make sense to add a brief note to the downloads page explaining
>> this?
>>
>> I am assuming it would be too disruptive to change the package names to
>> something more descriptive like -built-against-hadoopx.x.tgz.
>>
>> Nick
>
>
>
> --
> Marcelo
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message