hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <owen.omal...@gmail.com>
Subject Re: zstd compression
Date Tue, 25 Jul 2017 13:17:54 GMT
I'd support asking Facebook to change it with both my hadoop and orc hats on. 

.. Owen

> On Jul 24, 2017, at 23:43, Sean Busbey <busbey@cloudera.com> wrote:
> 
> Nope. Once I found out HBase's use was compliant as an optional runtime
> dependency I stopped looking.
> 
>> On Jul 24, 2017 7:22 PM, "Andrew Wang" <andrew.wang@cloudera.com> wrote:
>> 
>> I think it'd still be worth asking FB to relicense zstandard. Being able
>> to bundle it in the release would make it easier to use, since I doubt
>> there are zstandard packages in the default OS repos.
>> 
>> Sean, have you already filed an issue with zstandard?
>> 
>> On Mon, Jul 17, 2017 at 1:30 PM, Jason Lowe <jlowe@yahoo-inc.com.invalid>
>> wrote:
>> 
>>> I think we are OK to leave support for the zstd codec in the Hadoop code
>>> base.  I asked Chris Mattman for clarification, noting that the support for
>>> the zstd codec requires the user to install the zstd headers and libraries
>>> and then configure it to be included in the native Hadoop build.  The
>>> Hadoop releases are not shipping any zstd code (e.g.: headers or libraries)
>>> nor does it require zstd as a mandatory dependency.  Here's what he said:
>>> 
>>> 
>>> On Monday, July 17, 2017 11:07 AM, Chris Mattmann <mattmann@apache.org>
>>> wrote:
>>> 
>>>> Hi Jason,
>>>> 
>>>> This sounds like an optional dependency on a Cat-X software. This isn’t
>>> the only type of compression
>>>> that is allowed within Hadoop, correct? If it is truly optional and you
>>> have gone to that level of detail
>>>> below to make the user opt in, and if we are not shipping zstd with our
>>> products (source code releases),
>>>> then this is an acceptable usage.
>>>> 
>>>> Cheers,
>>>> Chris
>>> 
>>> 
>>> So I think we are in the clear with respect to zstd usage as long as we
>>> keep it as an optional codec where the user needs to get the headers and
>>> libraries for zstd and configure it into the native Hadoop build.
>>> 
>>> Jason
>>> 
>>> On Monday, July 17, 2017 9:44 AM, Sean Busbey <busbey@cloudera.com>
>>> wrote:
>>> 
>>> 
>>> 
>>> I know that the HBase community is also looking at what to do about
>>> 
>>> our inclusion of zstd. We've had it in releases since late 2016. My
>>> 
>>> plan was to request that they relicense it.
>>> 
>>> 
>>> Perhaps the Hadoop PMC could join HBase in the request?
>>> 
>>> 
>>> On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer
>>> 
>>> <aw@effectivemachines.com> wrote:
>>> 
>>>> 
>>> 
>>>>        It looks like HADOOP-13578 added Facebook's zstd compression
>>> codec.  Unfortunately, that codec is using the same 3-clause BSD (LICENSE
>>> file) + patent grant license (PATENTS file) that React is using and RocksDB
>>> was using.
>>> 
>>>> 
>>> 
>>>>        Should that code get reverted?
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> ---------------------------------------------------------------------
>>> 
>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> 
>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> busbey
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> 
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> 
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>> 
>>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message