hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Lowe <jl...@yahoo-inc.com.INVALID>
Subject Re: zstd compression
Date Mon, 17 Jul 2017 20:30:29 GMT
I think we are OK to leave support for the zstd codec in the Hadoop code base.  I asked Chris
Mattman for clarification, noting that the support for the zstd codec requires the user to
install the zstd headers and libraries and then configure it to be included in the native
Hadoop build.  The Hadoop releases are not shipping any zstd code (e.g.: headers or libraries)
nor does it require zstd as a mandatory dependency.  Here's what he said:


On Monday, July 17, 2017 11:07 AM, Chris Mattmann <mattmann@apache.org> wrote:

> Hi Jason,
> 
> This sounds like an optional dependency on a Cat-X software. This isn’t the only type
of compression
> that is allowed within Hadoop, correct? If it is truly optional and you have gone to
that level of detail
> below to make the user opt in, and if we are not shipping zstd with our products (source
code releases),
> then this is an acceptable usage.
> 
> Cheers,
> Chris


So I think we are in the clear with respect to zstd usage as long as we keep it as an optional
codec where the user needs to get the headers and libraries for zstd and configure it into
the native Hadoop build.

Jason

On Monday, July 17, 2017 9:44 AM, Sean Busbey <busbey@cloudera.com> wrote:



I know that the HBase community is also looking at what to do about

our inclusion of zstd. We've had it in releases since late 2016. My

plan was to request that they relicense it.


Perhaps the Hadoop PMC could join HBase in the request?


On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer

<aw@effectivemachines.com> wrote:

>

>         It looks like HADOOP-13578 added Facebook's zstd compression codec.  Unfortunately,
that codec is using the same 3-clause BSD (LICENSE file) + patent grant license (PATENTS file)
that React is using and RocksDB was using.

>

>         Should that code get reverted?

>

>

>

> ---------------------------------------------------------------------

> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org

> For additional commands, e-mail: common-dev-help@hadoop.apache.org

>




-- 

busbey


---------------------------------------------------------------------

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org

For additional commands, e-mail: common-dev-help@hadoop.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message