hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Json.org licensing, amazon-AWS and Jackson versions
Date Mon, 07 Nov 2016 18:14:18 GMT

https://issues.apache.org/jira/browse/HADOOP-13794: JSON.org<http://JSON.org> license
is now forbidden by the ASF From distribution.

Which means we can't make any Hadoop releases with the AWS SDK JARs < =1.11.0 in them,
meaning https://issues.apache.org/jira/browse/HADOOP-13050 has moved up from a minor issue
to a blocker, and are going to have to worry about the older branches.

1. The latest amazon-AWS SDKs absolutely do not work with shipping jackson version: it even
references artifacts that don't appear until  Jackson 2.3.3; and needs to on a later version
than that to actually work.
2. AWS SDK updates have generally needed code changes (example: HADOOP-12269)

For 2.8.x we can increment the AWS SDK, and take this as a time to increment jackson, which
an XEE vulnerability was hinting at anwyay ( https://issues.apache.org/jira/browse/HADOOP-12705)
. I know this has a risk of problems, but Sean Mackrory has done the due diliegence to show
that Jackson 2.7.8 doesn't break existing API use in Hadoop; after that jackson goes incompatible

For Branch 2.6.x we may just want to take the easy way out, and not bundle the (very dated)
AWS JAR; just strip it out of the final set of artifacts to include in the project dist, and
tell people that if they want to use s3a in 2.6.x (which I think people should really avoid,
given it to too 2.7.1 to stabilize), then they need to manually install it.

Which leaves Hadoop 2.7.x, doesn't it? What to do? People are using s3a, it's working well,
and putting the AWS JARs are going to cause problems. But pushing up a Jackson update in a
2.7.x update is going to be traumatic.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message