flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LINZ, Arnaud" <AL...@bouyguestelecom.fr>
Subject RE: Building with Hadoop 3
Date Tue, 03 Mar 2020 17:02:55 GMT
Have you shared it somewhere on the web already?

De : vino yang <yanghua1127@gmail.com>
Envoyé : mercredi 4 décembre 2019 11:55
À : Márton Balassi <balassi.marton@gmail.com>
Cc : Chesnay Schepler <chesnay@apache.org>; Foster, Craig <foscraig@amazon.com>;
user@flink.apache.org; dev@flink.apache.org
Objet : Re: Building with Hadoop 3

Hi Marton,

Thanks for your explanation. Personally, I look forward to your contribution!


Márton Balassi <balassi.marton@gmail.com<mailto:balassi.marton@gmail.com>> 于2019年12月4日周三
Wearing my Cloudera hat I can tell you that we have done this exercise for our distros of
the  3.0 and 3.1 Hadoop versions. We have not contributed these back just yet, but we are
open to do so. If the community is interested we can contribute those changes back to flink-shaded
and suggest the necessay changes to flink too. The task was not overly complex, but it certainly
involved a bit of dependency hell. :-)

Right now we are focused on internal timelines, but we could invest into contributing this
back in the end of January timeframe if the community deems this a worthwhile effort.


On Wed, Dec 4, 2019 at 10:00 AM Chesnay Schepler <chesnay@apache.org<mailto:chesnay@apache.org>>
There's no JIRA and no one actively working on it. I'm not aware of any investigations on
the matter; hence the first step would be to just try it out.

A flink-shaded artifact isn't a hard requirement; Flink will work with any 2.X hadoop distribution
(provided that there aren't any dependency clashes).

On 03/12/2019 18:22, Foster, Craig wrote:
I don’t see a JIRA for Hadoop 3 support. I see a comment on a JIRA here from a year ago
that no one is looking into Hadoop 3 support [1]. Is there a document or JIRA that now exists
which would point to what needs to be done to support Hadoop 3? Right now builds with Hadoop
3 don’t work obviously because there’s no flink-shaded-hadoop-3 artifacts.


[1] https://issues.apache.org/jira/browse/FLINK-11086


L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice
ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation
ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message,
merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent
this message cannot therefore be held liable for its content nor attachments. Any unauthorized
use or dissemination is prohibited. If you are not the intended recipient of this message,
then please delete it and notify the sender.
View raw message