nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Herrera <chris.herrer...@gmail.com>
Subject Re: [DISCUSS] Apache NiFi distribution has grown too large
Date Fri, 12 Jan 2018 21:33:47 GMT
I very much like the solution proposed by Bryan below. This would allow for a cleaner docker
image as well, while still proving the functionality as needed. For sure, the extension registry
will be great, but in the mean time this is an adequate mid step.

Regards,
Chris

On Jan 12, 2018, 2:52 PM -0600, Bryan Bende <bbende@gmail.com>, wrote:
> Long term I'd like to see the extension registry take form and have
> that be the solution (#3).
>
> In the more near term, we could separate all of the NARs, except for
> framework and maybe standard processors & services, into a separate
> git repo.
>
> In that new git repo we could organize things like Joe N just
> described according to some kind of functional grouping. Each of these
> functional bundles could produce its own tar/zip which we can make
> available for download.
>
> That would separate the release cycles between core NiFi and the other
> NARs, and also avoid having any single binary artifact that gets too
> large.
>
>
>
> On Fri, Jan 12, 2018 at 3:43 PM, Joseph Niemiec <josephxsxn@gmail.com> wrote:
> > just a random thought.
> >
> > Drop In Lib packs... All the Hadoop ones in one package for example that
> > can be added to a slim Nifi install. Another may be for Cloud, or Database
> > Interactions, Integration (JMS, FTP, etc) of course defining these groups
> > would be the tricky part... Or perhaps some type of installer which allows
> > you to elect which packages to download to add to the slim install?
> >
> >
> > On Fri, Jan 12, 2018 at 3:10 PM, Joe Witt <joe.witt@gmail.com> wrote:
> >
> > > Team,
> > >
> > > The NiFi convenience binary (tar.gz/zip) size has grown to 1.1GB now
> > > in the latest release. Apache infra expanded it to 1.6GB allowance
> > > for us but has stated this is the last time.
> > > https://issues.apache.org/jira/browse/INFRA-15816
> > >
> > > We need consider:
> > > 1) removing old nars/less commonly used nars/or particularly massive
> > > nars from the assembly we distribute by default. Folks can still use
> > > these things if they want just not from our convenience binary
> > > 2) collapsing nars with highly repeating deps
> > > 3) Getting the extension registry baked into the Flow Registry then
> > > moving to separate releases for extension bundles. The main release
> > > then would be just the NiFi framework.
> > >
> > > Any other ideas ?
> > >
> > > I'll plan to start identifying candiates for removal soon.
> > >
> > > Thanks
> > > Joe
> > >
> >
> >
> >
> > --
> > Joseph

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message