nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Psaltis <psaltis.and...@gmail.com>
Subject Re: Pulling out hair over slf4j...
Date Tue, 09 May 2017 17:42:25 GMT
Russell,
I was going to echo the same thing Jeff mentioned. I built a little test
app that uses the mave-share plugin and also exclude slf4j in the same way
suggested earlier. This worked as expected, properly excluding the slf4j
jar from the tika jar when shading my test app.

On Tue, May 9, 2017 at 1:20 PM, Jeff <jtswork@gmail.com> wrote:

> Tika, in the 1.0 tag, has slf4j-log4j12 [1] as a provided dependency,
> version 1.5.6 (which I see gets brought in to your maven repo).  Later
> versions of Tika are using maven-shade-plugin, but it doesn't look like 1.0
> does.
>
> Have you tried excluding that dependency from tika-app?
>
> Oddly, tika-bundle uses a different version of slf4j-simple [2] for tests
> (1.6.1).
>
> [1] https://github.com/apache/tika/blob/1.0/tika-app/pom.xml#L53
> [2] https://github.com/apache/tika/blob/1.0/tika-bundle/pom.xml#L99
>
> On Tue, May 9, 2017 at 10:03 AM Russell Bateman <russ@windofkeltia.com>
> wrote:
>
> > Thanks, Andrew.I haven't tried that before (a new diagnostic tool in my
> > quiver).
> >
> > I did it, scraped the IntelliJ IDEA test console, then split lines on
> > colons and grep'd for /slf4j/ to get this:
> >
> > *master ~/notes $ fgrep slf4j debug-console.log *
> > :/home/russ/.m2/repository/org/slf4j/slf4j-api/1.7.25/
> slf4j-api-1.7.25.jar
> >
> > :/home/russ/.m2/repository/org/slf4j/slf4j-simple/1.7.25/
> slf4j-simple-1.7.25.jar
> >
> > :/home/russ/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.
> 25/jcl-over-slf4j-1.7.25.jar
> >
> > :/home/russ/.m2/repository/org/slf4j/log4j-over-slf4j/1.
> 7.25/log4j-over-slf4j-1.7.25.jar
> >
> > :/home/russ/.m2/repository/org/slf4j/jul-to-slf4j/1.7.25/
> jul-to-slf4j-1.7.25.jar
> > [Loaded org.slf4j.LoggerFactory from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > [Loaded org.slf4j.ILoggerFactory from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > [Loaded org.slf4j.helpers.SubstituteLoggerFactory from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > [Loaded org.slf4j.Logger from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > [Loaded org.slf4j.spi.LoggerFactoryBinder from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > [Loaded org.slf4j.impl.StaticLoggerBinder from
> >
> > file:/home/russ/.m2/repository/com/perfectsearchcorp/medical-
> filter/195/medical-filter-195.jar]
> > [Loaded org.slf4j.helpers.MessageFormatter from
> >
> > file:/home/russ/.m2/repository/org/apache/tika/
> tika-app/1.0/tika-app-1.0.jar]
> > java.lang.AssertionError: java.lang.NoSuchMethodError:
> >
> > org.slf4j.helpers.MessageFormatter.arrayFormat(
> Ljava/lang/String;[Ljava/lang/Object;)Lorg/slf4j/helpers/FormattingTuple;
> > Caused by: java.lang.NoSuchMethodError:
> >
> > org.slf4j.helpers.MessageFormatter.arrayFormat(
> Ljava/lang/String;[Ljava/lang/Object;)Lorg/slf4j/helpers/FormattingTuple;
> >
> >
> > I'm very concerned by Tika having hard-linked /slf4j/. Is that what's
> > going on? I know from experience that I cannot use any other version of
> > Tika.* Anyway, what does this tell you?
> >
> > I greatly appreciate your help and suggestions in this.
> >
> > Russ
> >
> > * I've been working on excerpting those bits of our code base that
> > consume Tika to put them into a different project (and NAR). It's not
> > been easy and I haven't finished yet doing it in the midst of pretty
> > tight deadlines. My former colleague left things pretty interdependent
> > in February when he left (and told me it was going to be a lot of work
> > to separate what we call "legacy" processors out from the rest).
> >
> > On 05/08/2017 08:37 PM, Andrew Psaltis wrote:
> > > Russell,
> > > Sorry this is so much of a pain. I wonder if you can pass using the JVM
> > > argument -verbose:class and then running the test can shed more light
> on
> > > which JAR is pulling it in.
> > >
> > > On Mon, May 8, 2017 at 5:14 PM, Russell Bateman <russ@windofkeltia.com
> >
> > > wrote:
> > >
> > >> I understand that no NiFi JAR comes with /slf4j/, but expects the
> > >> consuming build to provide it. That's the assumption I've been going
> on
> > and
> > >> it's why in my top-level /pom.xml/, I've specified [1.7.25] so that no
> > >> quibbling over version can occur and I specify these JARs in the root
> > >> /pom.xml/ so that no "renegade" submodule calls the dependency for its
> > own.
> > >>
> > >> The problem is that some version of slf4j-api-x.y.z.jar doesn't
> provide
> > a
> > >> MessageFormatter class containing an arrayFormat() method, right?
> > Somehow,
> > >> though, this impoverished version is the one bound to my code despite
> my
> > >> marking every dependency with Andrew's exclusion statements?
> > >>
> > >> There is nothing giving this away; in IntelliJ IDEA, under External
> > >> Libraries in the Project pane, I see only 1.7.25 of these libraries,
> no
> > >> additional versions. I wonder if I should see more versions if in fact
> > >> that's what's going on?
> > >>
> > >> I tried wiping /~/.m2/repository/org/slf4j/, after mvn clean compile
> of
> > my
> > >> project, I see all those versions have come back to
> > >> /~/.m2/repository/org/slf4j/. So, at the root of my project, I created
> > >> /hard-repository/, and stuffed it with the slf4j-relevant JARs
> arranged
> > in
> > >> their repository accoutrements (paths, files, everything), then added
> > >>
> > >>     <repositories>
> > >>        <repository>
> > >>          <id>slf4j</id>
> > >>          <name>slf4j repo</name>
> > >>
> >  <url>file://${project.build.directory}/hard-repository/maven_repo</url>
> > >>       </repository>
> > >>     </repositories>
> > >>
> > >> ...first, so that this dependency would be satisfied--not that this
> > should
> > >> be necessary. So, I took this back out as superfluous.
> > >>
> > >> To me, this means that something else I'm linking has already
> > hard-linked
> > >> some version of slf4j-api older than (maybe 1.7.x--whenever
> > >> MessageFormatter acquired this new method) which would prevent me from
> > >> getting 1.7.25 and yet, IntelliJ's External Libraries list and,
> > examining
> > >> the module dependencies tab for each individual module I see only
> > 1.7.25.
> > >>
> > >> I am stumped. I guess I could attempt to link in the NiFi test
> framework
> > >> in such a way as to be able to debug down into it better to watch
> fail.
> > >> I've stepped down in using IDEA's decompiler, which I would think
> > adequate,
> > >> but I don't see exactly what's amiss.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On 05/08/2017 11:10 AM, Bryan Bende wrote:
> > >>
> > >>> Ok thanks for all the info, I am not totally sure what is going, but
> I
> > >>> can tell you how the logging JARs are setup in NiFi and maybe that
> > >>> will shed some light on things...
> > >>>
> > >>> In the root pom for NiFi there is a dependencyManagement section that
> > >>> declares slf4j-api, jul-to-slf4j, log4j-over-slf4j, and
> jcl-over-slf4j
> > >>> all as provided dependencies to ensure no NARs actually bundle these
> > >>> since they will be provided directly in the lib directory and
> > >>> available to all NARs, this also forces them all to
> > >>> ${org.slf4j.version} which in master is 1.7.25:
> > >>>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>jcl-over-slf4j</artifactId>
> > >>>       <version>${org.slf4j.version}</version>
> > >>>       <scope>provided</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>log4j-over-slf4j</artifactId>
> > >>>       <version>${org.slf4j.version}</version>
> > >>>       <scope>provided</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>jul-to-slf4j</artifactId>
> > >>>       <version>${org.slf4j.version}</version>
> > >>>       <scope>provided</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>slf4j-api</artifactId>
> > >>>       <version>${org.slf4j.version}</version>
> > >>>       <scope>provided</scope>
> > >>> </dependency>
> > >>>
> > >>>
> > >>> Then the nifi-assembly pom.xml declares these using compile scope and
> > >>> puts them in the lib directory:
> > >>>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>jcl-over-slf4j</artifactId>
> > >>>       <scope>compile</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>jul-to-slf4j</artifactId>
> > >>>       <scope>compile</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>log4j-over-slf4j</artifactId>
> > >>>       <scope>compile</scope>
> > >>> </dependency>
> > >>> <dependency>
> > >>>       <groupId>org.slf4j</groupId>
> > >>>       <artifactId>slf4j-api</artifactId>
> > >>>       <scope>compile</scope>
> > >>> </dependency>
> > >>>
> > >>> Now for Unit tests, the root pom has a regular dependencies section
> > >>> that has slf4j-simple declared with test scope:
> > >>>
> > >>>       <dependency>
> > >>>           <groupId>org.slf4j</groupId>
> > >>>           <artifactId>slf4j-simple</artifactId>
> > >>>           <scope>test</scope>
> > >>>       </dependency>
> > >>>
> > >>> So every NAR that is with in the NiFi project will have slf4j-simples
> > >>> available for unit tests.
> > >>>
> > >>> Andrew Psaltis also mentions a good point about the different
> > >>> versions, I believe we just upgraded to 1.7.25 for the 1.2.0 release,
> > >>> but previous were on 1.7.12, but you should double check that.
> > >>>
> > >>>
> > >>>
> > >
> >
> >
>



-- 
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message