logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remko Popma <remko.po...@gmail.com>
Subject Re: Two Question for NoSql MongoDB appender
Date Sat, 17 Aug 2013 10:26:20 GMT
That's a tricky one. Well done!
(Much better than me: I'm on day two of my struggle with LOG4J2-315 and not
making much progress... :-)


On Sat, Aug 17, 2013 at 5:17 PM, Nick Williams <
nicholas@nicholaswilliams.net> wrote:

> Solved it!
>
> And you're never gonna believe what I learned tonight...(well, maybe you
> will)
>
> I solved the error by changing this:
>
> public final class MongoDBProvider implements
> NoSQLProvider<MongoDBConnection> {
>     ...
> +                try {
> +                    if (!database.authenticate(username,
> password.toCharArray())) {
> +                        LOGGER.error("Failed to authenticate against
> MongoDB server. Unknown error.");
> +                    }
> +                } catch (MongoException e) {
> +                    LOGGER.error("Failed to authenticate against MongoDB:
> " + e.getMessage(), e);
> +                } catch (IllegalStateException e) {
> +                    LOGGER.error("Factory-supplied MongoDB database
> connection already authenticated with different" +
> +                            "credentials but lost connection.");
> +                }
>     ...
> }
>
> To this:
>
> public final class MongoDBProvider implements
> NoSQLProvider<MongoDBConnection> {
>     ...
> +                MongoDBConnection.authenticate(database, username,
> password);
>     ...
> }
>
> public final class MongoDBConnection implements
> NoSQLConnection<BasicDBObject, MongoDBObject> {
>     ...
> +    static void authenticate(final DB database, final String username,
> final String password) {
> +        try {
> +            if (!database.authenticate(username, password.toCharArray()))
> {
> +                LOGGER.error("Failed to authenticate against MongoDB
> server. Unknown error.");
> +            }
> +        } catch (final MongoException e) {
> +            LOGGER.error("Failed to authenticate against MongoDB: " +
> e.getMessage(), e);
> +        } catch (final IllegalStateException e) {
> +            LOGGER.error("Factory-supplied MongoDB database connection
> already authenticated with different" +
> +                    "credentials but lost connection.");
> +        }
> +    }
>     ...
> }
>
> Crazy, right!? Here's what I've learned:
>
> The errors were occurring in tests for the Log4j 1.2 API and the SLF4J
> Bridge. These tests use the core Logger which triggers plugin discovery. In
> order to scan for annotations, plugin discovery loads the MongoDBProvider,
> CouchDBProvider, and JPAAppender classes, among many others, all of which
> have transitive dependencies that are not on the classpath for running the
> unit tests for Log4j 1.2 API and SLF4J. So how did it ever work in the
> first place?
>
> As you may already know, when Java loads a class it also automatically
> loads any classes it extends or implements, any classes that are the types
> of static members of that class, any static inner classes, and any classes
> used within the static initializer. It doesn't load any other classes that
> class uses in any methods or constructors or as instance members--like
> com.mongodb.DB or javax.persistence.*--until the code that uses them
> actually executes for the first time. Because of this, we can do something
> like load the MongoDBProvider class to scan for @Plugin annotations even
> though the com.mongodb classes it uses are not on the classpath (as long as
> they aren't static members of or extended by the MongoDBProvider, that is).
>
> However, Java has a special behavior with exceptions. Because we have
> these lovely things called checked exceptions that methods must declare to
> be thrown, exceptions are naturally part of a class's interface. Thus, when
> Java loads a class it must load the exceptions the class's methods might
> throw so that it can complete the interface in memory
> (java.lang.Class.getMethod("someMethod").getExceptionTypes()). Likely for
> performance reasons, it doesn't differentiate between exceptions that are
> actually declared to be thrown and exceptions that are just used (caught).
> ANY dependent classes that are exceptions are loaded when the class loads,
> even if they're just caught exceptions. This is why this all worked until I
> started using an exception from a transitive dependency within a plugin
> class (MongoDBProvider).
>
> (Incidentally, it's also why more advanced class-scanning projects like
> Spring, Hibernate and Tomcat don't load classes using a ClassLoader just to
> scan for annotations. Instead, they inspect the byte code manually to scan
> for annotations, preventing such class loading errors during discovery
> phases and also saving memory resources since Classes aren't usually
> garbage collected. It might be worthwhile to look into doing something
> similar in Log4j plugin discovery. I don't know how much effort would be
> involved.)
>
> I haven't confirmed any of this with JLS documentation because no amount
> of Google searching for the combination of "class loading" and "exception"
> brings up anything other than 10,000,000 people asking questions about
> what's wrong with their classpath. I simply can't find that needle in a
> planet full of haystacks. But my thorough experimentation has some pretty
> clear results. This is exactly what's happening.
>
> Nick
>
> On Aug 17, 2013, at 12:44 AM, Ralph Goers wrote:
>
> > I'll reiterate what I wrote. Catch the RuntimeException and then do
> >
> > if (e.class.getName().equals("com.mongodb.MongoException")) {
> >   LOGGER.error("...");
> > } else {
> >   throw e;
> > }
> >
> > This should give you the same behavior.
> >
> > Ralph
> >
> > On Aug 16, 2013, at 9:49 PM, Nick Williams wrote:
> >
> >> That approach concerns me. Catching RuntimeException essentially opens
> it up to thousands of possible exceptions that could be the cause, as
> opposed to looking for that exact cause. I suppose I don't have a choice,
> though. This apparently just isn't going to work.
> >>
> >> Definitely agreed that there is too much going on for a simple
> Exception class.
> >>
> >> :-/
> >>
> >> Nick
> >>
> >> On Aug 16, 2013, at 11:45 PM, Ralph Goers wrote:
> >>
> >>> After following the chain of stuff that gets brought in via BSONObject
> I still recommend the approach in my other email of just catching
> RuntimeException.  A bunch of other classes are being referenced, one of
> which is creating a static Logger from java.util.logging. I have no idea
> why that might fail but there is just way too much going on for a simple
> Exception class.
> >>>
> >>> Ralph
> >>>
> >>> On Aug 16, 2013, at 9:26 PM, Nick Williams wrote:
> >>>
> >>>>
> https://github.com/mongodb/mongo-java-driver/blob/master/src/main/com/mongodb/DB.java
> >>>>
> >>>> That also shows an import for org.bson.BSONObject, but the tests
> still run with just DB and no MongoException. org.bson is in the
> org.mongodb:mongo-java-driver JAR file. So, no, that's not the problem.
> There's something else...
> >>>>
> >>>> Nick
> >>>>
> >>>> On Aug 16, 2013, at 11:20 PM, Ralph Goers wrote:
> >>>>
> >>>>>
> https://github.com/mongodb/mongo-java-driver/blob/master/src/main/com/mongodb/MongoException.javashows
an import for org.bson.BSONObject.  The pom.xml for mongo-java-driver
> doesn't contain a transitive dependency for that and mvn dependency:tree on
> core doesn't show it.
> >>>>>
> >>>>> Ralph
> >>>>>
> >>>>>
> >>>>> On Aug 16, 2013, at 3:48 PM, Nick Williams wrote:
> >>>>>
> >>>>>> Guys, I'm having a hard time with this simple fix that should
have
> taken five minutes. I'm getting test failures due to NoClassDefFoundErrors
> that shouldn't happen.
> >>>>>>
> >>>>>> Here are the tests in error:
> >>>>>>   CategoryTest.setupClass:52 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testTraceWithException:415 ? NoClassDefFound
> com/mongodb/MongoExcep...
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testLog:459 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testRB1:295 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testRB2:314 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testRB3:334 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testTrace:388 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testAdditivity1:119 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testAdditivity2:144 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testAdditivity3:183 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testIsTraceEnabled:443 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.testExists:355 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggerTest.tearDown:75 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggingTest.setupClass:44 ? NoClassDefFound
> com/mongodb/MongoException
> >>>>>>   LoggingTest.cleanupClass:49 NullPointer
> >>>>>>
> >>>>>> Here's the code I added:
> >>>>>>
> >>>>>>                 try {
> >>>>>>                     if (!database.authenticate(username,
> password.toCharArray())) {
> >>>>>>                         LOGGER.error("Failed to authenticate
> against MongoDB server. Unknown error.");
> >>>>>>                     }
> >>>>>>                 } catch (MongoException e) {
> >>>>>>                     LOGGER.error("Failed to authenticate against
> MongoDB: " + e.getMessage(), e);
> >>>>>>                 } catch (IllegalStateException e) {
> >>>>>>                     LOGGER.error("Factory-supplied MongoDB database
> connection already authenticated with different" +
> >>>>>>                             "credentials but lost connection.");
> >>>>>>                 }
> >>>>>>
> >>>>>> Problem is, "database" is an instance of com.mongodb.DB, which
is
> in the same JAR as com.mongodb.MongoException. If I remove this code, the
> tests pass. How is this possible? The DB instance is there with or without
> this new code, which means the JAR is on the classpath, which means
> MongoException should be on the classpath.
> >>>>>>
> >>>>>> Very confused...
> >>>>>>
> >>>>>> Nick
> >>>>>>
> >>>>>> On Aug 16, 2013, at 5:13 PM, Gary Gregory wrote:
> >>>>>>
> >>>>>>> Thank you for the update Nick!
> >>>>>>> :)
> >>>>>>> Gary
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Aug 16, 2013 at 5:39 PM, Nick Williams <
> nicholas@nicholaswilliams.net> wrote:
> >>>>>>> Answers inline.
> >>>>>>>
> >>>>>>> On Aug 14, 2013, at 2:10 AM, YuCheng Ting wrote:
> >>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> I use beta8 log4j2 and wrote log4j2.xml like example
in document (
> http://logging.apache.org/log4j/2.x/manual/appenders.html#NoSQLAppender ):
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> <appenders>
> >>>>>>>>     <NoSql name="databaseAppender">
> >>>>>>>>         <MongoDb databaseName="applicationDb"
> collectionName="applicationLog"
> >>>>>>>>             server="mongo.example.org"
> >>>>>>>>             username="loggingUser" password="abc123"
/>
> >>>>>>>>     </NoSql>
> >>>>>>>> </appenders>
> >>>>>>>
> >>>>>>> Yep. That's correct.
> >>>>>>>
> >>>>>>>> but I get the two exception:
> >>>>>>>>
> >>>>>>>> 1, "can't serialize class org.apache.logging.log4j.Level"
> exception in (BasicBSONEncoder.java:270), I read the code and add follow
> code in my project before logging, it gone.
> >>>>>>>>
> >>>>>>>>
> BSON.addEncodingHook(org.apache.logging.log4j.Level.class, new
> Transformer() {
> >>>>>>>>             @Override
> >>>>>>>>             public Object transform(Object o) {
> >>>>>>>>                 return o.toString();
> >>>>>>>>             }
> >>>>>>>>         });
> >>>>>>>
> >>>>>>> This bug was reported and fixed a few weeks ago. The fix
will be
> in the next version, or you can compile locally.
> https://issues.apache.org/jira/browse/LOG4J2-330
> >>>>>>>
> >>>>>>>> 2, “not authorized for insert test.log”, because
my MongoDB need
> auth to write, but the the "username" and "password" attributes in
> log4j2.xml is nearly useless, after I read source code, found it NOT auth in
> >>>>>>>>
> >>>>>>>>
> org.apache.logging.log4j.core.appender.db.nosql.mongo.MongoDBProvider.createNoSQLProvider
> >>>>>>>> source code line 181 after check username and password
and
> com.mongodb.DB.authenticate never be called.
> >>>>>>>
> >>>>>>> This is a bug. I'm reporting it and fixing it now. The fix
will be
> in the next version, or you can compile locally (after I get the change
> committed, of course).
> >>>>>>>
> >>>>>>>> so I change log4j2.xml :
> >>>>>>>>
> >>>>>>>> <NoSql name="mongodb">
> >>>>>>>>              <MongoDb collectionName="log" databaseName="test"
> >>>>>>>>
>  factoryClassName="com.yuchs.test.log4j.MainTest"
> >>>>>>>>                         factoryMethodName="getMongoClient"
/>
> >>>>>>>> </NoSql>
> >>>>>>>>
> >>>>>>>> and create MongoClient and call com.mongodb.DB.authenticate
> method in com.yuchs.test.log4j.MainTest.getMongoClient.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> This is my question:
> >>>>>>>>
> >>>>>>>> 1, Why not add BSON.addEncodingHook code into log4j2
project to
> avoid basic exception ? or another rule of method I don't know ?
> >>>>>>>>
> >>>>>>>> 2, Why not auth DB in log4j2 project if password and
username is
> set in log4j2.xml ? or another rule of method I don't know ?
> >>>>>>>>
> >>>>>>>> Thanks everyone!
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> >>>>>>> Java Persistence with Hibernate, Second Edition
> >>>>>>> JUnit in Action, Second Edition
> >>>>>>> Spring Batch in Action
> >>>>>>> Blog: http://garygregory.wordpress.com
> >>>>>>> Home: http://garygregory.com/
> >>>>>>> Tweet! http://twitter.com/GaryGregory
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>
>

Mime
View raw message