lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <>
Subject Re: LogFormatter
Date Tue, 03 Jan 2006 13:19:32 GMT

Please post this to the Nutch list, not the Lucene one.


On Jan 3, 2006, at 5:42 AM, Daniel Feinstein wrote:

> Hi,
> This is my first email to the community. In our (  
> project we integrated multiple lucene indexes and a nutch package.  
> We use the packages for about two years. Lucene is working great  
> and we could use it as a "black box". Unfortunately, we had to  
> modify Nutch package for our usage because of some leaks and a  
> design problem in LogFormatter class. This fact force us to use a  
> customized version of Nutch and avoid to update versions. At the  
> last week we decided to move to the latest version of Nutch but  
> discovered that the LogFormatter still have the same problem (leaks  
> I believe has been fixed).
> The problem:
> In Nutch project LogFormatter has duplicated functionality:
> 1) Logger records format and
> 2) Severe error handler
> The first usage is standard and usually could be overwritten by a  
> user of the package by modifying file.
> The second usage is much more problematic because it affects the  
> behavior of the whole application (not only Nutch package). To  
> support the error handling LogFormatter enforce usage of the  
> formatter class by all classes of the whole application which uses  
> Nutch package. This is done by overwriting all the system handlers  
> (class java.util.logging.Handler). This operation prevents the  
> application to use its own log formatter. Also this cause  
> LogFormatter.hasLoggedSevere() to be sensitive to all severe  
> records in the big system but not only to relevant. More than that  
> this flag, LogFormatter.loggedSevere is never cleaned what means if  
> an application had one, even unrelated severe record, tools like  
> Fetcher will never run until the application will be restarted.
> I would like to suggest the following solutions:
> 1) To separate the functionality of log formatting and error  
> handling or
> 2) Change LogFormatter class to be affected only by nutch package  
> functions
> For my opinion the first solution is much better especially if  
> error handling will be encapsulated for each task. I have found the  
> following usages of LogFormatter.hasLoggedSevere():
> - Fetcher
> - URLFilterChecker
> - ParseSegment
> Unfortunately I'm not familiar enough with the usages above to  
> implement this solution that why I suggest the second one. The  
> attached file is our implementation of LogFormatter class which we  
> use for more than a year. I hope this change will be accepted by  
> the community.
> Daniel
> _____________________
> Daniel Feinstein
> Lead Software Engineer
> __________________________
> This email may contain confidential and privileged material for the  
> sole use of the intended recipient. Any review or distribution by  
> others is strictly prohibited. If you are not the intended  
> recipient please contact the sender and delete all copies.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message