lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DM Smith <>
Subject Re: Results (Re: Survey: Lucene and Java 1.4 vs. 1.5)
Date Tue, 20 Jun 2006 18:01:21 GMT
I would like to suggest that a central core Lucene be identified and 
that core be maintained as compatible for Java 1.4.

It has also been stated that J2ME compatibility is a future goal. It 
would be nice to consider that in defining a central core. (BTW, there 
are two J2ME standards, one is a subset of Java 1.3 and the newer is a 
subset of Java 1.4, but it is not widely implemented on target devices. 
Since Lucene does not support J2ME today, perhaps the 1.4  version is 

Outside of that central core, I think Java 5, even Java 6 is just fine. 
And probably this is where most of the contributions will be.

As I said earlier, I am not a Lucene developer, but a Lucene user.

Here is an example of what I use (very trivial):
For indexing:

IndexWriter writer = new IndexWriter(tempPath.getCanonicalPath(), new SimpleAnalyzer(), true)
loop over all (verseReference,verseText) pairs
    Document doc = new Document();
    doc.add(new Field(FIELD_NAME, verseReference, Field.Store.YES, Field.Index.NO));
    doc.add(new Field(FIELD_BODY, new StringReader(verseText)));
end loop

And for searching:

IndexSearcher searcher = new IndexSearcher(path);
Analyzer analyzer = new SimpleAnalyzer();
QueryParser parser = new QueryParser(LuceneIndex.FIELD_BODY, analyzer);
Query query = parser.parse(search);
Hits hits =;
for (int i = 0; i < hits.length(); i++)
    Verse verse = VerseFactory.fromString(hits.doc(i).get(LuceneIndex.FIELD_NAME));
    // PassageTally understands a score of 0 as the verse not participating
    int score = (int) (hits.score(i) * 100 + 1);
    tally.add(verse, score);

I just don't see why Java 5 needs to be behind this kind of usage.

See below for more responses.

Robert Engels wrote:
> To set the record straight, I think the Lucene product and community are
> fantastic. Period.
> I was also not the one who starting in with what could be termed
> 'aggressive' language.
> Our company does not fully support 1.5. I was the loudest voice against the
> move to 1.5.
> After almost 2 years I now back the move. Why? Several reasons:
> 1. Sun is very slow, if at all to fix bugs in 1.4 (of which there are many).
> For example, the current problems in Lucene regarding ThreadLocals. Although
> this is not a bug per se, it is probably not intuitive or desired behavior.
> The Lucene developers have been forced to both diagnose and create
> workarounds "problems" already fixed in 1.5. The licensing of Java does not
> allow for the easy fix bugs by non-Sun developers.

I'm not certain, but IIRC earlier messages in the first Java 5 thread, 
this was not a change that prevented compiling under Java 5 for a Java 
1.4 target.

I think that this is an example of where we need to be clear about 
runtime compatibility. Java 1.4 programs compiled with a Java 1.4 
compiler run better under Java 5. Programs that don't use Java 5 
features can be compiled with Java 1.4 compatibility using the Java 5 

As long as the bugs are fixed in Java 5 and it can be cross-compiled for 
Java 1.4, the fix becomes available under a Java 1.4 jre.

> 2. The type safe collections are far more efficient to program/debug with.

I personally find this to be the case, but it does not change "business 
requirements" of a target application.

> 3. The standardized concurrent facilities can be of great benefit to
> multithreaded programs.

These can be used without going to Java 5. I have been using them since 
Java 1.3. Granted it would be a dependency and a first for Lucene. The 
license clearly places the cpj code in the public domain. This means 
that it can be distributed within the Lucene jar.

> 4. It is what students graduating from college understand and use.
> 5. It is what the currently available books explain and use.
True, but they are still taught the "old" way. Anyone needing to 
maintain or enhance existing pre Java 5 code will have to know the old 
way. Given the caliper of developers here and those that provide 
patches, I doubt that there is anyone that would have any difficulty 
writing Java 1.4 code.

> It just seems that many people believe that if there is ONE person (or a
> minority) that can't switch, then Lucene cannot switch. It seems that Bob is
> in this category of never being able to switch (I am fairly certain 1.5 will
> probably never be released for OS 9) - does that mean that Lucene developers
> can never use 1.5 features? What about the argument I made of using
> alternative algorithms that may not be as useable on older, slower machines.

Java 5 will never be released for MacOS 10.3. This OS is still current, 
supported and widely used.
I mentioned MacOS 9, because it is a business requirement for me. And 
yes, it will be many year before we will drop support for it. And during 
those years, I would like to have the benefit of bug fixes, performance 
enhancements and new features (such as leading wild card searches, if 
that ever is done.)
BTW, Java 6 is beta for MacOS 10.4.

> We have users that cannot upgrade to the latest release of our software
> because of inadequate hardware (or unwillingness to change).

I have run a "lucene" application on a Win98 laptop with a 333Mhz CPU, 
4200rpm harddrive and 32M ram. The performance of the searches is just 
fine. Old hardware is not the issue.

The issue is availability of Java for the OS that runs on the old 
hardware. (And for the record Java 5 runs just fine on that laptop.)

The other issue is that we do not control the environment owned by our 
users. As application developers, we use lucene, so we are lucene users. 
But our users don't care what we use. All they care is whether our 
application works for them in their environment.

>  They have 3
> choices. Stick with the old, upgrade their machines, or move to a
> competitor. The last of which we surely do not want, but that is their
> choice. At the same time, if we did not continue to grow and evolve our
> software, we would lose far more of our customers to competitors.
> This is open-source software. A clear majority has voted. Why can't we
> listen to them and move forward???

The vote was not for whether Lucene should go to Java 5, but whether the 
application base can migrate to Java 5 within a year.

The vote was very telling. 1/3rd of the voters are java 1.4 and cannot 
go to Java 5 in the next 12 months. This does not represent "ONE person" 
but potentially many more people than the 45 that did "vote".

> -----Original Message-----
> From: Dan Armbrust [] 
> Sent: Tuesday, June 20, 2006 8:42 AM
> To:
> Subject: Re: Results (Re: Survey: Lucene and Java 1.4 vs. 1.5)
> Robert Engels wrote:
>> People making these arguments against 1.5 sound really ill-informed, 
>> or lazy. Neither of which is good for open-source development.
> Preface - I'm not a lucene developer - just an interested user.
> I don't know - it seems to me that it is the 1.5 crowd that is making the
> lazy argument.  You are in effect, saying, that the highly skilled
> developers who would be making lucene contributions are unable or unwilling
> to write 1.4 java code?  Come on... it really not that hard. 
> Which set is being lazy?  I'll stop the name calling now, and try to make a
> better point.
> I have some applications that I have written in 1.5 - and yes - it is nice.
> But I also have other applications (that use Lucene) that are written to be
> 1.4 compatible.  And they need to stay that way for quite some time to come.
> Why?  Many reasons.  The first - because they implement an official HL7
> specification - and the specification says that the implementation needs to
> support Java 1.4.
> Also, at my place of employment we have about 40,000 desktop computers that
> are all centrally managed - down to every point release of every single
> piece of software.  There are multiple applications using java that are
> installed on these machines.  Each application has to be certified and fully
> tested with a newer version of java before a newer version of java can be
> installed.  As you can imagine, that severely hampers the pace of java
> updates.  We are just getting 1.4 installed on these machines now.  When you
> are managing that many machines in a clinical environment - you have to play
> it safe.  There are no upgrades for an upgrades sake, or for syntactic
> sugar.  There has to be a real problem to even get the process started.  I'm
> sure many other people have similar situations.
> Also - I don't know much about the Java mobile platform - but I thought I
> had read before that they are limited to the 1.3 or 1.4 feature set? 
> If this is true, do we really want to remove an entire ecosystem of
> potential users?  Over syntactic sugar?
> While I'm not completely opposed to the argument that I should just have to
> stay with the Lucene 2.0.x release with applications that need to run in 1.4
> environments - Lucene is an integral part of that code.  If performance
> improvements are made to the core, I want those in my code. 
>   If bugs are found and fixed - I want those fixes too.  As a matter of fact
> - until the 2.0 release, I was using a build from the trunk because of a bug
> that I found in Lucene, (and someone else was gracious enough to fix for
> me).  Lucene is a low level library that is used to build many great
> applications.  If you make the jump to 1.5 today - you are going to be
> leaving people behind.  And judging by the poll, you are going to be leaving
> a fairly significant number of people behind. 
> Lucene has great policy on not breaking backwards compatibility in their API
> - why should this be looked at any differently?
>  > Rather than having the 1.5 developers having to waste their time  >
> "thinking" in 1.4 when their work is predominately being performed  > using
> 1.5 features/compilers/tools.
> I don't think that the caliber of developers that are working on the Lucene
> core are going to be slowed down any by using 1.4 syntax over 1.5.  (It
> actually takes longer to type in all of those generics :)  All of my tools -
> Eclipse and Java 1.5 - have a check box that will cause them to generate 1.4
> compatible code.  Its really _not_ a big deal to write 1.4 code even if you
> are used to 1.5.  This particular argument just isn't compelling to me.
> My personal opinion for the path that Lucene should take:
> Core bugs fixes must be 1.4 compatible.
> Core improvements must be 1.4 compatible.
> Contrib / sandbox can be 1.5 or 1.6.
> Of course, at some point - Lucene Core does need to advance.  But I don't
> just don't feel that syntactic sugar in 1.5 is enough of a reason to break
> backwards compatibility.  I haven't followed 1.6 - I don't know what the new
> features are there.  Assuming that there are great new features in 1.6 -
> that would improve the lucene core if they were used -
>   I think that that is when this issue gets revisited.
> This isn't the type of question that should be decided by a poll.  This
> should be decided by thoughtfully looking at the consequences of each
> choice.  For me - the negative consequences of choosing 1.5 - leaving behind
> a lot of users - is much worse than the negative consequences of staying at
> 1.4 - making a couple dozen highly skilled developers check an extra box in
> their lucene development environments?
> If any developers have actually read this far (sorry - it got kind of
> long) - thanks again for all of your great work - Lucene is a great tool
> - and a great community.
> Dan
> --
> ****************************
> Daniel Armbrust
> Biomedical Informatics
> Mayo Clinic Rochester
> daniel.armbrust(at)
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message