lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: NewIndexModifier - - - DeletingIndexWriter
Date Wed, 14 Feb 2007 09:08:56 GMT

: My concern is, to get the raw speed of Lucene, you got to get to the basics.
: If we start to apply layers upon layers of code to just mask off the
: internals of Lucene, it will not do any good.

one of the reasons for the approach of this NewIndexModifier/IndexWriter
approach to supporting deletions was to prevent the need to layers upon
layers of code.

furthermore: i haven't seen any evidence thta these changes adversely
affect people who want to use "the basics" you describe: an IndexReader
for deletes and term info, and IndexWriter for adds.  if you have concerns
about the performance impacts (either in speed, or index size, or any
other metric) please point them out.

fundementally Lucene as library has a wide user base with various levels
of skill -- while the "Expert" level APIs might require a very in depth
understanding of how the internals work, there do need to be more
"non-Expert" level APIs ... when possible, it certainly makes sense for
these APIs to live in other classes/packages/contribs (or in some cases,
other Lucene sub-projects).  the 2.0 version of IndexModifier is an
example at an attempt to provide a "high level" api that didn't modify the
"low level" IndexWriter and IndexReader classes -- but from what i can
tlel it's not very useful because of the performance issues.

tradeoffs have to be made -- if making some API additions with no
performance cost to "expert" users makes it easy for novice users to get
big performace wins, then what's the down side?

: I think the Lucene library should be designed simple and efficient in order
: to allow tweaking for raw speed. That's the spirit for large scale search
: engines, right? Even Google file system has to sacrifice some design for raw
: speed, i.e., files are append-only.

as i said before: if you have any specific concerns about the performance
impacts to existing methods of these changes, please raise them; if you
see anyway in which these changes impact your ability to tweak your app
for raw speed, please elaborate.





-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message