[ https://issues.apache.org/jira/browse/LUCENE-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788189#action_12788189
]
Robert Muir commented on LUCENE-1606:
-------------------------------------
btw, Thanks to Uwe, Mike, Mark for all the help here!
> Automaton Query/Filter (scalable regex)
> ---------------------------------------
>
> Key: LUCENE-1606
> URL: https://issues.apache.org/jira/browse/LUCENE-1606
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Search
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: Flex Branch
>
> Attachments: automaton.patch, automatonMultiQuery.patch, automatonmultiqueryfuzzy.patch,
automatonMultiQuerySmart.patch, automatonWithWildCard.patch, automatonWithWildCard2.patch,
BenchWildcard.java, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606_nodep.patch
>
>
> Attached is a patch for an AutomatonQuery/Filter (name can change if its not suitable).
> Whereas the out-of-box contrib RegexQuery is nice, I have some very large indexes (100M+
unique tokens) where queries are quite slow, 2 minutes, etc. Additionally all of the existing
RegexQuery implementations in Lucene are really slow if there is no constant prefix. This
implementation does not depend upon constant prefix, and runs the same query in 640ms.
> Some use cases I envision:
> 1. lexicography/etc on large text corpora
> 2. looking for things such as urls where the prefix is not constant (http:// or ftp://)
> The Filter uses the BRICS package (http://www.brics.dk/automaton/) to convert regular
expressions into a DFA. Then, the filter "enumerates" terms in a special way, by using the
underlying state machine. Here is my short description from the comments:
> The algorithm here is pretty basic. Enumerate terms but instead of a binary accept/reject
do:
>
> 1. Look at the portion that is OK (did not enter a reject state in the DFA)
> 2. Generate the next possible String and seek to that.
> the Query simply wraps the filter with ConstantScoreQuery.
> I did not include the automaton.jar inside the patch but it can be downloaded from http://www.brics.dk/automaton/
and is BSD-licensed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-1606.
---------------------------------
Resolution: Fixed
Fix Version/s: (was: 3.1)
Flex Branch
Committed revision 888891.
> Automaton Query/Filter (scalable regex)
> ---------------------------------------
>
> Key: LUCENE-1606
> URL: https://issues.apache.org/jira/browse/LUCENE-1606
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Search
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: Flex Branch
>
> Attachments: automaton.patch, automatonMultiQuery.patch, automatonmultiqueryfuzzy.patch,
automatonMultiQuerySmart.patch, automatonWithWildCard.patch, automatonWithWildCard2.patch,
BenchWildcard.java, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch, LUCENE-1606-flex.patch,
LUCENE-1606-flex.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch, LUCENE-1606.patch,
LUCENE-1606.patch, LUCENE-1606_nodep.patch
>
>
> Attached is a patch for an AutomatonQuery/Filter (name can change if its not suitable).
> Whereas the out-of-box contrib RegexQuery is nice, I have some very large indexes (100M+
unique tokens) where queries are quite slow, 2 minutes, etc. Additionally all of the existing
RegexQuery implementations in Lucene are really slow if there is no constant prefix. This
implementation does not depend upon constant prefix, and runs the same query in 640ms.
> Some use cases I envision:
> 1. lexicography/etc on large text corpora
> 2. looking for things such as urls where the prefix is not constant (http:// or ftp://)
> The Filter uses the BRICS package (http://www.brics.dk/automaton/) to convert regular
expressions into a DFA. Then, the filter "enumerates" terms in a special way, by using the
underlying state machine. Here is my short description from the comments:
> The algorithm here is pretty basic. Enumerate terms but instead of a binary accept/reject
do:
>
> 1. Look at the portion that is OK (did not enter a reject state in the DFA)
> 2. Generate the next possible String and seek to that.
> the Query simply wraps the filter with ConstantScoreQuery.
> I did not include the automaton.jar inside the patch but it can be downloaded from http://www.brics.dk/automaton/
and is BSD-licensed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788169#action_12788169
]
Simon Willnauer commented on LUCENE-2139:
-----------------------------------------
Chris, I have a couple of issues with your patch. It seems that you renamed a couple of files
which doesn't work well with patches for some reason. I will comment on this later again.
The other thing is that you use 1.6 classes like [http://java.sun.com/javase/6/docs/api/java/util/concurrent/LinkedBlockingDeque.html|LinkedBlockingDeque]
but we should try to keep the contrib 1.5 dependent.
could you fix those 1.6 references please.
simon
> Cleanup and Improvement of Spatial Contrib
> ------------------------------------------
>
> Key: LUCENE-2139
> URL: https://issues.apache.org/jira/browse/LUCENE-2139
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/spatial
> Affects Versions: 3.1
> Reporter: Chris Male
> Assignee: Simon Willnauer
> Attachments: LUCENE-2139.patch
>
>
> The current spatial contrib can be improved by adding documentation, tests, removing
unused classes and code, repackaging the classes and improving the performance of the distance
filtering. The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
> Other improvements involve adding better support for different distance units, different
distance calculators and different data formats (whether it be lat/long fields, geohashes,
or something else in the future).
> Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2123:
----------------------------------
Attachment: LUCENE-2123.patch
LUCENE-2123-flex.patch
Here the final patches with updated JavaDocs. I want to apply them in this form to trunk and
flex. If nobody objects I will do this tomorrow.
WIth this patch, FuzzyQuery will always highlight correctly, as it can be switched to boolean
query rewrite mode.
> Move FuzzyQuery rewrite as separate RewriteMode into MTQ, was: Highlighter fails to highlight
FuzzyQuery
> --------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2123
> URL: https://issues.apache.org/jira/browse/LUCENE-2123
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/highlighter
> Affects Versions: Flex Branch
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: Flex Branch
>
> Attachments: LUCENE-2123-flex.patch, LUCENE-2123-flex.patch, LUCENE-2123-flex.patch,
LUCENE-2123-flex.patch, LUCENE-2123.patch
>
>
> As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in
flex since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is:
Allow MTQ to change rewrite method and make FUZZY_REWRITE public for that.
> The rewrite mode will live in MTQ as TOP_TERMS_SCORING_BOOLEAN_REWRITE. Also the code
will be refactored to make heavy reuse of term enumeration code and only plug in the PQ for
filtering the top terms.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2123:
----------------------------------
Description:
As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in flex
since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is: Allow
MTQ to change rewrite method and make FUZZY_REWRITE public for that.
The rewrite mode will live in MTQ as TOP_TERMS_SCORING_BOOLEAN_REWRITE. Also the code will
be refactored to make heavy reuse of term enumeration code and only plug in the PQ for filtering
the top terms.
was:
As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in flex
since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is: Allow
MTQ to change rewrite method and make FUZZY_REWRITE public for that.
Summary: Move FuzzyQuery rewrite as separate RewriteMode into MTQ, was: Highlighter
fails to highlight FuzzyQuery (was: Highlighter fails to highlight FuzzyQuery)
Trunk patch comes soon.
> Move FuzzyQuery rewrite as separate RewriteMode into MTQ, was: Highlighter fails to highlight
FuzzyQuery
> --------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2123
> URL: https://issues.apache.org/jira/browse/LUCENE-2123
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/highlighter
> Affects Versions: Flex Branch
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: Flex Branch
>
> Attachments: LUCENE-2123-flex.patch, LUCENE-2123-flex.patch, LUCENE-2123-flex.patch
>
>
> As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in
flex since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is:
Allow MTQ to change rewrite method and make FUZZY_REWRITE public for that.
> The rewrite mode will live in MTQ as TOP_TERMS_SCORING_BOOLEAN_REWRITE. Also the code
will be refactored to make heavy reuse of term enumeration code and only plug in the PQ for
filtering the top terms.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788130#action_12788130
]
Chris Male commented on LUCENE-2139:
------------------------------------
I have also included LUCENE-1934 in this, and tried to include LUCENE-1930 but was unable
to get 1930 to work.
> Cleanup and Improvement of Spatial Contrib
> ------------------------------------------
>
> Key: LUCENE-2139
> URL: https://issues.apache.org/jira/browse/LUCENE-2139
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/spatial
> Affects Versions: 3.1
> Reporter: Chris Male
> Assignee: Simon Willnauer
> Attachments: LUCENE-2139.patch
>
>
> The current spatial contrib can be improved by adding documentation, tests, removing
unused classes and code, repackaging the classes and improving the performance of the distance
filtering. The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
> Other improvements involve adding better support for different distance units, different
distance calculators and different data formats (whether it be lat/long fields, geohashes,
or something else in the future).
> Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Male updated LUCENE-2139:
-------------------------------
Attachment: LUCENE-2139.patch
Added patch.
Couple of TODOs still noted in the patch related to distances. We need to decide what distances
we are going to use for the radius and circumference of the Earth and then use them in SpatialConstants.
Currently the SpatialConstants values are taken from Wikipedia and other sites, yet differ
from some of the distances in the coded.
Also the patch doesn't seem to remove a couple of empty packages. Too many changes in 1 patch
confusing the IDE I think. Help cleaning this up would be appreciated.
> Cleanup and Improvement of Spatial Contrib
> ------------------------------------------
>
> Key: LUCENE-2139
> URL: https://issues.apache.org/jira/browse/LUCENE-2139
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/spatial
> Affects Versions: 3.1
> Reporter: Chris Male
> Assignee: Simon Willnauer
> Attachments: LUCENE-2139.patch
>
>
> The current spatial contrib can be improved by adding documentation, tests, removing
unused classes and code, repackaging the classes and improving the performance of the distance
filtering. The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
> Other improvements involve adding better support for different distance units, different
distance calculators and different data formats (whether it be lat/long fields, geohashes,
or something else in the future).
> Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Siiiiggggh. I'll look at it again tonight.... On Wed, Dec 9, 2009 at 9:13 AM, Robert Muir (JIRA) <jira@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/LUCENE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788100#action_12788100] > > Robert Muir commented on LUCENE-2122: > ------------------------------------- > > Hi Erick, in the Date tools test I think you can delete the public static > Collection<Locale[]> data(), I think you might have accidentally included > it? > > > > Use JUnit4 capabilites for more thorough Locale testing for classes > deriving from LocalizedTestCase > > > --------------------------------------------------------------------------------------------------- > > > > Key: LUCENE-2122 > > URL: https://issues.apache.org/jira/browse/LUCENE-2122 > > Project: Lucene - Java > > Issue Type: Improvement > > Components: Other > > Affects Versions: 3.1 > > Reporter: Erick Erickson > > Assignee: Erick Erickson > > Priority: Minor > > Fix For: 3.1 > > > > Attachments: LUCENE-2122-r2.patch, LUCENE-2122-r3.patch, > LUCENE-2122.patch > > > > > > Use the @Parameterized capabilities of Junit4 to allow more extensive > testing of Locales. > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-dev-help@lucene.apache.org > >
[ https://issues.apache.org/jira/browse/LUCENE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788100#action_12788100
]
Robert Muir commented on LUCENE-2122:
-------------------------------------
Hi Erick, in the Date tools test I think you can delete the public static Collection<Locale[]>
data(), I think you might have accidentally included it?
> Use JUnit4 capabilites for more thorough Locale testing for classes deriving from LocalizedTestCase
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2122
> URL: https://issues.apache.org/jira/browse/LUCENE-2122
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Other
> Affects Versions: 3.1
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2122-r2.patch, LUCENE-2122-r3.patch, LUCENE-2122.patch
>
>
> Use the @Parameterized capabilities of Junit4 to allow more extensive testing of Locales.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir reassigned LUCENE-2122:
-----------------------------------
Assignee: Robert Muir (was: Erick Erickson)
> Use JUnit4 capabilites for more thorough Locale testing for classes deriving from LocalizedTestCase
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-2122
> URL: https://issues.apache.org/jira/browse/LUCENE-2122
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Other
> Affects Versions: 3.1
> Reporter: Erick Erickson
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2122-r2.patch, LUCENE-2122-r3.patch, LUCENE-2122.patch
>
>
> Use the @Parameterized capabilities of Junit4 to allow more extensive testing of Locales.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788098#action_12788098
]
Marvin Humphrey commented on LUCENE-2126:
-----------------------------------------
> These methods should only be able to call the read/write methods (which this
> issue moves to DataInput/Output), but not methods like close(), seek() etc..
Ah, so that's what it is.
In that case, let me vote my (non-binding) -1. I don't believe that the
enforcement of such a restriction justifies the complexity cost of adding a
new class to the public API.
First, adding yet another class to the hierarchy steepens the learning curve
for users and contributors. If you aren't in the rarefied echelon of
exceptional brilliance occupied by people named Michael who work for IBM :),
the gradual accumulation of complexity in the Lucene code base matters. Inch
by inch, things move out of reach.
Second, changing things now for what seems to me like a minor reason makes it
harder to refactor the class hierarchy in the future when other, more
important reasons are inevitably discovered.
For LUCENE-2125, I recommend two possible options.
* Do nothing and assume that the sort of advanced user who writes a posting
codec won't do something incredibly stupid like call indexInput.close().
* Add a note to the docs for writing posting codecs indicating which sort of
of IO methods you ought not to call.
> once we see a need to allow users to extend DataInput/Output outside of
> Lucene we can go ahead and make the additional changes that are mentioned in
> your in my comments here.
In Lucy, there are three tiers of IO usage:
* For low-level IO, use FileHandle.
* For most applications, use InStream's encoder/decoder methods.
* For performance-critical inner-loop material (e.g. posting decoders,
SortCollector), access the raw memory-mapped IO buffer using
InStream_Buf()/InStream_Advance_Buf() and use static inline functions
such as NumUtil_decode_c32 (which does no bounds checking) from
Lucy::Util::NumberUtils.
While you can extend InStream to add a codec, that's not generally the best
way to go about it, because adding a method to InStream requires that all of
your users both use your InStream class and use a subclassed Folder which
overrides the Folder_Open_In() factory method (analogous to
Directory.openInput()). Better is to use the extension point provided by
InStream_Buf()/InStream_Advance_Buf() and write a utility function which
accepts an InStream as an argument.
I don't expect and am not advocating that Lucene adopt the same IO hierarchy
as Lucy, but I wanted to provide an example of other reasons why you might
change things. (What I'd really like to see is for Lucene to come up with
something *better* than the Lucy IO hierarchy.)
One of the reasons Lucene has so many backwards compatibility headaches is
because the public APIs are so extensive and thus constitute such an elaborate
set of backwards compatibility promises. IMO, DataInput and DataOutput do
not offer sufficient benefit to compensate for the increased intricacy they add
to that backwards compatibility contract.
> Split up IndexInput and IndexOutput into DataInput and DataOutput
> -----------------------------------------------------------------
>
> Key: LUCENE-2126
> URL: https://issues.apache.org/jira/browse/LUCENE-2126
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: Flex Branch
> Reporter: Michael Busch
> Assignee: Michael Busch
> Priority: Minor
> Fix For: Flex Branch
>
> Attachments: lucene-2126.patch
>
>
> I'd like to introduce the two new classes DataInput and DataOutput
> that contain all methods from IndexInput and IndexOutput that actually
> decode or encode data, such as readByte()/writeByte(),
> readVInt()/writeVInt().
> Methods like getFilePointer(), seek(), close(), etc., which are not
> related to data encoding, but to files as input/output source stay in
> IndexInput/IndexOutput.
> This patch also changes ByteSliceReader/ByteSliceWriter to extend
> DataInput/DataOutput. Previously ByteSliceReader implemented the
> methods that stay in IndexInput by throwing RuntimeExceptions.
> See also LUCENE-2125.
> All tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-2124.
---------------------------------
Resolution: Fixed
website updated in revision 888803
> move JDK collation to core, ICU collation to ICU contrib
> --------------------------------------------------------
>
> Key: LUCENE-2124
> URL: https://issues.apache.org/jira/browse/LUCENE-2124
> Project: Lucene - Java
> Issue Type: Task
> Components: contrib/*, Search
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2124.patch, LUCENE-2124.patch
>
>
> As mentioned on the list, I propose we move the JDK-based CollationKeyFilter/CollationKeyAnalyzer,
currently located in contrib/collation into core for collation support (language-sensitive
sorting)
> These are not much code (the heavy duty stuff is already in core, IndexableBinaryString).
> And I would also like to move the ICUCollationKeyFilter/ICUCollationKeyAnalyzer (along
with the jar file they depend on) also currently located in contrib/collation into a contrib/icu.
> This way, we can start looking at integrating other functionality from ICU into a fully-fleshed
out icu contrib.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-2100.
-------------------------------------
Resolution: Fixed
committed in revision 888799
thanks robert for review
> Make contrib analyzers final
> ----------------------------
>
> Key: LUCENE-2100
> URL: https://issues.apache.org/jira/browse/LUCENE-2100
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2, 2.4, 2.4.1, 2.9, 2.9.1,
3.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2100.patch, LUCENE-2100.patch
>
>
> The analyzers in contrib/analyzers should all be marked final. None of the Analyzers
should ever be subclassed - users should build their own analyzers if a different combination
of filters and Tokenizers is desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788084#action_12788084
]
Robert Muir commented on LUCENE-2100:
-------------------------------------
patch looks good to me!
> Make contrib analyzers final
> ----------------------------
>
> Key: LUCENE-2100
> URL: https://issues.apache.org/jira/browse/LUCENE-2100
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2, 2.4, 2.4.1, 2.9, 2.9.1,
3.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2100.patch, LUCENE-2100.patch
>
>
> The analyzers in contrib/analyzers should all be marked final. None of the Analyzers
should ever be subclassed - users should build their own analyzers if a different combination
of filters and Tokenizers is desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer updated LUCENE-2100:
------------------------------------
Attachment: LUCENE-2100.patch
Updated to latest trunk
> Make contrib analyzers final
> ----------------------------
>
> Key: LUCENE-2100
> URL: https://issues.apache.org/jira/browse/LUCENE-2100
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2, 2.4, 2.4.1, 2.9, 2.9.1,
3.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2100.patch, LUCENE-2100.patch
>
>
> The analyzers in contrib/analyzers should all be marked final. None of the Analyzers
should ever be subclassed - users should build their own analyzers if a different combination
of filters and Tokenizers is desired.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Does someone with commit rights want to pick this up? I've incorporated the changes suggested by Robert (Thanks!) and think it's ready to go..... Erick
[ https://issues.apache.org/jira/browse/LUCENE-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788077#action_12788077
]
Shai Erera commented on LUCENE-2104:
------------------------------------
I think that if I move those lines (in NativeFSLock.release()):
{code}
if (!path.delete())
throw new LockReleaseFailedException("failed to delete " + path);
{code}
to outside the if(lockExists()) section, this should work? Because then the new NativeFSLock
will attempt to release an lock that's held by someone else, and fail. If the lock exists
for some reason, but nobody is holding it, that line should succeed.
In order to test it, I think I'll need to spawn two processes, which is trickier. Let me know
what you think about the fix in the meantime.
> IndexWriter.unlock does does nothing if NativeFSLockFactory is used
> -------------------------------------------------------------------
>
> Key: LUCENE-2104
> URL: https://issues.apache.org/jira/browse/LUCENE-2104
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Shai Erera
> Fix For: 3.1
>
>
> If NativeFSLockFactory is used, IndexWriter.unlock will return, silently doing nothing.
The reason is that NativeFSLockFactory's makeLock always creates a new NativeFSLock. NativeFSLock's
release first checks if its lock is not null. However, only if obtain() is called, that lock
is not null. So release actually does nothing, and so IndexWriter.unlock does not delete the
lock, or fail w/ exception.
> This is only a problem in NativeFSLock, and not in other Lock implementations, at least
as I was able to see.
> Need to think first how to reproduce in a test, and then fix it. I'll work on it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-2117.
-------------------------------------
Resolution: Fixed
committed in revision 888787
thanks robert
> Fix SnowballAnalyzer casing behavior for Turkish Language
> ---------------------------------------------------------
>
> Key: LUCENE-2117
> URL: https://issues.apache.org/jira/browse/LUCENE-2117
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2117.patch, LUCENE-2117.patch
>
>
> LUCENE-2102 added a new TokenFilter to handle Turkish unique casing behavior correctly.
We should fix the casing behavior in SnowballAnalyzer too as it supports a TurkishStemmer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788054#action_12788054
]
Robert Muir commented on LUCENE-2124:
-------------------------------------
Committed revision 888780.
I will keep this open until i regen the website and commit the changes.
> move JDK collation to core, ICU collation to ICU contrib
> --------------------------------------------------------
>
> Key: LUCENE-2124
> URL: https://issues.apache.org/jira/browse/LUCENE-2124
> Project: Lucene - Java
> Issue Type: Task
> Components: contrib/*, Search
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2124.patch, LUCENE-2124.patch
>
>
> As mentioned on the list, I propose we move the JDK-based CollationKeyFilter/CollationKeyAnalyzer,
currently located in contrib/collation into core for collation support (language-sensitive
sorting)
> These are not much code (the heavy duty stuff is already in core, IndexableBinaryString).
> And I would also like to move the ICUCollationKeyFilter/ICUCollationKeyAnalyzer (along
with the jar file they depend on) also currently located in contrib/collation into a contrib/icu.
> This way, we can start looking at integrating other functionality from ICU into a fully-fleshed
out icu contrib.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2123:
----------------------------------
Attachment: LUCENE-2123-flex.patch
Now i also made the strange anonymous inner class a named inner class to get rid of the strange
boolean holder, implemented by an array.
> Highlighter fails to highlight FuzzyQuery
> -----------------------------------------
>
> Key: LUCENE-2123
> URL: https://issues.apache.org/jira/browse/LUCENE-2123
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/highlighter
> Affects Versions: Flex Branch
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: Flex Branch
>
> Attachments: LUCENE-2123-flex.patch, LUCENE-2123-flex.patch, LUCENE-2123-flex.patch
>
>
> As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in
flex since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is:
Allow MTQ to change rewrite method and make FUZZY_REWRITE public for that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2123:
----------------------------------
Attachment: LUCENE-2123-flex.patch
More refactoring. No also AUTO_REWRITE uses the new TermCollector. It gets less and less code.
> Highlighter fails to highlight FuzzyQuery
> -----------------------------------------
>
> Key: LUCENE-2123
> URL: https://issues.apache.org/jira/browse/LUCENE-2123
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/highlighter
> Affects Versions: Flex Branch
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: Flex Branch
>
> Attachments: LUCENE-2123-flex.patch, LUCENE-2123-flex.patch
>
>
> As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in
flex since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is:
Allow MTQ to change rewrite method and make FUZZY_REWRITE public for that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2123:
----------------------------------
Attachment: LUCENE-2123-flex.patch
Here a refactoring of the rewrite modes in Flex. I'll port to trunk, too.
FuzzyQuery now uses per default TOP_TERMS_SCORING_BOOLEAN_REWRITE which is part of MTQ and
can now also be used by e.g. MoreLikeThis.
> Highlighter fails to highlight FuzzyQuery
> -----------------------------------------
>
> Key: LUCENE-2123
> URL: https://issues.apache.org/jira/browse/LUCENE-2123
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/highlighter
> Affects Versions: Flex Branch
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: Flex Branch
>
> Attachments: LUCENE-2123-flex.patch
>
>
> As FuzzyQuery does not allow to change the rewrite mode, highlighter fails with UOE in
flex since LUCENE-2110, because it changes the rewrite mode to Boolean query. The fix is:
Allow MTQ to change rewrite method and make FUZZY_REWRITE public for that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Ingersoll resolved LUCENE-1512.
-------------------------------------
Resolution: Fixed
Fix Version/s: 2.9
Lucene Fields: (was: [New])
> Incorporate GeoHash in contrib/spatial
> --------------------------------------
>
> Key: LUCENE-1512
> URL: https://issues.apache.org/jira/browse/LUCENE-1512
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/spatial
> Reporter: patrick o'leary
> Assignee: Ryan McKinley
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1512.patch, LUCENE-1512.patch
>
>
> Based on comments from Yonik and Ryan in SOLR-773
> GeoHash provides the ability to store latitude / longitude values in a single field consistent
hash field.
> Which elements the need to maintain 2 field caches for latitude / longitude fields, reducing
the size of an index
> and the amount of memory needed for a spatial search.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788040#action_12788040
]
Simon Willnauer commented on LUCENE-2124:
-----------------------------------------
Robert patch looks good to me!
Go for it!
> move JDK collation to core, ICU collation to ICU contrib
> --------------------------------------------------------
>
> Key: LUCENE-2124
> URL: https://issues.apache.org/jira/browse/LUCENE-2124
> Project: Lucene - Java
> Issue Type: Task
> Components: contrib/*, Search
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2124.patch, LUCENE-2124.patch
>
>
> As mentioned on the list, I propose we move the JDK-based CollationKeyFilter/CollationKeyAnalyzer,
currently located in contrib/collation into core for collation support (language-sensitive
sorting)
> These are not much code (the heavy duty stuff is already in core, IndexableBinaryString).
> And I would also like to move the ICUCollationKeyFilter/ICUCollationKeyAnalyzer (along
with the jar file they depend on) also currently located in contrib/collation into a contrib/icu.
> This way, we can start looking at integrating other functionality from ICU into a fully-fleshed
out icu contrib.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788037#action_12788037
]
Simon Willnauer commented on LUCENE-1512:
-----------------------------------------
Is this isssue still relevant? seems like it has been committted
> Incorporate GeoHash in contrib/spatial
> --------------------------------------
>
> Key: LUCENE-1512
> URL: https://issues.apache.org/jira/browse/LUCENE-1512
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/spatial
> Reporter: patrick o'leary
> Assignee: Ryan McKinley
> Priority: Minor
> Attachments: LUCENE-1512.patch, LUCENE-1512.patch
>
>
> Based on comments from Yonik and Ryan in SOLR-773
> GeoHash provides the ability to store latitude / longitude values in a single field consistent
hash field.
> Which elements the need to maintain 2 field caches for latitude / longitude fields, reducing
the size of an index
> and the amount of memory needed for a spatial search.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer reassigned LUCENE-2139:
---------------------------------------
Assignee: Simon Willnauer
> Cleanup and Improvement of Spatial Contrib
> ------------------------------------------
>
> Key: LUCENE-2139
> URL: https://issues.apache.org/jira/browse/LUCENE-2139
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/spatial
> Affects Versions: 3.1
> Reporter: Chris Male
> Assignee: Simon Willnauer
>
> The current spatial contrib can be improved by adding documentation, tests, removing
unused classes and code, repackaging the classes and improving the performance of the distance
filtering. The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
> Other improvements involve adding better support for different distance units, different
distance calculators and different data formats (whether it be lat/long fields, geohashes,
or something else in the future).
> Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788034#action_12788034
]
Simon Willnauer commented on LUCENE-2139:
-----------------------------------------
cant wait to see you patch - its gonna be huge I guess :)
I will be here to help you splitting it apart and get you good work into contrib/spatial
> Cleanup and Improvement of Spatial Contrib
> ------------------------------------------
>
> Key: LUCENE-2139
> URL: https://issues.apache.org/jira/browse/LUCENE-2139
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/spatial
> Affects Versions: 3.1
> Reporter: Chris Male
> Assignee: Simon Willnauer
>
> The current spatial contrib can be improved by adding documentation, tests, removing
unused classes and code, repackaging the classes and improving the performance of the distance
filtering. The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
> Other improvements involve adding better support for different distance units, different
distance calculators and different data formats (whether it be lat/long fields, geohashes,
or something else in the future).
> Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Cleanup and Improvement of Spatial Contrib
------------------------------------------
Key: LUCENE-2139
URL: https://issues.apache.org/jira/browse/LUCENE-2139
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/spatial
Affects Versions: 3.1
Reporter: Chris Male
The current spatial contrib can be improved by adding documentation, tests, removing unused
classes and code, repackaging the classes and improving the performance of the distance filtering.
The latter will incorporate the multi-threaded functionality introduced in LUCENE-1732.
Other improvements involve adding better support for different distance units, different distance
calculators and different data formats (whether it be lat/long fields, geohashes, or something
else in the future).
Patch to be added soon.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-2107.
-------------------------------------
Resolution: Fixed
Fix Version/s: 2.9.1
3.0
The missing maven artifacts for the fast-vector-highlighter contrib of
Lucene Java in version 2.9.1 and 3.0.0 are now available at:
http://repo1.maven.org/maven2/org/apache/lucene/
http://repo2.maven.org/maven2/org/apache/lucene/
> Add contrib/fast-vector-highlighter to Maven central repo
> ---------------------------------------------------------
>
> Key: LUCENE-2107
> URL: https://issues.apache.org/jira/browse/LUCENE-2107
> Project: Lucene - Java
> Issue Type: Task
> Components: contrib/*
> Affects Versions: 2.9.1, 3.0
> Reporter: Chas Emerick
> Assignee: Simon Willnauer
> Fix For: 3.0, 2.9.1
>
> Attachments: LUCENE-2107.patch
>
>
> I'm not at all familiar with the Lucene build/deployment process, but it would be very
nice if releases of the fast vector highlighter were pushed to the maven central repository,
as is done with other contrib modules.
> (Issue filed at the request of Grant Ingersoll.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
nice - I closed the issue. thanks uwe On Wed, Dec 9, 2009 at 10:28 AM, Uwe Schindler <uwe@thetaphi.de> wrote: > Hi all, > > The missing maven artifacts for the fast-vector-highlighter contrib of > Lucene Java in version 2.9.1 and 3.0.0 are now available at: > > http://repo1.maven.org/maven2/org/apache/lucene/ > http://repo2.maven.org/maven2/org/apache/lucene/ > > Uwe > > ----- > Uwe Schindler > uschindler@apache.org > Apache Lucene Java Committer > Bremen, Germany > http://lucene.apache.org/java/docs/ > >> From: Uwe Schindler [mailto:uwe@thetaphi.de] >> Sent: Tuesday, December 08, 2009 10:41 PM >> To: java-dev@lucene.apache.org; general@lucene.apache.org >> Subject: RE: [VOTE] Push fast-vector-highlighter mvn artifacts for 3.0.0 >> and 2.9.1 >> >> I got 3 binding votes from Grant, Mike, and Ted (and one from Simon, who >> was >> a big help on Sunday evening when I created the artifacts), so I push the >> maven artifacts onto the rsync repo in few minutes. >> >> Thanks! >> >> ----- >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: uwe@thetaphi.de >> >> > -----Original Message----- >> > From: Uwe Schindler [mailto:uwe@thetaphi.de] >> > Sent: Tuesday, December 08, 2009 7:03 PM >> > To: java-dev@lucene.apache.org >> > Subject: [VOTE] Push fast-vector-highlighter mvn artifacts for 3.0.0 and >> > 2.9.1 >> > >> > Sorry, >> > >> > I initially didn't want to start a vote, as Grant only proposed to >> "maybe >> > start one". But nobody responded (esp. to the questions in this mail) I >> > ask >> > again, an I will start the vote for now. >> > >> > >> ========================================================================== >> > == >> > Please vote, that the missing artifacts for of fast-verctor-highlighter >> of >> > Lucene Java 2.9.1 and 3.0.0 should be pushed to repoX.maven.org. >> > >> > You can find the artifacts here: >> > http://people.apache.org/~uschindler/staging-area/ >> > >> > This dir contains only the maven folder to be copied to maven-rsync dir >> on >> > p.a.o. The top-level version in the maven metadata is 3.0.0, which >> > conforms >> > to the current state on maven (so during merging both folders during >> > build, >> > I set preference to metadata.xml of 3.0.0). >> > >> > All files are signed by my PGP key (even the 2.9.1 ones; that release >> was >> > originally built by Mike McCandless). >> > >> ========================================================================== >> > == >> > >> > What I additionally found out until now (because Simon nagged me): >> > >> > If you compare the JAR files inside the binary ZIP file from the apache >> > archive and the JAR files directly published on maven (for the other >> > contribs), the MD5s/SHA1s are different even as they are created from >> the >> > same source code (because the timestamps inside the JAR are different, >> for >> > 2.9.1 another JDK compiler/platform was used). This interestingly does >> not >> > apply to lucene-core.jar in 3.0. Because of that I see no problem with >> > this >> > maven release, even that they are not the orginal JAR files from the >> > binary >> > distrib. >> > >> > What is not nice, is that the svn revision number in the manifest is >> > different, but else is exactly the same, see my comments below in >> earlier >> > mails about changing the ant script for showing the SVN rev of the last >> > changed file. >> > >> > So if nobody objects to release these rebuild jar files, all signed by >> my >> > key, I would like to simply put them on the maven-rsync folder. >> > >> > Uwe >> > >> > ----- >> > Uwe Schindler >> > H.-H.-Meier-Allee 63, D-28213 Bremen >> > http://www.thetaphi.de >> > eMail: uwe@thetaphi.de >> > >> > >> > > -----Original Message----- >> > > From: Chris Hostetter [mailto:hossman_lucene@fucit.org] >> > > Sent: Tuesday, December 08, 2009 6:48 PM >> > > To: java-dev@lucene.apache.org >> > > Subject: Re: (NAG) Push fast-vector-highlighter mvn artifacts for 3.0 >> > and >> > > 2.9 >> > > >> > > >> > > : What to do now, any votes on adding the missing maven artifacts for >> > > : fast-vector-highlighter to 2.9.1 and 3.0.0 on the apache maven >> > > reposititory? >> > > >> > > It's not even clear to me that anything special needs to be done >> before >> > > publishing those jars to maven. 2.9.1 and 3.0.0 were already voted on >> > and >> > > released -- including all of the source code in them. >> > > >> > > The safest bet least likely to anger the process gods is just to call >> a >> > > vote (new thread with VOTE in the subject) and cast a vote ... >> > considering >> > > the sources has already been reviewed it should go pretty quick. >> > > >> > > : >> > > : > I rebuilt the maven-dir for 2.9.1 and 3.0.0, merged them (3.0.0 is >> > > top- >> > > : > level >> > > : > version) and extracted only fast-vector-highlighter: >> > > : > >> > > : > http://people.apache.org/~uschindler/staging-area/ >> > > : > >> > > : > I will copy this dir to the maven folder on people.a.o, when I got >> > > votes >> > > : > (how many)? At least someone should check the signatures. >> > > : > >> > > : > By the way, we have a small error in our ant build.xml that >> inserts >> > > : > svnversion into the manifest file. This version is not the version >> > of >> > > the >> > > : > last changed item (would be svnversion -c) but the current svn >> > > version, >> > > : > even >> > > : > that I checked out the corresponding tags. It's no problem at all, >> > but >> > > not >> > > : > very nice. >> > > : > >> > > : > Maybe we should change build.xml to call "svnversion -c" in >> future, >> > to >> > > get >> > > : > the real number. >> > > : > >> > > : > Uwe >> > > : > >> > > : > ----- >> > > : > Uwe Schindler >> > > : > H.-H.-Meier-Allee 63, D-28213 Bremen >> > > : > http://www.thetaphi.de >> > > : > eMail: uwe@thetaphi.de >> > > : > >> > > : > >> > > : > > -----Original Message----- >> > > : > > From: Grant Ingersoll [mailto:gsingers@apache.org] >> > > : > > Sent: Saturday, December 05, 2009 10:26 PM >> > > : > > To: java-dev@lucene.apache.org >> > > : > > Subject: Re: Push fast-vector-highlighter mvn artifacts for 3.0 >> > and >> > > 2.9 >> > > : > > >> > > : > > I suppose we could put up the artifacts on a dev site and then >> we >> > > could >> > > : > > vote to release both of them pretty quickly. I think that >> should >> > be >> > > : > easy >> > > : > > to do, since it pretty much only involves verifying the jar and >> > the >> > > : > > signatures. >> > > : > > >> > > : > > On Dec 5, 2009, at 1:03 PM, Simon Willnauer wrote: >> > > : > > >> > > : > > > hi folks, >> > > : > > > The maven artifacts for fast-vector-highlighter have never >> been >> > > pushed >> > > : > > > since it was released because there were no pom.xml.template >> > > inside >> > > : > > > the module. I added a pom file a day ago in the context of >> > > : > > > LUCENE-2107. I already talked to uwe and grant how to deal >> with >> > > this >> > > : > > > issues and if we should push the artifact for Lucene 2.9 / >> 3.0. >> > > Since >> > > : > > > this is only a metadata file we could consider rebuilding the >> > > : > > > artefacts and publish them for those releases. I can not >> > remember >> > > that >> > > : > > > anything like that happened before, so we should discuss how >> to >> > > deal >> > > : > > > with this situation and if we should wait until 3.1. >> > > : > > > >> > > : > > > simon >> > > : > > > >> > > : > > > -------------------------------------------------------------- >> -- >> > -- >> > > --- >> > > : > > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > > : > > > For additional commands, e-mail: java-dev- >> help@lucene.apache.org >> > > : > > > >> > > : > > >> > > : > > >> > > : > > >> > > : > > ---------------------------------------------------------------- >> -- >> > -- >> > > - >> > > : > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > > : > > For additional commands, e-mail: java-dev-help@lucene.apache.org >> > > : > >> > > : > >> > > : > >> > > : > ------------------------------------------------------------------ >> -- >> > - >> > > : > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > > : > For additional commands, e-mail: java-dev-help@lucene.apache.org >> > > : >> > > : >> > > : >> > > : -------------------------------------------------------------------- >> - >> > > : To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > > : For additional commands, e-mail: java-dev-help@lucene.apache.org >> > > : >> > > >> > > >> > > >> > > -Hoss >> > > >> > > >> > > --------------------------------------------------------------------- >> > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > > For additional commands, e-mail: java-dev-help@lucene.apache.org >> > >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org >> > For additional commands, e-mail: java-dev-help@lucene.apache.org >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-dev-help@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788018#action_12788018
]
Michael McCandless commented on LUCENE-2138:
--------------------------------------------
Could we maybe instead factor out ReaderPool from IW, and somehow enable this extensibility,
there?
This would be the first step in LUCENE-2026, I guess.
The mergedSegmentWarmer should then also go into ReaderPool.
> Allow custom index readers when using IndexWriter.getReader
> -----------------------------------------------------------
>
> Key: LUCENE-2138
> URL: https://issues.apache.org/jira/browse/LUCENE-2138
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 3.0
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2138.patch
>
>
> This is needed for backwards compatible support with Solr, and is a spin-off from SOLR-1606.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788001#action_12788001
]
Michael Busch commented on LUCENE-2126:
---------------------------------------
The main reason why I'd like to separate DataInput/Output from IndexInput/Output now is LUCENE-2125.
Users should be able to implement methods that serialize/deserialize attributes into/from
a postinglist. These methods should only be able to call the read/write methods (which this
issue moves to DataInput/Output), but not methods like close(), seek() etc..
Thanks for spending time reviewing this and giving feedback from Lucy land, Marvin!
I think I will go ahead and commit this, and once we see a need to allow users to extend DataInput/Output
outside of Lucene we can go ahead and make the additional changes that are mentioned in your
in my comments here.
So I will commit this tomorrow if nobody objects.
> Split up IndexInput and IndexOutput into DataInput and DataOutput
> -----------------------------------------------------------------
>
> Key: LUCENE-2126
> URL: https://issues.apache.org/jira/browse/LUCENE-2126
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: Flex Branch
> Reporter: Michael Busch
> Assignee: Michael Busch
> Priority: Minor
> Fix For: Flex Branch
>
> Attachments: lucene-2126.patch
>
>
> I'd like to introduce the two new classes DataInput and DataOutput
> that contain all methods from IndexInput and IndexOutput that actually
> decode or encode data, such as readByte()/writeByte(),
> readVInt()/writeVInt().
> Methods like getFilePointer(), seek(), close(), etc., which are not
> related to data encoding, but to files as input/output source stay in
> IndexInput/IndexOutput.
> This patch also changes ByteSliceReader/ByteSliceWriter to extend
> DataInput/DataOutput. Previously ByteSliceReader implemented the
> methods that stay in IndexInput by throwing RuntimeExceptions.
> See also LUCENE-2125.
> All tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Hi all, The missing maven artifacts for the fast-vector-highlighter contrib of Lucene Java in version 2.9.1 and 3.0.0 are now available at: http://repo1.maven.org/maven2/org/apache/lucene/ http://repo2.maven.org/maven2/org/apache/lucene/ Uwe ----- Uwe Schindler uschindler@apache.org Apache Lucene Java Committer Bremen, Germany http://lucene.apache.org/java/docs/ > From: Uwe Schindler [mailto:uwe@thetaphi.de] > Sent: Tuesday, December 08, 2009 10:41 PM > To: java-dev@lucene.apache.org; general@lucene.apache.org > Subject: RE: [VOTE] Push fast-vector-highlighter mvn artifacts for 3.0.0 > and 2.9.1 > > I got 3 binding votes from Grant, Mike, and Ted (and one from Simon, who > was > a big help on Sunday evening when I created the artifacts), so I push the > maven artifacts onto the rsync repo in few minutes. > > Thanks! > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: uwe@thetaphi.de > > > -----Original Message----- > > From: Uwe Schindler [mailto:uwe@thetaphi.de] > > Sent: Tuesday, December 08, 2009 7:03 PM > > To: java-dev@lucene.apache.org > > Subject: [VOTE] Push fast-vector-highlighter mvn artifacts for 3.0.0 and > > 2.9.1 > > > > Sorry, > > > > I initially didn't want to start a vote, as Grant only proposed to > "maybe > > start one". But nobody responded (esp. to the questions in this mail) I > > ask > > again, an I will start the vote for now. > > > > > ========================================================================== > > == > > Please vote, that the missing artifacts for of fast-verctor-highlighter > of > > Lucene Java 2.9.1 and 3.0.0 should be pushed to repoX.maven.org. > > > > You can find the artifacts here: > > http://people.apache.org/~uschindler/staging-area/ > > > > This dir contains only the maven folder to be copied to maven-rsync dir > on > > p.a.o. The top-level version in the maven metadata is 3.0.0, which > > conforms > > to the current state on maven (so during merging both folders during > > build, > > I set preference to metadata.xml of 3.0.0). > > > > All files are signed by my PGP key (even the 2.9.1 ones; that release > was > > originally built by Mike McCandless). > > > ========================================================================== > > == > > > > What I additionally found out until now (because Simon nagged me): > > > > If you compare the JAR files inside the binary ZIP file from the apache > > archive and the JAR files directly published on maven (for the other > > contribs), the MD5s/SHA1s are different even as they are created from > the > > same source code (because the timestamps inside the JAR are different, > for > > 2.9.1 another JDK compiler/platform was used). This interestingly does > not > > apply to lucene-core.jar in 3.0. Because of that I see no problem with > > this > > maven release, even that they are not the orginal JAR files from the > > binary > > distrib. > > > > What is not nice, is that the svn revision number in the manifest is > > different, but else is exactly the same, see my comments below in > earlier > > mails about changing the ant script for showing the SVN rev of the last > > changed file. > > > > So if nobody objects to release these rebuild jar files, all signed by > my > > key, I would like to simply put them on the maven-rsync folder. > > > > Uwe > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: uwe@thetaphi.de > > > > > > > -----Original Message----- > > > From: Chris Hostetter [mailto:hossman_lucene@fucit.org] > > > Sent: Tuesday, December 08, 2009 6:48 PM > > > To: java-dev@lucene.apache.org > > > Subject: Re: (NAG) Push fast-vector-highlighter mvn artifacts for 3.0 > > and > > > 2.9 > > > > > > > > > : What to do now, any votes on adding the missing maven artifacts for > > > : fast-vector-highlighter to 2.9.1 and 3.0.0 on the apache maven > > > reposititory? > > > > > > It's not even clear to me that anything special needs to be done > before > > > publishing those jars to maven. 2.9.1 and 3.0.0 were already voted on > > and > > > released -- including all of the source code in them. > > > > > > The safest bet least likely to anger the process gods is just to call > a > > > vote (new thread with VOTE in the subject) and cast a vote ... > > considering > > > the sources has already been reviewed it should go pretty quick. > > > > > > : > > > : > I rebuilt the maven-dir for 2.9.1 and 3.0.0, merged them (3.0.0 is > > > top- > > > : > level > > > : > version) and extracted only fast-vector-highlighter: > > > : > > > > : > http://people.apache.org/~uschindler/staging-area/ > > > : > > > > : > I will copy this dir to the maven folder on people.a.o, when I got > > > votes > > > : > (how many)? At least someone should check the signatures. > > > : > > > > : > By the way, we have a small error in our ant build.xml that > inserts > > > : > svnversion into the manifest file. This version is not the version > > of > > > the > > > : > last changed item (would be svnversion -c) but the current svn > > > version, > > > : > even > > > : > that I checked out the corresponding tags. It's no problem at all, > > but > > > not > > > : > very nice. > > > : > > > > : > Maybe we should change build.xml to call "svnversion -c" in > future, > > to > > > get > > > : > the real number. > > > : > > > > : > Uwe > > > : > > > > : > ----- > > > : > Uwe Schindler > > > : > H.-H.-Meier-Allee 63, D-28213 Bremen > > > : > http://www.thetaphi.de > > > : > eMail: uwe@thetaphi.de > > > : > > > > : > > > > : > > -----Original Message----- > > > : > > From: Grant Ingersoll [mailto:gsingers@apache.org] > > > : > > Sent: Saturday, December 05, 2009 10:26 PM > > > : > > To: java-dev@lucene.apache.org > > > : > > Subject: Re: Push fast-vector-highlighter mvn artifacts for 3.0 > > and > > > 2.9 > > > : > > > > > : > > I suppose we could put up the artifacts on a dev site and then > we > > > could > > > : > > vote to release both of them pretty quickly. I think that > should > > be > > > : > easy > > > : > > to do, since it pretty much only involves verifying the jar and > > the > > > : > > signatures. > > > : > > > > > : > > On Dec 5, 2009, at 1:03 PM, Simon Willnauer wrote: > > > : > > > > > : > > > hi folks, > > > : > > > The maven artifacts for fast-vector-highlighter have never > been > > > pushed > > > : > > > since it was released because there were no pom.xml.template > > > inside > > > : > > > the module. I added a pom file a day ago in the context of > > > : > > > LUCENE-2107. I already talked to uwe and grant how to deal > with > > > this > > > : > > > issues and if we should push the artifact for Lucene 2.9 / > 3.0. > > > Since > > > : > > > this is only a metadata file we could consider rebuilding the > > > : > > > artefacts and publish them for those releases. I can not > > remember > > > that > > > : > > > anything like that happened before, so we should discuss how > to > > > deal > > > : > > > with this situation and if we should wait until 3.1. > > > : > > > > > > : > > > simon > > > : > > > > > > : > > > -------------------------------------------------------------- > -- > > -- > > > --- > > > : > > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > > : > > > For additional commands, e-mail: java-dev- > help@lucene.apache.org > > > : > > > > > > : > > > > > : > > > > > : > > > > > : > > ---------------------------------------------------------------- > -- > > -- > > > - > > > : > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > > : > > For additional commands, e-mail: java-dev-help@lucene.apache.org > > > : > > > > : > > > > : > > > > : > ------------------------------------------------------------------ > -- > > - > > > : > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > > : > For additional commands, e-mail: java-dev-help@lucene.apache.org > > > : > > > : > > > : > > > : -------------------------------------------------------------------- > - > > > : To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > > : For additional commands, e-mail: java-dev-help@lucene.apache.org > > > : > > > > > > > > > > > > -Hoss > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > > For additional commands, e-mail: java-dev-help@lucene.apache.org > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org > > For additional commands, e-mail: java-dev-help@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787989#action_12787989
]
Uwe Schindler commented on LUCENE-2089:
---------------------------------------
Cool! The code looks quite simple (but maybe this is because of n=1). But FuzzyQuery with
n>1 are used seldom, or not? And how slow it is?
> explore using automaton for fuzzyquery
> --------------------------------------
>
> Key: LUCENE-2089
> URL: https://issues.apache.org/jira/browse/LUCENE-2089
> Project: Lucene - Java
> Issue Type: Wish
> Components: Search
> Reporter: Robert Muir
> Assignee: Mark Miller
> Priority: Minor
> Attachments: Moman-0.2.1.tar.gz, TestFuzzy.java
>
>
> Mark brought this up on LUCENE-1606 (i will assign this to him, I know he is itching
to write that nasty algorithm)
> we can optimize fuzzyquery by using AutomatonTermEnum, here is my idea
> * up front, calculate the maximum required K edits needed to match the users supplied
float threshold.
> * for at least common K (1,2,3, etc) we should use automatontermenum. if its outside
of that, maybe use the existing slow logic. At high K, it will seek too much to be helpful
anyway.
> i modified my wildcard benchmark to generate random fuzzy queries.
> * Pattern: 7N stands for NNNNNNN, etc.
> * AvgMS_DFA: this is the time spent creating the automaton (constructor)
> ||Pattern||Iter||AvgHits||AvgMS(old)||AvgMS (new,total)||AvgMS_DFA||
> |7N|10|64.0|4155.9|38.6|20.3|
> |14N|10|0.0|2511.6|46.0|37.9|
> |28N|10|0.0|2506.3|93.0|86.6|
> |56N|10|0.0|2524.5|304.4|298.5|
> as you can see, this prototype is no good yet, because it creates the DFA in a slow way.
right now it creates an NFA, and all this wasted time is in NFA->DFA conversion.
> So, for a very long string, it just gets worse and worse. This has nothing to do with
lucene, and here you can see, the TermEnum is fast (AvgMS - AvgMS_DFA), there is no problem
there.
> instead we should just build a DFA to begin with, maybe with this paper: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.16.652
> we can precompute the tables with that algorithm up to some reasonable K, and then I
think we are ok.
> the paper references using http://portal.acm.org/citation.cfm?id=135907 for linear minimization,
if someone wants to implement this they should not worry about minimization.
> in fact, we need to at some point determine if AutomatonQuery should even minimize FSM's
at all, or if it is simply enough for them to be deterministic with no transitions to dead
states. (The only code that actually assumes minimal DFA is the "Dumb" vs "Smart" heuristic
and this can be rewritten as a summation easily). we need to benchmark really complex DFAs
(i.e. write a regex benchmark) to figure out if minimization is even helping right now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787973#action_12787973
]
Wouter Heijke commented on LUCENE-1815:
---------------------------------------
I'm happily using now for some time:
http://code.google.com/p/geospatialweb/source/browse/trunk/geohash/src/Geohash.java
> Geohash encode/decode floating point problems
> ---------------------------------------------
>
> Key: LUCENE-1815
> URL: https://issues.apache.org/jira/browse/LUCENE-1815
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/spatial
> Affects Versions: 2.9
> Reporter: Wouter Heijke
> Priority: Minor
>
> i'm finding the Geohash support in the spatial package to be rather unreliable.
> Here is the outcome of a test that encodes/decodes the same lat/lon and geohash a few
times.
> the format:
> action geohash=(latitude, longitude)
> the result:
> encode u173zq37x014=(52.3738007,4.8909347)
> decode u173zq37x014=(52.373799999999996,4.890934)
> encode u173zq37rpbw=(52.373799999999996,4.890934)
> decode u173zq37rpbw=(52.373799999999996,4.8909329999999995)
> encode u173zq37qzzy=(52.373799999999996,4.8909329999999995)
> if I now change to the google code implementation:
> encode u173zq37x014=(52.3738007,4.8909347)
> decode u173zq37x014=(52.37380061298609,4.890934377908707)
> encode u173zq37x014=(52.37380061298609,4.890934377908707)
> decode u173zq37x014=(52.37380061298609,4.890934377908707)
> encode u173zq37x014=(52.37380061298609,4.890934377908707)
> Note the differences between the geohashes in both situations and the lat/lon's!
> Now things get worse if you work on low-precision geohashes:
> decode u173=(52.0,4.0)
> encode u14zg429yy84=(52.0,4.0)
> decode u14zg429yy84=(52.0,3.999999)
> encode u14zg429ywx6=(52.0,3.999999)
> and google:
> decode u173=(52.20703125,4.5703125)
> encode u17300000000=(52.20703125,4.5703125)
> decode u17300000000=(52.20703125,4.5703125)
> encode u17300000000=(52.20703125,4.5703125)
> We are using geohashes extensively and will now use the google code version unfortunately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787934#action_12787934
]
patrick o'leary commented on LUCENE-1815:
-----------------------------------------
What google code are you working with?
> Geohash encode/decode floating point problems
> ---------------------------------------------
>
> Key: LUCENE-1815
> URL: https://issues.apache.org/jira/browse/LUCENE-1815
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/spatial
> Affects Versions: 2.9
> Reporter: Wouter Heijke
> Priority: Minor
>
> i'm finding the Geohash support in the spatial package to be rather unreliable.
> Here is the outcome of a test that encodes/decodes the same lat/lon and geohash a few
times.
> the format:
> action geohash=(latitude, longitude)
> the result:
> encode u173zq37x014=(52.3738007,4.8909347)
> decode u173zq37x014=(52.373799999999996,4.890934)
> encode u173zq37rpbw=(52.373799999999996,4.890934)
> decode u173zq37rpbw=(52.373799999999996,4.8909329999999995)
> encode u173zq37qzzy=(52.373799999999996,4.8909329999999995)
> if I now change to the google code implementation:
> encode u173zq37x014=(52.3738007,4.8909347)
> decode u173zq37x014=(52.37380061298609,4.890934377908707)
> encode u173zq37x014=(52.37380061298609,4.890934377908707)
> decode u173zq37x014=(52.37380061298609,4.890934377908707)
> encode u173zq37x014=(52.37380061298609,4.890934377908707)
> Note the differences between the geohashes in both situations and the lat/lon's!
> Now things get worse if you work on low-precision geohashes:
> decode u173=(52.0,4.0)
> encode u14zg429yy84=(52.0,4.0)
> decode u14zg429yy84=(52.0,3.999999)
> encode u14zg429ywx6=(52.0,3.999999)
> and google:
> decode u173=(52.20703125,4.5703125)
> encode u17300000000=(52.20703125,4.5703125)
> decode u17300000000=(52.20703125,4.5703125)
> encode u17300000000=(52.20703125,4.5703125)
> We are using geohashes extensively and will now use the google code version unfortunately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-2089:
--------------------------------
Attachment: TestFuzzy.java
attached is a testcase that builds n=1 for some string the slow way.
you can use this to verify that some faster method is correct, via assertEquals
> explore using automaton for fuzzyquery
> --------------------------------------
>
> Key: LUCENE-2089
> URL: https://issues.apache.org/jira/browse/LUCENE-2089
> Project: Lucene - Java
> Issue Type: Wish
> Components: Search
> Reporter: Robert Muir
> Assignee: Mark Miller
> Priority: Minor
> Attachments: Moman-0.2.1.tar.gz, TestFuzzy.java
>
>
> Mark brought this up on LUCENE-1606 (i will assign this to him, I know he is itching
to write that nasty algorithm)
> we can optimize fuzzyquery by using AutomatonTermEnum, here is my idea
> * up front, calculate the maximum required K edits needed to match the users supplied
float threshold.
> * for at least common K (1,2,3, etc) we should use automatontermenum. if its outside
of that, maybe use the existing slow logic. At high K, it will seek too much to be helpful
anyway.
> i modified my wildcard benchmark to generate random fuzzy queries.
> * Pattern: 7N stands for NNNNNNN, etc.
> * AvgMS_DFA: this is the time spent creating the automaton (constructor)
> ||Pattern||Iter||AvgHits||AvgMS(old)||AvgMS (new,total)||AvgMS_DFA||
> |7N|10|64.0|4155.9|38.6|20.3|
> |14N|10|0.0|2511.6|46.0|37.9|
> |28N|10|0.0|2506.3|93.0|86.6|
> |56N|10|0.0|2524.5|304.4|298.5|
> as you can see, this prototype is no good yet, because it creates the DFA in a slow way.
right now it creates an NFA, and all this wasted time is in NFA->DFA conversion.
> So, for a very long string, it just gets worse and worse. This has nothing to do with
lucene, and here you can see, the TermEnum is fast (AvgMS - AvgMS_DFA), there is no problem
there.
> instead we should just build a DFA to begin with, maybe with this paper: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.16.652
> we can precompute the tables with that algorithm up to some reasonable K, and then I
think we are ok.
> the paper references using http://portal.acm.org/citation.cfm?id=135907 for linear minimization,
if someone wants to implement this they should not worry about minimization.
> in fact, we need to at some point determine if AutomatonQuery should even minimize FSM's
at all, or if it is simply enough for them to be deterministic with no transitions to dead
states. (The only code that actually assumes minimal DFA is the "Dumb" vs "Smart" heuristic
and this can be rewritten as a summation easily). we need to benchmark really complex DFAs
(i.e. write a regex benchmark) to figure out if minimization is even helping right now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787876#action_12787876
]
Marvin Humphrey commented on LUCENE-2126:
-----------------------------------------
I spent a long time today trying to understand why DataInput and
DataOutput are justified so that I could formulate an intelligent reply,
but I had to give up. :\ Please carry on.
> Split up IndexInput and IndexOutput into DataInput and DataOutput
> -----------------------------------------------------------------
>
> Key: LUCENE-2126
> URL: https://issues.apache.org/jira/browse/LUCENE-2126
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: Flex Branch
> Reporter: Michael Busch
> Assignee: Michael Busch
> Priority: Minor
> Fix For: Flex Branch
>
> Attachments: lucene-2126.patch
>
>
> I'd like to introduce the two new classes DataInput and DataOutput
> that contain all methods from IndexInput and IndexOutput that actually
> decode or encode data, such as readByte()/writeByte(),
> readVInt()/writeVInt().
> Methods like getFilePointer(), seek(), close(), etc., which are not
> related to data encoding, but to files as input/output source stay in
> IndexInput/IndexOutput.
> This patch also changes ByteSliceReader/ByteSliceWriter to extend
> DataInput/DataOutput. Previously ByteSliceReader implemented the
> methods that stay in IndexInput by throwing RuntimeExceptions.
> See also LUCENE-2125.
> All tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[ https://issues.apache.org/jira/browse/LUCENE-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Rutherglen updated LUCENE-2138:
-------------------------------------
Attachment: LUCENE-2138.patch
IndexWriter has an API to set and get the IndexReaderFactory which creates SegmentReaders.
> Allow custom index readers when using IndexWriter.getReader
> -----------------------------------------------------------
>
> Key: LUCENE-2138
> URL: https://issues.apache.org/jira/browse/LUCENE-2138
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 3.0
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2138.patch
>
>
> This is needed for backwards compatible support with Solr, and is a spin-off from SOLR-1606.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Allow custom index readers when using IndexWriter.getReader
-----------------------------------------------------------
Key: LUCENE-2138
URL: https://issues.apache.org/jira/browse/LUCENE-2138
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 3.0
Reporter: Jason Rutherglen
Priority: Minor
Fix For: 3.1
This is needed for backwards compatible support with Solr, and is a spin-off from SOLR-1606.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org