lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Karich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1729) Date Facet now override time parameter
Date Thu, 02 Dec 2010 17:07:12 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966181#action_12966181
] 

Peter Karich commented on SOLR-1729:
------------------------------------

Peter Sturge,

in SOLR-1709 you said that you are working with branch3x I checked it out from here:
https://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x

but this 1729 patch didn't apply cleanly*. 

When I tried the 1.4.1 release it is ok, but the tests fail due to**

What could be wrong?

Regards,
Peter.



*
solr_branch_3x/solr$ patch -p0 < solr-1.4.0-solr-1729.patch 
patching file src/java/org/apache/solr/request/SimpleFacets.java
Hunk #1 succeeded at 245 (offset 28 lines).
Hunk #2 succeeded at 280 (offset 28 lines).
Hunk #3 FAILED at 582.
Hunk #4 FAILED at 652.
2 out of 4 hunks FAILED -- saving rejects to file src/java/org/apache/solr/request/SimpleFacets.java.rej
patching file src/java/org/apache/solr/request/UnInvertedField.java
Hunk #2 succeeded at 40 with fuzz 1 (offset 1 line).
Hunk #3 succeeded at 440 (offset 5 lines).
Hunk #4 succeeded at 557 (offset 5 lines).
patching file src/common/org/apache/solr/common/params/FacetParams.java
Hunk #1 FAILED at 175.
1 out of 1 hunk FAILED -- saving rejects to file src/common/org/apache/solr/common/params/FacetParams.java.rej




**
    [junit] Running org.apache.solr.TestTrie
    [junit]  xml response was: <?xml version="1.0" encoding="UTF-8"?>
    [junit] <response>
    [junit] <lst name="responseHeader"><int name="status">0</int><int
name="QTime">157</int></lst><result name="response" numFound="15" start="0"><doc><float
name="id">0.0</float><date name="tdate">2010-12-02T00:00:00Z</date><double
name="tdouble">0.0</double><float name="tfloat">0.0</float><int name="tint">0</int><long
name="tlong">2147483647</long></doc><doc><float name="id">1.0</float><date
name="tdate">2010-12-03T00:00:00Z</date><double name="tdouble">2.33</double><float
name="tfloat">31.11</float><int name="tint">1</int><long name="tlong">2147483648</long></doc><doc><float
name="id">2.0</float><date name="tdate">2010-12-04T00:00:00Z</date><double
name="tdouble">4.66</double><float name="tfloat">124.44</float><int
name="tint">2</int><long name="tlong">2147483649</long></doc><doc><float
name="id">3.0</float><date name="tdate">2010-12-05T00:00:00Z</date><double
name="tdouble">6.99</double><float name="tfloat">279.99</float><int
name="tint">3</int><long name="tlong">2147483650</long></doc><doc><float
name="id">4.0</float><date name="tdate">2010-12-06T00:00:00Z</date><double
name="tdouble">9.32</double><float name="tfloat">497.76</float><int
name="tint">4</int><long name="tlong">2147483651</long></doc><doc><float
name="id">5.0</float><date name="tdate">2010-12-07T00:00:00Z</date><double
name="tdouble">11.65</double><float name="tfloat">777.75</float><int
name="tint">5</int><long name="tlong">2147483652</long></doc><doc><float
name="id">6.0</float><date name="tdate">2010-12-08T00:00:00Z</date><double
name="tdouble">13.98</double><float name="tfloat">1119.96</float><int
name="tint">6</int><long name="tlong">2147483653</long></doc><doc><float
name="id">7.0</float><date name="tdate">2010-12-09T00:00:00Z</date><double
name="tdouble">16.310000000000002</double><float name="tfloat">1524.39</float><int
name="tint">7</int><long name="tlong">2147483654</long></doc><doc><float
name="id">8.0</float><date name="tdate">2010-12-10T00:00:00Z</date><double
name="tdouble">18.64</double><float name="tfloat">1991.04</float><int
name="tint">8</int><long name="tlong">2147483655</long></doc><doc><float
name="id">9.0</float><date name="tdate">2010-12-11T00:00:00Z</date><double
name="tdouble">20.97</double><float name="tfloat">2519.9102</float><int
name="tint">9</int><long name="tlong">2147483656</long></doc><doc><float
name="id">10.0</float><date name="tdate">2010-12-02T00:00:00Z</date><double
name="tdouble">0.0</double><float name="tfloat">0.0</float><int name="tint">0</int><long
name="tlong">2147483647</long></doc><doc><float name="id">20.0</float><date
name="tdate">2010-12-03T00:00:00Z</date><double name="tdouble">2.33</double><float
name="tfloat">31.11</float><int name="tint">1</int><long name="tlong">2147483648</long></doc><doc><float
name="id">30.0</float><date name="tdate">2010-12-04T00:00:00Z</date><double
name="tdouble">4.66</double><float name="tfloat">124.44</float><int
name="tint">2</int><long name="tlong">2147483649</long></doc><doc><float
name="id">40.0</float><date name="tdate">2010-12-05T00:00:00Z</date><double
name="tdouble">6.99</double><float name="tfloat">279.99</float><int
name="tint">3</int><long name="tlong">2147483650</long></doc><doc><float
name="id">50.0</float><date name="tdate">2010-12-06T00:00:00Z</date><double
name="tdouble">9.32</double><float name="tfloat">497.76</float><int
name="tint">4</int><long name="tlong">2147483651</long></doc></result><lst
name="facet_counts"><lst name="facet_queries"/><lst name="facet_fields"><lst
name="tint"><int name="0">2</int><int name="1">2</int><int name="2">2</int><int
name="3">2</int><int name="4">2</int><int name="5">1</int><int
name="6">1</int><int name="7">1</int><int name="8">1</int><int
name="9">1</int></lst><lst name="tlong"><int name="2147483647">2</int><int
name="2147483648">2</int><int name="2147483649">2</int><int name="2147483650">2</int><int
name="2147483651">2</int><int name="2147483652">1</int><int name="2147483653">1</int><int
name="2147483654">1</int><int name="2147483655">1</int><int name="2147483656">1</int></lst><lst
name="tfloat"><int name="0.0">2</int><int name="31.11">2</int><int
name="124.44">2</int><int name="279.99">2</int><int name="497.76">2</int><int
name="777.75">1</int><int name="1119.96">1</int><int name="1524.39">1</int><int
name="1991.04">1</int><int name="2519.9102">1</int></lst><lst
name="tdouble"><int name="0.0">2</int><int name="2.33">2</int><int
name="4.66">2</int><int name="6.99">2</int><int name="9.32">2</int><int
name="11.65">1</int><int name="13.98">1</int><int name="16.310000000000002">1</int><int
name="18.64">1</int><int name="20.97">1</int></lst></lst><lst
name="facet_dates"><lst name="tdate"><int name="2010-12-02T00:00:00Z">2</int><int
name="2010-12-03T00:00:00Z">2</int><int name="2010-12-04T00:00:00Z">2</int><int
name="2010-12-05T00:00:00Z">2</int><int name="2010-12-06T00:00:00Z">1</int><int
name="2010-12-07T00:00:00Z">1</int><str name="gap">+1DAY</str><date
name="end">2010-12-08T00:00:00Z</date></lst></lst></lst>
    [junit] </response>
    [junit] 
    [junit]  request was: facet.date.start=NOW/DAY&facet=true&q=*:*&facet.date=tdate&facet.date.gap=%2B1DAY&facet.field=tint&facet.field=tlong&facet.field=tfloat&facet.field=tdouble&facet.date.end=NOW/DAY%2B6DAYS&rows=15)
    [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 4,948 sec
    [junit] Test org.apache.solr.TestTrie FAILED




    [junit] Running org.apache.solr.request.SimpleFacetsTest
    [junit]  xml response was: <?xml version="1.0" encoding="UTF-8"?>
    [junit] <response>
    [junit] <lst name="responseHeader"><int name="status">0</int><int
name="QTime">11</int></lst><result name="response" numFound="14" start="0"/><lst
name="facet_counts"><lst name="facet_queries"/><lst name="facet_fields"/><lst
name="facet_dates"><lst name="bday"><int name="1976-07-01T00:00:00Z">0</int><int
name="1976-07-02T00:00:00Z">0</int><int name="1976-07-03T00:00:00Z">2</int><int
name="1976-07-04T00:00:00Z">2</int><int name="1976-07-05T00:00:00Z">1</int><int
name="1976-07-06T00:00:00Z">0</int><int name="1976-07-07T00:00:00Z">0</int><int
name="1976-07-08T00:00:00Z">0</int><int name="1976-07-09T00:00:00Z">0</int><int
name="1976-07-10T00:00:00Z">0</int><int name="1976-07-11T00:00:00Z">0</int><int
name="1976-07-12T00:00:00Z">1</int><int name="1976-07-13T00:00:00Z">1</int><int
name="1976-07-14T00:00:00Z">0</int><int name="1976-07-15T00:00:00Z">2</int><int
name="1976-07-16T00:00:00Z">0</int><int name="1976-07-17T00:00:00Z">0</int><int
name="1976-07-18T00:00:00Z">0</int><int name="1976-07-19T00:00:00Z">0</int><int
name="1976-07-20T00:00:00Z">0</int><int name="1976-07-21T00:00:00Z">1</int><int
name="1976-07-22T00:00:00Z">0</int><int name="1976-07-23T00:00:00Z">0</int><int
name="1976-07-24T00:00:00Z">0</int><int name="1976-07-25T00:00:00Z">0</int><int
name="1976-07-26T00:00:00Z">0</int><int name="1976-07-27T00:00:00Z">0</int><int
name="1976-07-28T00:00:00Z">0</int><int name="1976-07-29T00:00:00Z">0</int><int
name="1976-07-30T00:00:00Z">1</int><int name="1976-07-31T00:00:00Z">0</int><str
name="gap">+1DAY</str><date name="end">1976-08-01T00:00:00Z</date><int
name="before">2</int><int name="after">1</int><int name="between">11</int></lst></lst></lst>
    [junit] </response>
    [junit] 
    [junit]  request was: facet.date.start=1976-07-01T00:00:00.000Z&facet=true&q=*:*&facet.date=bday&facet.date.other=all&facet.date.gap=%2B1DAY&facet.date.end=1976-07-01T00:00:00.000Z%2B1MONTH&rows=0)
    [junit] Tests run: 6, Failures: 1, Errors: 0, Time elapsed: 6,996 sec
    [junit] Test org.apache.solr.request.SimpleFacetsTest FAILED


> Date Facet now override time parameter
> --------------------------------------
>
>                 Key: SOLR-1729
>                 URL: https://issues.apache.org/jira/browse/SOLR-1729
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>         Environment: Solr 1.4
>            Reporter: Peter Sturge
>            Priority: Minor
>         Attachments: FacetParams.java, SimpleFacets.java, solr-1.4.0-solr-1729.patch,
UnInvertedField.java
>
>
> This PATCH introduces a new query parameter that tells a (typically, but not necessarily)
remote server what time to use as 'NOW' when calculating date facets for a query (and, for
the moment, date facets *only*) - overriding the default behaviour of using the local server's
current time.
> This gets 'round a problem whereby an explicit time range is specified in a query (e.g.
timestamp:[then0 TO then1]), and date facets are required for the given time range (in fact,
any explicit time range). 
> Because DateMathParser performs all its calculations from 'NOW', remote callers have
to work out how long ago 'then0' and 'then1' are from 'now', and use the relative-to-now values
in the facet.date.xxx parameters. If a remote server has a different opinion of NOW compared
to the caller, the results will be skewed (e.g. they are in a different time-zone, not time-synced
etc.).
> This becomes particularly salient when performing distributed date faceting (see SOLR-1709),
where multiple shards may all be running with different times, and the faceting needs to be
aligned.
> The new parameter is called 'facet.date.now', and takes as a parameter a (stringified)
long that is the number of milliseconds from the epoch (1 Jan 1970 00:00) - i.e. the returned
value from a System.currentTimeMillis() call. This was chosen over a formatted date to delineate
it from a 'searchable' time and to avoid superfluous date parsing. This makes the value generally
a programatically-set value, but as that is where the use-case is for this type of parameter,
this should be ok.
> NOTE: This parameter affects date facet timing only. If there are other areas of a query
that rely on 'NOW', these will not interpret this value. This is a broader issue about setting
a 'query-global' NOW that all parts of query analysis can share.
> Source files affected:
> FacetParams.java   (holds the new constant FACET_DATE_NOW)
> SimpleFacets.java  getFacetDateCounts() NOW parameter modified
> This PATCH is mildly related to SOLR-1709 (Distributed Date Faceting), but as it's a
general change for date faceting, it was deemed deserving of its own patch. I will be updating
SOLR-1709 in due course to include the use of this new parameter, after some rfc acceptance.
> A possible enhancement to this is to detect facet.date fields, look for and match these
fields in queries (if they exist), and potentially determine automatically the required time
skew, if any. There are a whole host of reasons why this could be problematic to implement,
so an explicit facet.date.now parameter is the safest route.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message