lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Rafalovitch <arafa...@gmail.com>
Subject Re: DIH Issues
Date Tue, 25 Apr 2017 16:26:25 GMT
I wonder if it is possible to write a component/URP/something that
will intercept exceptions like these and dump out full record.

Regards,
   Alex.
----
http://www.solr-start.com/ - Resources for Solr users, new and experienced


On 25 April 2017 at 12:19, Erick Erickson <erickerickson@gmail.com> wrote:
> You say your SQL database always has the values, but does the output
> from the SQL query you actually use have them? I've been fooled before
> by the query I form "somehow" doesn't have a value for all fields I
> expect.
>
> You could also crank the logging level up enough to see the docs that
> are indexed, although that would probably only confirm that the fields
> weren't in the docs which you know already, not tell you why they are
> missing. Pull the SQL out and run it independently perhaps?
>
> I sound a bit like a broken record, but this is why I like SolrJ, I
> can actually debug that:
> https://lucidworks.com/2012/02/14/indexing-with-solrj/
>
> Best,
> Erick
>
> On Tue, Apr 25, 2017 at 8:28 AM, AJ Lemke <aj.lemke@securitylabs.com> wrote:
>> Thanks for the thought Alex!
>> The fields that have this happen most often are numeric and boolean fields. These
fields have real data (id numbers, true/false, etc.)
>>
>> AJ
>>
>> -----Original Message-----
>> From: Alexandre Rafalovitch [mailto:arafalov@gmail.com]
>> Sent: Tuesday, April 25, 2017 8:27 AM
>> To: solr-user <solr-user@lucene.apache.org>
>> Subject: Re: DIH Issues
>>
>> Maybe the content gets simplified away between the database and the Solr schema.
For example if your field contains just spaces and you have UpdateRequestProcessors to do
trim and removal of empty fields?
>>
>> Schemaless mode will remove empty fields, but will not trim for example.
>>
>> Regards,
>>    Alex.
>> ----
>> http://www.solr-start.com/ - Resources for Solr users, new and experienced
>>
>>
>> On 25 April 2017 at 09:21, AJ Lemke <aj.lemke@securitylabs.com> wrote:
>>> Hey all,
>>>
>>> We are using 6.3.0 and we have issues with DIH throwing errors.  We are seeing
an intermittent issue where on a full index a single error will be thrown.  The error is always
"missing required field: fieldname".
>>> Our SQL database always has data in the field that comes up with the error. 
Most of the errors are coming on fields that SQL has marked as required.
>>>
>>> Would anyone have any hints or ideas where to look to remedy this situation.
>>>
>>> As always if you need more information let me know.
>>>
>>> Thanks
>>> AJ

Mime
View raw message