lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Chazalet <bchaza...@companywatch.net>
Subject Re: 8.3.0: Invalid UUID String while indexing document with a UUID field
Date Thu, 14 Nov 2019 13:09:16 GMT
I dug a little in the dataimport code, and there's a special case for
BigDecimal in the JdbcDataSource class, here exactly:
https://github.com/apache/lucene-solr/blob/faaee86efb01fa6e431fcb129cfb956c7d62d514/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java#L403

I believe we might need the same kind of logic for a UUID object coming
directly from the jdbc driver.

On Thu, 14 Nov 2019 at 13:46, Boris Chazalet <bchazalet@companywatch.net>
wrote:

> Thanks for your response Jörn. Yes, I saw the prefix and I suspect this is
> the problem. But I'm not doing anything special in the DIH config, this is
> a minimalized version of it:
>
> <dataConfig>
>     <dataSource type="JdbcDataSource"
>                 driver="org.postgresql.Driver"
>                 url="jdbc:postgresql://redacted"
>                 readOnly="false" autoCommit="false"
> transactionIsolation="TRANSACTION_READ_COMMITTED"
> holdability="CLOSE_CURSORS_AT_COMMIT" application_name="solr"
> prepareThreshold="0" />
>     <document>
>         <entity name="index" pk="cinumber" transformer="RegexTransformer"
>                 query="
>                 SELECT myuuidfield, mypk
>                 FROM {{solr__datasource}}
>                 WHERE
>                     '${dataimporter.request.clean}' != 'false' OR
> lastmodified  > '${dataimporter.last_index_time}'::timestamp - '3
> hours'::interval
>                 ">
>             <field column="myuuidfield" name="myuuidfield" />
>             <field column="mypk" name="mypk" />
>         </entity>
>     </document>
> </dataConfig>
>
> The field is a UUID in the database, so it's definitely valid and without
> prefix. Where can I double check for myself of the DataImportHandler
> seralises an UUID in the code?
>
>
>
> On Thu, 14 Nov 2019 at 13:38, Jörn Franke <jornfranke@gmail.com> wrote:
>
>> It seems there is a prefix java.util.UUID: in front of your UUID. Any
>> idea where it comes from? Is it also like this in the database? Is your
>> import handler maybe receiving a java object java.util.UUID and it is not
>> converted correctly to string?
>>
>> > Am 14.11.2019 um 11:52 schrieb Boris Chazalet <
>> bchazalet@companywatch.net>:
>> >
>> > 
>> >
>> > Hi,
>> >
>> > I'm running into an issue with Solr 8.3.0: it fails at indexing a
>> schema with UUID field.
>> >
>> > I'm using a SolrCloud setup with 3 instances, and I'm using the DIH to
>> fetch and index the data from a postgres database.
>> >
>> > In schema.xml I have:
>> >
>> >     <field name="myuuidfield" type="uuid" uninvertible="false"
>> indexed="true" stored="true" multiValued="false" required="false"/>
>> >     <fieldType name="uuid" class="solr.UUIDField"/>
>> >
>> > The data-config is a simple select, the uuid field is of UUID type in
>> postgres.
>> >
>> > I was running 7.7.2 until now, and first noticed the problem there. But
>> given the number of things around UUIDs fixed in the latest version, I
>> thought I'd try 8.3.0 first. The same problem arises while running the DIH.
>> >
>> > Note that I have another core with a uuid field, which I am indexing
>> externally (i.e. not from the DIH) and I haven't had a problem there, so
>> I'm suspecting the problem might be in the DIH logic, but with no certainty.
>> >
>> > Below is a truncated version of exception's stacktrace I see in the
>> logs. I can provide the full one if necessary. Is this a legitimate bug?
>> What can I do to help tracking down the problem?
>> >
>> > 2019-11-13 17:29:55.430 ERROR (qtp1990098664-15) [c:db_c s:shard1
>> r:core_node5 x:db_c_shard1_replica_n2] o.a.s.s.HttpSolrCall
>> null:org.apache.solr.common.SolrException: ERROR: [doc=1] Error adding
>> field 'myuuidfield'='java.util.UUID:afa9cf35-0b2d-e811-89a7-0025900429ba'
>> msg=Error while creating field
>> 'myuuidfield{type=uuid,properties=indexed,stored,omitNorms,omitTermFreqAndPositions,useDocValuesAsStored}'
>> from value 'java.util.UUID:afa9cf35-0b2d-e811-89a7-0025900429ba'
>> > at
>> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:215)
>> > at
>> org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:109)
>> > at
>> org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:969)
>> > ...
>> > Caused by: org.apache.solr.common.SolrException: Error while creating
>> field
>> 'myuuidfield{type=uuid,properties=indexed,stored,omitNorms,omitTermFreqAndPositions,useDocValuesAsStored}'
>> from value 'java.util.UUID:4ee3992e-0b2d-e811-89a7-0025900429ba'
>> >         at
>> org.apache.solr.schema.FieldType.createField(FieldType.java:291)
>> >         at
>> org.apache.solr.schema.StrField.createFields(StrField.java:48)
>> >         at
>> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:65)
>> >         at
>> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:171)
>> >         ... 69 more
>> > Caused by: org.apache.solr.common.SolrException: Invalid UUID String:
>> 'java.util.UUID:4ee3992e-0b2d-e811-89a7-0025900429ba'
>> >         at
>> org.apache.solr.schema.UUIDField.toInternal(UUIDField.java:88)
>> >         at
>> org.apache.solr.schema.FieldType.createField(FieldType.java:289)
>> >         ... 72 more
>> >
>> > Kind Regards,
>> > Boris
>> >
>> > This message is intended only for the addressee and unless otherwise
>> stated is commercial in confidence and may contain information that is
>> privileged.  Where all recipients are in the companywatch.net domain,
>> this communication is classified as Confidential.  Unauthorised use is
>> strictly prohibited and may be unlawful. If you are not the addressee, you
>> should not read, copy, disclose or otherwise use this message, except for
>> the purpose of delivery to the addressee. If you have received this in
>> error, please delete and advise us immediately. Although Company Watch
>> makes every reasonable effort to keep its network and systems free from
>> viruses, the company accepts no responsibility for computer viruses
>> transmitted through this mail or in any attachments. It is your
>> responsibility to virus scan any attachments we send to you.
>> >
>> > Company Watch Limited is a company registered in England & Wales with
>> company number 3597613
>> > Centurion House, 37 Jewry Street, London, EC3N 2ER
>> >
>> > Please consider the environment before printing this email
>> >
>>
>
>
> --
>
>
> *Boris Chazalet*Senior developer and problem solver
> [image: Co_watch_signature]
> T: +44 (0)20 3740 9402
> E: bchazalet@companywatch.net
>
>
>
>
>

-- 


*Boris Chazalet*Senior developer and problem solver
[image: Co_watch_signature]
T: +44 (0)20 3740 9402
E: bchazalet@companywatch.net

-- 
This message is
intended only for the addressee and unless otherwise stated 
is commercial in confidence and may contain information that is privileged. 
 Where all recipients are in the companywatch.net <http://companywatch.net> 
domain, this communication is classified as Confidential.  Unauthorised use 
is strictly prohibited and may be
unlawful. If you are not the addressee, 
you should not read, copy,
disclose or otherwise use this message, except 
for the purpose of delivery to
the addressee. If you have received this in 
error, please delete and
advise us immediately. Although Company Watch 
makes every reasonable
effort to keep its network and systems free from 
viruses, the company accepts
no responsibility for computer viruses 
transmitted through this mail or in any
attachments. It is your 
responsibility to virus scan any attachments we
send to you. 



Company 
Watch Limited is a company registered in England & Wales with
company 
number 3597613

Centurion House, 37 Jewry Street, London, EC3N 2ER




Please consider the environment before printing this email  




Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message