lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (SOLR-9579) Reuse lucene FieldType in createField flow during ingestion
Date Wed, 12 Oct 2016 00:29:20 GMT

     [ https://issues.apache.org/jira/browse/SOLR-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless reopened SOLR-9579:
--------------------------------------

> Reuse lucene FieldType in createField flow during ingestion
> -----------------------------------------------------------
>
>                 Key: SOLR-9579
>                 URL: https://issues.apache.org/jira/browse/SOLR-9579
>             Project: Solr
>          Issue Type: Improvement
>          Components: Schema and Analysis
>    Affects Versions: 6.x, master (7.0)
>         Environment: This has been primarily tested on Windows 8 and Windows Server 2012
R2
>            Reporter: John Call
>            Priority: Minor
>              Labels: gc, memory, reuse
>             Fix For: master (7.0)
>
>         Attachments: SOLR-9579.patch, SOLR-9579.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> During ingestion createField in FieldType is being called for each field on each document.
For the subclasses of FieldType without their own implementation of createField the lucene
version of FieldType is created to be stored along with the value. However the lucene FieldType
object is identical when created from the same SchemaField. In testing ingestion of one million
rows with 22 field each we were creating 22 million lucene FieldType objects when only 22
are needed. Solr should lazily initialize a lucene FieldType for each SchemaField and reuse
them for future ingestion. Not only does this relieve memory usage but also relieves significant
pressure on the gc.
> There are also subclasses of Solr FieldType which create separate Lucene FieldType for
stored fields instead of reusing the static in StoredField.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message