lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julio Castillo" <jcasti...@edgenuity.com>
Subject RE: [jira] Updated: (SOLR-469) Data Import RequestHandler
Date Wed, 11 Jun 2008 15:54:54 GMT
 Shalin,
Thanks for consolidating the patch.

Any idea, when the dB Import request handler will be part of the nightly
build?

Thanks again

** julio

-----Original Message-----
From: Shalin Shekhar Mangar (JIRA) [mailto:jira@apache.org] 
Sent: Wednesday, June 11, 2008 8:43 AM
To: jcastillo@edgenuity.com
Subject: [jira] Updated: (SOLR-469) Data Import RequestHandler


     [
https://issues.apache.org/jira/browse/SOLR-469?page=com.atlassian.jira.plugi
n.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar updated SOLR-469:
---------------------------------------

    Attachment: SOLR-469.patch

A new patch file (SOLR-469.patch) consisting of some important bug fixes and
minor enhancements. The changes and the corresponding classes are given
below

*Changes*
* Set fetch size to Integer.MIN_VALUE if batchSize in configuration is -1 as
per Patrick's suggestion -- JdbcDataSource
* Transformers can add a boost to a document by adding a key/value pair
row.put("$docBoost", 2.0f) from any entity -- DocBuilder,SolrWriter and
DataImportHandler
* Fixes for infinite loop in SqlEntityProcessor when delta query fails for
some reason and NullPointerException is thrown in EntityProcessorBase --
EntityProcessorBase
* Fix for NullPointerException in TemplateTransformer and corresponding test
-- TemplateTransformer, TestTemplateTransformer
* Enhancement for specifying table.column syntax for pk attribute in entity
as per issue reported by Chris Moser and Olivier Poitrey --
SqlEntityProcessor,TestSqlEntityProcessor2
* Fix for NullPointerException in XPathRecordReader when attribute specified
through xpath is null -- XPathRecordReader, TestXPathRecordReader
* Enhancement to DataSource interface to provide a close method --
DataSource, FileDataSource, HttpDataSource, MockDataSource
* Context interface has a new method getDataSource(String entityName) for
getting a new DataSource instance for the given entity -- Context,
ContextImpl, DataImporter, DocBuilder
* FileListEntityProcessor implements olderThan and newerThan filtering
parameters -- FileListEntityProcessor, TestFileListEntityProcessor
* Debug Mode can be disabled from solrconfig.xml by enableDebug=false --
DataImporter, DataImportHandler
* Running statistics are exposed on the Solr Statistics page in addition to
cumulative statictics -- DataImportHandler, DocBuilder

> Data Import RequestHandler
> --------------------------
>
>                 Key: SOLR-469
>                 URL: https://issues.apache.org/jira/browse/SOLR-469
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 1.3
>            Reporter: Noble Paul
>            Assignee: Grant Ingersoll
>             Fix For: 1.3
>
>         Attachments: SOLR-469-contrib.patch, SOLR-469.patch, 
> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, 
> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch
>
>
> We need a RequestHandler Which can import data from a DB or other
dataSources into the Solr index .Think of it as an advanced form of
SqlUpload Plugin (SOLR-103).
> The way it works is as follows.
>     * Provide a configuration file (xml) to the Handler which takes in the
necessary SQL queries and mappings to a solr schema
>           - It also takes in a properties file for the data source
configuraution
>     * Given the configuration it can also generate the solr schema.xml
>     * It is registered as a RequestHandler which can take two commands
do-full-import, do-delta-import
>           -  do-full-import - dumps all the data from the Database into
the index (based on the SQL query in configuration)
>           - do-delta-import - dumps all the data that has changed since
last import. (We assume a modified-timestamp column in tables)
>     * It provides a admin page
>           - where we can schedule it to be run automatically at regular
intervals
>           - It shows the status of the Handler (idle, full-import, 
> delta-import)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message