lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Optimizing Dataimport from Oracle; cursor sharing; changing oracle session parameters
Date Tue, 15 Aug 2017 14:33:02 GMT
I presume you're using Data Import Handler? An alternative when you
get into complex imports is to use a SolrJ client, here's a sample.
That way you can use whatever tools the particular JDBC connector will
allow and can be much faster.

https://lucidworks.com/2012/02/14/indexing-with-solrj/

Best,
Erick

On Tue, Aug 15, 2017 at 7:09 AM, Mannott, Birgit <B.Mannott@klopotek.de> wrote:
> Hi,
>
> I'm using solr 6.6.0 and I have to do a complex data import from an oracle db concerning
3.500.000 data rows.
> For each row I have 15 additional entities. That means that more than 52 Million selects
are send to the database.
> For every select that is done I optimized the oracle execution path by creating indizes.
> The execution plans are ok.
> But the import still lasts 12 hours.
>
> I think, the main remaining problem is that oracle cursor sharing is not used and that
for every select a hard parse is done.
> Solr does not use binding variables. This would be the easiest way to use oracle cursor
sharing. But I didn't find anything about influencing the way solr builds select statements.
> I could force oracle cursor sharing without binding variables but I have to do this configuration
for the session. I'm not allowed to change the configuration of the whole database system.
>
> Is there a way to execute a command like "ALTER SESSION SET cursor_sharing = FORCE;"
after getting the connection for processing an entity?
>
> Thanks,
> Birgit

Mime
View raw message