lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avlesh Singh <avl...@gmail.com>
Subject Re: DIH full-import with fetchSize(Integer.MIN_VALUE) taking long time to start processing rows
Date Thu, 05 Nov 2009 13:20:56 GMT
>
> Parallelly I have another process wich is doing lots of inserts to that
> table (I also had it before but with less number of inserts). Could this be
> causing some bloking that makes the query take that long? In case not, and
> advice what could make to take so long until I start see rows beeing
> processed?
>
Sounds scary! With innodb engine you are causing a table level lock with
each insert (assuming your table has an auto-increment column). With
frequent inserts you are of-course delaying the read time.
Why would you want to do this kind of an operation in the very first place?
Can't you use views for indexing?

Cheers
Avlesh

On Thu, Nov 5, 2009 at 6:18 PM, Marc Sturlese <marc.sturlese@gmail.com>wrote:

>
> I have been using fetchSize(Integer.MIN_VALUE) for a long time and it was
> working perfect until now. I use MySQL, java 1.6,
> mysql-connector-java-5.1.7-bin.jar and InoDB tables.
> Since a month ago when the query is executed it will take a long time
> untill
> it starts processing the results from the resultSet. The query matches
> about
> 2M rows. It use to take 10 min untill rows processing started. Now it’s
> taking about 2 hours.
> Parallelly I have another process wich is doing lots of inserts to that
> table (I also had it before but with less number of inserts). Could this be
> causing some bloking that makes the query take that long? In case not, and
> advice what could make to take so long until I start see rows beeing
> processed?
> Thanks in advance.
> --
> View this message in context:
> http://old.nabble.com/DIH-full-import-with-fetchSize%28Integer.MIN_VALUE%29-taking-long-time-to-start-processing-rows-tp26213642p26213642.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message