lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Sturlese <marc.sturl...@gmail.com>
Subject Re: Deduplication patch not working in nightly build
Date Mon, 05 Jan 2009 11:44:58 GMT

Donig this fix I get the same error :(

I am going to try to set up the last nigthly build... let's see if I have
better luck.

The thing is it stop indexing at the doc num 150.000 aprox... and give me
that mysql exception error... Without DeDuplication patch I can index 2
milion docs without problems...

I am pretty lost with this... :(


Shalin Shekhar Mangar wrote:
> 
> Yes I meant the 05/01/2008 build. The fix is a one line change
> 
> Add the following as the last line of DataConfig.Entity.clearCache()
> dataSrc = null;
> 
> 
> 
> On Mon, Jan 5, 2009 at 4:22 PM, Marc Sturlese
> <marc.sturlese@gmail.com>wrote:
> 
>>
>> Shalin you mean I should test the 05/01/2008 nighlty? maybe with this one
>> works? If the fix you did is not really big can u tell me where in the
>> source is and what is it for? (I have been debuging and tracing a lot the
>> dataimporthandler source and I I would like to know what the imporovement
>> is
>> about if it is not a problem...)
>>
>> Thanks!
>>
>>
>> Shalin Shekhar Mangar wrote:
>> >
>> > Marc, I've just committed a fix which may have caused the bug. Can you
>> use
>> > svn trunk (or the next nightly build) and confirm?
>> >
>> > On Mon, Jan 5, 2009 at 3:10 PM, Noble Paul നോബിള്‍ नोब्ळ्
<
>> > noble.paul@gmail.com> wrote:
>> >
>> >> looks like a bug w/ DIH with the recent fixes.
>> >> --Noble
>> >>
>> >> On Mon, Jan 5, 2009 at 2:36 PM, Marc Sturlese
>> <marc.sturlese@gmail.com>
>> >> wrote:
>> >> >
>> >> > Hey there,
>> >> > I was using the Deduplication patch with Solr 1.3 release and
>> >> everything
>> >> was
>> >> > working perfectly. Now I upgraded to a nigthly build (20th december)
>> to
>> >> be
>> >> > able to use new facet algorithm and other stuff and DeDuplication is
>> >> not
>> >> > working any more. I have followed exactly the same steps to apply
>> the
>> >> patch
>> >> > to the source code. I am geting this error:
>> >> >
>> >> > WARNING: Error reading data
>> >> > com.mysql.jdbc.CommunicationsException: Communications link failure
>> due
>> >> to
>> >> > underlying exception:
>> >> >
>> >> > ** BEGIN NESTED EXCEPTION **
>> >> >
>> >> > java.io.EOFException
>> >> >
>> >> > STACKTRACE:
>> >> >
>> >> > java.io.EOFException
>> >> >        at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905)
>> >> >        at
>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2404)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771)
>> >> >        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289)
>> >> >        at
>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352)
>> >> >        at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388)
>> >> >
>> >> >
>> >> > ** END NESTED EXCEPTION **
>> >> > Last packet sent to the server was 202481 ms ago.
>> >> >        at
>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771)
>> >> >        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289)
>> >> >        at
>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352)
>> >> >        at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388)
>> >> > Jan 5, 2009 10:06:16 AM
>> >> org.apache.solr.handler.dataimport.JdbcDataSource
>> >> > logError
>> >> > WARNING: Exception while closing result set
>> >> > com.mysql.jdbc.CommunicationsException: Communications link failure
>> due
>> >> to
>> >> > underlying exception:
>> >> >
>> >> > ** BEGIN NESTED EXCEPTION **
>> >> >
>> >> > java.io.EOFException
>> >> >
>> >> > STACKTRACE:
>> >> >
>> >> > java.io.EOFException
>> >> >        at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905)
>> >> >        at
>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2351)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771)
>> >> >        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289)
>> >> >        at
>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150)
>> >> >        at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488)
>> >> >        at com.mysql.jdbc.ResultSet.close(ResultSet.java:736)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388)
>> >> >
>> >> >
>> >> > ** END NESTED EXCEPTION **
>> >> >
>> >> > Last packet sent to the server was 202482 ms ago.
>> >> >        at
>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862)
>> >> >        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771)
>> >> >        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289)
>> >> >        at
>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352)
>> >> >        at
>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150)
>> >> >        at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488)
>> >> >        at com.mysql.jdbc.ResultSet.close(ResultSet.java:736)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407)
>> >> >        at
>> >> >
>> >>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388)
>> >> >
>> >> > Mark do you know if I should to some extra hack to run DeDuplication
>> >> patch
>> >> > in the new solr version (nightly)? Has something changed in the
>> source
>> >> that
>> >> > make it not work or it's simply that i am doing something wrong?
>> >> >
>> >> > Thanks in advanced
>> >> > --
>> >> > View this message in context:
>> >>
>> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21287327.html
>> >> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> --Noble Paul
>> >>
>> >
>> >
>> >
>> > --
>> > Regards,
>> > Shalin Shekhar Mangar.
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21288939.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> 

-- 
View this message in context: http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21289643.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message