lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <karl.wri...@nokia.com>
Subject Repeat to the right list: Solr spewage and possible re-entrancy problem?
Date Mon, 07 Jun 2010 14:35:30 GMT
Hi folks,

This morning I was experimenting with using multiple threads while indexing some 20,000,000
records worth of content.  In fact, my test spun up some 50 threads, and happily chugged away
for a couple of hours before I saw the following output from my test code:

>>>>>>
Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to index record 6469124
Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to index record 6469551
Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to index record 6470592
Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to index record 6472454
java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:168)
        at HttpPoster.getResponse(HttpPoster.java:280)
        at HttpPoster.indexPost(HttpPoster.java:191)
        at ParseAndLoad$PostThread.run(ParseAndLoad.java:638)
<<<<<<

Looking at the solr-side output, I see nothing interesting at all:

>>>>>>
Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={literal.nokia_longitude=9.78518981933594&literal.nokia_phone=%2B497971910474&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_district=Münster&literal.nokia_placerating=0&literal.id=6472724&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=1&literal.nokia_ppid=276u0wyw-c8cb7f4d6cd84a639a4e7d3570bf8814&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9985514322917&literal.nokia_postalcode=74405&literal.nokia_street=WeinhaldenstraÃe&literal.nokia_title=Dorfgemeinschaft+Münster+e.V.&literal.nokia_category=261}
status=0 QTime=1
Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={literal.nokia_longitude=9.76717020670573&literal.nokia_phone=%2B497971950725&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=0&literal.id=6472737&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=13&literal.nokia_ppid=276u0wyw-d3bed6449fcb41b0adc50ae08e041f8d&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9974405924479&literal.nokia_fax=%2B497971950712&literal.nokia_postalcode=74405&literal.nokia_street=KochstraÃe&literal.nokia_title=BayWa+AG+Bau-+%26+Gartenmarkt&literal.nokia_category=194}
status=0 QTime=0
Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={literal.nokia_longitude=9.77591044108073&literal.nokia_phone=%2B49797124009&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_district=Unterrot&literal.nokia_placerating=0&literal.id=6472739&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=28&literal.nokia_ppid=276u0wyw-d534d7a9235a4edf878d5e32a34bad8b&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9791788736979&literal.nokia_fax=%2B49797123431&literal.nokia_postalcode=74405&literal.nokia_street=HauptstraÃe&literal.nokia_title=Gastel+R.&literal.nokia_category=5}
status=0 QTime=1
Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={literal.nokia_longitude=9.76935&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=5&literal.id=6472698&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=15&literal.nokia_ppid=276u0wyw-9544100e68d74162aff54783b9376134&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9981&literal.nokia_postalcode=74405&literal.nokia_street=KanzleistraÃe&literal.nokia_tag=Steuerberater&literal.nokia_tag=Business+%26+Service&literal.nokia_title=Consultis+GmbH&literal.nokia_category=215}
status=0 QTime=92
Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={literal.nokia_longitude=9.77173970540364&literal.nokia_phone=%2B4979713238&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=0&literal.id=6472699&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=37&literal.nokia_ppid=276u0wyw-9600016fd0d248c9b442111838350f64&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9987182617188&literal.nokia_fax=%2B497971911639&literal.nokia_postalcode=74405&literal.nokia_street=KarlstraÃe&literal.nokia_title=Videothek,+5th+avenue+Peltekis+Apostolos&literal.nokia_category=5}
status=0 QTime=93
<<<<<<

It is unlikely (but, of course, not out of the question) that this hiccup is due to some reentrancy
problem in my test code.  It is much more likely to be some kind of a Solr multi-threaded
race condition - especially since it looks like a number of requests all failed at precisely
the same time.  This is a Solr 1.5 build from mid-late March, FWIW.  Does anyone know of an
extractingUpdateRequestHandler re-entrancy bug of this kind?

Thanks,
Karl






Mime
View raw message