incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Williams (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BLUR-320) Some reducers slow to a crawl
Date Tue, 17 Dec 2013 13:32:07 GMT
Tim Williams created BLUR-320:
---------------------------------

             Summary: Some reducers slow to a crawl
                 Key: BLUR-320
                 URL: https://issues.apache.org/jira/browse/BLUR-320
             Project: Apache Blur
          Issue Type: Bug
          Components: Blur MapReduce
    Affects Versions: 0.2.0, 0.2.1, 0.2.2
            Reporter: Tim Williams
            Priority: Critical


Still working to reproduce this condition, if others are seeing something similar please describe
it here so that we can narrow in on and hopefully reliably reproduce it...

In a m/r job, the majority of reducers are fairly quickly successful and complete.  Then a
small subset slow to an absolute crawl.  There are some overflow rows - but not an abnormal
amount as compared to other successful reducers.  It indexes locally, and I notice in the
main index directory there are around 14k files which I imagine is an important clue.  

I changed two things at once to get the job to complete successfully in a reasonable amount
of time.  I limited the row size to max of 250k records and added a multiplier of 4.  



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message