incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Updated] (BLUR-320) Some reducers slow to a crawl
Date Thu, 17 Jul 2014 14:51:04 GMT


Aaron McCurry updated BLUR-320:

    Fix Version/s: 0.2.2

> Some reducers slow to a crawl
> -----------------------------
>                 Key: BLUR-320
>                 URL:
>             Project: Apache Blur
>          Issue Type: Bug
>          Components: Blur MapReduce
>    Affects Versions: 0.2.0, 0.2.1, 0.2.2
>            Reporter: Tim Williams
>            Priority: Critical
>             Fix For: 0.2.2
> Still working to reproduce this condition, if others are seeing something similar please
describe it here so that we can narrow in on and hopefully reliably reproduce it...
> In a m/r job, the majority of reducers are fairly quickly successful and complete.  Then
a small subset slow to an absolute crawl.  There are some overflow rows - but not an abnormal
amount as compared to other successful reducers.  It indexes locally, and I notice in the
main index directory there are around 14k files which I imagine is an important clue.  
> I changed two things at once to get the job to complete successfully in a reasonable
amount of time.  I limited the row size to max of 250k records and added a multiplier of 4.

This message was sent by Atlassian JIRA

View raw message