incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (BLUR-320) Some reducers slow to a crawl
Date Thu, 17 Jul 2014 14:51:04 GMT

     [ https://issues.apache.org/jira/browse/BLUR-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aaron McCurry closed BLUR-320.
------------------------------

    Resolution: Fixed

This issue was actually fixed in 0.2.2 (0.2.3).

> Some reducers slow to a crawl
> -----------------------------
>
>                 Key: BLUR-320
>                 URL: https://issues.apache.org/jira/browse/BLUR-320
>             Project: Apache Blur
>          Issue Type: Bug
>          Components: Blur MapReduce
>    Affects Versions: 0.2.0, 0.2.1, 0.2.2
>            Reporter: Tim Williams
>            Priority: Critical
>             Fix For: 0.2.2
>
>
> Still working to reproduce this condition, if others are seeing something similar please
describe it here so that we can narrow in on and hopefully reliably reproduce it...
> In a m/r job, the majority of reducers are fairly quickly successful and complete.  Then
a small subset slow to an absolute crawl.  There are some overflow rows - but not an abnormal
amount as compared to other successful reducers.  It indexes locally, and I notice in the
main index directory there are around 14k files which I imagine is an important clue.  
> I changed two things at once to get the job to complete successfully in a reasonable
amount of time.  I limited the row size to max of 250k records and added a multiplier of 4.
 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message