lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-8497) Merge Indexes doesn't close input directories on HDFS, causing HDFSLocalityReporter to log warnings once input directory is deleted.
Date Mon, 18 Jan 2016 14:18:39 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105333#comment-15105333
] 

Mark Miller commented on SOLR-8497:
-----------------------------------

That HDFSLocalityReporter has a really ugly performance bug currently I think. It was disabled
by default after it was added. You probably want to move to a version that it allows it to
be turned off.

bq. the input directories on HDFS do not get closed (only released by the CachingDirectoryFactory).

Hmm...perhaps we just also need to mark them as done and we are not? I'll take a look when
I can.


> Merge Indexes doesn't close input directories on HDFS, causing HDFSLocalityReporter to
log warnings once input directory is deleted.
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-8497
>                 URL: https://issues.apache.org/jira/browse/SOLR-8497
>             Project: Solr
>          Issue Type: Bug
>          Components: Hadoop Integration, SolrCloud
>    Affects Versions: 4.10.3
>         Environment: Cloudera Search (CDH 5.4.5 Solr 4.10.3)
>            Reporter: Sivlio Sanchez
>            Priority: Minor
>
> After a Merge Indexes, the input directories on HDFS do not get closed (only released
by the CachingDirectoryFactory). This causes the HDFSLocalityReporter to continue monitoring
the input directories even after they are cleaned up/deleted. This results in a large volume
of logged warnings on the Solr node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message