drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5811) Large number of "Failure finding Drillbit" messages when using MFS
Date Mon, 09 Oct 2017 22:49:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197829#comment-16197829
] 

ASF GitHub Bot commented on DRILL-5811:
---------------------------------------

Github user vrozov commented on a diff in the pull request:

    https://github.com/apache/drill/pull/965#discussion_r143593333
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/BlockMapBuilder.java
---
    @@ -104,12 +104,16 @@ public BlockMapReader(FileStatus status, boolean blockify) {
         @Override
         protected List<CompleteFileWork> runInner() throws Exception {
           final List<CompleteFileWork> work = Lists.newArrayList();
    +
    +      final Set<String> noDrillbitHosts = logger.isDebugEnabled() ? Sets.<String>newHashSet()
: null;
    --- End diff --
    
    Consider moving `noDillbitHosts` to `BlockMapBuilder` class (use `Sets.newConcurrentHashSet()`
in this case) as it does not seem to belong to `BlockMapReader`. With such change, other changes
are not necessary and likely this will allow reducing repeated log messages even further.
Drop `<Sring>` from `Sets.<String>newHashSet()`.


> Large number of "Failure finding Drillbit" messages when using MFS
> ------------------------------------------------------------------
>
>                 Key: DRILL-5811
>                 URL: https://issues.apache.org/jira/browse/DRILL-5811
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.12.0
>            Reporter: Paul Rogers
>            Assignee: Timothy Farkas
>            Priority: Minor
>             Fix For: 1.12.0
>
>
> A query was run on a single node of a three-node cluster running the MapR file system
(MFS.) When planning the query, zillions of messages of the following form appear in the log:
> {code}
> 2017-09-21 16:18:19,401 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] INFO  o.a.d.exec.store.dfs.FileSelection
- FileSelection.getStatuses() took 0 ms, numFiles: 1
> 2017-09-21 16:18:19,408 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG o.a.d.e.s.schedule.BlockMapBuilder
- Failure finding Drillbit running on host qa-node113.qa.lab.  Skipping affinity to that host.
> 2017-09-21 16:18:19,409 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG o.a.d.e.s.schedule.BlockMapBuilder
- FileWork group (maprfs:///drill/testdata/resource-manager/250wide.tbl,0) max bytes 17132639127
> 2017-09-21 16:18:19,409 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG o.a.d.e.s.schedule.BlockMapBuilder
- Took 1 ms to set endpoint bytes
> {code}
> Might we put such messages a TRACE level, leaving an aggregate message at DEBUG level,
something like:
> {code}
> 123 files to be read using remote reads because of missing Drillbit processes. Enable
trace logging for details.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message