accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Tubbs (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACCUMULO-395) Map reduce reading from Accumulo is not running mapper locally
Date Fri, 25 Apr 2014 18:59:17 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Christopher Tubbs updated ACCUMULO-395:
---------------------------------------

    Summary: Map reduce reading from Accumulo is not running mapper locally  (was: Map reduce
reading from Accumulo is not running mapper locally )

> Map reduce reading from Accumulo is not running mapper locally
> --------------------------------------------------------------
>
>                 Key: ACCUMULO-395
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-395
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.3.5
>            Reporter: Keith Turner
>            Assignee: Keith Turner
>            Priority: Critical
>              Labels: 14_qa_bug
>             Fix For: 1.3.6, 1.4.0
>
>
> Noticed the continuous ingest verification map reduce job was running slow.  Looking
into I noticed that scans were not running locally.  Not sure if this is map reduce issue
or AccumuloInputFormat issue.  Not only are the scans not local, sometimes a single tablet
server will get a lot scans from many mappers overwhelming it.
> {noformat}
> $ accumulo shell -u root -p xxxx -e "listscans -np" 
>  TABLET SERVER        | CLIENT               | AGE      | LAST     | STATE  | TYPE  |
USER    | TABLE   | COLUMNS   | TABLET    | ITERATORS  | ITERATOR OPTIONS
>    xxx.xxx.xxx.8:9997 |  xxx.xxx.xxx.4:59715 |    2m42s |      8ms |   IDLE |SINGLE |
   root |      ci |        [] |6;23a0cc;2380ca |        [] | {}
>    xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.12:49054 |     5m3s |      7ms |   IDLE |SINGLE |
   root |      ci |        [] |6;1ea0a4c;1e80ac |        [] | {}
>    xxx.xxx.xxx.8:9997 |  xxx.xxx.xxx.9:57157 |    2m27s |      3ms |RUNNING |SINGLE |
   root |      ci |        [] |6;3a00b;39e0bb |        [] | {}
>    xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.10:47880 |    2m58s |      3ms |RUNNING |SINGLE |
   root |      ci |        [] |6;0aa0ab;0a80aa |        [] | {}
>    xxx.xxx.xxx.8:9997 |  xxx.xxx.xxx.9:57101 |    3m12s |    157ms |RUNNING |SINGLE |
   root |      ci |        [] |6;04a05f;04805a |        [] | {}
>    xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.13:34053 |    6m20s |    180ms |   IDLE |SINGLE |
   root |      ci |        [] |6;34a0cd;3480bd |        [] | {}
>    xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.10:47862 |    3m18s |     48ms |   IDLE |SINGLE |
   root |      ci |        [] |6;3ea08a;3e808f |        [] | {}
>    xxx.xxx.xxx.8:9997 |  xxx.xxx.xxx.7:42593 |    2m35s |      2ms |RUNNING |SINGLE |
   root |      ci |        [] |6;46a0c5;4680c5 |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.5:34764 |    2m11s |    177ms |   IDLE |SINGLE |
   root |      ci |        [] |6;78007;77e07c |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.8:43458 |    1m51s |    139ms |   IDLE |SINGLE |
   root |      ci |        [] |6;3f803b;3f604e |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.4:40674 |     3m0s |      1ms |RUNNING |SINGLE |
   root |      ci |        [] |6;0ea08b;0e808f |        [] | {}
>   xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.13:49562 |    2m31s |    111ms |RUNNING |SINGLE |
   root |      ci |        [] |6;0ba086;0b8086 |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.8:43274 |    2m38s |     35ms |   IDLE |SINGLE |
   root |      ci |        [] |6;57a024;578025 |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.6:39526 |    5m11s |    133ms |   IDLE |SINGLE |
   root |      ci |        [] |6;16a0c9;1680cc |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.9:40769 |     5m0s |     11ms |   IDLE |SINGLE |
   root |      ci |        [] |6;3fa034;3f803b |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.6:60691 |    2m59s |     11ms |   IDLE |SINGLE |
   root |      ci |        [] |6;1ba0dbe;1b80da |        [] | {}
>   xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.10:35987 |     5m4s |     31ms |   IDLE |SINGLE |
   root |      ci |        [] |6;72a066;72806a |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.9:40839 |    3m18s |      8ms |   IDLE |SINGLE |
   root |      ci |        [] |6;71a05e;71805b |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.8:43189 |    4m36s |    135ms |   IDLE |SINGLE |
   root |      ci |        [] |6;75a07d;758075 |        [] | {}
>   xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.12:45476 |     2m5s |    112ms |   IDLE |SINGLE |
   root |      ci |        [] |6;48007;47e08 |        [] | {}
>   xxx.xxx.xxx.13:9997 |  xxx.xxx.xxx.7:33791 |  4s981ms |     85ms |RUNNING |SINGLE |
   root |      ci |        [] |6;5f8036;5f6044c |        [] | {}
>   xxx.xxx.xxx.10:9997 |  xxx.xxx.xxx.6:38304 |    2m38s |     18ms |   IDLE |SINGLE |
   root |      ci |        [] |6;7ea059;7e8058 |        [] | {}
>   xxx.xxx.xxx.10:9997 |  xxx.xxx.xxx.5:44152 |     2m3s |      7ms |   IDLE |SINGLE |
   root |      ci |        [] |6;6e809f;6e60a8b |        [] | {}
>   xxx.xxx.xxx.10:9997 |  xxx.xxx.xxx.5:44058 |    2m23s |      7ms |   IDLE |SINGLE |
   root |      ci |        [] |6;7a00d;79e0d7f |        [] | {}
>   xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.13:52463 |    2m35s |      6ms |   IDLE |SINGLE |
   root |      ci |        [] |6;77a089;778089 |        [] | {}
>   xxx.xxx.xxx.10:9997 |  xxx.xxx.xxx.5:44086 |    2m19s |      1ms |RUNNING |SINGLE |
   root |      ci |        [] |6;4a00c;49e0c3 |        [] | {}
>    xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.13:43610 |    2m12s |    179ms |   IDLE |SINGLE |
   root |      ci |        [] |6;3800b;37e0bf |        [] | {}
>    xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.13:55025 |    4m59s |    152ms |   IDLE |SINGLE |
   root |      ci |        [] |6;2ba1568;2b815c |        [] | {}
>    xxx.xxx.xxx.7:9997 |  xxx.xxx.xxx.6:44613 |  1s520ms |    518ms |RUNNING |SINGLE |
   root |      ci |        [] |6;0f802f;0f6047 |        [] | {}
>    xxx.xxx.xxx.7:9997 |  xxx.xxx.xxx.9:33954 |    3m23s |      3ms |RUNNING |SINGLE |
   root |      ci |        [] |6;4da0b4;4d80b9 |        [] | {}
>    xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:57942 |    2m18s |    224ms |   IDLE |SINGLE |
   root |      ci |        [] |6;0a00a;09e0a2 |        [] | {}
>    xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:58004 |     2m9s |    164ms |   IDLE |SINGLE |
   root |      ci |        [] |6;68016;67e16c |        [] | {}
>    xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.10:57480 |     3m3s |     80ms |RUNNING |SINGLE |
   root |      ci |        [] |6;17a0bc;1780bf |        [] | {}
>    xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.12:59552 |    1m59s |    145ms |   IDLE |SINGLE |
   root |      ci |        [] |6;1880d8;1860cc |        [] | {}
>    xxx.xxx.xxx.6:9997 |  xxx.xxx.xxx.5:47196 |    2m29s |     15ms |   IDLE |SINGLE |
   root |      ci |        [] |6;1a00d;19e0dd |        [] | {}
>    xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:58101 |    1m39s |     43ms |   IDLE |SINGLE |
   root |      ci |        [] |6;4f8035;4f6045 |        [] | {}
>    xxx.xxx.xxx.6:9997 |  xxx.xxx.xxx.4:36684 |     3m3s |     97ms |   IDLE |SINGLE |
   root |      ci |        [] |6;1ca0df;1c80dc |        [] | {}
>    xxx.xxx.xxx.6:9997 |  xxx.xxx.xxx.9:45627 |    3m15s |      4ms |RUNNING |SINGLE |
   root |      ci |        [] |6;07a022;078025 |        [] | {}
>    xxx.xxx.xxx.5:9997 |  xxx.xxx.xxx.6:37328 |    4m35s |     93ms |   IDLE |SINGLE |
   root |      ci |        [] |6;2da0e5;2d80ed |        [] | {}
>    xxx.xxx.xxx.5:9997 |  xxx.xxx.xxx.4:46787 |    4m24s |    204ms |   IDLE |SINGLE |
   root |      ci |        [] |6;39a0c7d;3980c8 |        [] | {}
>    xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.12:39133 |    9m18s |    451ms |   IDLE |SINGLE |
   root |      ci |        [] |6;25a147;25813c |        [] | {}
>    xxx.xxx.xxx.5:9997 |  xxx.xxx.xxx.4:46827 |    3m21s |      3ms |RUNNING |SINGLE |
   root |      ci |        [] |6;52a069;528062 |        [] | {}
>    xxx.xxx.xxx.5:9997 |  xxx.xxx.xxx.8:49713 |    2m29s |      4ms |   IDLE |SINGLE |
   root |      ci |        [] |6;2aa18b;2a818b |        [] | {}
>   xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.12:55427 |    2m39s |      1ms |RUNNING |SINGLE |
   root |      ci |        [] |6;45a0c6;4580c7 |        [] | {}
>   xxx.xxx.xxx.12:9997 |  xxx.xxx.xxx.9:40025 |     6m6s |     66ms |   IDLE |SINGLE |
   root |      ci |        [] |6;44a0bf;4480b9 |        [] | {}
>   xxx.xxx.xxx.12:9997 |  xxx.xxx.xxx.7:46896 |    4m48s |      5ms |   IDLE |SINGLE |
   root |      ci |        [] |6;6ea092;6e809f |        [] | {}
>   xxx.xxx.xxx.12:9997 |  xxx.xxx.xxx.5:59397 |    2m26s |    189ms |   IDLE |SINGLE |
   root |      ci |        [] |6;5a004;59e048d |        [] | {}
>   xxx.xxx.xxx.12:9997 |  xxx.xxx.xxx.8:47096 |    4m33s |      5ms |   IDLE |SINGLE |
   root |      ci |        [] |6;6ca11f;6c8117 |        [] | {}
>    xxx.xxx.xxx.4:9997 |  xxx.xxx.xxx.6:47355 |    3m23s |      8ms |   IDLE |SINGLE |
   root |      ci |        [] |6;06a063;068068 |        [] | {}
>    xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.12:56529 |    2m11s |      4ms |RUNNING |SINGLE |
   root |      ci |        [] |6;1800a;17e0ad |        [] | {}
>    xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.12:56401 |    2m48s |     47ms |   IDLE |SINGLE |
   root |      ci |        [] |6;15a0bf;1580bd |        [] | {}
>    xxx.xxx.xxx.4:9997 |  xxx.xxx.xxx.9:44547 |    3m21s |      6ms |   IDLE |SINGLE |
   root |      ci |        [] |6;5ea049;5e8043 |        [] | {}
>    xxx.xxx.xxx.4:9997 |  xxx.xxx.xxx.8:58391 |     5m6s |    236ms |   IDLE |SINGLE |
   root |      ci |        [] |6;28a19a;288196 |        [] | {}
>    xxx.xxx.xxx.9:9997 |  xxx.xxx.xxx.8:60543 |    2m41s |     43ms |   IDLE |SINGLE |
   root |      ci |        [] |6;53a069;538068 |        [] | {}
>    xxx.xxx.xxx.9:9997 |  xxx.xxx.xxx.4:46148 |     3m6s |    235ms |   IDLE |SINGLE |
   root |      ci |        [] |6;03a05c;03805e |        [] | {}
>    xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.12:52594 |     2m8s |    319ms |   IDLE |SINGLE |
   root |      ci |        [] |6;28019;27e198 |        [] | {}
>    xxx.xxx.xxx.9:9997 |  xxx.xxx.xxx.7:51462 |    2m59s |    115ms |   IDLE |SINGLE |
   root |      ci |        [] |6;19a0eb;1980ea |        [] | {}
>    xxx.xxx.xxx.9:9997 |  xxx.xxx.xxx.5:48253 |     2m5s |    195ms |   IDLE |SINGLE |
   root |      ci |        [] |6;6f803a;6f60496 |        [] | {}
>    xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.10:43025 |    2m24s |      3ms |   IDLE |SINGLE |
   root |      ci |        [] |6;2a018;29e188 |        [] | {}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message