drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Victoria Markman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4325) ForemanException: One or more nodes lost connectivity during query
Date Fri, 04 Mar 2016 17:43:40 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180230#comment-15180230
] 

Victoria Markman commented on DRILL-4325:
-----------------------------------------

[~jacques@dremio.com] I will give it a try.

> ForemanException: One or more nodes lost connectivity during query
> ------------------------------------------------------------------
>
>                 Key: DRILL-4325
>                 URL: https://issues.apache.org/jira/browse/DRILL-4325
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 1.5.0
>            Reporter: Victoria Markman
>         Attachments: drillbit.log.133, drillbit.log.134, drillbit.log.135, drillbit.log.136,
stats.133.tar, stats.134.tar, stats.135.tar, stats.136.tar, zookeeper.log
>
>
> The picture pretty much looks like this: bunch of queries are running (usually something
more involved than just simple functional tests),usually  tpch or tpcds  with lots of major
fragments, like query74 from tpcds . Zookeeper decides that particular node is dead and queries
that were running at the time of the connection loss are failed by drill ( which is correct
behavior, I think )
> It seems that I can reliably reproduce this issue when I bump up number of concurrently
running queries and make all of them go to the same forman node (I don't really imply here
that  planning is to blame, just seems to reproduce easier)
> On my 4 node cluster I can pretty much reproduce this problem relaiably by running: 
> run.sh -s Advanced/tpcds/tpcds_sf100/original -g smoke -t 600 -n 10 
> {code}
> 2016-01-28 16:30:20,146 [29554d63-b478-6bae-f0f6-435d9f33ffdf:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d63-b478-6bae-f0f6-435d9f33ffdf: select * from sys.version
> 2016-01-28 16:30:22,844 [29554d61-2789-babb-54e5-22b701bf2f64:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d61-2789-babb-54e5-22b701bf2f64: select * from sys.drillbits
> 2016-01-28 16:30:23,281 [29554d60-5bbd-dae1-c38d-21708ad37fbe:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d60-5bbd-dae1-c38d-21708ad37fbe: alter system set `planner.enable_decimal_data_type`
= true
> 2016-01-28 16:30:24,889 [29554d5e-d243-6299-3103-58b180135854:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-d243-6299-3103-58b180135854: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:24,931 [29554d5e-b395-14aa-42a4-f6f248059363:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-b395-14aa-42a4-f6f248059363: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:24,964 [29554d5f-24ac-cf00-714c-7419d3894af0:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5f-24ac-cf00-714c-7419d3894af0: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:24,998 [29554d5e-ae92-6306-3495-be5cb7f98139:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-ae92-6306-3495-be5cb7f98139: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,040 [29554d5e-1a20-3d6d-143b-0ee3bcd4aa11:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-1a20-3d6d-143b-0ee3bcd4aa11: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,073 [29554d5d-e7b4-c61c-9735-ce37938aa47d:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-e7b4-c61c-9735-ce37938aa47d: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,106 [29554d5d-823b-0536-e4df-4c6cef64b3e4:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-823b-0536-e4df-4c6cef64b3e4: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,131 [29554d5e-099c-3acd-477e-ee4bece4dc4e:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-099c-3acd-477e-ee4bece4dc4e: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,184 [29554d5d-b87b-fadd-d5bb-a5d0bba03671:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-b87b-fadd-d5bb-a5d0bba03671: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,205 [29554d5d-97a8-c577-76b7-01bb6c5f5e48:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-97a8-c577-76b7-01bb6c5f5e48: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:30:25,432 [29554d5e-353a-27cf-9c93-969f9e8866da:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-353a-27cf-9c93-969f9e8866da: -- start query 55 in stream
0 using template query55.tpl 
> 2016-01-28 16:30:25,509 [29554d5d-c872-a1be-8386-bf5500ba6726:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-c872-a1be-8386-bf5500ba6726: -- start query 76 in stream
0 using template query76.tpl 
> 2016-01-28 16:30:25,564 [29554d5d-a2ad-bb2a-cfd3-0ec498ef4da9:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-a2ad-bb2a-cfd3-0ec498ef4da9: -- start query 46 in stream
0 using template query46.tpl 
> 2016-01-28 16:30:25,665 [29554d5d-9ffe-96e8-7b4b-0e4214554185:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-9ffe-96e8-7b4b-0e4214554185: -- start query 21 in stream
0 using template query21.tpl 
> 2016-01-28 16:30:25,714 [29554d5d-a051-b35c-078f-2872372aa7bb:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-a051-b35c-078f-2872372aa7bb: -- start query 34 in stream
0 using template query34.tpl 
> 2016-01-28 16:30:25,738 [29554d5d-9dd8-3477-9ce0-ac5f3323195d:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5d-9dd8-3477-9ce0-ac5f3323195d: -- start query 68 in stream
0 using template query68.tpl 
> 2016-01-28 16:30:25,861 [29554d5e-2d2e-81bb-4427-696d27f8deb5:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-2d2e-81bb-4427-696d27f8deb5: -- start query 74 in stream
0 using template query74.tpl 
> 2016-01-28 16:30:25,910 [29554d5e-71d6-5cf5-3ec6-58d07ce81ed5:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-71d6-5cf5-3ec6-58d07ce81ed5: -- start query 33 in stream
0 using template query33.tpl 
> 2016-01-28 16:30:26,012 [29554d5e-4fab-a069-ea09-5d1c561664fe:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5e-4fab-a069-ea09-5d1c561664fe: -- start query 50 in stream
0 using template query50.tpl 
> 2016-01-28 16:30:26,047 [29554d5c-d2ce-153d-4f5b-a8c26f3d256d:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554d5c-d2ce-153d-4f5b-a8c26f3d256d: -- start query 52 in stream
0 using template query52.tpl 
> 2016-01-28 16:32:43,453 [29554cd4-40a8-101a-b418-a91898d720b6:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554cd4-40a8-101a-b418-a91898d720b6: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:32:44,303 [29554cd2-8fe4-9ca4-2b3f-3d591f2144c4:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554cd2-8fe4-9ca4-2b3f-3d591f2144c4: -- start query 91 in stream
0 using template query91.tpl 
> 2016-01-28 16:32:47,965 [29554ccf-db1d-c69b-7dc5-703c1a03f623:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554ccf-db1d-c69b-7dc5-703c1a03f623: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:32:49,119 [29554cce-68d0-6ec4-cbb9-192430099659:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554cce-68d0-6ec4-cbb9-192430099659: -- start query 59 in stream
0 using template query59.tpl 
> 2016-01-28 16:33:01,153 [29554cc1-9ad9-902d-ea5e-f8520b77bd8a:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554cc1-9ad9-902d-ea5e-f8520b77bd8a: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:33:02,119 [29554cc1-74cc-89e8-cbb9-a0c5961d1018:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554cc1-74cc-89e8-cbb9-a0c5961d1018: -- start query 3 in stream
0 using template query3.tpl 
> 2016-01-28 16:35:31,837 [29554c2b-9ebc-9509-96fd-a504657a516f:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554c2b-9ebc-9509-96fd-a504657a516f: use `dfs.tpcds_sf100_parquet_views`
> 2016-01-28 16:35:34,231 [29554c29-0e51-3d3c-dfa5-358d67233d9b:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 29554c29-0e51-3d3c-dfa5-358d67233d9b: -- start query 66 in stream
0 using template query66.tpl 
> 2016-01-28 16:36:13,623 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:3:74.
> 2016-01-28 16:36:13,624 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:6:18.
> 2016-01-28 16:36:13,636 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:4:38.
> 2016-01-28 16:36:13,663 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:6:50.
> 2016-01-28 16:36:13,664 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9dd8-3477-9ce0-ac5f3323195d:1:2.
> 2016-01-28 16:36:13,664 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-a2ad-bb2a-cfd3-0ec498ef4da9:6:30.
> 2016-01-28 16:36:13,665 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:1:10.
> 2016-01-28 16:36:13,665 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:6:6.
> 2016-01-28 16:36:13,665 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9ffe-96e8-7b4b-0e4214554185:3:22.
> 2016-01-28 16:36:13,666 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:3:86.
> 2016-01-28 16:36:13,666 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9ffe-96e8-7b4b-0e4214554185:1:46.
> 2016-01-28 16:36:13,666 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:3:30.
> 2016-01-28 16:36:13,666 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:7:34.
> 2016-01-28 16:36:13,667 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-c872-a1be-8386-bf5500ba6726:1:50.
> 2016-01-28 16:36:13,667 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-a051-b35c-078f-2872372aa7bb:1:10.
> 2016-01-28 16:36:13,668 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:7:10.
> 2016-01-28 16:36:13,669 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:3:90.
> 2016-01-28 16:36:13,670 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:22:41.
> 2016-01-28 16:36:13,670 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:6:26.
> 2016-01-28 16:36:13,670 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:3:18.
> 2016-01-28 16:36:13,671 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:20:37.
> 2016-01-28 16:36:13,671 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-c872-a1be-8386-bf5500ba6726:1:10.
> 2016-01-28 16:36:13,767 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:5:2.
> 2016-01-28 16:36:13,768 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9ffe-96e8-7b4b-0e4214554185:1:14.
> 2016-01-28 16:36:13,768 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:5:54.
> 2016-01-28 16:36:13,768 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:5:86.
> 2016-01-28 16:36:13,769 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:3:62.
> 2016-01-28 16:36:13,812 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:3:82.
> 2016-01-28 16:36:13,823 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:8:29.
> 2016-01-28 16:36:13,862 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-c872-a1be-8386-bf5500ba6726:1:86.
> 2016-01-28 16:36:13,950 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9ffe-96e8-7b4b-0e4214554185:3:6.
> 2016-01-28 16:36:13,991 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9ffe-96e8-7b4b-0e4214554185:1:66.
> 2016-01-28 16:36:13,992 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9dd8-3477-9ce0-ac5f3323195d:3:62.
> 2016-01-28 16:36:13,992 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:15:13.
> 2016-01-28 16:36:13,992 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:8:2.
> 2016-01-28 16:36:13,993 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:1:54.
> 2016-01-28 16:36:13,993 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:5:30.
> 2016-01-28 16:36:13,993 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-a2ad-bb2a-cfd3-0ec498ef4da9:6:34.
> 2016-01-28 16:36:13,993 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-9dd8-3477-9ce0-ac5f3323195d:6:38.
> 2016-01-28 16:36:13,994 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:6:34.
> 2016-01-28 16:36:14,095 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5d-a2ad-bb2a-cfd3-0ec498ef4da9:3:54.
> 2016-01-28 16:36:14,155 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:6:74.
> 2016-01-28 16:36:14,155 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-2d2e-81bb-4427-696d27f8deb5:6:46.
> 2016-01-28 16:36:14,155 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:2:2.
> 2016-01-28 16:36:14,156 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554cce-68d0-6ec4-cbb9-192430099659:3:34.
> 2016-01-28 16:36:14,156 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:0:0.
> 2016-01-28 16:36:14,157 [Curator-ServiceCache-0] WARN  o.a.d.e.w.fragment.FragmentExecutor
- Foreman atsqa4-133.qa.lab no longer active.  Cancelling fragment 29554d5e-4fab-a069-ea09-5d1c561664fe:3:54.
> 2016-01-28 16:36:14,367 [Curator-ServiceCache-0] ERROR o.a.drill.exec.work.foreman.Foreman
- SYSTEM ERROR: ForemanException: One more more nodes lost connectivity during query.  Identified
nodes were [atsqa4-133.qa.lab:31010].
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: ForemanException: One
more more nodes lost connectivity during query.  Identified nodes were [atsqa4-133.qa.lab:31010].
> 	at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:746)
[drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:858)
[drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:790)
[drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:792)
[drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:909) [drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman.access$2700(Foreman.java:110) [drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> 	at org.apache.drill.exec.work.foreman.Foreman$StateListener.moveToState(Foreman.java:1183)
[drill-java-exec-1.5.0-SNAPSHOT.jar:1.5.0-SNAPSHOT]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException: One more more nodes lost
connectivity during query.  Identified nodes were [atsqa4-133.qa.lab:31010].
> {code}
> Will post logs shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message