drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aditya Kishore" <adityakish...@gmail.com>
Subject Re: Review Request 27271: DRILL-1592: Detect drillbit node failure and cancel affected running queries
Date Tue, 11 Nov 2014 02:11:17 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27271/#review60731
-----------------------------------------------------------



exec/java-exec/src/main/java/org/apache/drill/exec/coord/ClusterCoordinator.java
<https://reviews.apache.org/r/27271/#comment102119>

    To avoid ambiguity from the event names (registered, unregistered), would you mind changing
the function names to `addDrillbitStatusListener()` and `removeDrillbitStatusListener()`.



exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/DrillbitStatusListener.java
<https://reviews.apache.org/r/27271/#comment102122>

    Thanks for separating the events out.
    
    However instead of calling these function for a set of drillbits, I think we should make
one call for each individual drillbit that joins or leaves the cluster.
    
    Somthing like
    
    ```
    public void drillbitUnregistered(CoordinationProtos.DrillbitEndpoint unregisteredDrillbit);
    
    public void drillbitRegistered(CoordinationProtos.DrillbitEndpoint registeredDrillbit);
    
    ```
    
    What do you think?


- Aditya Kishore


On Nov. 10, 2014, 5:20 p.m., Jinfeng Ni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27271/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2014, 5:20 p.m.)
> 
> 
> Review request for drill and Jacques Nadeau.
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Use cluster's ZK to keep track the available drillbit node. Whenever there is a node
change detected by ClusterCoordinator, such change will be notified to either Foreman or the
non-root Fragment's FragmentExecutor. The notification would lead to drillbit to cancel the
affected queries.
> 
> Basic design is to register a DrillbitStatusListener with ClusterCoordinator. QueryManager
or FragmentExecutor will implemented a different DrilbitStatusListener. 
> 1. For Foreman's DrillbitStatusListener, it will check if the active drillbits still
contain all the drillbit runnning the queries. If not, send cancel requet to non-root fragments.
> 2. For Non-root fragment's DrillbitStatusListener, it will check if the foreman drillbit
is in the active drillbit. If not, cancel itself.
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/coord/ClusterCoordinator.java 508a5b2

>   exec/java-exec/src/main/java/org/apache/drill/exec/coord/local/LocalClusterCoordinator.java
035c1aa 
>   exec/java-exec/src/main/java/org/apache/drill/exec/coord/zk/ZKClusterCoordinator.java
7f538d2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 9b78c1d

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java ea48b05 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/DrillbitStatusListener.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0979f34

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java b200edc

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
37074d8 
> 
> Diff: https://reviews.apache.org/r/27271/diff/
> 
> 
> Testing
> -------
> 
> Unit test. 
> 
> functional / TPCH SF100. 
> 
> 
> Thanks,
> 
> Jinfeng Ni
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message