giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavan Kumar Athivarapu" <pava...@outlook.com>
Subject Re: Review Request 21987: Detect crashes of Netty threads
Date Thu, 29 May 2014 18:40:29 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21987/#review44270
-----------------------------------------------------------



findbugs-exclude.xml
<https://reviews.apache.org/r/21987/#comment78623>

    why are u removing these stuff,
    wouldn't it be fine to just add GraphTaskManager to allow exit



giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
<https://reviews.apache.org/r/21987/#comment78624>

    where is conf used in this method?



giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/ExceptionHandler.java
<https://reviews.apache.org/r/21987/#comment78626>

    this catches exceptions only for inbound conenctions, do u want to do this for outbound
as well.
    
    in that case would it not be better to just piggy back on the counters & override
exceptioncaught in those methods?
    
    I am fine with creating new classes or just overriding the method in counters


- Pavan Kumar Athivarapu


On May 28, 2014, 8:05 p.m., Sergey Edunov wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21987/
> -----------------------------------------------------------
> 
> (Updated May 28, 2014, 8:05 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Repository: giraph-git
> 
> 
> Description
> -------
> 
> When some of the request processing threads fails, the worker gets stuck but the job
doesn't fail and it has to be killed manually. We should detect netty thread crashes and fail
the job automatically.
> 
> 
> Diffs
> -----
> 
>   findbugs-exclude.xml e0466f7 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java cb36c3e

>   giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyServer.java 14d4ea8 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java adb96cb

>   giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/ExceptionHandler.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
601cd2f 
>   giraph-core/src/main/java/org/apache/giraph/graph/GraphMapper.java c86a024 
>   giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java ad5fc91 
>   giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java 90dc9f3 
>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java aff7084 
>   giraph-core/src/main/java/org/apache/giraph/yarn/GiraphYarnTask.java f4719cc 
>   giraph-core/src/test/java/org/apache/giraph/comm/ConnectionTest.java e771e36 
>   giraph-core/src/test/java/org/apache/giraph/comm/MockExceptionHandler.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java 236bc88 
>   giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java fcdfa5c 
>   giraph-core/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java c026cf8 
> 
> Diff: https://reviews.apache.org/r/21987/diff/
> 
> 
> Testing
> -------
> 
> Run some production jobs with this change. 
> Also introduced random bugs in deserialization logic and confirmed that job fails. 
> 
> 
> Thanks,
> 
> Sergey Edunov
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message