drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam" <skat...@maprtech.com>
Subject Re: Review Request 34541: DRILL-3147: tpcds-sf1-parquet query 73 causes memory leak
Date Thu, 21 May 2015 17:08:48 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34541/#review84727
-----------------------------------------------------------



exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
<https://reviews.apache.org/r/34541/#comment136065>

    Is there an alternative implementation? I feel like this method should not be part of
FragmentContext.



exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
<https://reviews.apache.org/r/34541/#comment136067>

    Why a list? .contains(...) is linear time.



exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
<https://reviews.apache.org/r/34541/#comment136071>

    Unnecessary "synchronized"?


- Sudheesh Katkam


On May 21, 2015, 4:30 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34541/
> -----------------------------------------------------------
> 
> (Updated May 21, 2015, 4:30 p.m.)
> 
> 
> Review request for drill, Chris Westin and Jacques Nadeau.
> 
> 
> Bugs: DRILL-3147
>     https://issues.apache.org/jira/browse/DRILL-3147
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> - FragmentContext.close() waits 100ms before closing the allocator to give enough time
to the rpc layer to properly release any batch that was just transfered to this fragment's
allocator
> - each time a fragment A sends a "receiver finished" to fragment B, fragment B id will
be added to FragmentContext.ignoredSenders list
> - refactored UnorderedReceiverBatch.informSenders() and MergingRecordBatch.informSenders()
by moving this method to FragmentContext
> - DataServer.send() uses FragmentContext.ignoredSenders to decide if a batch should be
passed to the fragment or discarded right away
> - BaseRawBatchBuffer methods enqueue() and kill() are now synchronized
> - TestTpcdsSf1Leak test reproduces the leak, it's ignored by default because it requires
a large dataset
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 1cbe886

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
baf9bda 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
684f715 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java 80d2d6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
11b6cc8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
b770a33 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestTpcdsSf1Leaks.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/34541/diff/
> 
> 
> Testing
> -------
> 
> unit tests and tpch100 are passing. Couldn't run functional yet
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message