drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdelhakim deneche" <adene...@gmail.com>
Subject Review Request 34541: DRILL-3147: tpcds-sf1-parquet query 73 causes memory leak
Date Thu, 21 May 2015 12:29:07 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34541/
-----------------------------------------------------------

Review request for drill, Chris Westin and Jacques Nadeau.


Bugs: DRILL-3147
    https://issues.apache.org/jira/browse/DRILL-3147


Repository: drill-git


Description
-------

- FragmentContext.close() waits 100ms before closing the allocator to give enough time to
the rpc layer to properly release any batch that was just transfered to this fragment's allocator
- each time a fragment A sends a "receiver finished" to fragment B, fragment B id will be
added to FragmentContext.ignoredSenders list
- refactored UnorderedReceiverBatch.informSenders() and MergingRecordBatch.informSenders()
by moving this method to FragmentContext
- DataServer.send() uses FragmentContext.ignoredSenders to decide if a batch should be passed
to the fragment or discarded right away
- BaseRawBatchBuffer methods enqueue() and kill() are now synchronized
- TestTpcdsSf1Leak test reproduces the leak, it's ignored by default because it requires a
large dataset


Diffs
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 1cbe886 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
baf9bda 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
684f715 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java 80d2d6e 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java 11b6cc8

  exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
b770a33 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestTpcdsSf1Leaks.java PRE-CREATION


Diff: https://reviews.apache.org/r/34541/diff/


Testing
-------

unit tests and tpch100 are passing. Couldn't run functional yet


Thanks,

abdelhakim deneche


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message