drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinfeng Ni (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-4911) SimpleParallelizer should avoid plan serialization for logging purpose when debug logging is not enabled.
Date Wed, 28 Sep 2016 17:48:20 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jinfeng Ni updated DRILL-4911:
------------------------------
    Description: 
SimpleParallelizer and subclass have code to put serialized plan in debug log. However, in
case when the physical plan fragment is very large (table is huge with large # of partitions,
files, etc), the overhead of serialization is not negligible. The code seems to always do
serialization, regardless whether the debug flag is turned on. 

To reduce the overhead, SimpleParallelizer and subclass should only do plan serialization
for logging purpose when debug logging is turned on.


  was:
SimpleParallelizer and subclass has code to put serialized plan in debug log. However, in
case when the physical plan fragment is very large (table is huge with large # of partitions,
files, etc), the overhead of serialization is not negligible. The code seems to always do
serialization, regardless whether the debug flag is turned on. 

To reduce the overhead, SimpleParallelizer and subclass should only do plan serialization
for logging purpose when debug logging is turned on.



> SimpleParallelizer should avoid plan serialization for logging purpose when debug logging
is not enabled.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4911
>                 URL: https://issues.apache.org/jira/browse/DRILL-4911
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>            Reporter: Jinfeng Ni
>            Priority: Minor
>
> SimpleParallelizer and subclass have code to put serialized plan in debug log. However,
in case when the physical plan fragment is very large (table is huge with large # of partitions,
files, etc), the overhead of serialization is not negligible. The code seems to always do
serialization, regardless whether the debug flag is turned on. 
> To reduce the overhead, SimpleParallelizer and subclass should only do plan serialization
for logging purpose when debug logging is turned on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message