drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6197) Duplicate entries in inputProfiles of minor fragments for specific operators
Date Wed, 28 Feb 2018 22:05:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381098#comment-16381098
] 

ASF GitHub Bot commented on DRILL-6197:
---------------------------------------

Github user kkhatua commented on the issue:

    https://github.com/apache/drill/pull/1141
  
    @amansinha100  please review


> Duplicate entries in inputProfiles of minor fragments for specific operators
> ----------------------------------------------------------------------------
>
>                 Key: DRILL-6197
>                 URL: https://issues.apache.org/jira/browse/DRILL-6197
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Monitoring
>    Affects Versions: 1.12.0
>            Reporter: Kunal Khatua
>            Assignee: Kunal Khatua
>            Priority: Minor
>             Fix For: 1.13.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Minor fragments for the following operators show duplicate entries of the inputProfile
({{org.apache.drill.exec.ops.OperatorStats}} instance) when viewed in the Profile UI.
> e.g
> {code:json}
> {
>     ...
> 	"query": "select * from sys.version",
>     ...
> 	        [ ...
> 			{
> 				"inputProfile": [{
> 					"records": 0,
> 					"batches": 0,
> 					"schemas": 0
> 				}],
> 				"operatorId": 0,
> 				"operatorType": 13,
> 				"setupNanos": 0,
> 				"processNanos": 0,
> 				"peakLocalMemoryAllocated": 27131904,
> 				"waitNanos": 0
> 			},
> 			{
> 				"inputProfile": [{
> 					"records": 1,
> 					"batches": 1,
> 					"schemas": 1
> 				}],
> 				"operatorId": 0,
> 				"operatorType": 13,
> 				"setupNanos": 0,
> 				"processNanos": 752448,
> 				"peakLocalMemoryAllocated": 27131904,
> 				"metric": [{
> 					"metricId": 0,
> 					"longValue": 178
> 				}],
> 				"waitNanos": 889492
> 			}]
> 	...
> }
> {code}
> {{operatorType: 13}} is the screen operator, for which there can be only one inputProfile.
> It turns out that by default, all minor fragments' operators are provide a list of inputProfiles
by {{org.apache.drill.exec.ops.FragmentStats.newOperatorStats(OpProfileDef, BufferAllocator)}}.
However, for the following 4 operators, the {{org.apache.drill.exec.physical.impl.BaseRootExec}}
constructors also inject {{OperatorStats}}. 
> {code:java}
> org.apache.drill.exec.proto.beans.CoreOperatorType.SCREEN
> org.apache.drill.exec.proto.beans.CoreOperatorType.SINGLE_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.BROADCAST_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.HASH_PARTITION_SENDER
> {code}
> All updates to the inputProfiles are done by the latter, while the former only reports
zero values.
> The workaround is to have {{org.apache.drill.exec.ops.FragmentStats}} skip injecting
the {{org.apache.drill.exec.ops.OperatorStats}} instance for these operators



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message