drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-6197) Duplicate entries in inputProfiles of minor fragments for specific operators
Date Fri, 02 Mar 2018 06:11:00 GMT

     [ https://issues.apache.org/jira/browse/DRILL-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kunal Khatua updated DRILL-6197:
--------------------------------
    Labels: ready-to-commit  (was: )

> Duplicate entries in inputProfiles of minor fragments for specific operators
> ----------------------------------------------------------------------------
>
>                 Key: DRILL-6197
>                 URL: https://issues.apache.org/jira/browse/DRILL-6197
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Monitoring
>    Affects Versions: 1.12.0
>            Reporter: Kunal Khatua
>            Assignee: Kunal Khatua
>            Priority: Minor
>              Labels: ready-to-commit
>             Fix For: 1.13.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Minor fragments for the following operators show duplicate entries of the inputProfile
({{org.apache.drill.exec.ops.OperatorStats}} instance) when viewed in the Profile UI.
> e.g
> {code:json}
> {
>     ...
> 	"query": "select * from sys.version",
>     ...
> 	        [ ...
> 			{
> 				"inputProfile": [{
> 					"records": 0,
> 					"batches": 0,
> 					"schemas": 0
> 				}],
> 				"operatorId": 0,
> 				"operatorType": 13,
> 				"setupNanos": 0,
> 				"processNanos": 0,
> 				"peakLocalMemoryAllocated": 27131904,
> 				"waitNanos": 0
> 			},
> 			{
> 				"inputProfile": [{
> 					"records": 1,
> 					"batches": 1,
> 					"schemas": 1
> 				}],
> 				"operatorId": 0,
> 				"operatorType": 13,
> 				"setupNanos": 0,
> 				"processNanos": 752448,
> 				"peakLocalMemoryAllocated": 27131904,
> 				"metric": [{
> 					"metricId": 0,
> 					"longValue": 178
> 				}],
> 				"waitNanos": 889492
> 			}]
> 	...
> }
> {code}
> {{operatorType: 13}} is the screen operator, for which there can be only one inputProfile.
> It turns out that by default, all minor fragments' operators are provide a list of inputProfiles
by {{org.apache.drill.exec.ops.FragmentStats.newOperatorStats(OpProfileDef, BufferAllocator)}}.
However, for the following 4 operators, the {{org.apache.drill.exec.physical.impl.BaseRootExec}}
constructors also inject {{OperatorStats}}. 
> {code:java}
> org.apache.drill.exec.proto.beans.CoreOperatorType.SCREEN
> org.apache.drill.exec.proto.beans.CoreOperatorType.SINGLE_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.BROADCAST_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.HASH_PARTITION_SENDER
> {code}
> All updates to the inputProfiles are done by the latter, while the former only reports
zero values.
> The workaround is to have {{org.apache.drill.exec.ops.FragmentStats}} skip injecting
the {{org.apache.drill.exec.ops.OperatorStats}} instance for these operators



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message