drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timothy Farkas (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-6325) Memory usage stats may not be updated correctly for an operator.
Date Wed, 11 Apr 2018 20:03:00 GMT
Timothy Farkas created DRILL-6325:

             Summary: Memory usage stats may not be updated correctly for an operator.
                 Key: DRILL-6325
                 URL: https://issues.apache.org/jira/browse/DRILL-6325
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Timothy Farkas
            Assignee: Timothy Farkas

The memory usage stats for an operator in a query profile may not be up to date, even after
the query is completed. This is caused by two possible issues:

1 Memory usage is polled by a thread once every 5 seconds. However the final memory usage
of the operator may not be reported if the operator completes before the next polling cycle.
2 There is a race condition in the Foreman where one thread sets the metrics on an object
and another thread reads the metrics object to include in a query profile. The thread that
reads the metrics object may read old metrics since access to the variable is not synchronized.

There are two fixes for this

1 Poll the stats one more time after the fragment root completes but before the operator is
2 Synchronize the methods on the FragmentData class.

This message was sent by Atlassian JIRA

View raw message