hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-4542) add filter info to slow query logging
Date Tue, 31 Jan 2012 21:22:30 GMT

     [ https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HBASE-4542:
-------------------------------

    Attachment: D1539.1.patch

zhiqiu requested code review of "[jira] [HBASE-4542] [89-fb] Add filter info to slow query
logging".
Reviewers: Kannan, madhuvaidya, mbautin, JIRA

  Slow opertaion log does not provide enough information when a filter is
  present. The followings are done to add the filter info:

  1) Added toString() method for filters inheriting FilterBase, this
  affect 22 filters and their subclasses. The info added includes the
  filter's name and its members. For example, for TimestampsFilter, we'll
  output its class name as well as the defined timestamps.

  2) Added a field 'filter' in Get::toMap() and Scan::toMap() to enable
  the logging of filter info.

  Task ID: #750975

  Blame Rev:

TEST PLAN
  1. Run and passed unit-tests to make sure it does not break things

  2. Run kannan's script to trigger the slow operation logging, checked
  for each filter to make sure the filter info was logged. To be more
  detailed, the output log are as following (only 'filter' filed is put
  here for ease of reading):

  "filter":"TimestampsFilter (3/3): [2, 3, 5]"
  "filter":"TimestampsFilter (5/6): [2, 3, 5, 7, 11]"
  "filter":"ColumnPrefixFilter col2"
  "filter":"ColumnRangeFilter [col2a, col2b]"
  "filter":"ColumnCountGetFilter 8"
  "filter":"ColumnPaginationFilter (4, 4)"
  "filter":"InclusiveStopFilter row"
  "filter":"PrefixFilter row"
  "filter":"PageFilter 1"
  "filter":"SkipFilter TimestampsFilter (1/1): [1000]"
  "filter":"WhileMatchFilter TimestampsFilter (3/3): [2, 3, 5]"
  "filter":"KeyOnlyFilter"
  "filter":"FirstKeyOnlyFilter"
  "filter":"MultipleColumnPrefixFilter (3/3): [a, b, c]"
  "filter":"DependentColumnFilter (family, qualifier, true, LESS, value)"
  "filter":"FamilyFilter (LESS, value)"
  "filter":"QualifierFilter (LESS, value)"
  "filter":"RowFilter (LESS, value)"
  "filter":"ValueFilter (LESS, value)"
  "filter":"KeyOnlyFilter"
  "filter":"FirstKeyOnlyFilter"
  "filter":"SingleColumnValueFilter (family, qualifier, EQUAL, value)"
  "filter":"SingleColumnValueExcludeFilter (family, qualifier, EQUAL,
  value)"
  "filter":"FilterList AND (2/2): [KeyOnlyFilter, FirstKeyOnlyFilter]"
  Please check ~zhiqiu/Codes/scripts/testFilter.rb for the testing script.

  3. Added unit test cases to TestOperation to verify the filters'
  toString() method works well.

  Revert Plan:

  Tags:

REVISION DETAIL
  https://reviews.facebook.net/D1539

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/Get.java
  src/main/java/org/apache/hadoop/hbase/client/Scan.java
  src/main/java/org/apache/hadoop/hbase/filter/ColumnCountGetFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/ColumnPrefixFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/ColumnRangeFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/CompareFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/DependentColumnFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java
  src/main/java/org/apache/hadoop/hbase/filter/FilterList.java
  src/main/java/org/apache/hadoop/hbase/filter/InclusiveStopFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/MultipleColumnPrefixFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/PageFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/SkipFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/TimestampsFilter.java
  src/main/java/org/apache/hadoop/hbase/filter/WhileMatchFilter.java
  src/test/java/org/apache/hadoop/hbase/client/TestOperation.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/3231/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.

                
> add filter info to slow query logging
> -------------------------------------
>
>                 Key: HBASE-4542
>                 URL: https://issues.apache.org/jira/browse/HBASE-4542
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Kannan Muthukkaruppan
>            Assignee: Madhuwanti Vaidya
>         Attachments: D1263.2.patch, D1539.1.patch
>
>
> Slow query log doesn't report filters in effect.
> For example:
> {code}
> (operationTooSlow): \
> {"processingtimems":3468,"client":"10.138.43.206:40035","timeRange": [0,9223372036854775807],\
> "starttimems":1317772005821,"responsesize":42411, \
> "class":"HRegionServer","table":"myTable","families":{"CF1":"ALL"]},\
> "row":"6c3b8efa132f0219b7621ed1e5c8c70b","queuetimems":0,\
> "method":"get","totalColumns":1,"maxVersions":1,"storeLimit":-1}
> {code}
> the above would suggest that all columns of myTable:CF1 are being requested for the given
row. But in reality there could be filters in effect (such as ColumnPrefixFilter, ColumnRangeFilter,
TimestampsFilter() etc.). We should enhance the slow query log to capture & report this
information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message