hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0
Date Wed, 29 Nov 2017 18:37:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271320#comment-16271320
] 

Haibo Chen commented on YARN-7581:
----------------------------------

Based on the discussion I had with our internal HBase folks, the issue has to do how we construct
HBase Filters in ATSv2.
Specifically, in the case of TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters(),
the query that
is failing the test,       
{code}
URI uri = URI.create("http://localhost:" + getServerPort() + "/ws/v2/" +
          "timeline/clusters/cluster1/apps/application_1111111111_1111/" +
          "entities/type1?conffilters=config_param1%20eq%20value1%20OR%20" +
          "config_param1%20eq%20value3");
{code}
generates the following HBase filter
{code}
FilterList AND (2/2): [
  FilterList AND (1/1): [
    FilterList OR (2/2): [
      SingleColumnValueFilter (c, config_param1, EQUAL, "value1"),
      SingleColumnValueFilter (c, config_param1, EQUAL, "value3")
    ]
  ],
  FilterList OR (1/1): [
    FilterList AND (5/5): [
      FamilyFilter (EQUAL, i),
      QualifierFilter (NOT_EQUAL, e!),
      QualifierFilter (NOT_EQUAL, i!),
      QualifierFilter (NOT_EQUAL, s!),
      QualifierFilter (NOT_EQUAL, r!)
    ]
  ]
]
{code}
and  when the SingleColumnValueFilter are created (in TimelineFilterUtils.createHBaseSingleColValueFilter()),
we call setFilterIfMissing(true)  so that the whole row will be skipped if SCVF does not see
 config_param1.



> ATSv2 does not construct HBase filters correctly in HBase 2.0
> -------------------------------------------------------------
>
>                 Key: YARN-7581
>                 URL: https://issues.apache.org/jira/browse/YARN-7581
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: ATSv2
>    Affects Versions: 3.0.0-beta1
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters()
start to fail after we upgrade HBase to 2.0-alpha4 (To reproduce locally, apply YARN-7581.prelim.patch
that is attached in YARN-7346 and run the atsv2 unit tests)
> *Error Message*
> [ERROR] Failures:
> [ERROR]   TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266
expected:<2> but was:<0>
> [ERROR]   TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523
expected:<1> but was:<0>



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message