Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 29 Nov 2017 18:37:00 +0000 (UTC)
From: "Haibo Chen (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13121656.1511964557000.347682.1511980620089@Atlassian.JIRA>
In-Reply-To: <JIRA.13121656.1511964557000@Atlassian.JIRA>
References: <JIRA.13121656.1511964557000@Atlassian.JIRA> <JIRA.13121656.1511964557759@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-7581) ATSv2 does not construct HBase
 filters correctly in HBase 2.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 29 Nov 2017 18:37:05 -0000


    [ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271320#comment-16271320 ] 

Haibo Chen commented on YARN-7581:
----------------------------------

Based on the discussion I had with our internal HBase folks, the issue has to do how we construct HBase Filters in ATSv2.
Specifically, in the case of TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters(), the query that
is failing the test,       
{code}
URI uri = URI.create("http://localhost:" + getServerPort() + "/ws/v2/" +
          "timeline/clusters/cluster1/apps/application_1111111111_1111/" +
          "entities/type1?conffilters=config_param1%20eq%20value1%20OR%20" +
          "config_param1%20eq%20value3");
{code}
generates the following HBase filter
{code}
FilterList AND (2/2): [
  FilterList AND (1/1): [
    FilterList OR (2/2): [
      SingleColumnValueFilter (c, config_param1, EQUAL, "value1"),
      SingleColumnValueFilter (c, config_param1, EQUAL, "value3")
    ]
  ],
  FilterList OR (1/1): [
    FilterList AND (5/5): [
      FamilyFilter (EQUAL, i),
      QualifierFilter (NOT_EQUAL, e!),
      QualifierFilter (NOT_EQUAL, i!),
      QualifierFilter (NOT_EQUAL, s!),
      QualifierFilter (NOT_EQUAL, r!)
    ]
  ]
]
{code}
and  when the SingleColumnValueFilter are created (in TimelineFilterUtils.createHBaseSingleColValueFilter()),
we call setFilterIfMissing(true)  so that the whole row will be skipped if SCVF does not see  config_param1.


> ATSv2 does not construct HBase filters correctly in HBase 2.0
> -------------------------------------------------------------
>
>                 Key: YARN-7581
>                 URL: https://issues.apache.org/jira/browse/YARN-7581
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: ATSv2
>    Affects Versions: 3.0.0-beta1
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() start to fail after we upgrade HBase to 2.0-alpha4 (To reproduce locally, apply YARN-7581.prelim.patch that is attached in YARN-7346 and run the atsv2 unit tests)
> *Error Message*
> [ERROR] Failures:
> [ERROR]   TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 expected:<2> but was:<0>
> [ERROR]   TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 expected:<1> but was:<0>


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org