hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5070) upgrade HBase version for first merge
Date Sat, 11 Jun 2016 01:53:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325635#comment-15325635

Joep Rottinghuis commented on YARN-5070:

Yes I think (eventually) we need to do that, although at the moment we don't honor those.
I think we're aware of this and will need to add that to the initial call to nextRaw to ensure
we set all these values in the builder.

See these previous comment:
bq. Btw. should we also honor regionScanner.getMaxResultSize()

bq. Will be uploading a patch very shortly today. Although not being used today in our scan,
we still need to check that values are being respected for MaxResultSize and getMaxResultsPerColumnFamily.

If we're ok do deal with these additional parameters, that we previously (before this particular
patch) also didn't deal with, in a separate patch then we can possibly continue on with this
current approach for the initial merge.

This does beg the question why we defer getMaxResultSize() from the scanner whereas we hang
on to the batchSize through a member initialized in the constructor.
  public long getMaxResultSize() {
    if (regionScanner == null) {
      throw new IllegalStateException(
          "RegionScanner.isFilterDone() called when the flow "
              + "scanner's scanner is not a RegionScanner");
    return regionScanner.getMaxResultSize();

We could arguably get both from the scanner and not have that as an argument to the constructor.
On two calls in the call hierarchy we actually get the value put into the constructor from
the scanner to begin with, in other cases we set it to -1.
I don't know if that -1 is needed to visit all cell versions or if this limit refers to rowkey/column
qualifier cell counts.

> upgrade HBase version for first merge
> -------------------------------------
>                 Key: YARN-5070
>                 URL: https://issues.apache.org/jira/browse/YARN-5070
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-5070-YARN-2928.01.patch, YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
> Currently we set the HBase version for the timeline service storage to 1.0.1. This is
a fairly old version, and there are reasons to upgrade to a newer version. We should upgrade

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message