hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5070) upgrade HBase version for first merge
Date Sat, 11 Jun 2016 01:34:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325613#comment-15325613
] 

Vrushali C commented on YARN-5070:
----------------------------------

bq. l.120: From the javadoc, it appears that ScannerContext keeps track of the progress towards
the limits.  If the progress should be monitored across multiple invocations of nextRaw(List<Cell>)
.  I'm not sure if this will do that.

Yes, I believe the progress to be tracked is within the context of the invocation of the "next"
call, not across.
 
The documentation in the ScannerContext class says
https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java;h=29bffd26753795f33b90f31e9b77a5d1387e5cd7;hb=refs/heads/branch-1.1

{code}
 * ScannerContext instances encapsulate limit tracking AND progress towards those limits during
 * invocations of {@link InternalScanner#next(java.util.List)} and
 * {@link RegionScanner#next(java.util.List)}.

{code}

For the flow run coprocessor, the nextRaw/next functions call the nextInternal function which
is the one that actually does the iteration. Hence the batch limit is set up here.

bq. Are we even supposed to create instances of ScannerContext? Am I off? I'm basically not
sure what is the correct way of using the ScannerContext.
An example of how the ScannerContext is being used in the hbase region server code:
https://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/StoreFlusher.html

bq. Are we certain that batchLimit is the correct one to use in ScannerContext? 
batchLimit is the one that tracks the batch size during the scan’s next, hence we are using
that.


> upgrade HBase version for first merge
> -------------------------------------
>
>                 Key: YARN-5070
>                 URL: https://issues.apache.org/jira/browse/YARN-5070
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-5070-YARN-2928.01.patch, YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
YARN-5070-YARN-2928.04.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1. This is
a fairly old version, and there are reasons to upgrade to a newer version. We should upgrade
it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message