hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4062) Add the flush and compaction functionality via coprocessors and scanners for flow run table
Date Tue, 08 Mar 2016 02:38:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184269#comment-15184269
] 

Sangjin Lee commented on YARN-4062:
-----------------------------------

Sorry [~vrushalic] it took a while to get to this. It looks good for the most part. I do have
some questions and minor comments.

(YarnConfiguration.java)
- please define it in yarn-default.xml as well

(FlowScannerOperation.java)
- l.38, 43: typo: "Comapction" -> "compaction"

(FlowRunColumnPrefix.java)
- l.43: if we're removing the aggregation operation from the only enum entry, should we remove
the aggregation operation ({{aggOp}}) completely from this class?

(FlowRunCoprocessor.java)
- l.268: Logging of {{request.isMajor()}} seems a bit cryptic. It would print strings like
"Compactionrequest= ... true for ObserverContext ...". Should we do something like {{request.isMajor()
? "major compaction" : "minor compaction"}} instead? And did you mean it to be an INFO logging
statement?

(TimelineStorageUtils.java)
- l.524: nit: we probably don't need that beginning space

(FlowScanner.java)
- l.79: we should not set the action by default, as it is always passed in through the constructor
- l.178: Shouldn't this be {{cellLimit}} still? {{limit}} is not an argument of this method.
- l.220: could you elaborate on why this change is needed? I'm generally not too clear on
the difference between {{cellLimit}} and {{limit}}.  How are these values different and under
what conditions?
- l.233: missing the row key value?
- l.381-382: should we throw an exception as this is not possible (currently)?
- l.484: Although this would be true for the most part, {{sum.longValue() > 0L}} is not
equivalent to summations being performed on compaction here, especially if there were negative
individual values for example. Should we introduce a boolean flag to denote the case where
summations were performed instead?

(TestHBaseStorageFlowRunCompaction.java)
- there are 3 unused imports


> Add the flush and compaction functionality via coprocessors and scanners for flow run
table
> -------------------------------------------------------------------------------------------
>
>                 Key: YARN-4062
>                 URL: https://issues.apache.org/jira/browse/YARN-4062
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4062-YARN-2928.04.patch, YARN-4062-YARN-2928.05.patch, YARN-4062-YARN-2928.06.patch,
YARN-4062-YARN-2928.07.patch, YARN-4062-YARN-2928.1.patch, YARN-4062-feature-YARN-2928.01.patch,
YARN-4062-feature-YARN-2928.02.patch, YARN-4062-feature-YARN-2928.03.patch
>
>
> As part of YARN-3901, coprocessor and scanner is being added for storing into the flow_run
table. It also needs a flush & compaction processing in the coprocessor and perhaps a
new scanner to deal with the data during flushing and compaction stages. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message