hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5139) Compute (weighted) median using AggregateProtocol
Date Mon, 23 Jan 2012 17:12:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191261#comment-13191261

Hadoop QA commented on HBASE-5139:

-1 overall.  Here are the results of testing the latest attachment 
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/836//console

This message is automatically generated.
> Compute (weighted) median using AggregateProtocol
> -------------------------------------------------
>                 Key: HBASE-5139
>                 URL: https://issues.apache.org/jira/browse/HBASE-5139
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Zhihong Yu
>            Assignee: Zhihong Yu
>         Attachments: 5139-v2.txt, 5139.addendum
> Suppose cf:cq1 stores numeric values and optionally cf:cq2 stores weights. This task
finds out the median value among the values of cf:cq1 (See http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/R.basic/html/weighted.median.html)
> This can be done in two passes.
> The first pass utilizes AggregateProtocol where the following tuple is returned from
each region:
> (partial-sum-of-values, partial-sum-of-weights)
> The start rowkey (supplied by coprocessor framework) would be used to sort the tuples.
This way we can determine which region (called R) contains the (weighted) median. partial-sum-of-weights
can be 0 if unweighted median is sought
> The second pass involves scanning the table, beginning with startrow of region R and
computing partial (weighted) sum until the threshold of S/2 is crossed. The (weighted) median
is returned.
> However, this approach wouldn't work if there is mutation in the underlying table between
pass one and pass two.
> In that case, sequential scanning seems to be the solution which is slower than the above

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message