cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-1096) Sequential splits causing unbalanced MapReduce load
Date Sun, 16 May 2010 22:33:42 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-1096:
--------------------------------------

       Issue Type: Improvement  (was: Bug)
    Fix Version/s: 0.6.2
         Priority: Minor  (was: Major)

> Sequential splits causing unbalanced MapReduce load
> ---------------------------------------------------
>
>                 Key: CASSANDRA-1096
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1096
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Hadoop
>    Affects Versions: 0.6
>            Reporter: Joost Ouwerkerk
>            Priority: Minor
>             Fix For: 0.6.2
>
>         Attachments: CASSANDRA-1096.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Since CassandraInputFormat returns an ordered list of splits, when there are many splits
(e.g. hundreds or more) the load on cassandra is horribly unbalanced.  e.g. if I have 30 tasks
processing 600 splits, then the rows for the first 30 splits are all located on the same one
or two nodes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message