cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cyril Scetbon (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task
Date Mon, 27 May 2013 22:09:20 GMT


Cyril Scetbon commented on CASSANDRA-5544:

[~alexliu68] I did some tests with more than 64k row and had only one mapper for the whole
cluster. Even if we have less than 64k rows, why don't we have at least one mapper per node
(in my case replication_factor=1) to work on rows using data locality. Vnodes are enabled
on my cluster, can there be a relation with this option ?
> Hadoop jobs assigns only one mapper in task 
> --------------------------------------------
>                 Key: CASSANDRA-5544
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hadoop
>    Affects Versions: 1.2.1
>         Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
>            Reporter: Shamim Ahmed
>            Assignee: Alex Liu
>         Attachments: Screen Shot 2013-05-26 at 4.49.48 PM.png
> We have got very strange beheviour of hadoop cluster after upgrading 
> Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of Cassandra, where
three of them are hodoop slaves. Now when we are submitting job through Pig script, only one
map assigns in task running on one of the hadoop slaves regardless of 
> volume of data (already tried with more than million rows).
> Configure of pig as follows:
> export PIG_HOME=/oracle/pig-0.10.0
> export PIG_CONF_DIR=${HADOOP_HOME}/conf
> export PIG_RPC_PORT=9160
> export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
> Also we have these following properties in hadoop:
>  <property>
>  <name></name>
>  <value>10</value>
>  </property>
>  <property>
>  <name></name>
>  <value>4</value>
>  </property>

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message