cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thamizh <tceg...@yahoo.co.in>
Subject Re: multi-node cassandra config doubt
Date Wed, 24 Aug 2011 11:34:41 GMT
Hi Aaron,

This is yet to be resolved. 

I have set-up Cassandra multi node clustering and facing issues in pushing HDFS data to Cassandra.
When I ran "MapReduce" progrma I am getting UnknownHostException.

In hadoop(0.20.1), I have configured node01-as master and node01, node02 & node03 as slaves.

In Cassandra(0.8.4), the installation & configurations has been done. when I issue nodetool
ring command I could see the ring and also the KEYSPACES & COLUMNFAMILYS have got distributed.

o/p: nodetool
$bin/nodetool -h node02 ring
Address         DC          Rack        Status State   Load           
Owns    Token                                      

                                                                              
161930152162677484001961360738128229499     
198.168.0.1     datacenter1 rack1       Up     Normal  132.28 MB      
12.48%  13027320554261208311902766005835168982      
198.168.0.2     datacenter1 rack1       Up     Normal  99.34 MB       
75.07%  140745249930211229277235689500208693608     
198.168.0.3     datacenter1 rack1       Up     Normal  66.21 KB       
12.45%  161930152162677484001961360738128229499     
nutch@lab02:/code/apache-cassandra-0.8.4$ 


Here are the hadoop config.

        job4.setOutputFormatClass(ColumnFamilyOutputFormat.class);
        ConfigHelper.setOutputColumnFamily(job4.getConfiguration(), KEYSPACE,COLUMN_FAMILY
);
        ConfigHelper.setRpcPort(job4.getConfiguration(), ""9160);
        ConfigHelper.setInitialAddress(job4.getConfiguration(), "node01");
        ConfigHelper.setPartitioner(job4.getConfiguration(), "org.apache.cassandra.dht.RandomPartitioner");

Bleow is an exception message:

Error: java.net.UnknownHostException: /198.168.0.3
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:849)
    at java.net.InetAddress.getAddressFromNameService(InetAddress.java:1200)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1153)
    at java.net.InetAddress.getAllByName(InetAddress.java:1083)
    at java.net.InetAddress.getAllByName(InetAddress.java:1019)
    at java.net.InetAddress.getByName(InetAddress.java:969)
    at org.apache.cassandra.client.RingCache.refreshEndpointMap(RingCache.java:93)
    at org.apache.cassandra.client.RingCache.<init>(RingCache.java:67)
    at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.<init>(ColumnFamilyRecordWriter.java:98)
    at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.<init>(ColumnFamilyRecordWriter.java:92)
    at org.apache.cassandra.hadoop.ColumnFamilyOutputFormat.getRecordWriter(ColumnFamilyOutputFormat.java:132)
    at org.apache.cassandra.hadoop.ColumnFamilyOutputFormat.getRecordWriter(ColumnFamilyOutputFormat.java:62)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

note: Same /etc/hosts file has been used across all the nodes.

Kindly help me to resolve this issue?


Regards,

  Thamizhannal P

--- On Wed, 24/8/11, aaron morton <aaron@thelastpickle.com> wrote:

From: aaron morton <aaron@thelastpickle.com>
Subject: Re: multi-node cassandra config doubt
To: user@cassandra.apache.org
Date: Wednesday, 24 August, 2011, 2:40 PM

Did you get this sorted ? 
At a guess I would say there are no nodes listed in the Hadoop JobConf.
Cheers

-----------------Aaron MortonFreelance Cassandra Developer@aaronmortonhttp://www.thelastpickle.com



On 23/08/2011, at 9:51 PM, Thamizh wrote:
Hi All,

This is regarding multi-node cluster configuration doubt.

I have configured 3 nodes of cluster using Cassandra-0.8.4 and getting error when I ran Map/Reduce
job which uploads records from HDFS to Cassandra.

Here are my 3 nodes cluster config file (cassandra.yaml) for Cassandra:

node01:
    seeds: "node01,node02,node03"
    auto_bootstrap: false
    listen_address: 192.168.0.1
    rpc_address: 192.168.0.1


node02:

seeds: "node01,node02,node03"
auto_bootstrap: true
listen_address: 192.168.0.2
rpc_address: 192.168.0.2


node03:
seeds: "node01,node02,node03"
auto_bootstrap: true
listen_address: 192.168.0.3
rpc_address: 192.168.0.3

When I ran M/R program, I am getting below error
11/08/23 04:37:00 INFO
 mapred.JobClient:  map 100% reduce 11%
11/08/23 04:37:06 INFO mapred.JobClient:  map 100% reduce 22%
11/08/23 04:37:09 INFO mapred.JobClient:  map 100% reduce 33%
11/08/23 04:37:14 INFO mapred.JobClient: Task Id : attempt_201104211044_0719_r_000000_0, Status
: FAILED
java.lang.NullPointerException
    at org.apache.cassandra.client.RingCache.getRange(RingCache.java:130)
    at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.write(ColumnFamilyRecordWriter.java:125)
    at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter.write(ColumnFamilyRecordWriter.java:60)
    at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
    at CassTblUploader$TblUploadReducer.reduce(CassTblUploader.java:90)
    at CassTblUploader$TblUploadReducer.reduce(CassTblUploader.java:1)
  
  at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:563)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)


Is anything wrong on my cassandra.yaml file?

I followed http://wiki.apache.org/cassandra/MultinodeCluster for cluster configuration.

Regards,
Thamizhannal

Mime
View raw message