cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Wang <>
Subject RE: pig + hadoop
Date Wed, 20 Apr 2011 02:54:01 GMT
Did you set PIG_RPC_PORT in your I was seeing this error for a while before
I added that.


From: pob []
Sent: Tuesday, April 19, 2011 6:42 PM
Subject: Re: pig + hadoop

Hey Aaron,

I read it, and all of 3 env variables was exported. The results are same.

2011/4/20 aaron morton <<>>
Am guessing but here goes. Looks like the cassandra RPC port is not set, did you follow these
steps in contrib/pig/README.txt

Finally, set the following as environment variables (uppercase,
underscored), or as Hadoop configuration variables (lowercase, dotted):
* PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening on
* PIG_INITIAL_ADDRESS or cassandra.thrift.address : initial address to connect to
* PIG_PARTITIONER or cassandra.partitioner.class : cluster partitioner

Hope that helps.

On 20 Apr 2011, at 11:28, pob wrote:


I did cluster configuration by When I run
pig example-script.pig
-x local, everything is fine and i get correct results.

Problem is occurring with -x mapreduce

Im getting those errors :>

2011-04-20 01:24:21,791 [main] ERROR - ERROR: java.lang.NumberFormatException:
2011-04-20 01:24:21,792 [main] ERROR - 1 map reduce
job(s) failed!
2011-04-20 01:24:21,793 [main] INFO - Script Statistics:

Failed to read data from "cassandra://Keyspace1/Standard1"

Failed to produce result in "hdfs://ip:54310/tmp/temp-1383865669/tmp-1895601791"

Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_201104200056_0005   ->      null,
null    ->      null,

2011-04-20 01:24:21,793 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!
2011-04-20 01:24:21,803 [main] ERROR - ERROR 1066: Unable
to open iterator for alias topnames. Backend error : java.lang.NumberFormatException: null

thats from jobtasks web management - error  from task directly:

java.lang.RuntimeException: java.lang.NumberFormatException: null
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(
at org.apache.hadoop.mapred.MapTask.runNewMapper(
at org.apache.hadoop.mapred.Child.main(
Caused by: java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(
at java.lang.Integer.parseInt(
at org.apache.cassandra.hadoop.ConfigHelper.getRpcPort(
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(
... 5 more

Any suggestions where should be problem?


View raw message