cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Howe (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-3552) Pig data objects returned by CassandraStorage behave irrationally.
Date Thu, 01 Dec 2011 23:08:40 GMT
Pig data objects returned by CassandraStorage behave irrationally.
------------------------------------------------------------------

                 Key: CASSANDRA-3552
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3552
             Project: Cassandra
          Issue Type: Bug
          Components: Contrib
    Affects Versions: 1.0.3
         Environment: Ubuntu
            Reporter: Chris Howe


When I try to perform computations on data that I get back from CassandraStorage in Pig, I
see inexplicable results.

For example, on a column family that has UTF8Type as the key validator, I do the following:

A = LOAD 'cassandra://keyspace/colfam' USING CassandraStorage();
B = FOREEACH A GENERATE (chararray) key;
STORE B INTO 'tempfile';
C = LOAD 'tempfile' AS (key:chararray);
D1 = FOREACH B GENERATE SUBSTRING(key,0,10);
D2 = FOREACH C GENERATE SUBSTRING(key,0,10);

DUMP D1;
DUMP D2;


For D1 I get
()
()
()
()
()

For D2 I get:
(a)
(b x y)
(b)
(a b c)
(a c b)


Clearly something has gone awry!

I have tried many workarounds and other functions. TOKENIZE has an entirely different behavior:

E = FOREACH B GENERATE TOKENIZE(key)

Ultimately this throws an exception:
2011-12-01 15:01:56,007 [Thread-149] WARN  org.apache.hadoop.mapred.LocalJobRunner - job_local_0010
org.apache.pig.backend.executionengine.ExecException: ERROR 2114: Expected input to be chararray,
but got org.apache.pig.data.DataByteArray
	at org.apache.pig.builtin.TOKENIZE.exec(TOKENIZE.java:62)
	at org.apache.pig.builtin.TOKENIZE.exec(TOKENIZE.java:43)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:245)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:287)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:338)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message