cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-7241) Pig test fails on 2.1 branch
Date Wed, 28 May 2014 16:23:03 GMT


Sylvain Lebresne updated CASSANDRA-7241:

    Attachment: 0001-Revert-5417-changes-to-hadoop-stuff.txt

bq. If we roll back the cell changes in Pig/Hadoop, those errors should be gone. We can create
new Hadoop Reader/InputFormat using cell implementation if we have to, for hadoop tasks are
running in a separate process. I don't see using Cell instead of thrift Column really makes
much difference

Let's clear some confusion here. Pig/Hadoop in Cassandra 2.0 (and in 2.1 pre- and post-CASSANDRA-5417)
*does not* use the thrift Column class, it uses {{org.apache.cassandra.db.Column}}. I very
much agree that it *should* be using the thrift Column class (or it's own custom class) but
it's simply not the case and it is *not* because of CASSANDRA-5417. The only reason the Cell
class is used now is because that's how we renamed the {{org.apache.cassandra.db.Column}}
(that part was not even part of CASSANDRA-5417 so the renaming itself is definitively not
involved here). There is no roll back to using the thrift Column class since it was never
used in the first place.

Now back to CASSANDRA-5417, there were a few changes whose goal was to make proper use of
the different cellname types since I hadn't got that the use of the Column class was not linked
to anything internal. And at least in the case of ColumnFamilyRecordReader it assumes dense
cell names where it probably shouldn't. Those changes were not really needed and attaching
a patch to somewhat revert them. This "might" fix ThriftColumnFamilyDataTypeTest but I actually
don't get reliable failure of that test on 2.1 so I'm not entirely sure (it failed twice out
of 5 runs with the error you pasted on current 2.1 but passed on 5 runs with the attached

> Pig test fails on 2.1 branch
> ----------------------------
>                 Key: CASSANDRA-7241
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Alex Liu
>            Assignee: Sylvain Lebresne
>             Fix For: 2.1 rc1
>         Attachments: 0001-Revert-5417-changes-to-hadoop-stuff.txt, repro-cli.txt, repro-pig.txt
> run ant pig-test on cassandra-2.1 branch. There are many tests failed. I trace it a little
and find out Pig test fails starts from
> commit.
> It looks like storage changes break Pig tests.

This message was sent by Atlassian JIRA

View raw message