incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Shook <jsh...@gmail.com>
Subject Re: CRUD test
Date Sat, 24 Jul 2010 21:36:35 GMT
Just to clarify, microseconds may be used, but they provide the same
behavior as milliseconds if they aren't using a higher time resolution
underneath. In some cases, the microseconds are generated simply as
milliseconds * 1000, which doesn't actually fix any sequencing bugs.

On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <rantav@gmail.com> wrote:
> Hi Oleg, I didn't follow up the entire thread, but just to let you know that
> the 0.6.* version of the CLI uses microsec as the time unit for timestamps.
> Hector also uses micros to match that, however, previous versions of hector
> (as well as the CLI) used milliseconds, not micro.
> So if you're using hector version 0.6.0-11 or earlier, or by any chance in
> some other ways are mixing milisec in your app (are you using
> System.currentTimeMili() somewhere?) then the behavior you're seeing is
> expected.
>
> On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <jshook@gmail.com> wrote:
>>
>> I think you are getting it.
>>
>> As far as what means what at which level, it's really about using them
>> consistently in every case. The [row] key (or [row] key range) is a
>> top-level argument for all of the operations, since it is the "key" to
>> mapping the set of responsible nodes. The key is the part of the name
>> of any column which most affects how the load is apportioned in the
>> cluster, so it is used very early in request processing.
>>
>>
>> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> <Peter.Minearo@reardencommerce.com> wrote:
>> > Consequentially the remove should look like:
>> >
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("Best Western".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp1,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("Econolodge".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp2,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:Peter.Minearo@Reardencommerce.com]
>> > Sent: Fri 7/23/2010 2:17 PM
>> > To: user@cassandra.apache.org
>> > Subject: RE: CRUD test
>> >
>> > CORRECTION:
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("Best Western".getBytes());
>> > cp1.setColumn("name".getBytes());
>> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(),
>> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:Peter.Minearo@Reardencommerce.com]
>> > Sent: Friday, July 23, 2010 2:14 PM
>> > To: user@cassandra.apache.org
>> > Subject: RE: CRUD test
>> >
>> > Interesting!! Let me rephrase to make sure I understood what is going
>> > on:
>> >
>> > When Inserting data via the "insert" function/method:
>> >
>> > void insert(string keyspace, string key, ColumnPath column_path, binary
>> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >
>> > The "key" parameter is the actual Key to the "Row", which contains
>> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >
>> >
>> >
>> > INCORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("hotel".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> > CORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("name".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jonathan Shook [mailto:jshook@gmail.com]
>> > Sent: Friday, July 23, 2010 1:49 PM
>> > To: user@cassandra.apache.org
>> > Subject: Re: CRUD test
>> >
>> > Correct.
>> >
>> > After the initial insert,
>> >
>> > cassandra> get Keyspace1.Super2['name']
>> > => (super_column=hotel,
>> >     (column=Best Western, value=Best Western of SF,
>> > timestamp=1279916772571)
>> >     (column=Econolodge, value=Econolodge of SF,
>> > timestamp=1279916772573)) Returned 1 results.
>> >
>> > ... and ...
>> >
>> > cassandra> get Keyspace1.Super2['hotel']
>> > Returned 0 results.
>> >
>> >
>> >
>> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> > <Peter.Minearo@reardencommerce.com> wrote:
>> >> The Model Should look like:
>> >>
>> >>
>> >> Super2 = {
>> >>        hotel: {
>> >>                        Best Western: {name: "Best Western of
SF"}
>> >>                        Econolodge: {name: "Econolodge of SF"}
>> >>        }
>> >> }
>> >>
>> >> Are the CRUD Operations not referencing this correctly?
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jonathan Shook [mailto:jshook@gmail.com]
>> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> To: user@cassandra.apache.org
>> >> Subject: Re: CRUD test
>> >>
>> >> There seem to be data consistency bugs in the test.  Are "name" and
>> >> "hotel" being used in a pair-wise way?
>> >> Specifically, the first test is using creating one and checking for the
>> >> other.
>> >>
>> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <oleg.tsvinev@gmail.com>
>> >> wrote:
>> >>> Johathan,
>> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >>> work for me. Can you provide a simplest CRUD test possible that works?
>> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <jshook@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> I suspect that it is still your timestamps.
>> >>>> You can verify this with a fake timestamp generator that is simply
>> >>>> incremented on each getTimestamp().
>> >>>>
>> >>>> 1 millisecond is a long time for code that is wrapped tightly in
a
>> >>>> test. You are likely using the same logical time stamp for multiple
>> >>>> operations.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >>>> <Peter.Minearo@reardencommerce.com> wrote:
>> >>>> > I am able to reproduce his problem. If you take the default
>> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily
with
>> >>>> > the code below.  You will see that the data is not getting
created
>> >>>> > once you run the delete.  It seems to not allow you to create
data
>> >>>> > via Thrift.  HOWEVER, data can be created via the command
line
>> >>>> > tool.
>> >>>> >
>> >>>> > import java.io.UnsupportedEncodingException;
>> >>>> > import java.util.List;
>> >>>> >
>> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >>>> > import org.apache.cassandra.thrift.Column;
>> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >>>> > import org.apache.thrift.TException; import
>> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >>>> > import org.apache.thrift.transport.TSocket;
>> >>>> > import org.apache.thrift.transport.TTransport;
>> >>>> >
>> >>>> >
>> >>>> > public class CrudTest {
>> >>>> >
>> >>>> >
>> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >>>> >
>> >>>> >
>> >>>> >        public static void main(String[] args) {
>> >>>> >                CrudTest client = new CrudTest();
>> >>>> >
>> >>>> >                try {
>> >>>> >                        client.run();
>> >>>> >                } catch (Exception e) {
>> >>>> >                        e.printStackTrace();
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        public void run() throws TException,
>> >>>> > InvalidRequestException, UnavailableException,
>> >>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>> >>>> > {
>> >>>> >                TTransport tr = new TSocket("localhost",
9160);
>> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >>>> >                Cassandra.Client client = new
>> >>>> > Cassandra.Client(proto);
>> >>>> >                tr.open();
>> >>>> >
>> >>>> >                System.out.println("******** CREATING
DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** DELETING
DATA
>> >>>> > *********");
>> >>>> >                deleteData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** CREATING
DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >
>> >>>> >                tr.close();
>> >>>> >          }
>> >>>> >
>> >>>> >
>> >>>> >        private void createData(Cassandra.Client client)
throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >>>> >
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                          "name",
>> >>>> >                          cp1,
>> >>>> >                          "Best Western of SF".getBytes(),
>> >>>> >                          System.currentTimeMillis(),
>> >>>> >                          ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                                  "name",
>> >>>> >                                  cp2,
>> >>>> >                                  "Econolodge
of SF".getBytes(),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void deleteData(Cassandra.Client client)
throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >
>> >>>> >                client.remove(KEYSPACE,
>> >>>> >                                  "hotel",
>> >>>> >                                  new ColumnPath("Super2"),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ONE);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                SliceRange sliceRange = new SliceRange();
>> >>>> >                sliceRange.setStart(new byte[] {});
>> >>>> >                sliceRange.setFinish(new byte[] {});
>> >>>> >
>> >>>> >                SlicePredicate slicePredicate = new
>> >>>> > SlicePredicate();
>> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >>>> >
>> >>>> >                getData(client, slicePredicate);
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client,
>> >>>> > SlicePredicate
>> >>>> > slicePredicate) throws InvalidRequestException,
>> >>>> > UnavailableException, TimedOutException, TException {
>> >>>> >                List<ColumnOrSuperColumn> coscList
=
>> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                if (coscList.isEmpty()) {
>> >>>> >                        System.out.println("Column
Or Super Column
>> >>>> > is EMPTY");
>> >>>> >                }
>> >>>> >
>> >>>> >                for (ColumnOrSuperColumn cosc: coscList)
{
>> >>>> >
>> >>>> >                        if (cosc == null) {
>> >>>> >                                System.out.println("NULL
RETURN
>> >>>> > VALUE");
>> >>>> >                        }
>> >>>> >
>> >>>> >                        SuperColumn superColumn
=
>> >>>> > cosc.getSuper_column();
>> >>>> >
>> >>>> >                        if (superColumn == null)
{
>> >>>> >                                System.out.println("Super
Column is
>> >>>> > NULL");
>> >>>> >                        }
>> >>>> >                        else {
>> >>>> >                                showSuperColumnInfo(superColumn);
>> >>>> >                        }
>> >>>> >
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn)
{
>> >>>> >                System.out.println("######## Super Columns
>> >>>> > ###########");
>> >>>> >                System.out.println("Super Column Name
= " + new
>> >>>> > String(superColumn.getName()));
>> >>>> >                List<Column> columnList = superColumn.getColumns();
>> >>>> >
>> >>>> >                System.out.println("--------- Start
Columns
>> >>>> > -----------");
>> >>>> >                for (Column column: columnList) {
>> >>>> >                        System.out.println("Column
Name = " + new
>> >>>> > String(column.getName()));
>> >>>> >                        System.out.println("Column
Value = " + new
>> >>>> > String(column.getValue()));
>> >>>> >                }
>> >>>> >                System.out.println("--------- End Columns
>> >>>> > -----------");
>> >>>> >
>> >>>> > System.out.println("##################################");
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> > }
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > -----Original Message-----
>> >>>> > From: Oleg Tsvinev [mailto:oleg.tsvinev@gmail.com]
>> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >>>> > To: user@cassandra.apache.org
>> >>>> > Subject: Re: CRUD test
>> >>>> >
>> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >>>> > cassandra-cli, I cannot insert anything back with my code.
Insert
>> >>>> > code works does not throw any exceptions but when I read just
>> >>>> > inserted columns I get NotFoundException at the last line:
>> >>>> >
>> >>>> >            client = borrowClient();
>> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >>>> > CONSISTENCY_LEVEL);
>> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category,
cp);
>> >>>> >
>> >>>> > It makes me think that once I remove supercolumn it cannot
be
>> >>>> > created again.
>> >>>> >
>> >>>> >
>> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <zodiaczx6@gmail.com>
>> >>>> > wrote:
>> >>>> >
>> >>>> > Have you checked the timestamp you're using for the subsequent
>> >>>> > inserts is higher than that used in the delete?
>> >>>> >
>> >>>> >
>> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >>>> > <oleg.tsvinev@gmail.com>
>> >>>> > wrote:
>> >>>> >> Hi there,
>> >>>> >> I'm try...
>> >>>> > --
>> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from
the
>> >>>> > chandelier wearing a boar's head.
>> >>>> >
>> >>>> > Something which you, I, and everyone else would call "Tuesday",
of
>> >>>> > course.
>> >>>> >
>> >>>> >
>> >>>
>> >>>
>> >>
>> >
>> >
>
>

Mime
View raw message