incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Tsvinev <oleg.tsvi...@gmail.com>
Subject Re: CRUD test
Date Sun, 25 Jul 2010 05:54:05 GMT
Thank you guys for your help!

Yes, I am using System.currentTimeMillis() in my CRUD test. Even though I'm
still using it my tests now run as expected. I do not use cassandra-cli
anymore.

@Ran great job on Hector, I wish there was more documentation but I managed.

@Jonathan, what is the recommended time source? I use batch_mutation to
insert and update multiple columns atomically. Do I have to use
the batch_mutation for deletion, too?

On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <jshook@gmail.com> wrote:

> Just to clarify, microseconds may be used, but they provide the same
> behavior as milliseconds if they aren't using a higher time resolution
> underneath. In some cases, the microseconds are generated simply as
> milliseconds * 1000, which doesn't actually fix any sequencing bugs.
>
> On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <rantav@gmail.com> wrote:
> > Hi Oleg, I didn't follow up the entire thread, but just to let you know
> that
> > the 0.6.* version of the CLI uses microsec as the time unit for
> timestamps.
> > Hector also uses micros to match that, however, previous versions of
> hector
> > (as well as the CLI) used milliseconds, not micro.
> > So if you're using hector version 0.6.0-11 or earlier, or by any chance
> in
> > some other ways are mixing milisec in your app (are you using
> > System.currentTimeMili() somewhere?) then the behavior you're seeing is
> > expected.
> >
> > On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <jshook@gmail.com>
> wrote:
> >>
> >> I think you are getting it.
> >>
> >> As far as what means what at which level, it's really about using them
> >> consistently in every case. The [row] key (or [row] key range) is a
> >> top-level argument for all of the operations, since it is the "key" to
> >> mapping the set of responsible nodes. The key is the part of the name
> >> of any column which most affects how the load is apportioned in the
> >> cluster, so it is used very early in request processing.
> >>
> >>
> >> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
> >> <Peter.Minearo@reardencommerce.com> wrote:
> >> > Consequentially the remove should look like:
> >> >
> >> >
> >> > ColumnPath cp1 = new ColumnPath("Super2");
> >> >                cp1.setSuper_column("Best Western".getBytes());
> >> >
> >> >                client.remove(KEYSPACE,
> >> >                                  "hotel",
> >> >                                  cp1,
> >> >                                  System.currentTimeMillis(),
> >> >                                  ConsistencyLevel.ONE);
> >> >
> >> >                ColumnPath cp2 = new ColumnPath("Super2");
> >> >                cp2.setSuper_column("Econolodge".getBytes());
> >> >
> >> >                client.remove(KEYSPACE,
> >> >                                  "hotel",
> >> >                                  cp2,
> >> >                                  System.currentTimeMillis(),
> >> >                                  ConsistencyLevel.ONE);
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Peter Minearo [mailto:Peter.Minearo@Reardencommerce.com]
> >> > Sent: Fri 7/23/2010 2:17 PM
> >> > To: user@cassandra.apache.org
> >> > Subject: RE: CRUD test
> >> >
> >> > CORRECTION:
> >> >
> >> > ColumnPath cp1 = new ColumnPath("Super2");
> >> > cp1.setSuper_column("Best Western".getBytes());
> >> > cp1.setColumn("name".getBytes());
> >> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(),
> >> > System.currentTimeMillis(), ConsistencyLevel.ALL);
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Peter Minearo [mailto:Peter.Minearo@Reardencommerce.com]
> >> > Sent: Friday, July 23, 2010 2:14 PM
> >> > To: user@cassandra.apache.org
> >> > Subject: RE: CRUD test
> >> >
> >> > Interesting!! Let me rephrase to make sure I understood what is going
> >> > on:
> >> >
> >> > When Inserting data via the "insert" function/method:
> >> >
> >> > void insert(string keyspace, string key, ColumnPath column_path,
> binary
> >> > value, i64 timestamp, ConsistencyLevel consistency_level)
> >> >
> >> > The "key" parameter is the actual Key to the "Row", which contains
> >> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
> >> >
> >> >
> >> >
> >> > INCORRECT:
> >> > ColumnPath cp1 = new ColumnPath("Super2");
> >> > cp1.setSuper_column("hotel".getBytes());
> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
> >> > "name", cp1, "Best Western of SF".getBytes(),
> System.currentTimeMillis(),
> >> > ConsistencyLevel.ALL);
> >> >
> >> >
> >> > CORRECT:
> >> > ColumnPath cp1 = new ColumnPath("Super2");
> >> > cp1.setSuper_column("name".getBytes());
> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
> >> > "hotel", cp1, "Best Western of SF".getBytes(),
> System.currentTimeMillis(),
> >> > ConsistencyLevel.ALL);
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Jonathan Shook [mailto:jshook@gmail.com]
> >> > Sent: Friday, July 23, 2010 1:49 PM
> >> > To: user@cassandra.apache.org
> >> > Subject: Re: CRUD test
> >> >
> >> > Correct.
> >> >
> >> > After the initial insert,
> >> >
> >> > cassandra> get Keyspace1.Super2['name']
> >> > => (super_column=hotel,
> >> >     (column=Best Western, value=Best Western of SF,
> >> > timestamp=1279916772571)
> >> >     (column=Econolodge, value=Econolodge of SF,
> >> > timestamp=1279916772573)) Returned 1 results.
> >> >
> >> > ... and ...
> >> >
> >> > cassandra> get Keyspace1.Super2['hotel']
> >> > Returned 0 results.
> >> >
> >> >
> >> >
> >> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
> >> > <Peter.Minearo@reardencommerce.com> wrote:
> >> >> The Model Should look like:
> >> >>
> >> >>
> >> >> Super2 = {
> >> >>        hotel: {
> >> >>                        Best Western: {name: "Best Western of SF"}
> >> >>                        Econolodge: {name: "Econolodge of SF"}
> >> >>        }
> >> >> }
> >> >>
> >> >> Are the CRUD Operations not referencing this correctly?
> >> >>
> >> >>
> >> >>
> >> >> -----Original Message-----
> >> >> From: Jonathan Shook [mailto:jshook@gmail.com]
> >> >> Sent: Friday, July 23, 2010 1:34 PM
> >> >> To: user@cassandra.apache.org
> >> >> Subject: Re: CRUD test
> >> >>
> >> >> There seem to be data consistency bugs in the test.  Are "name" and
> >> >> "hotel" being used in a pair-wise way?
> >> >> Specifically, the first test is using creating one and checking for
> the
> >> >> other.
> >> >>
> >> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <
> oleg.tsvinev@gmail.com>
> >> >> wrote:
> >> >>> Johathan,
> >> >>> I followed your suggestion. Unfortunately, CRUD test still does
not
> >> >>> work for me. Can you provide a simplest CRUD test possible that
> works?
> >> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <jshook@gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> I suspect that it is still your timestamps.
> >> >>>> You can verify this with a fake timestamp generator that is
simply
> >> >>>> incremented on each getTimestamp().
> >> >>>>
> >> >>>> 1 millisecond is a long time for code that is wrapped tightly
in a
> >> >>>> test. You are likely using the same logical time stamp for
multiple
> >> >>>> operations.
> >> >>>>
> >> >>>>
> >> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
> >> >>>> <Peter.Minearo@reardencommerce.com> wrote:
> >> >>>> > I am able to reproduce his problem. If you take the default
> >> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily
with
> >> >>>> > the code below.  You will see that the data is not getting
> created
> >> >>>> > once you run the delete.  It seems to not allow you to
create
> data
> >> >>>> > via Thrift.  HOWEVER, data can be created via the command
line
> >> >>>> > tool.
> >> >>>> >
> >> >>>> > import java.io.UnsupportedEncodingException;
> >> >>>> > import java.util.List;
> >> >>>> >
> >> >>>> > import org.apache.cassandra.thrift.Cassandra;
> >> >>>> > import org.apache.cassandra.thrift.Column;
> >> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
> >> >>>> > import org.apache.cassandra.thrift.ColumnParent;
> >> >>>> > import org.apache.cassandra.thrift.ColumnPath;
> >> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
> >> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
> >> >>>> > import org.apache.cassandra.thrift.NotFoundException;
> >> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
> >> >>>> > import org.apache.cassandra.thrift.SliceRange;
> >> >>>> > import org.apache.cassandra.thrift.SuperColumn;
> >> >>>> > import org.apache.cassandra.thrift.TimedOutException;
> >> >>>> > import org.apache.cassandra.thrift.UnavailableException;
> >> >>>> > import org.apache.thrift.TException; import
> >> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
> >> >>>> > import org.apache.thrift.protocol.TProtocol;
> >> >>>> > import org.apache.thrift.transport.TSocket;
> >> >>>> > import org.apache.thrift.transport.TTransport;
> >> >>>> >
> >> >>>> >
> >> >>>> > public class CrudTest {
> >> >>>> >
> >> >>>> >
> >> >>>> >        private static final String KEYSPACE = "Keyspace1";
> >> >>>> >
> >> >>>> >
> >> >>>> >        public static void main(String[] args) {
> >> >>>> >                CrudTest client = new CrudTest();
> >> >>>> >
> >> >>>> >                try {
> >> >>>> >                        client.run();
> >> >>>> >                } catch (Exception e) {
> >> >>>> >                        e.printStackTrace();
> >> >>>> >                }
> >> >>>> >
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> >        public void run() throws TException,
> >> >>>> > InvalidRequestException, UnavailableException,
> >> >>>> > UnsupportedEncodingException, NotFoundException,
> TimedOutException
> >> >>>> > {
> >> >>>> >                TTransport tr = new TSocket("localhost",
9160);
> >> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
> >> >>>> >                Cassandra.Client client = new
> >> >>>> > Cassandra.Client(proto);
> >> >>>> >                tr.open();
> >> >>>> >
> >> >>>> >                System.out.println("******** CREATING DATA
> >> >>>> > *********");
> >> >>>> >                createData(client);
> >> >>>> >                getData(client);
> >> >>>> >                System.out.println();
> >> >>>> >                System.out.println("******** DELETING DATA
> >> >>>> > *********");
> >> >>>> >                deleteData(client);
> >> >>>> >                getData(client);
> >> >>>> >                System.out.println();
> >> >>>> >                System.out.println("******** CREATING DATA
> >> >>>> > *********");
> >> >>>> >                createData(client);
> >> >>>> >                getData(client);
> >> >>>> >
> >> >>>> >                tr.close();
> >> >>>> >          }
> >> >>>> >
> >> >>>> >
> >> >>>> >        private void createData(Cassandra.Client client)
throws
> >> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
> >> >>>> > TException {
> >> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
> >> >>>> >                cp1.setSuper_column("hotel".getBytes());
> >> >>>> >                cp1.setColumn("Best Western".getBytes());
> >> >>>> >
> >> >>>> >
> >> >>>> >                client.insert(KEYSPACE,
> >> >>>> >                          "name",
> >> >>>> >                          cp1,
> >> >>>> >                          "Best Western of SF".getBytes(),
> >> >>>> >                          System.currentTimeMillis(),
> >> >>>> >                          ConsistencyLevel.ALL);
> >> >>>> >
> >> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
> >> >>>> >                cp2.setSuper_column("hotel".getBytes());
> >> >>>> >                cp2.setColumn("Econolodge".getBytes());
> >> >>>> >
> >> >>>> >                client.insert(KEYSPACE,
> >> >>>> >                                  "name",
> >> >>>> >                                  cp2,
> >> >>>> >                                  "Econolodge of SF".getBytes(),
> >> >>>> >                                  System.currentTimeMillis(),
> >> >>>> >                                  ConsistencyLevel.ALL);
> >> >>>> >
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> >        private void deleteData(Cassandra.Client client)
throws
> >> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
> >> >>>> > TException {
> >> >>>> >
> >> >>>> >                client.remove(KEYSPACE,
> >> >>>> >                                  "hotel",
> >> >>>> >                                  new ColumnPath("Super2"),
> >> >>>> >                                  System.currentTimeMillis(),
> >> >>>> >                                  ConsistencyLevel.ONE);
> >> >>>> >
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> >        private void getData(Cassandra.Client client) throws
> >> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
> >> >>>> > TException {
> >> >>>> >                SliceRange sliceRange = new SliceRange();
> >> >>>> >                sliceRange.setStart(new byte[] {});
> >> >>>> >                sliceRange.setFinish(new byte[] {});
> >> >>>> >
> >> >>>> >                SlicePredicate slicePredicate = new
> >> >>>> > SlicePredicate();
> >> >>>> >                slicePredicate.setSlice_range(sliceRange);
> >> >>>> >
> >> >>>> >                getData(client, slicePredicate);
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> >        private void getData(Cassandra.Client client,
> >> >>>> > SlicePredicate
> >> >>>> > slicePredicate) throws InvalidRequestException,
> >> >>>> > UnavailableException, TimedOutException, TException {
> >> >>>> >                List<ColumnOrSuperColumn> coscList
=
> >> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
> >> >>>> > slicePredicate, ConsistencyLevel.ALL);
> >> >>>> >
> >> >>>> >                if (coscList.isEmpty()) {
> >> >>>> >                        System.out.println("Column Or Super
Column
> >> >>>> > is EMPTY");
> >> >>>> >                }
> >> >>>> >
> >> >>>> >                for (ColumnOrSuperColumn cosc: coscList)
{
> >> >>>> >
> >> >>>> >                        if (cosc == null) {
> >> >>>> >                                System.out.println("NULL
RETURN
> >> >>>> > VALUE");
> >> >>>> >                        }
> >> >>>> >
> >> >>>> >                        SuperColumn superColumn =
> >> >>>> > cosc.getSuper_column();
> >> >>>> >
> >> >>>> >                        if (superColumn == null) {
> >> >>>> >                                System.out.println("Super
Column
> is
> >> >>>> > NULL");
> >> >>>> >                        }
> >> >>>> >                        else {
> >> >>>> >                                showSuperColumnInfo(superColumn);
> >> >>>> >                        }
> >> >>>> >
> >> >>>> >                }
> >> >>>> >
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn)
> {
> >> >>>> >                System.out.println("######## Super Columns
> >> >>>> > ###########");
> >> >>>> >                System.out.println("Super Column Name =
" + new
> >> >>>> > String(superColumn.getName()));
> >> >>>> >                List<Column> columnList =
> superColumn.getColumns();
> >> >>>> >
> >> >>>> >                System.out.println("--------- Start Columns
> >> >>>> > -----------");
> >> >>>> >                for (Column column: columnList) {
> >> >>>> >                        System.out.println("Column Name
= " + new
> >> >>>> > String(column.getName()));
> >> >>>> >                        System.out.println("Column Value
= " + new
> >> >>>> > String(column.getValue()));
> >> >>>> >                }
> >> >>>> >                System.out.println("--------- End Columns
> >> >>>> > -----------");
> >> >>>> >
> >> >>>> > System.out.println("##################################");
> >> >>>> >        }
> >> >>>> >
> >> >>>> >
> >> >>>> > }
> >> >>>> >
> >> >>>> >
> >> >>>> >
> >> >>>> >
> >> >>>> > -----Original Message-----
> >> >>>> > From: Oleg Tsvinev [mailto:oleg.tsvinev@gmail.com]
> >> >>>> > Sent: Thu 7/22/2010 1:56 PM
> >> >>>> > To: user@cassandra.apache.org
> >> >>>> > Subject: Re: CRUD test
> >> >>>> >
> >> >>>> > Yes, and that was the issue. But now after I delete a
row from
> >> >>>> > cassandra-cli, I cannot insert anything back with my code.
Insert
> >> >>>> > code works does not throw any exceptions but when I read
just
> >> >>>> > inserted columns I get NotFoundException at the last line:
> >> >>>> >
> >> >>>> >            client = borrowClient();
> >> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
> >> >>>> > CONSISTENCY_LEVEL);
> >> >>>> >            ColumnPath cp = new ColumnPath(application);
> >> >>>> >            cp.setSuper_column(uuid.getBytes());
> >> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category,
> cp);
> >> >>>> >
> >> >>>> > It makes me think that once I remove supercolumn it cannot
be
> >> >>>> > created again.
> >> >>>> >
> >> >>>> >
> >> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <zodiaczx6@gmail.com>
> >> >>>> > wrote:
> >> >>>> >
> >> >>>> > Have you checked the timestamp you're using for the subsequent
> >> >>>> > inserts is higher than that used in the delete?
> >> >>>> >
> >> >>>> >
> >> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
> >> >>>> > <oleg.tsvinev@gmail.com>
> >> >>>> > wrote:
> >> >>>> >> Hi there,
> >> >>>> >> I'm try...
> >> >>>> > --
> >> >>>> > Maybe she awoke to see the roommate's boyfriend swinging
from the
> >> >>>> > chandelier wearing a boar's head.
> >> >>>> >
> >> >>>> > Something which you, I, and everyone else would call "Tuesday",
> of
> >> >>>> > course.
> >> >>>> >
> >> >>>> >
> >> >>>
> >> >>>
> >> >>
> >> >
> >> >
> >
> >
>

Mime
View raw message