incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Sanderson <asander...@lingotek.com>
Subject Re: Problems with subcolumn retrieval after upgrade from 0.6 to 0.7
Date Mon, 18 Apr 2011 16:12:41 GMT
Ok, I made the changes and tried again.  Here is the before modifying my
method using a simple get, confirmed the same output in the cli:

DEBUG [pool-1-thread-2] 2011-04-18 09:37:23,910 CassandraServer.java (line
279) get
DEBUG [pool-1-thread-2] 2011-04-18 09:37:23,911 StorageProxy.java (line 322)
Command/ConsistencyLevel is SliceByNamesReadCommand(table='DocStore',
key=64316337663662392d313432352d346661622d623037342d353537346335346361653038,
columnParent='QueryPath(columnFamilyName='Tran
slationsByTarget', superColumnName='java.nio.HeapByteBuffer[pos=95 lim=211
cap=244]', columnName='null')',
columns=[7c2f5d5b-b370-42e1-a6a2-77fc721440fe,])/ALL
DEBUG [pool-1-thread-2] 2011-04-18 09:37:23,911 ReadCallback.java (line 84)
Blockfor/repair is 1/true; setting up requests to localhost/127.0.0.1
DEBUG [pool-1-thread-2] 2011-04-18 09:37:23,911 StorageProxy.java (line 345)
reading data locally
DEBUG [ReadStage:4] 2011-04-18 09:37:23,911 StorageProxy.java (line 450)
LocalReadRunnable reading SliceByNamesReadCommand(table='DocStore',
key=64316337663662392d313432352d346661622d623037342d353537346335346361653038,
columnParent='QueryPath(columnFamilyName='Translatio
nsByTarget', superColumnName='java.nio.HeapByteBuffer[pos=95 lim=211
cap=244]', columnName='null')',
columns=[7c2f5d5b-b370-42e1-a6a2-77fc721440fe,])
DEBUG [pool-1-thread-2] 2011-04-18 09:37:23,912 StorageProxy.java (line 395)
Read: 1 ms.
ERROR [pool-1-thread-2] 2011-04-18 09:37:23,912 Cassandra.java (line 2665)
Internal error processing get
java.lang.AssertionError
        at
org.apache.cassandra.thrift.CassandraServer.get(CassandraServer.java:300)
        at
org.apache.cassandra.thrift.Cassandra$Processor$get.process(Cassandra.java:2655)
        at
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555)
        at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:206)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)

And here is the after...it succeeds here but still gives me multiple
subcolumns in the response.  Same behavior, it seems, I'm just sidestepping
the original AssertionError:

DEBUG [pool-1-thread-6] 2011-04-18 09:50:26,617 CassandraServer.java (line
232) get_slice
DEBUG [pool-1-thread-6] 2011-04-18 09:50:26,617 StorageProxy.java (line 322)
Command/ConsistencyLevel is SliceByNamesReadCommand(table='DocStore',
key=64316337663662392d313432352d346661622d623037342d353537346335346361653038,
columnParent='QueryPath(columnFamilyName='TranslationsByTarget',
superColumnName='java.nio.HeapByteBuffer[pos=101 lim=217 cap=259]',
columnName='null')', columns=[7c2f5d5b-b370-42e1-a6a2-77fc721440fe,])/ALL
DEBUG [pool-1-thread-6] 2011-04-18 09:50:26,617 ReadCallback.java (line 84)
Blockfor/repair is 1/true; setting up requests to localhost/127.0.0.1
DEBUG [pool-1-thread-6] 2011-04-18 09:50:26,617 StorageProxy.java (line 345)
reading data locally
DEBUG [ReadStage:3] 2011-04-18 09:50:26,618 StorageProxy.java (line 450)
LocalReadRunnable reading SliceByNamesReadCommand(table='DocStore',
key=64316337663662392d313432352d346661622d623037342d353537346335346361653038,
columnParent='QueryPath(columnFamilyName='TranslationsByTarget',
superColumnName='java.nio.HeapByteBuffer[pos=101 lim=217 cap=259]',
columnName='null')', columns=[7c2f5d5b-b370-42e1-a6a2-77fc721440fe,])
DEBUG [pool-1-thread-6] 2011-04-18 09:50:26,618 StorageProxy.java (line 395)
Read: 0 ms.

My comparators are relatively simple.  Basically I have a schema that
required heterogenous columns, but I needed to be able to deserialize them
in unique ways.  So there is always a type byte that precedes the bytes of
the data.  The supercolumn in this case is a general data type, which
happens to represent a serializable object:

  public void validate(ByteBuffer bytes)
    throws MarshalException
  {
    if(bytes.remaining() == 0)
      return;

    validateDataType(bytes.get(bytes.position()));
    return;
  }

  public int compare(ByteBuffer bytes1, ByteBuffer bytes2)
  {
    if (bytes1.remaining() == 0)
      return bytes2.remaining() == 0 ? 0 : -1;
    else if (bytes2.remaining() == 0)
      return 1;
    else
    {
      // compare type
bytes

      byte T1 = bytes1.get(bytes1.position());
      byte T2 = bytes2.get(bytes2.position());
      if (T1 != T2)
        return (T1 - T2);

      // compare
values

      return ByteBufferUtil.compareUnsigned(bytes1, bytes2);
    }
  }

The subcolumn is similar...just a UUID with a type byte prefix:

  public void validate(ByteBuffer bytes)
    throws MarshalException
  {
    if(bytes.remaining() == 0)
      return;

    validateDataType(bytes.get(bytes.position()));
    if((bytes.remaining() - 1) == 0)
      return;
    else if((bytes.remaining() - 1) != 16)
      throw new MarshalException("UUID value must be exactly 16 bytes");
  }

  public int compare(ByteBuffer bytes1, ByteBuffer bytes2)
  {
    if (bytes1.remaining() == 0)
      return bytes2.remaining() == 0 ? 0 : -1;
    else if (bytes2.remaining() == 0)
      return 1;
    else
    {
      // compare type
bytes

      byte T1 = bytes1.get(bytes1.position());
      byte T2 = bytes2.get(bytes2.position());
      if (T1 != T2)
        return (T1 - T2);

      // compare
values

      UUID U1 = getUUID(bytes1, bytes1.position()+1);
      UUID U2 = getUUID(bytes2, bytes2.position()+1);
      return U1.compareTo(U2);
    }
  }

  static UUID getUUID(ByteBuffer bytes, int pos)
  {
    long msBits = bytes.getLong(pos);
    long lsBits = bytes.getLong(pos+8);
    return new UUID(msBits, lsBits);
  }

All of my buffer reads are done by index, the position shouldn't be changing
at all.

Abe Sanderson

On Sat, Apr 16, 2011 at 5:38 PM, aaron morton <aaron@thelastpickle.com>wrote:

> Can you run the same request as a get_slice naming the column in the
> SlicePredicate and see what comes back ?
>
> Can you reproduce the fault with logging set at DEBUG and send the logs ?
>
> Also, whats the compare function like for your custom type ?
>
> Cheers
> Aaron
>
>
> On 16 Apr 2011, at 07:34, Abraham Sanderson wrote:
>
> > I'm having some issues with a few of my ColumnFamilies after a cassandra
> upgrade/import from 0.6.1 to 0.7.4.  I followed the instructions to upgrade
> and everything seem to work OK...until I got into the application and
> noticed some wierd behavior.  I was getting the following stacktrace in
> cassandra occassionally when I did get operations for a single subcolumn for
> some of the Super type CFs:
> >
> > ERROR 12:56:05,669 Internal error processing get
> > java.lang.AssertionError
> >         at org.apache.cassandra.thrift.
> > CassandraServer.get(CassandraServer.java:300)
> >         at
> org.apache.cassandra.thrift.Cassandra$Processor$get.process(Cassandra.java:2655)
> >         at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555)
> >         at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:206)
> >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >         at java.lang.Thread.run(Thread.java:636)
> >
> > The assertion that is failing is the check that only one column is
> retrieved by the get.  I did some debugging with the cli and a remote
>  debugger and found a few interesting patterns.  First, the problem does not
> seem consistently duplicatable.  If one supercolumn is affected though, it
> will happen more frequently for subcolumns that when sorted appear at the
> beginning of the range.  For columns near the end of the range, it seems to
> be more intermittent, and almost never occurs when I step through the code
> line by line.  The only factor I can think of that might cause issues is
> that I am using custom data types for all supercolumns and columns.  I
> originally thought I might be reading past the end of the ByteBuffer, but I
> have quadrupled checked that this is not the case.
> >
> > Abe Sanderson
>
>

Mime
View raw message