Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7BD8EFCD1 for ; Thu, 11 Apr 2013 13:39:15 +0000 (UTC) Received: (qmail 771 invoked by uid 500); 11 Apr 2013 13:37:58 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 99577 invoked by uid 500); 11 Apr 2013 13:37:49 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 92607 invoked by uid 99); 11 Apr 2013 13:34:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Apr 2013 13:34:16 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sylvain@datastax.com designates 209.85.192.178 as permitted sender) Received: from [209.85.192.178] (HELO mail-pd0-f178.google.com) (209.85.192.178) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Apr 2013 13:34:12 +0000 Received: by mail-pd0-f178.google.com with SMTP id w11so861317pde.37 for ; Thu, 11 Apr 2013 06:33:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=DK42kKiqel/UYiTci3DUNPvfb4Ti1/wKIlVwFk0JPO4=; b=DfMrjymBeXjjgsR7VgRrD3gREyyUsyCj1WB0vubJNmZWzQ1zeh/QNUVDrgtc1ozpnP jGy1Odh614A+bkN8PgzozcJplrnCQFxg2P8e1LUqyrMfriECgC1GnoJsYreCIFwnvtie hUsHM7cLWMJFMpPW22XNRqu5XnF3tquLtTDtss/yPygg1X+5zjfDZsmToOXUaL9pq4z5 63va8aTKBSGeMsd7FWoHpFEF52oau9yBXdAsmZEsXRPQIwgTj/Z949mJJ3LMs6ylF8PW aGh/U5IDkFFarOKWdAGtR5WWDuVpNqnUGGfYx9QXhBRzXQcfPgmxQHiBOvFRNFk+4TJd facA== MIME-Version: 1.0 X-Received: by 10.66.144.69 with SMTP id sk5mr9907270pab.69.1365687231697; Thu, 11 Apr 2013 06:33:51 -0700 (PDT) Received: by 10.68.134.131 with HTTP; Thu, 11 Apr 2013 06:33:51 -0700 (PDT) In-Reply-To: References: Date: Thu, 11 Apr 2013 15:33:51 +0200 Message-ID: Subject: Re: Blobs in CQL? From: Sylvain Lebresne To: "user@cassandra.apache.org" Cc: Gabriel Ciuloaica Content-Type: multipart/alternative; boundary=047d7b6dcedc51eb7604da15d6b7 X-Gm-Message-State: ALoCoQkQgxumtsZFMHamfwoSWY5C9sh4/aVPzsPrTZq4unF4qpYt0J/fZOS2L5g0dfC++22yk8gN X-Virus-Checked: Checked by ClamAV on apache.org --047d7b6dcedc51eb7604da15d6b7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable > I assume I'm doing something wrong in the select. Am I incorrectly using > the ResultSet? > You're incorrectly using the returned ByteBuffer. But you should not feel bad, that API kinda sucks. The short version is that .array() returns the backing array of the ByteBuffer. But there is no guarantee that you'll have a one-to-one correspondence between the valid content of the ByteBuffer and the backing array, the backing array can be bigger in particular (long story short, this allows multiple ByteBuffer to share the same backing array, which can avoid doing copies). I also note that there is no guarantee that .array() will work unless you've called .hasArray(). Anyway, what you could do is: ByteBuffer bb =3D resultSet.one().getBytes("data"); byte[] data =3D new byte[bb.remaining()]; bb.get(data); Alternatively, you can use the result of .array(), but you should only consider the bb.remaining() bytes starting at bb.arrayOffset() + bb.position() (where bb is the returned ByteBuffer). -- Sylvain > > -brian > > On Thu, Apr 11, 2013 at 9:09 AM, Brian O'Neill wro= te: > >> Yep, it worked like a charm. (PreparedStatement avoided the hex >> conversion) >> >> But now, I'm seeing a few extra bytes come back in the select=85. >> (I'll keep digging, but maybe you have some insight?) >> >> I see this: >> >> ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao: >> repository.add() byte.length()=3D[259804] >> >> ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao: >> repository.get() [foo.jpeg] byte.length()=3D[259861] >> >> (Notice the length's don't match up) >> >> Using this code: >> >> public void addContent(String key, byte[] data) >> >> throws NoHostAvailableException { >> >> LOG.error("repository.add() byte.length()=3D[" + data.length + "= ]" >> ); >> >> String statement =3D "INSERT INTO " + KEYSPACE + "." + TABLE + "= (key, >> data) VALUES (?, ?)"; >> >> PreparedStatement ps =3D session.prepare(statement); >> >> BoundStatement bs =3D ps.bind(key, ByteBuffer.wrap(data)); >> >> session.execute(bs); >> >> } >> >> >> public byte[] getContent(String key) throws NoHostAvailableException >> { >> >> Query select =3D select("data").from(KEYSPACE, TABLE).where(eq( >> "key", key)); >> >> ResultSet resultSet =3D session.execute(select); >> >> byte[] data =3D resultSet.one().getBytes("data").array(); >> >> LOG.error("repository.get() [" + key + "] byte.length()=3D[" + >> data.length + "]"); >> >> return data; >> >> } >> >> --- >> >> Brian O'Neill >> >> Lead Architect, Software Development >> >> *Health Market Science* >> >> *The Science of Better Results* >> >> 2700 Horizon Drive =95 King of Prussia, PA =95 19406**** >> >> M: 215.588.6024 =95 @boneill42 =95 >> >> healthmarketscience.com >> >> >> This information transmitted in this email message is for the intended >> recipient only and may contain confidential and/or privileged material. = If >> you received this email in error and are not the intended recipient, or = the >> person responsible to deliver it to the intended recipient, please conta= ct >> the sender at the email above and delete this email and any attachments = and >> destroy any copies thereof. Any review, retransmission, dissemination, >> copying or other use of, or taking any action in reliance upon, this >> information by persons or entities other than the intended recipient is >> strictly prohibited.**** >> >> ** ** >> >> >> From: Sylvain Lebresne >> Reply-To: >> Date: Thursday, April 11, 2013 8:48 AM >> To: "user@cassandra.apache.org" >> Cc: Gabriel Ciuloaica >> Subject: Re: Blobs in CQL? >> >> >> Hopefully, the prepared statement doesn't do the conversion. >>> >> >> It does not. >> >> >>> (I'm not sure if it is a limitation of the CQL protocol itself) >>> >>> thanks again, >>> -brian >>> >>> >>> >>> --- >>> Brian O'Neill >>> Lead Architect, Software Development >>> Health Market Science >>> The Science of Better Results >>> 2700 Horizon Drive =95 King of Prussia, PA =95 19406 >>> M: 215.588.6024 =95 @boneill42 =95 >>> healthmarketscience.com >>> >>> This information transmitted in this email message is for the intended >>> recipient only and may contain confidential and/or privileged material. >>> If >>> you received this email in error and are not the intended recipient, or >>> the person responsible to deliver it to the intended recipient, please >>> contact the sender at the email above and delete this email and any >>> attachments and destroy any copies thereof. Any review, retransmission, >>> dissemination, copying or other use of, or taking any action in relianc= e >>> upon, this information by persons or entities other than the intended >>> recipient is strictly prohibited. >>> >>> >>> >>> >>> >>> >>> >>> On 4/11/13 8:34 AM, "Gabriel Ciuloaica" wrote: >>> >>> >I'm not using the query builder but the PreparedStatement. >>> > >>> >Here is the sample code: https://gist.github.com/devsprint/5363023 >>> > >>> >Gabi >>> >On 4/11/13 3:27 PM, Brian O'Neill wrote: >>> >> Great! >>> >> >>> >> Thanks Gabriel. Do you have an example? (are using QueryBuilder?) >>> >> I couldn't find the part of the API that allowed you to pass in the >>> >>byte >>> >> array. >>> >> >>> >> -brian >>> >> >>> >> --- >>> >> Brian O'Neill >>> >> Lead Architect, Software Development >>> >> Health Market Science >>> >> The Science of Better Results >>> >> 2700 Horizon Drive =80 King of Prussia, PA =80 19406 >>> >> M: 215.588.6024 =80 @boneill42 = =80 >>> >> healthmarketscience.com >>> >> >>> >> This information transmitted in this email message is for the intend= ed >>> >> recipient only and may contain confidential and/or privileged >>> material. >>> >>If >>> >> you received this email in error and are not the intended recipient, >>> or >>> >> the person responsible to deliver it to the intended recipient, plea= se >>> >> contact the sender at the email above and delete this email and any >>> >> attachments and destroy any copies thereof. Any review, >>> retransmission, >>> >> dissemination, copying or other use of, or taking any action in >>> reliance >>> >> upon, this information by persons or entities other than the intende= d >>> >> recipient is strictly prohibited. >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> On 4/11/13 8:25 AM, "Gabriel Ciuloaica" wrote= : >>> >> >>> >>> Hi Brian, >>> >>> >>> >>> I'm using the blobs to store images in cassandra(1.2.3) using the >>> >>> java-driver version 1.0.0-beta1. >>> >>> There is no need to convert a byte array into hex. >>> >>> >>> >>> Br, >>> >>> Gabi >>> >>> >>> >>> On 4/11/13 3:21 PM, Brian O'Neill wrote: >>> >>>> I started playing around with the CQL driver. >>> >>>> Has anyone used blobs with it yet? >>> >>>> >>> >>>> Are you forced to convert a byte[] to hex? >>> >>>> (e.g. I have a photo that I want to store in C* using the >>> java-driver >>> >>>> API) >>> >>>> >>> >>>> -brian >>> >>>> >>> >>>> -- >>> >>>> Brian ONeill >>> >>>> Lead Architect, Health Market Science ( >>> http://healthmarketscience.com) >>> >>>> mobile:215.588.6024 >>> >>>> blog: http://brianoneill.blogspot.com/ >>> >>>> twitter: @boneill42 >>> >> >>> > >>> >>> >>> >> > > > -- > Brian ONeill > Lead Architect, Health Market Science (http://healthmarketscience.com) > > mobile:215.588.6024 > blog: http://brianoneill.blogspot.com/ > twitter: @boneill42 > --047d7b6dcedc51eb7604da15d6b7 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable

=
I assume I'm doing something wrong in the select. =A0Am I incorrec= tly using the ResultSet?

You= 9;re incorrectly using the returned ByteBuffer. But you should not feel bad= , that API kinda
sucks.

The short version is= that .array() returns the backing array of the ByteBuffer. But there is no=
guarantee that you'll have a one-to-one correspondence= between the valid content of the
ByteBuffer and the backing array, the backing array can be bigge= r in particular (long story short,
this allows multiple Byt= eBuffer to share the same backing array, which can avoid doing copies).

I also note that there is no guarantee that= .array() will work unless you've called .hasArray().
<= br>
Anyway, what you could do is:
ByteBuffe= r bb =3D=A0resultSet.one().getBytes("data");
byte[] data =3D new byte[bb.remaining()];
bb= .get(data);

Alternatively, you can use= the result of .array(), but you should only consider the bb.remaining()=A0=
bytes starting at bb.arrayOffset() + bb.position() (where bb is = the returned ByteBuffer).=A0

--
Sy= lvain

=A0

-brian

On Thu, Apr 11, 2013 at 9:09 AM, Brian O'Neill <bone@al= umni.brown.edu> wrote:
Yep, it worked like a charm. =A0(Prep= aredStatement avoided the hex conversion)

But now, I'm seeing a few extra bytes come back in = the select=85.
(I'll keep digging, but maybe you have some in= sight?)

I see this:

ERROR [2013-04-11 13:05:03,461] com.skookle.dao.RepositoryDao: repository.a= dd() byte.length()=3D[259804]

ERROR [2013-04-11 13:08:08,487] com.skookle.dao.RepositoryDao: repository.g= et() [foo.jpeg] byte.length()=3D[259861]


(Notice the length's don't match up)

Using this code:

=A0 =A0=A0public void addContent(String key, byte[] data)

=A0 =A0 =A0 =A0 = =A0 =A0 throws NoHostAvailable= Exception {

=A0 =A0 =A0 =A0 LOG.error("repository.add() byte.length()=3D[" + data.length + "]");

=A0 =A0 =A0 =A0 String statement =3D = "INSERT INTO " + <= span style=3D"color:rgb(3,47,195)">KEYSPACE + "." + <= span style=3D"color:rgb(3,47,195)">TABLE + "(key, data) VALUES (?, ?)";

=A0 =A0 =A0 =A0 = PreparedStatement ps =3D session= .prepare(statement);

=A0 =A0 =A0 =A0 = BoundStatement bs =3D ps.bind(key, ByteBuffer.wrap(data));

=A0 =A0 =A0 =A0 = session.execute(bs);

= =A0 =A0 }

=

=A0 =A0 public byte[] getContent(String key) throws NoHostAvailableException {

=A0 =A0 =A0 =A0 = Query select =3D select("data&quo= t;).from(KEYSPACE, TABLE).where(eq("key", key));

=A0 =A0 =A0 =A0 = ResultSet resultSet =3D session.= execute(select);

=A0 =A0 =A0 =A0 = byte[] data =3D resultSet.one(= ).getBytes("data").ar= ray();

=A0 =A0 =A0 =A0 LOG.error("repository.get() [" + key += "] byte.length()=3D[" + = data.length + "]"= );

=A0 =A0 =A0 =A0 = return data;

=A0 =A0 }


---

Brian O'Neill

Lead Arc= hitect, Software Development

Healt= h Market=A0Science

The Science of Better Results

2700 Hor= izon Drive=A0=95= =A0King of Prussia, PA= =A0=95=A019406

M: 215.588.6024=A0=95 @boneill42=A0=A0=95=A0=A0

hea= lthmarketscience.com


T= his information transmitted in this email message is for the intended recip= ient only and may contain confidential and/or privileged material. If you r= eceived this email in error and are not the intended recipient, or the pers= on responsible to deliver it to the intended recipient, please contact the = sender at the email above and delete this email and any attachments and des= troy any copies thereof. Any review, retransmission, dissemination, copying= or other use of, or taking any action in reliance upon, this information b= y persons or entities other than the intended recipient is strictly prohibi= ted.

=A0


From: Sylvain Lebresne <sylvain@datastax.com= >
Reply-To: <user@cassandra.apache.or= g>
Date: Thursday, April 11, 2013 8:4= 8 AM
To: "user@cassandra.apache.org" <= user@cassandra.apache.org>
Cc: Gabriel Ciuloaica <gciuloaica@gmail.com= >
Subject: Re: Blobs in CQL?=


Hopefully, the prepared statement doesn't do the conversion.

It does not.
=A0
(I'm not sure if it is a limitation of the CQL protocol itself)

thanks again,
-brian



---
Brian O'Neill
Lead Architect, Software Development
Health Market Science
The Science of Better Results
2700 Horizon Drive =95 King of Prussia, PA =95 19406
M: 21= 5.588.6024 =95 @boneill42 <http://www.twitter.com/boneill42> =A0=95
healthmarketscie= nce.com

This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If<= br> you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended
recipient is strictly prohibited.







On 4/11/13 8:34 AM, "Gabriel Ciuloaica" <gciuloaica@gmail.com> wro= te:

>I'm not using the query builder but the PreparedStatement.
>
>Here is the sample code: https://gist.github.com/devsprint/5363023
>
>Gabi
>On 4/11/13 3:27 PM, Brian O'Neill wrote:
>> Great!
>>
>> Thanks Gabriel. =A0Do you have an example? (are using QueryBuilder= ?)
>> I couldn't find the part of =A0the API that allowed you to pas= s in the
>>byte
>> array.
>>
>> -brian
>>
>> ---
>> Brian O'Neill
>> Lead Architect, Software Development
>> Health Market Science
>> The Science of Better Results
>> 2700 Horizon Drive =80 King of Prussia, PA =80 19406
>> M: 215.588.6024 =80 @boneill42 <http://www.twitter.com/boneill42> =A0= =80
>> healt= hmarketscience.com
>>
>> This information transmitted in this email message is for the inte= nded
>> recipient only and may contain confidential and/or privileged mate= rial.
>>If
>> you received this email in error and are not the intended recipien= t, or
>> the person responsible to deliver it to the intended recipient, pl= ease
>> contact the sender at the email above and delete this email and an= y
>> attachments and destroy any copies thereof. Any review, retransmis= sion,
>> dissemination, copying or other use of, or taking any action in re= liance
>> upon, this information by persons or entities other than the inten= ded
>> recipient is strictly prohibited.
>>
>>
>>
>>
>>
>>
>>
>> On 4/11/13 8:25 AM, "Gabriel Ciuloaica" <gciuloaica@gmail.com> = wrote:
>>
>>> Hi Brian,
>>>
>>> I'm using the blobs to store images in cassandra(1.2.3) us= ing the
>>> java-driver version 1.0.0-beta1.
>>> There is no need to convert a byte array into hex.
>>>
>>> Br,
>>> Gabi
>>>
>>> On 4/11/13 3:21 PM, Brian O'Neill wrote:
>>>> I started playing around with the CQL driver.
>>>> Has anyone used blobs with it yet?
>>>>
>>>> Are you forced to convert a byte[] to hex?
>>>> (e.g. I have a photo that I want to store in C* using the = java-driver
>>>> API)
>>>>
>>>> -brian
>>>>
>>>> --
>>>> Brian ONeill
>>>> Lead Architect, Health Market Science (http://healthmarketscience.com)
>>>> mobile:215.588.6024
>>>> blog: http://brianoneill.blogspot.com/
>>>> twitter: @boneill42
>>
>



=



--
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketsc= ience.com)

mobile:215.588.6024
blog: http:/= /brianoneill.blogspot.com/
twitter: @boneill42

--047d7b6dcedc51eb7604da15d6b7--