Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6190410A4A for ; Thu, 10 Apr 2014 12:21:38 +0000 (UTC) Received: (qmail 17073 invoked by uid 500); 10 Apr 2014 12:21:37 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 16834 invoked by uid 500); 10 Apr 2014 12:21:36 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 16822 invoked by uid 99); 10 Apr 2014 12:21:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Apr 2014 12:21:35 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jgresock@gmail.com designates 209.85.213.48 as permitted sender) Received: from [209.85.213.48] (HELO mail-yh0-f48.google.com) (209.85.213.48) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Apr 2014 12:21:31 +0000 Received: by mail-yh0-f48.google.com with SMTP id z6so3683368yhz.21 for ; Thu, 10 Apr 2014 05:21:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=eHV7VO7qJTy0EU87SOdxgx5bteglnmQ5QcUT4pUt9Es=; b=HF5T2n60QAqiW5MUlTzTD26H56rSmJiG9EDg2GYi9LduV1QdnJe47YQeH1K+IyywXg wCXqZTqUYVhEE80KkFJMEem/Z4HBhv+fkFdZePYaJje+/30/Wd9Afq1QE7tfN8nkhYjk Eq4a4yVJbSsV0MTppYys8wtnO4kw1k0nr/QJEmCQIJNNeE7d4S32uE/IRyHc+fH+Uy/w vRPQvvyQpCSDJAMylf71y44aCRtN8dIEi2MDZhVhDbpUB93DkP6epVhF2B4gy4t2Frz/ OrIsNPVUA7I5agoQgFbTJRT8iacgjG2NNH8s8OF+TfH/mhYH9d7DudmE9kb2PPd38PB5 GK8A== MIME-Version: 1.0 X-Received: by 10.236.117.98 with SMTP id i62mr22532657yhh.81.1397132470579; Thu, 10 Apr 2014 05:21:10 -0700 (PDT) Received: by 10.170.161.213 with HTTP; Thu, 10 Apr 2014 05:21:10 -0700 (PDT) In-Reply-To: <1397131487883-8985.post@n5.nabble.com> References: <1397128244549-8981.post@n5.nabble.com> <1397129556277-8983.post@n5.nabble.com> <1397131487883-8985.post@n5.nabble.com> Date: Thu, 10 Apr 2014 08:21:10 -0400 Message-ID: Subject: Re: Stream fed accumulo From: Joe Gresock To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=20cf3011de279cf56904f6af409c X-Virus-Checked: Checked by ClamAV on apache.org --20cf3011de279cf56904f6af409c Content-Type: text/plain; charset=ISO-8859-1 Right, but you can set the byte[] buffer size to as large or small as you want. For example, if you use BufferedInputStream, it defaults to an 8K buffer -- this would be similar. This was a good solution for us, since Accumulo doesn't inherently support streaming at the moment. On Thu, Apr 10, 2014 at 8:04 AM, pdread wrote: > > Your still passing byte[] to your mutations which means your process > allocated space for that buffer. I was hoping for > > public void myPut(Stream mydata. String key, String type) > { > Mutation mutation = new Mutation(generateRowId(key, type)); > mutation.put(DATA_CF, encoder.encode(sequenceNum), visibility, timestamp, > mydata); > } > > Then under the covers the accumulo client streams the data to where ever > the > data goes. Maybe similar to how one would use the DataHandler in a REST > service to pass a stream in a REST call. > > Thanks > > Paul > > > > -- > View this message in context: > http://apache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981p8985.html > Sent from the Users mailing list archive at Nabble.com. > -- I know what it is to be in need, and I know what it is to have plenty. I have learned the secret of being content in any and every situation, whether well fed or hungry, whether living in plenty or in want. I can do all this through him who gives me strength. *-Philippians 4:12-13* --20cf3011de279cf56904f6af409c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Right, but you can set the byte[] buffer size to as large = or small as you want. =A0For example, if you use BufferedInputStream, it de= faults to an 8K buffer -- this would be similar. =A0This was a good solutio= n for us, since Accumulo doesn't inherently support streaming at the mo= ment.


On Thu, Apr 1= 0, 2014 at 8:04 AM, pdread <paul.read@siginttech.com>= wrote:

Your still passing byte[] to your mutations which means your process
allocated space for that buffer. I was hoping for

public void myPut(Stream mydata. String key, String type)
{
=A0 Mutation mutation =3D new Mutation(generateRowId(key, type));
=A0 mutation.put(DATA_CF, encoder.encode(sequenceNum), visibility, timestam= p,
mydata);
}

Then under the covers the accumulo client streams the data to where ever th= e
data goes. Maybe similar to how one would use the DataHandler in a REST
service to pass a stream in a REST call.

Thanks

Paul



--
View this message in context: http://a= pache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981p8985.html
Sent from the Users mailing list ar= chive at Nabble.com.



--
=
I know what it is to be in need, and I know what it is to have plenty. = =A0I have learned the secret of being content in any and every situation, w= hether well fed or hungry,=A0whether living in plenty or i= n want.=A0=A0I can do all this through him who gives me strengt= h.=A0=A0=A0 -Philippians 4:12-13
--20cf3011de279cf56904f6af409c--