Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 27BB89104 for ; Thu, 2 Aug 2012 17:50:25 +0000 (UTC) Received: (qmail 62598 invoked by uid 500); 2 Aug 2012 17:50:25 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 62571 invoked by uid 500); 2 Aug 2012 17:50:25 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 62563 invoked by uid 99); 2 Aug 2012 17:50:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Aug 2012 17:50:25 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of wilhelm.von.cloud@accumulo.net does not designate 209.85.216.169 as permitted sender) Received: from [209.85.216.169] (HELO mail-qc0-f169.google.com) (209.85.216.169) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Aug 2012 17:50:17 +0000 Received: by qcsd16 with SMTP id d16so6310860qcs.0 for ; Thu, 02 Aug 2012 10:49:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=rXpFkKhfVH3JzUpHC5WK6VwF79H82oI/KZqP9Ib4Lkk=; b=IRK+ofkYgBQreTUPmBRhv6+J7tpp4w8FGdS1qWC+jtt0pfg4J4IM0j8seJks4D1i0X MS8Sbjt8jUIwbAZrsNcBW5Yp6tBwCikSyHTVowbGrC/ukWvZFq4gg4jaStlaWXnljoXK +rvv+Tf6DWI5/wKoYXGgUS/33bzg+ePbUxlt6N5IxwVM/7/52D5g4LK/hpThwK40/a43 mi2LxK60iib3Ul+2mB7CHJiV9H8bgAoKgFxrLVMIILnEEibjT2QWbhBFS+ScVwzVJAHC Xnj0U4gpGf2cOJqh39r44GPG7jdf66bqckfVoQLlElCALveUVN26fRhMbHkHNO/DgI3B +dGw== MIME-Version: 1.0 Received: by 10.60.29.169 with SMTP id l9mr39483974oeh.14.1343929796709; Thu, 02 Aug 2012 10:49:56 -0700 (PDT) Received: by 10.60.41.71 with HTTP; Thu, 2 Aug 2012 10:49:56 -0700 (PDT) X-Originating-IP: [144.51.26.95] In-Reply-To: References: Date: Thu, 2 Aug 2012 10:49:56 -0700 Message-ID: Subject: Re: Accumulo 1.4 Memory Issues From: William Slacum To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=e89a8ff2566622cf0804c64c0aae X-Gm-Message-State: ALoCoQmvI4kefnXtNWz+vCM0r2v0tDLrGGjZOhNkh5d3oe4ZHhxQUYy7L0Nq2vWIAe3xVpAglihU X-Virus-Checked: Checked by ClamAV on apache.org --e89a8ff2566622cf0804c64c0aae Content-Type: text/plain; charset=ISO-8859-1 Is it the TServer bombing out or your client or both? How often are you flushing your writer? On Thu, Aug 2, 2012 at 10:34 AM, Matt Parker wrote: > for my small test case, I'm storing some basic data in three tables: > > nodes - spatial index (id, list of child nodes, whether it's a leaf node ) > image metadata - (id, bounding box coordinates, a text string of the > bounding box) > link - linking table that tells which images correspond to specific nodes. > > The image data isn't being stored in Accumulo, yet. > > > > On Thu, Aug 2, 2012 at 1:25 PM, Marc Parisi wrote: > >> are you using native maps? if so, are they being used? >> >> >> On Thu, Aug 2, 2012 at 1:16 PM, Matt Parker wrote: >> >>> I setup a single instance Accumulo server. >>> >>> I can load 32K rows of image metadata without issue. >>> >>> I have another set of routines that build a dynamic spatial index, where >>> nodes are inserted/updated/deleted over time. >>> These operations are typically done one at a time, where each >>> batchwriter are closed after use. >>> >>> It loads maybe a couple hundred operations, and then it dies with an >>> OutOfMemory error when trying to close a batchwriter. >>> >>> I tried uping the memery settings on my client and on the tserver, but >>> the results were the same. >>> >>> Outside of Accumulo, I can build the whole index in memory without any >>> special JVM memory settings. I was wondering whether anyone else had run >>> into a similar issue? >>> >> >> > --e89a8ff2566622cf0804c64c0aae Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Is it the TServer bombing out or your client or both? How often are you flu= shing your writer?

On Thu, Aug 2, 2012 at= 10:34 AM, Matt Parker <parker20121@gmail.com> wrote:
for my small test case, I'm storing some= basic data in three tables:

nodes - spatial index (id, list of chil= d nodes, whether it's a leaf node )
image metadata - (id, bounding box coordinates, a text string of the boundi= ng box)
link - linking table that tells which images correspond to specific nodes.<= br>
The image data isn't being stored in Accumulo, yet.



On Thu, A= ug 2, 2012 at 1:25 PM, Marc Parisi <marc@accumulo.net> wrote= :
are you using native maps? if so, are they b= eing used?


--e89a8ff2566622cf0804c64c0aae--