Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AB42A9110 for ; Thu, 2 Aug 2012 17:53:43 +0000 (UTC) Received: (qmail 69040 invoked by uid 500); 2 Aug 2012 17:53:43 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 69011 invoked by uid 500); 2 Aug 2012 17:53:43 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 69001 invoked by uid 99); 2 Aug 2012 17:53:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Aug 2012 17:53:43 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of marc@accumulo.net does not designate 209.85.214.169 as permitted sender) Received: from [209.85.214.169] (HELO mail-ob0-f169.google.com) (209.85.214.169) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Aug 2012 17:53:37 +0000 Received: by obhx4 with SMTP id x4so17039028obh.0 for ; Thu, 02 Aug 2012 10:53:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=4aZKXKluHOk2TqKaWTLw40v6JJpOV501xW7yFH9RVMQ=; b=OHuxwPLmg0rGOFSuN+CSyU8RFzwkXVNbI8R0WBPkkdc8BYSBIgTCqucxQGl2kX39yc KeupEGvFBQgPtS2q0xFNHxHE9H5bHRmeEroGpOwIpA99geesZYSL+N5CQb2yEO/RRwwU 1Lk2xSePSmI1GzyF2JMYnhZ8e8ezx2YSvVca3jfSRNkBBK/OrUpaDlPY9r+RVBmga48r +eQ4JuxUzVXfcq08DWYojSeXf1RwVa7Dl2h4FCCE7aBrJ4VNJOmi7l/2oO7TApab0j+l 5Et6G65ztlCBJINUkNKKMFN4MhSRo81xOvdHR28wXl6Rke5V60RMaOplQys8lSbw9VCo ZJHg== MIME-Version: 1.0 Received: by 10.182.222.39 with SMTP id qj7mr39102492obc.16.1343929996108; Thu, 02 Aug 2012 10:53:16 -0700 (PDT) Received: by 10.76.18.201 with HTTP; Thu, 2 Aug 2012 10:53:16 -0700 (PDT) X-Originating-IP: [63.239.65.11] In-Reply-To: References: Date: Thu, 2 Aug 2012 13:53:16 -0400 Message-ID: Subject: Re: Accumulo 1.4 Memory Issues From: Marc Parisi To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=f46d0444738305627404c64c16c4 X-Gm-Message-State: ALoCoQkg+kIM+lAAyKuVCILIHRsflk8+97IFSm6KU7Wo6auYAkxQ/4m2tAHyzql6ifJYxD8mqSF/ X-Virus-Checked: Checked by ClamAV on apache.org --f46d0444738305627404c64c16c4 Content-Type: text/plain; charset=ISO-8859-1 Check your log ( /logs/tserver_<>.log ) and see if you can find reference to memory failures, either in the native map interface or the JVM ( perhaps there is a misconfiguration on the tserver causing the JVM to fail )? On Thu, Aug 2, 2012 at 1:49 PM, William Slacum < wilhelm.von.cloud@accumulo.net> wrote: > Is it the TServer bombing out or your client or both? How often are you > flushing your writer? > > > On Thu, Aug 2, 2012 at 10:34 AM, Matt Parker wrote: > >> for my small test case, I'm storing some basic data in three tables: >> >> nodes - spatial index (id, list of child nodes, whether it's a leaf node ) >> image metadata - (id, bounding box coordinates, a text string of the >> bounding box) >> link - linking table that tells which images correspond to specific nodes. >> >> The image data isn't being stored in Accumulo, yet. >> >> >> >> On Thu, Aug 2, 2012 at 1:25 PM, Marc Parisi wrote: >> >>> are you using native maps? if so, are they being used? >>> >>> >>> On Thu, Aug 2, 2012 at 1:16 PM, Matt Parker wrote: >>> >>>> I setup a single instance Accumulo server. >>>> >>>> I can load 32K rows of image metadata without issue. >>>> >>>> I have another set of routines that build a dynamic spatial index, >>>> where nodes are inserted/updated/deleted over time. >>>> These operations are typically done one at a time, where each >>>> batchwriter are closed after use. >>>> >>>> It loads maybe a couple hundred operations, and then it dies with an >>>> OutOfMemory error when trying to close a batchwriter. >>>> >>>> I tried uping the memery settings on my client and on the tserver, but >>>> the results were the same. >>>> >>>> Outside of Accumulo, I can build the whole index in memory without any >>>> special JVM memory settings. I was wondering whether anyone else had run >>>> into a similar issue? >>>> >>> >>> >> > --f46d0444738305627404c64c16c4 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Check your log ( /logs/tserver_<<hostname>>.log ) and see if yo= u can find reference to memory failures, either in the native map interface= or the JVM ( perhaps there is a misconfiguration on the tserver causing th= e JVM to fail )?

On Thu, Aug 2, 2012 at 1:49 PM, William Slac= um <wilhelm.von.cloud@accumulo.net> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex"> Is it the TServer bombing out or your client or both? How often are you flu= shing your writer?


On Thu, Aug 2, 2012 at 10:34 AM, Matt Parker <parker= 20121@gmail.com> wrote:
for my small test case, I'm storing some= basic data in three tables:

nodes - spatial index (id, list of chil= d nodes, whether it's a leaf node )
image metadata - (id, bounding box coordinates, a text string of the boundi= ng box)
link - linking table that tells which images correspond to specific nodes.<= br>
The image data isn't being stored in Accumulo, yet.


On Thu, Aug 2, 2012 at 1:25 PM, Marc Pa= risi <marc@accumulo.net> wrote:
are you using native maps? if so, are they b= eing used?



--f46d0444738305627404c64c16c4--