Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5163FDB9B for ; Wed, 12 Sep 2012 16:33:12 +0000 (UTC) Received: (qmail 72350 invoked by uid 500); 12 Sep 2012 16:33:12 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 72303 invoked by uid 500); 12 Sep 2012 16:33:11 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 72294 invoked by uid 99); 12 Sep 2012 16:33:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Sep 2012 16:33:11 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [149.8.64.32] (HELO mclmx2.mail.saic.com) (149.8.64.32) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Sep 2012 16:33:05 +0000 Received: from dc04p-em-sbg01.saic.com ([149.8.64.21] [149.8.64.21]) by mclmx2.mail.saic.com with ESMTP id BT-MMP-6636976 for user@accumulo.apache.org; Wed, 12 Sep 2012 12:32:34 -0400 X-AuditID: 9508406d-b7f816d00000772b-98-5050b922d25b Received: from 0015-its-exbh03.us.saic.com (mcl-sixl-nat.saic.com [149.8.64.21]) by dc04p-em-sbg01.saic.com (Symantec Messaging Gateway) with SMTP id D8.0D.30507.229B0505; Wed, 12 Sep 2012 12:32:34 -0400 (EDT) Received: from 0905-its-exmp01.us.saic.com ([10.42.208.45]) by 0015-its-exbh03.us.saic.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 12 Sep 2012 12:32:34 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CD9104.366462AD" Subject: RE: Running Accumulo straight from Memory Date: Wed, 12 Sep 2012 12:32:31 -0400 Message-Id: <7B6B9A998E54184F9E0CEED43DD504090391E2C2@0905-its-exmp01.us.saic.com> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Running Accumulo straight from Memory Thread-Index: Ac2QZJ7yeTj5sKG8Q1uM3NXDSXoR6wAnzyrQ References: <7B6B9A998E54184F9E0CEED43DD504090391DDD7@0905-its-exmp01.us.saic.com> From: "Moore, Matthew J." To: X-OriginalArrivalTime: 12 Sep 2012 16:32:34.0314 (UTC) FILETIME=[3674E6A0:01CD9104] X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprGIsWRmVeSWpSXmKPExsUylcNBVFdpZ0CAwbwGIYv1O44wOzB6TNt0 nTGAMYrLJiU1J7MstUjfLoEro/HAZtaCFf4Vj38tZm1gvOrWxcjJISFgIvHi7SQmCFtM4sK9 9WxdjFwcQgKXGSWm7r7GCJIQEljCKHFyexREkZ7EssVTWEBsZgEtiSOXmsBqeAUEJU7OfAIU 5wCKh0ss+mENEhYWMJJ42dwHVsIioCqx4MofJojyIIndfxaAxTkFAiUOdX1igRgvKLFo9h5m mHv+7XrIBmKLCBhLNBxYyARhG0ns/34U6s7pjBLzv94GG8QG9MyKVWehGuQkLk74zQRxv6rE 2U8XoYYGSxz8/JNpAqPoLCQvzELywiyEFyBMPYm2jYwQ1doSyxa+ZoawdSX+P5/DjCy+gJF9 FaN4SrKBSYFuaq5ucVK6gaFecWJmsl5yfu4mRkhs5e5g/D7Z/BCjAAejEg+vQJ5/gBBrYllx Ze4hRgkOZiUR3iNLAgKEeFMSK6tSi/Lji0pzUosPMUpzsCiJ8yrx3vEXEkhPLEnNTk0tSC2C yTJxcEo1MB58/F+Z+7FO44qLn/4naScp79+wap3un+5Vzovd/ixsXGv79tQa04JpmheN3uff uJv2wXbuLOajz3cdWrX1+q/tTkdXKHwXOv1PrnW3X1OA/pPJTR88zwVNXHqn1n+PeohJ4LJD f1on7oxL8p2rclv9zNUg9qtCHz5sVTy+hGvxsY7Ly4SaH7IeVGIpzkg01GIuKk4EAKNJPTWp AgAA X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. ------_=_NextPart_001_01CD9104.366462AD Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Adam, It does look like we are the first to try this. We are trying to keep everything in memory and as a result there is no minor compactions, and probably major compactions to make tables larger. We tried this on SSDs using a file system and we were not getting the processing speeds that we had wanted. =20 Matt =20 =20 From: user-return-1330-MATTHEW.J.MOORE=3Dsaic.com@accumulo.apache.org [mailto:user-return-1330-MATTHEW.J.MOORE=3Dsaic.com@accumulo.apache.org] On Behalf Of Adam Fuchs Sent: Tuesday, September 11, 2012 5:30 PM To: user@accumulo.apache.org Subject: Re: Running Accumulo straight from Memory =20 Matthew, =20 I don't know of anyone who has done this, but I believe you could: 1. mount a RAM disk 2. point the hdfs core-site.xml fs.default.name property to file:/// 3. point the accumulo-site.xml instance.dfs.dir property to a directory on the RAM disk 4. disable the WAL for all tables by setting the accumulo-site.xml table.walog.enabled to false 5. initialize and start up accumulo as you regularly would and cross your fingers Of course, the "you may lose data" and "this is not an officially supported configuration" caveats apply. Out of curiosity, what would you be trying to accomplish with this configuration? =20 Adam =20 =20 On Tue, Sep 11, 2012 at 12:02 PM, Moore, Matthew J. wrote: Has anyone run Accumulo on a single server straight from memory? Probably using something like a Fusion IO drive. We are trying to use it without using an SSD or any spinning discs. =20 Matthew Moore Systems Engineer SAIC, ISBU Columbia, MD 410-312-2542 =20 =20 ------_=_NextPart_001_01CD9104.366462AD Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Adam,

It does look like we are the first to try this.  We are trying = to keep everything in memory and as a result there is no minor = compactions, and probably major compactions to make tables larger.  = We tried this on SSDs using a file system and we were not getting the = processing speeds that we had wanted.

 

Matt

 

 

From:= = user-return-1330-MATTHEW.J.MOORE=3Dsaic.com@accumulo.apache.org = [mailto:user-return-1330-MATTHEW.J.MOORE=3Dsaic.com@accumulo.apache.org] = On Behalf Of Adam Fuchs
Sent: Tuesday, September 11, = 2012 5:30 PM
To: user@accumulo.apache.org
Subject: = Re: Running Accumulo straight from Memory

 

Matthew,

 

I don't know of anyone who has done this, but I believe you = could:

1. mount a RAM disk

2. point the hdfs core-site.xml fs.default.name property to = file:///

3. point the accumulo-site.xml instance.dfs.dir property to a directory = on the RAM disk

4. disable the WAL for all tables by setting the accumulo-site.xml = table.walog.enabled to false

5. initialize and start up accumulo as you regularly would and cross = your fingers

Of course, the "you may lose data" and = "this is not an officially supported configuration" caveats = apply. Out of curiosity, what would you be trying to accomplish with = this configuration?

 

Adam

 

 

On Tue, = Sep 11, 2012 at 12:02 PM, Moore, Matthew J. <MATTHEW.J.MOORE@saic.com> = wrote:

Has anyone = run Accumulo on a single server straight from memory?  Probably = using something like a Fusion  IO drive.  We are trying to use = it without using an SSD or any spinning discs.

 <= /o:p>

Matthew = Moore

Sy= stems Engineer

SA= IC, ISBU

Co= lumbia, MD

410-312-2542

 <= /o:p>

 

------_=_NextPart_001_01CD9104.366462AD--