From user-return-7356-archive-asf-public=cust-asf.ponee.io@accumulo.apache.org Wed Aug 29 14:05:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1263A180657 for ; Wed, 29 Aug 2018 14:05:05 +0200 (CEST) Received: (qmail 90228 invoked by uid 500); 29 Aug 2018 12:05:05 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 90217 invoked by uid 99); 29 Aug 2018 12:05:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Aug 2018 12:05:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 97D0AC057F for ; Wed, 29 Aug 2018 12:05:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.119 X-Spam-Level: ** X-Spam-Status: No, score=2.119 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2pIzW3-EMW55 for ; Wed, 29 Aug 2018 12:05:03 +0000 (UTC) Received: from mail-oi0-f54.google.com (mail-oi0-f54.google.com [209.85.218.54]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 392B95F3E2 for ; Wed, 29 Aug 2018 12:05:03 +0000 (UTC) Received: by mail-oi0-f54.google.com with SMTP id k12-v6so8624224oiw.8 for ; Wed, 29 Aug 2018 05:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=oL+6N2NXEwJgJRl0FndGUF/ZpEquM1HM/wG2/1AM1zI=; b=Y7m12y3181Lp7YCZnH9Qbul/RzWSdl2+Tlrp/amspiXTABXinDO0IMYa6ByeDbzRsZ 02HYizBULHKaIOMbRoWSduhdSVRah1WZyEL55VLM66FxM6qwyQ8ErkAJan3T6QQHcffJ 26HjVbMmlO6IT5YTFaOFGHBvBrCK+wRsXKawioywSeHSSRn5JuextsChMsZgeraIK7Uz TC/49l25NG12PDuu2kQNSU6L/Fa4mlhbCYEGig5ZqlYfTf6dCSgARewnfEm9ODwoCcEQ JHmOmv6Cxotih2jiQ+cESYMrtpbU9wQDfFLyf/4sC++YeT56Q327aJjT2vO9gDxLnoL5 9MKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=oL+6N2NXEwJgJRl0FndGUF/ZpEquM1HM/wG2/1AM1zI=; b=P0yQBYsmawQ4Lme8jVohdal48Zdwe/0tOEvTpYxiy0UkFxQ8pFyfpy4qZXHVYhVWT4 bgHx5EEBEQ95145XAdk1Am/CjWoWyB7y0Uqlo37m+k4hKOfGl+8QWgdB02Yk10nDGYO/ yNZZxAOp6ottwboAWvFmdQHqY+6kpuQPYoYyiToHVT7RyCUTrzddMHp+DOXktukfkolf IsAUIicMf3Tnva+Hjig9Q6iPIKr4faOcP+q0m3XU6bEo45BF4wywnI89vL3j2WVm0sZN Tp7TznCzrhckR02r4/v2cV8sKn57OrXYzhSYcUHpaZ+ybkQrlNrj9nmjsWhj0F9VaZbl FyFA== X-Gm-Message-State: APzg51Dq2Cbr9S91pAXeiPEG0hjNVcGdAGCeJrM7G8nMrh3iiNW/TTOm 4zvQL3Tw6L7grk3ycg3zKd3+eVFXBqEdD/lhU7ghLkC2 X-Google-Smtp-Source: ANB0VdbazIlPeLTfUr0oSAvmmUWEs2pb9CLSyWOLQxPu6ccv3oRhAC3RedrDLbmnw6JZTfTalB306XRfh4g0DIGv7N4= X-Received: by 2002:aca:6c4f:: with SMTP id h76-v6mr2527671oic.214.1535544302260; Wed, 29 Aug 2018 05:05:02 -0700 (PDT) MIME-Version: 1.0 From: guy sharon Date: Wed, 29 Aug 2018 15:04:51 +0300 Message-ID: Subject: Accumulo performance on various hardware configurations To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary="000000000000ac56c5057491c45e" --000000000000ac56c5057491c45e Content-Type: text/plain; charset="UTF-8" hi, Continuing my performance benchmarks, I'm still trying to figure out if the results I'm getting are reasonable and why throwing more hardware at the problem doesn't help. What I'm doing is a full table scan on a table with 6M entries. This is Accumulo 1.7.4 with Zookeeper 3.4.12 and Hadoop 2.8.4. The table is populated by org.apache.accumulo.examples.simple.helloworld.InsertWithBatchWriter modified to write 6M entries instead of 50k. Reads are performed by "bin/accumulo org.apache.accumulo.examples.simple.helloworld.ReadData -i muchos -z localhost:2181 -u root -t hellotable -p secret". Here are the results I got: 1. 5 tserver cluster as configured by Muchos ( https://github.com/apache/fluo-muchos), running on m5d.large AWS machines (2vCPU, 8GB RAM) running CentOS 7. Master is on a separate server. Scan took 12 seconds. 2. As above except with m5d.xlarge (4vCPU, 16GB RAM). Same results. 3. Splitting the table to 4 tablets causes the runtime to increase to 16 seconds. 4. 7 tserver cluster running m5d.xlarge servers. 12 seconds. 5. Single node cluster on m5d.12xlarge (48 cores, 192GB RAM), running Amazon Linux. Configuration as provided by Uno ( https://github.com/apache/fluo-uno). Total time was 26 seconds. Offhand I would say this is very slow. I'm guessing I'm making some sort of newbie (possibly configuration) mistake but I can't figure out what it is. Can anyone point me to something that might help me find out what it is? thanks, Guy. --000000000000ac56c5057491c45e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
hi,

Continuing my performance benchmarks, I= 9;m still trying to figure out if the results I'm getting are reasonabl= e and why throwing more hardware at the problem doesn't help. What I= 9;m doing is a full table scan on a table with 6M entries. This is Accumulo= 1.7.4 with Zookeeper 3.4.12 and Hadoop 2.8.4. The table is populated by or= g.apache.accumulo.examples.simple.helloworld.InsertWithBatchWriter modified= to write 6M entries instead of 50k. Reads are performed by "bin/accum= ulo org.apache.accumulo.examples.simple.helloworld.ReadData -i muchos -z lo= calhost:2181 -u root -t hellotable -p secret". Here are the results I = got:

1. 5 tserver cluster as configured by Muchos (https://github.com/apache/flu= o-muchos), running on m5d.large AWS machines (2vCPU, 8GB RAM) running C= entOS 7. Master is on a separate server. Scan took 12 seconds.
2. As abo= ve except with m5d.xlarge (4vCPU, 16GB RAM). Same results.
3. Splitting = the table to 4 tablets causes the runtime to increase to 16 seconds.
4. = 7 tserver cluster running m5d.xlarge servers. 12 seconds.
5. Single node= cluster on m5d.12xlarge (48 cores, 192GB RAM), running Amazon Linux. Confi= guration as provided by Uno (https://github.com/apache/fluo-uno). Total time was 26 seconds.
Offhand I would say this is very slow. I'm guessing I'm making so= me sort of newbie (possibly configuration) mistake but I can't figure o= ut what it is. Can anyone point me to something that might help me find out= what it is?

thanks,
Guy.


--000000000000ac56c5057491c45e--