Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3CAA9187D2 for ; Thu, 25 Jun 2015 20:21:47 +0000 (UTC) Received: (qmail 4751 invoked by uid 500); 25 Jun 2015 20:21:46 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 4698 invoked by uid 500); 25 Jun 2015 20:21:46 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 4681 invoked by uid 99); 25 Jun 2015 20:21:46 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jun 2015 20:21:46 +0000 Received: from mail-yk0-f180.google.com (mail-yk0-f180.google.com [209.85.160.180]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 585E91A00B6 for ; Thu, 25 Jun 2015 20:21:46 +0000 (UTC) Received: by ykdr198 with SMTP id r198so46595509ykd.3 for ; Thu, 25 Jun 2015 13:21:45 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.13.214.1 with SMTP id y1mr14436872ywd.30.1435263705633; Thu, 25 Jun 2015 13:21:45 -0700 (PDT) Received: by 10.129.24.211 with HTTP; Thu, 25 Jun 2015 13:21:45 -0700 (PDT) In-Reply-To: References: Date: Thu, 25 Jun 2015 16:21:45 -0400 Message-ID: Subject: Re: Any limits on Accumulo namespaces/tables? From: Christopher To: Accumulo User List Content-Type: text/plain; charset=UTF-8 Correction: s/this because a big problem/this causes a big problem/ :) -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Jun 25, 2015 at 2:38 PM, Christopher wrote: > In theory, no, there aren't limits. In practice, you'd be limited by > the amount of memory you have available in ZooKeeper, and you may have > trouble (currently) with lots of ZooKeeper watchers which monitor > configuration changes for tables. > > There may be data structures in the API and throughout the code which > contain a complete list of tables, also, and that could hurt in terms > of memory. > > In short, we don't define a limit, but there may be bottlenecks you > might encounter. If you do find these, we may be able to make > improvements, so let us know. If this because a big problem, we may > impose some upper bound in the future, just so people can reason about > their environment better, and we could offer more concrete numbers on > what is feasible. > > -- > Christopher L Tubbs II > http://gravatar.com/ctubbsii > > > On Thu, Jun 25, 2015 at 2:31 PM, Konstantin Pelykh wrote: >> Hi everyone, >> >> Are there any limits on a number of namespaces and tables in Accumulo? >> For my current application I'm planning to use multiple namespaces to >> separate data for different clients. Each namespace will have 8 tables. >> Would it be posible to have 100,500, 1000 namespaces in Accumulo? How will >> such namespace layout impact performance of accumulo scanner? As far as I >> rememebr every time when scanner is created it gets the whole list of >> namespaces/tables from zookeeper and it slows down performace as number of >> namespaces increases. >> >> Thanks! >> Konstantin >> -------- >> Big Data / Search Consultant >> LinkedIn: linkedin.com/in/kpelykh >> Website: www.kpelykh.com