Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D15871896B for ; Thu, 13 Aug 2015 17:29:19 +0000 (UTC) Received: (qmail 69739 invoked by uid 500); 13 Aug 2015 17:29:16 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 69690 invoked by uid 500); 13 Aug 2015 17:29:16 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 69680 invoked by uid 99); 13 Aug 2015 17:29:16 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Aug 2015 17:29:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 309BFC2B69 for ; Thu, 13 Aug 2015 17:29:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.702 X-Spam-Level: ** X-Spam-Status: No, score=2.702 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KAM_ASCII_DIVIDERS=0.8, KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2ME4VPkH2BYQ for ; Thu, 13 Aug 2015 17:29:03 +0000 (UTC) Received: from mail-qk0-f172.google.com (mail-qk0-f172.google.com [209.85.220.172]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 82D1D31F4E for ; Thu, 13 Aug 2015 15:52:56 +0000 (UTC) Received: by qkbm65 with SMTP id m65so16743760qkb.2 for ; Thu, 13 Aug 2015 08:52:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=lF8M5GFHlS4+eqUtexB9Um+S28ju2plpNFgVoOX1A+o=; b=ACjcCX6x6tlcQfSgf5bmoxcDXzMFecxWdKi5kvkQW31s4Lt63cOZ4AIN6F+8O1l/0L Zk/k670MAaaqVgALTQXj5pIgMaJm0+L1Ru2d6tDzHUYpjsqtkPwFEOGJ+pTAzCtXKJt2 nIrU7HVeexRrjC1DTVNN5oDGBOn9Rq3ELUr1s22rg1Geh4ykyZzEd6GtDYA6AR3TaNaz //AN/UIa7SeOGydAZbgFZR35+uFDvb/UObasb2X7xPlpx7qYD4Y3hQP083DuNE85bV+m cCNhgoiL2q4ZnVqMP8TdDSrdlewBScBac9m0hjZNuTO72/4IU8KueNHXgSlFcR0QkWXj zb7w== X-Received: by 10.55.17.34 with SMTP id b34mr69732893qkh.97.1439481175757; Thu, 13 Aug 2015 08:52:55 -0700 (PDT) Received: from hw10447.local (pool-68-134-10-53.bltmmd.fios.verizon.net. [68.134.10.53]) by smtp.googlemail.com with ESMTPSA id 200sm1325334qhe.9.2015.08.13.08.52.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Aug 2015 08:52:54 -0700 (PDT) Message-ID: <55CCBD56.3080108@gmail.com> Date: Thu, 13 Aug 2015 11:52:54 -0400 From: Josh Elser User-Agent: Postbox 3.0.11 (Macintosh/20140602) MIME-Version: 1.0 To: user@accumulo.apache.org Subject: Re: Fetch Taking Longer Than Expected References: <002a01d0d4c2$f2a52ec0$d7ef8c40$@gmail.com> <55CB6433.7080006@gmail.com> <000601d0d526$53c53110$fb4f9330$@gmail.com> In-Reply-To: <000601d0d526$53c53110$fb4f9330$@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Looks like we have a SetGroupsCommand (setgroups) which is analogous to the Java API in TableOperations, but we don't have those nice add/remove commands for locality groups (likely as they're harder to implement atomically). Feel free to log a JIRA if you'd like to see add/remove instead of set. We can at least have the conversation on the matter -- there may be a reason I'm not aware of :) Daniel Ruiz wrote: > Thanks for the quick response. It does make sense, although it would be > nice to have an addgroup and delgroup (not sure if that exists in 1.7). > This was extremely helpful, thanks again! > > V/r, > -Daniel > > -----Original Message----- > From: Josh Elser [mailto:josh.elser@gmail.com] > Sent: Wednesday, August 12, 2015 9:20 AM > To: user@accumulo.apache.org > Subject: Re: Fetch Taking Longer Than Expected > > Yup, that would be expected. > > Remember that doing `scan -c ...` is an unbounded search over your > entire table. So, it takes approximately 3 minutes to read your > GUIDIndexTable. Because you have a single locality group, all of the > columns in your table are grouped together. > > One exercise that may be interesting for yourself is to create a > locality group that has your specific column family in it, compact your > GUIDIndexTable, and rerun your `scan -c` query. The speed should be > similar to your exact scan. Removing the locality group and > re-compacting the table should return the query time back to the slow 3 > minutes. > > Does that make sense? > > Daniel Ruiz wrote: >> Hi All, >> >> I am having an issue where column fetches are taking over a minute on >> 1.6.3. I don't believe this should be case and my experience in the past >> supports the idea that fetches should be very fast. >> >> For example we doing a scan on the table gives results instantly but >> doing a scan -c vesselmmsitext=2706758566 takes 2 minutes and 44 seconds >> (plus or minus 1 second). >> >> Figure 1.1. Generated Test Data on GUIDIndexTable >> >> Here is the table config >> >> > -----------+---------------------------------------------------------------+ > ---------------------------------------------------------------------------- > ----- >> SCOPE | NAME | VALUE >> >> > -----------+---------------------------------------------------------------+ > ---------------------------------------------------------------------------- > ----- >> default | table.balancer .............................................. >> | org.apache.accumulo.server.master.balancer.DefaultLoadBalancer >> >> default | table.bloom.enabled ......................................... >> | false >> >> default | table.bloom.error.rate ...................................... >> | 0.5% >> >> default | table.bloom.hash.type ....................................... >> | murmur >> >> default | table.bloom.key.functor ..................................... >> | org.apache.accumulo.core.file.keyfunctor.RowFunctor >> >> default | table.bloom.load.threshold .................................. | > 1 >> default | table.bloom.size ............................................ >> | 1048576 >> >> default | table.cache.block.enable .................................... >> | false >> >> default | table.cache.index.enable .................................... >> | true >> >> default | table.classpath.context ..................................... | >> >> default | table.compaction.major.everything.idle ...................... | > 1h >> default | table.compaction.major.ratio ................................ | > 3 >> default | table.compaction.minor.idle ................................. | > 5m >> default | table.compaction.minor.logs.threshold ....................... | > 3 >> table | table.constraint.1 .......................................... | >> org.apache.accumulo.core.constraints.DefaultKeySizeConstraint >> >> default | table.failures.ignore ....................................... >> | false >> >> default | table.file.blocksize ........................................ | > 0B >> default | table.file.compress.blocksize ............................... >> | 100K >> >> default | table.file.compress.blocksize.index ......................... >> | 128K >> >> default | table.file.compress.type .................................... | > gz >> default | table.file.max .............................................. | > 15 >> default | table.file.replication ...................................... | > 0 >> default | table.file.type ............................................. | > rf >> default | table.formatter ............................................. >> | org.apache.accumulo.core.util.format.DefaultFormatter >> >> default | table.groups.enabled ........................................ | >> >> default | table.interepreter .......................................... >> | org.apache.accumulo.core.util.interpret.DefaultScanInterpreter >> >> table | table.iterator.majc.AgeOffIterator##GUIDIndexTable .......... | >> 1,org.apache.accumulo.core.iterators.user.AgeOffFilter >> >> table | table.iterator.majc.AgeOffIterator##GUIDIndexTable.opt.ttl .. | >> 2592000000 >> >> table | table.iterator.majc.vers .................................... | >> 20,org.apache.accumulo.core.iterators.user.VersioningIterator >> >> table | table.iterator.majc.vers.opt.maxVersions .................... | 1 >> >> table | table.iterator.minc.AgeOffIterator##GUIDIndexTable .......... | >> 1,org.apache.accumulo.core.iterators.user.AgeOffFilter >> >> table | table.iterator.minc.AgeOffIterator##GUIDIndexTable.opt.ttl .. | >> 2592000000 >> >> table | table.iterator.minc.vers .................................... | >> 20,org.apache.accumulo.core.iterators.user.VersioningIterator >> >> table | table.iterator.minc.vers.opt.maxVersions .................... | 1 >> >> table | table.iterator.scan.AgeOffIterator##GUIDIndexTable .......... | >> 1,org.apache.accumulo.core.iterators.user.AgeOffFilter >> >> table | table.iterator.scan.AgeOffIterator##GUIDIndexTable.opt.ttl .. | >> 2592000000 >> >> ---------------------------------------------------------- hit any key >> to continue or 'q' to quit >> ---------------------------------------------------------- >> >> table | table.iterator.scan.vers .................................... | >> 20,org.apache.accumulo.core.iterators.user.VersioningIterator >> >> table | table.iterator.scan.vers.opt.maxVersions .................... | 1 >> >> default | table.majc.compaction.strategy .............................. >> | org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy >> >> default | table.scan.max.memory ....................................... >> | 512K >> >> table | @override ................................................ | 1M >> >> default | table.security.scan.visibility.default ...................... | >> >> default | table.split.threshold ....................................... | > 1G >> default | table.walog.enabled ......................................... >> | true >> >> > -----------+---------------------------------------------------------------+ > ---------------------------------------------------------------------------- > ----- >> More Table Info: >> >> GUIDIndexTable >> >> >> >> ONLINE >> >> >> >> 2 >> >> >> >> 0 >> >> >> >> 82.56M >> >> >> >> 810.00K >> >> >> >> 159 >> >> Please let me know if I am doing something wrong to if there is more >> information you need. >> >> V/r, >> >> -Daniel >> >