Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <664629431.3697231.1480583398056@mail.yahoo.com>
References: <CAJ3fcbDK+_aDkuNmUPA_tBX2WsM8ibUduzf-Z2CctkNvSNwwNg@mail.gmail.com>
 <CAJd_F2tnDDUpDj=o3+0-92p4venFNs0=YiYwCGhchD0fHSPDnw@mail.gmail.com>
 <1954769023.3376965.1480540502507@mail.yahoo.com> <CAJd_F2tn940=0fEo2fnyogFRi-Pv8AhLASOG18AODPTWmPVuQg@mail.gmail.com>
 <664629431.3697231.1480583398056@mail.yahoo.com>
From: Neelesh <neeleshs@gmail.com>
Date: Thu, 1 Dec 2016 08:04:21 -0800
Message-ID: <CAJd_F2viX9pyqnZo1nj4kxeAGrobTNC8G1KSi19mQ1Z9Nx+xZA@mail.gmail.com>
Subject: Re: Hbase on HDFS versus Cassandra
To: Ted Yu <ted_yu@yahoo.com>
Cc: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a1143198481b4b605429af98f
archived-at: Thu, 01 Dec 2016 16:04:33 -0000

--001a1143198481b4b605429af98f
Content-Type: text/plain; charset=UTF-8

Yes, PHOENIX-1718 has about the same information

On Dec 1, 2016 1:12 AM, "Ted Yu" <ted_yu@yahoo.com> wrote:

> w.r.t. "Unable to find cached index metadata" error, have you seen this ?
>
> http://search-hadoop.com/m/Phoenix/9UY0h2YBSOhgbflB1?
> subj=Re+Global+Secondary+Index+ERROR+2008+INT10+Unable+
> to+find+cached+index+metadata+PHOENIX+1718+
>
> Cheers
>
>
> On Wednesday, November 30, 2016 11:59 PM, Neelesh <neeleshs@gmail.com>
> wrote:
>
>
> Ted,
> we use HDP 2.3.4 (HBase 1.1.2, phoenix 4.4 - but with a lot of backports
> from later versions)
>
> The key of the data table is  <customerid- 11 bytes><userid- 36 bytes-
>  but it really is a right padded long due to historic data><event type
> long><timestamp><random long>
>
> The two global indexes are <customerid-11 bytes><event type
> long><timestamp> <user id> and <customerid-11 bytes><campaign id
> long><timestamp>
> Around 100B rows in the main table. The main issues we see are
>
> # Sudden spikes in queueSize - going all the way to 1G limit and staying
> there, without any correlated client traffic
> # Boatloads of these errors 2016-11-30 11:28:54,907 INFO
>  [RW.default.writeRpcServer.handler=43,queue=9,port=16020]
> util.IndexManagementUtil: Rethrowing org.apache.hadoop.hbase.DoNotRetryIOException:
> ERROR 2008 (INT10): ERROR 2008 (INT10): Unable to find cached index
> metadata.  key=120521194876100862 region=<region-key>. Index update failed
>
> We have cross datacenter WAL replication enabled.
> We saw PHOENIX-1718, and changed all recommended timeouts to 1 hour.  Our
> HBase version has HBase-11705. We also discovered that the queuesize is
> global (across general/replication/priority queues) and if it reaches the
> 1GB limit, calls to all queues will drop. That was interesting because even
> though the replication handlers have  a different queue, the size is
> counted globally, affecting others. Please correct me on this. I hope I'm
> wrong on this one :)
>
> Our challenge has been to understand what's HBase doing under various
> scenarios. We monitor call queue lengths, sizes and latencies as the
> primary alerting mechanism to tell us something is going on with HBase.
>
> Thanks!
> -neelesh
>
> On Wed, Nov 30, 2016 at 1:15 PM, Ted Yu <ted_yu@yahoo.com.invalid> wrote:
>
> Neelesh:Can you share more details about the sluggish cluster performance
> (such as version of hbase / phoenix, your schema, region server log
> snippet, stack traces, etc) ?
> As hbase / phoenix evolve, I hope the performance keeps getting better for
> your use case.
> Cheers
>
>     On Wednesday, November 30, 2016 10:07 AM, Neelesh <neeleshs@gmail.com>
> wrote:
>
>
>  We use both, in different capacities. Cassandra is an x-DC archive store
> with mostly batch writes and occasional key based reads. Hbase is for
> real-time event ingestion. Our experience so far on hbase + phoenix is that
> when it works, it is fast and scales like crazy. But if you ever hit a snag
> around data patterns, you will have a VERY hard time figuring out what's
> going on. A combination of global phoenix indexes and heavy writes leave an
> entire cluster sluggish, if there is a hint of hotspotting.
>
> On the other hand, we had a big struggle getting Cassandra when a node
> recovery was in progress. What with twice the amount of disk requirements
> during recovery etc. Other than that, it is quiet.
> But the access patterns are not the same.
>
> I think the old rule still stays. If you are already on hadoop , or
> interested in using/analysing data in several different ways, go with hbase
> . If you just need a big data store with a few predefined query patterns,
> Cassandra is good
>
> Of course, I'm biased towards HBase.
>
> On Nov 30, 2016 7:02 AM, "Mich Talebzadeh" <mich.talebzadeh@gmail.com>
> wrote:
>
> > Hi Guys,
> >
> > Used Hbase on HDFS reasonably well. Happy to to stick with it and more
> with
> > Hive/Phoenix views and Phoenix indexes where I can.
> >
> > I have a bunch of users now vocal about the use case for Cassandra and
> > whether it can do a better job than Hbase.
> >
> > Unfortunately I am no expert on Cassandra. However, some use case fit
> would
> > be very valuable.
> >
> > Thanks
> >
> > Dr Mich Talebzadeh
> >
> >
> >
> > LinkedIn * https://www.linkedin.com/ profile/view?id=
> <https://www.linkedin.com/profile/view?id=>
> > AAEAAAAWh2gBxianrbJd6zP6AcPCCd OABUrV8Pw
> > <https://www.linkedin.com/ profile/view?id=
> AAEAAAAWh2gBxianrbJd6zP6AcPCCd
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCd>
> > OABUrV8Pw>*
> >
> >
> >
> > http://talebzadehmich. wordpress.com
> <http://talebzadehmich.wordpress.com/>
> >
> >
> > *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> > loss, damage or destruction of data or any other property which may arise
> > from relying on this email's technical content is explicitly disclaimed.
> > The author will in no case be liable for any monetary damages arising
> from
> > such loss, damage or destruction.
> >
>
>
>
>
>
>
>
>

--001a1143198481b4b605429af98f--