hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tobe <tobeg3oo...@gmail.com>
Subject Re: How to verify data from MySQL and HBase
Date Tue, 12 Aug 2014 07:49:04 GMT
Thanks for replaying. @Serega

I know MySQL and HBase are reliable. What I want to validate is the date in
both MySQL and HBase. The upper-stream application and unexpected operation
may make it inconsistent.

I'm also wondering could sqoop validate the values from each database?
There's RowCountValidator but it's not suitable for us.

On Tue, Aug 12, 2014 at 3:23 PM, Serega Sheypak <serega.sheypak@gmail.com>

> you should design resilient ETL-processes. Also introduce post-ETL checks.
> There is no need to test MySQL or HBase. They are already tested.
> See this:
> http://www.slideshare.net/wyaddow/data-verification-in-qa-department-final
> Pretty old, but gives basic ideas. Nothing changed from that time.
> 2014-08-12 7:55 GMT+04:00 tobe <tobeg3oogle@gmail.com>:
> > Most of our users migrated their data form MySQL to HBase. Before they
> > totally trust HBase, they use MySQL and HBase at the same time. Sometimes
> > the data is inconsistent because they use it incorrectly or maybe
> there're
> > bugs of HBase. Anyway, we have to make sure the data from MySQL and HBase
> > is consistent.
> >
> > So how can we do that? Write a simple script or is there any general
> > method?
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message