hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Heng Chen <heng.chen.1...@gmail.com>
Subject Re: Data loss after compaction when a row has more than Integer.MAX_VALUE columns
Date Mon, 18 Jan 2016 07:05:46 GMT
The incoming-follow-nodes and outgoing-follow-nodes of one node exceed
Integer.MAX_VALUE,  unbelievable!

Is the performance OK if i request the number of incoming-follow-nodes?


2016-01-18 13:26 GMT+08:00 Toshihiro Suzuki <brfrn169@gmail.com>:

> Thank you for your reply.
>
> We are using hbase to store social graph data on the SNS we provide.
>
> Our use case was presented in HBasecon 2015.
>
> http://www.slideshare.net/HBaseCon/use-cases-session-6a
>
> Schema Design is the below,
>
> http://www.slideshare.net/HBaseCon/use-cases-session-6a/44
>
> Thanks,
> Toshihiro Suzuki.
>
>
> 2016-01-18 13:48 GMT+09:00 Ted Yu <yuzhihong@gmail.com>:
>
> > Interesting.
> >
> > Can you share your use case where more than Integer.MAX_VALUE columns are
> > needed.
> >
> > Consider filing a JIRA for the proposed change.
> >
> > On Sun, Jan 17, 2016 at 8:05 PM, Toshihiro Suzuki <brfrn169@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > We have lost the data in our development environment when a row has
> more
> > > than Integer.MAX_VALUE columns after compaction.
> > >
> > > I think the reason is type of StoreScanner's countPerRow is int.
> > >
> > >
> > >
> > >
> >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java#L67
> > >
> > >
> > > After changing the type to long, it seems to be fixed.
> > >
> > > What do you think about that?
> > >
> > >
> > > Thanks,
> > >
> > > Toshihiro Suzuki.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message