hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toshihiro Suzuki <brfrn...@gmail.com>
Subject Re: Data loss after compaction when a row has more than Integer.MAX_VALUE columns
Date Wed, 20 Jan 2016 03:17:36 GMT
> Is the performance OK if i request the number of incoming-follow-nodes?
Yes, I think no problem.

At least, in our use case, the performance is OK.

2016-01-18 16:05 GMT+09:00 Heng Chen <heng.chen.1986@gmail.com>:

> The incoming-follow-nodes and outgoing-follow-nodes of one node exceed
> Integer.MAX_VALUE,  unbelievable!
>
> Is the performance OK if i request the number of incoming-follow-nodes?
>
>
> 2016-01-18 13:26 GMT+08:00 Toshihiro Suzuki <brfrn169@gmail.com>:
>
> > Thank you for your reply.
> >
> > We are using hbase to store social graph data on the SNS we provide.
> >
> > Our use case was presented in HBasecon 2015.
> >
> > http://www.slideshare.net/HBaseCon/use-cases-session-6a
> >
> > Schema Design is the below,
> >
> > http://www.slideshare.net/HBaseCon/use-cases-session-6a/44
> >
> > Thanks,
> > Toshihiro Suzuki.
> >
> >
> > 2016-01-18 13:48 GMT+09:00 Ted Yu <yuzhihong@gmail.com>:
> >
> > > Interesting.
> > >
> > > Can you share your use case where more than Integer.MAX_VALUE columns
> are
> > > needed.
> > >
> > > Consider filing a JIRA for the proposed change.
> > >
> > > On Sun, Jan 17, 2016 at 8:05 PM, Toshihiro Suzuki <brfrn169@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We have lost the data in our development environment when a row has
> > more
> > > > than Integer.MAX_VALUE columns after compaction.
> > > >
> > > > I think the reason is type of StoreScanner's countPerRow is int.
> > > >
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java#L67
> > > >
> > > >
> > > > After changing the type to long, it seems to be fixed.
> > > >
> > > > What do you think about that?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Toshihiro Suzuki.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message