hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil Gupta <anilgupt...@gmail.com>
Subject Re: question on reducing number of versions
Date Mon, 27 Aug 2018 02:32:48 GMT
You should see a smaller t2 after major compaction if your table actually had versions over
18k.(as Ted mentioned)

Sent from my iPhone

> On Aug 26, 2018, at 5:20 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> 
> This depends on how far down you revise the max versions for table t2.
> If your data normally only reaches 15000 versions and you lower max
> versions to ~15000, there wouldn't be much saving.
> 
> FYI
> 
>> On Sun, Aug 26, 2018 at 3:52 PM Antonio Si <antonio.si@gmail.com> wrote:
>> 
>> Thanks Anil.
>> 
>> We are using hbase on s3. Yes, I understand 18000 is very high. We are in
>> the process of reducing it.
>> 
>> If I have a snapshot and I restore the table from this snapshot. Let's call
>> this table t1.
>> I then clone another table from the same snapshot, call it t2.
>> 
>> If I reduce the max versions of t2 and run a major compaction on t2, will I
>> see the decrease in table size for t2? If I compare the size of t2 and t1,
>> I should see a smaller size for t2?
>> 
>> Thanks.
>> 
>> Antonio.
>> 
>>> On Sun, Aug 26, 2018 at 3:33 PM Anil Gupta <anilgupta84@gmail.com> wrote:
>>> 
>>> You will need to do major compaction on table for the table to
>>> clean/delete up extra version.
>>> Btw, 18000 max version is a unusually high value.
>>> 
>>> Are you using hbase on s3 or hbase on hdfs?
>>> 
>>> Sent from my iPhone
>>> 
>>>> On Aug 26, 2018, at 2:34 PM, Antonio Si <antonio.si@gmail.com> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> I have a hbase table whose definition has a max number of versions set
>> to
>>>> 36000.
>>>> I have verified that there are rows which have more than 20000 versions
>>>> saved.
>>>> 
>>>> Now, I change the definition of the table and reduce the max number of
>>>> versions to 18000. Will I see the size of the table being reduced as I
>> am
>>>> not seeing that?
>>>> 
>>>> Also, after I reduce the max number of versions, I try to create a
>>>> snapshot, but I am getting a
>>>> com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.mo
>>>> 
>>>> del.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404;
>>>> Error Code: 404 Not Found;
>>>> 
>>>> 
>>>> What may be the cause of that?
>>>> 
>>>> I am using s3 as my storage.
>>>> 
>>>> 
>>>> Thanks in advance for your suggestions.
>>>> 
>>>> 
>>>> Antonio.
>>> 
>> 

Mime
View raw message