hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J Mohamed Zahoor <jmo...@gmail.com>
Subject Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
Date Fri, 10 Aug 2012 07:36:45 GMT
Sorry.. previous mail Not for this thread.

./zahoor

On Fri, Aug 10, 2012 at 12:54 PM, J Mohamed Zahoor <jmozah@gmail.com> wrote:

> Look at this post for more about Catalog Janitor
>
> http://blog.zahoor.in/2012/08/hbase-hmaster-architecture/
>
> ./zahoor
>
>
> On Fri, Aug 10, 2012 at 12:47 PM, J Mohamed Zahoor <jmozah@gmail.com>wrote:
>
>> Hi Lars,
>>
>> Will file it...
>>
>> ./Zahoor
>>
>>
>> On Fri, Aug 10, 2012 at 12:00 AM, lars hofhansl <lhofhansl@yahoo.com>wrote:
>>
>>> Hi zahoor,
>>>
>>> could you file a jira with what you found? It looks like a bug.
>>> Thanks.
>>>
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: J Mohamed Zahoor <jmozah@gmail.com>
>>> To: user@hbase.apache.org
>>> Cc:
>>> Sent: Thursday, August 9, 2012 2:03 AM
>>> Subject: Re: Hbase- Hadoop DFS size not decreased even after deleting a
>>> column family
>>>
>>> Hi
>>>
>>> I see this atleast in trunk... in TableDeleteFamilyHandler's
>>> constructor..
>>>
>>> public TableDeleteFamilyHandler(byte[] tableName, byte [] familyName,
>>>
>>>       Server server, final MasterServices masterServices)
>>> throwsIOException {
>>>
>>>     super(EventType.C_M_ADD_FAMILY, tableName, server, masterServices);
>>>
>>>     HTableDescriptor htd = getTableDescriptor();
>>>
>>>     this.familyName = hasColumnFamily(htd, familyName);
>>>
>>>   }
>>>
>>>
>>> The event type is added as C_M_ADD_FAMILY....
>>> May be this is the problem and it is there in 0.94.1 also !!!
>>>
>>> ./zahoor
>>>
>>>
>>> On Wed, Aug 8, 2012 at 5:33 PM, prem yadav <ipremyadav@gmail.com> wrote:
>>>
>>> > Hi Stack,
>>> > we have got a lot of these in the logs after we deleted the column
>>> family.
>>> >
>>> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
>>> Column
>>> > family ol does not exist in region <region>
>>> >         at
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:4759)
>>> >         at
>>> >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:2639)
>>> >         at
>>> >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2065)
>>> >         at
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1962)
>>> >         at
>>> >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3428)
>>> >         at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
>>> >         at
>>> >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >         at java.lang.reflect.Method.invoke(Method.java:601)
>>> >         at
>>> >
>>> >
>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>>> >         at
>>> >
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1386)
>>> > 2012-08-05 00:11:37,697 DEBUG
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> > rollbackMemstore rolled back 0 keyvalues from start:0 to end:1
>>> >
>>> >
>>> >
>>> >
>>> > On Tue, Aug 7, 2012 at 10:27 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> >
>>> > > Looking at TableDeleteFamilyHandler, I see:
>>> > >
>>> > >   protected void handleTableOperation(List<HRegionInfo> hris)
throws
>>> > > IOException {
>>> > >     // Update table descriptor in HDFS
>>> > >     HTableDescriptor htd =
>>> > >
>>>  this.masterServices.getMasterFileSystem().deleteColumn(tableName,
>>> > > familyName);
>>> > >     // Update in-memory descriptor cache
>>> > >     this.masterServices.getTableDescriptors().add(htd);
>>> > >
>>> > > MasterFileSystem.deleteColumn() calls:
>>> > >
>>> > >     HTableDescriptor htd =
>>> > > this.services.getTableDescriptors().get(tableName);
>>> > >     htd.removeFamily(familyName);
>>> > >
>>> > > I will dig some more.
>>> > >
>>> > > But looks like there is a bug.
>>> > >
>>> > > On Tue, Aug 7, 2012 at 9:21 AM, Stack <stack@duboce.net> wrote:
>>> > >
>>> > > > On Tue, Aug 7, 2012 at 9:59 AM, prem yadav <ipremyadav@gmail.com>
>>> > wrote:
>>> > > > > Hi Stack,
>>> > > > > yes . I can still see the column family.
>>> > > > >
>>> > > > > we ran the following command too:
>>> > > > >
>>> > > > > fs -dus /hbase/<table name>/<region hash>/<cf
that was deleted>
>>> and
>>> > it
>>> > > > > returned a large value which means that the space has not
been
>>> > > reclaimed.
>>> > > > > The hbase version we are running is 0.94.1.
>>> > > > >
>>> > > > > is there  a manual way to force reclaim the space?
>>> > > > >
>>> > > >
>>> > > > You could delete the column family from hdfs under each region.
>>> > > > Anything in the logs around the remove of the column family?
>>> Maybe we
>>> > > > just don't do the delete cleanup in hdfs when we drop a cf (I've
>>> not
>>> > > > checked)?
>>> > > >
>>> > > > St.Ack
>>> > > >
>>> > >
>>> >
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message