hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Benson <Martin.Ben...@jaywing.com>
Subject RE: Clear up Hive scratch directory
Date Fri, 24 Apr 2015 15:40:33 GMT
Hi Nitin,

Certainly – it was pretty straightforward! The process was just:


1)     Stop Hive (this is to ensure that there are no sessions open which may be expecting
to read/write to the scratch directory at that time)

2)     I think that by default the scratch directories are named tmp/hive-*username*. In my
case it was tmp/hive-hive that contained a load of bloat. If you’re unsure of the full path
you’ll be able to find it in the Hive configuration.

3)     Do a recursive delete to clear the contents of the scratch directory that you want
to empty using the hdfs command line

4)     Re-start Hive

Thanks,

Martin.

From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: 24 April 2015 09:44
To: user@hive.apache.org
Subject: Re: Clear up Hive scratch directory

Thanks Martin

Can you also mention steps you did to reclaim the hdfs data from temporary data ?

On Fri, Apr 24, 2015 at 12:21 PM, Martin Benson <Martin.Benson@jaywing.com<mailto:Martin.Benson@jaywing.com>>
wrote:
Hi All,

I just wanted to feedback that it does appear to be safe - I emptied the directory manually,
without adverse consequences.

Thanks,

Martin.
________________________________
From: Martin Benson<mailto:Martin.Benson@jaywing.com>
Sent: ‎20/‎04/‎2015 18:06
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Clear up Hive scratch directory
Hi,

One of my users tried to run an HUGE join, which failed due to a lack of space in HDFS. This
has resulted in a large amount of data remaining in the Hive scratch directory which I need
to clear down. I've tried setting hive.start.cleanup.scratchdir to true and restarting Hive,
but it didn't tidy it up. So, I'm wondering if it is safe to just delete the content of the
directory in HDFS (while Hive is stopped). Could anyone advise please?

Many thanks,

Martin.


Registered in England and Wales at Players House, 300 Attercliffe Common, Sheffield, S9 2AG.
Company number 05935923.

This email and its attachments are confidential and are intended solely for the use of the
addressed recipient.
Any views or opinions expressed are those of the author and do not necessarily represent Jaywing.
If you are not
the intended recipient, you must not forward or show this to anyone or take any action based
upon it.
Please contact the sender if you received this in error.


Registered in England and Wales at Players House, 300 Attercliffe Common, Sheffield, S9 2AG.
Company number 05935923.

This email and its attachments are confidential and are intended solely for the use of the
addressed recipient.
Any views or opinions expressed are those of the author and do not necessarily represent Jaywing.
If you are not
the intended recipient, you must not forward or show this to anyone or take any action based
upon it.
Please contact the sender if you received this in error.



--
Nitin Pawar



Registered in England and Wales at Players House, 300 Attercliffe Common, Sheffield, S9 2AG.
Company number 05935923.

This email and its attachments are confidential and are intended solely for the use of the
addressed recipient.
Any views or opinions expressed are those of the author and do not necessarily represent Jaywing.
If you are not
the intended recipient, you must not forward or show this to anyone or take any action based
upon it.
Please contact the sender if you received this in error.
Mime
View raw message