hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <sean.bus...@gmail.com>
Subject Re: hbase build sizes
Date Wed, 08 Mar 2017 21:37:34 GMT
Change is in place, I'll take a look at some build results tomorrow to
see if it's going well.

In the mean time, just a heads up to the committers that contributors
will have questions if things fail and the logs didn't get properly
zipped up.

@Ted Yu I also undid your change of keeping builds for only 3 days.
Please don't change the retention period; 3 days is not enough time
for us reasonably expect volunteer contributors to circle back on
getting their build results. If you updated the retention on any of
the other jobs please restore them.

On Wed, Mar 8, 2017 at 3:26 PM, Sean Busbey <sean.busbey@gmail.com> wrote:
> Okay, I'm reasonably sure this is just test logs. I'm updating our
> builds to zip up the test logs as a post-build action, which should
> bring down our space consumption by a couple of orders of magnitude.
>
>
> On Wed, Mar 8, 2017 at 3:03 PM, Sean Busbey <sean.busbey@gmail.com> wrote:
>> I grabbed one of the items Chris pointed me towards:
>>
>> https://builds.apache.org/job/PreCommit-HBASE-Build/5991/
>>
>> If you download the archived artifacts as a zip, it's ~61MiB. If you
>> uncompressed that zip file, you get ~1.3GiB of logs.
>>
>> I'm going to try to grab a couple more to make sure it's consistently test logs.
>>
>> On Wed, Mar 8, 2017 at 2:31 PM, Chris Lambertus <cml@apache.org> wrote:
>>> Hi HBase folks,
>>>
>>> Just wanted to advise you of some build size issues we noticed on the jenkins
master. It looks like over a span of about 2 weeks, the hbase builds are using a very large
amount of storage in the retained builds directory:
>>>
>>> 79G        PreCommit-HBASE-Build
>>> 72G        HBase-1.3-JDK8
>>> 68G        HBase-1.3-JDK7
>>> 60G        HBase-1.2-JDK7t
>>> 59G        HBase-1.2-JDK8
>>> 51G        HBase-1.2-IT
>>> 36G        HBase Website Link Ckecker
>>>
>>> Each stored build appears to use on the average of 600-1.1GB of disk space, and
there are over 200 builds in two weeks for PreCommit-HBASE-Build alone. Some are much smaller
(1-8MB.)
>>>
>>> We are running a bit low on storage on the jenkins master, so anything you could
help us do to reduce the footprint here would be appreciated.
>>>
>>> I’ve already discussed this a bit with Sean Busbey on Hipchat, he asked me
to follow up here.
>>>
>>> Thanks,
>>>
>>> -Chris
>>> ASF Infra
>>>
>>
>>
>>
>> --
>> Sean
>
>
>
> --
> Sean



-- 
Sean

Mime
View raw message