cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nalley <da...@gnsa.us>
Subject Re: jenkins disk full ?
Date Wed, 30 Sep 2015 09:40:24 GMT
SGTM, Feel free to.



On Wed, Sep 30, 2015 at 5:34 AM, Daan Hoogland <daan.hoogland@gmail.com> wrote:
> H David, I just tried and it works indeed. Forgot about that. Does it make
> sense to have a script containing du mail to dev@ if something like
> /var/lib/jenkins/workspace or / exceeds a threshold? (yes me volunteering)
>
> On Wed, Sep 30, 2015 at 10:48 AM, Raja Pullela <raja.pullela@citrix.com>
> wrote:
>
>> Btw, edison, pradeep, rayeesn - are not active anymore - talluri may be?
>>  I think.
>>
>> -----Original Message-----
>> From: David Nalley [mailto:david@gnsa.us]
>> Sent: Wednesday, September 30, 2015 2:01 PM
>> To: dev@cloudstack.apache.org
>> Cc: Pierre-Luc Dion <pdion891@apache.org>; talluri@apache.org
>> Subject: Re: jenkins disk full ?
>>
>> Right now these folks have access:
>>
>> ewanm:x:1002:1002::/home/ewanm:/bin/bash
>> prasanna:x:1003:1003::/home/prasanna:/bin/bash
>> salvatore:x:1004:1004::/home/salvatore:/bin/bash
>> edison:x:1005:1005::/home/edison:/bin/bash
>> pradeep:x:1006:1006::/home/pradeep:/bin/bash
>> rayeesn:x:1007:1007::/home/rayeesn:/bin/bash
>> hugot:x:1008:1008::/home/hugot:/bin/bash
>> talluri:x:1009:1009::/home/talluri:/bin/bash
>> dahn:x:1010:1010::/home/dahn:/bin/bash
>>
>> PL or other folks who want access, send me a ssh pub key off list.
>>
>> --David
>>
>> On Wed, Sep 30, 2015 at 4:20 AM, Rajani Karuturi <
>> Rajani.Karuturi@citrix.com> wrote:
>> > Thanks david. Do you know if anyone else has access to this? if not, can
>> you give access to PL please?
>> >
>> > ~Rajani
>> >
>> >
>> >
>> > On 30-Sep-2015, at 1:13 pm, David Nalley <david@gnsa.us> wrote:
>> >
>> >> The below is before I cleaned up a bit. We now have 3.2GB free.
>> >>
>> >> [root@domU-12-31-39-15-22-2D jobs]# du -h -d 1 1.3G
>> >> ./package-rhel63-master 200M ./cloudstack-4.3-maven-build-noredist
>> >> 638M ./package-rhel63-4.5
>> >> 1.1G ./cloudstack-4.4-systemvm64
>> >> 1.3M ./build-4.5
>> >> 8.0K ./cloudstack-master-with-patch
>> >> 186M ./build-test-archive-4.4
>> >> 2.9M ./test-smoke-matrix-master
>> >> 100K ./Create-Release-Packages
>> >> 1.5M ./test-packaging-4.4
>> >> 214M ./build-test-archive-master
>> >> 612K ./cloudstack.apt-get.eu-systemvm 88M ./built-test-archive 4.2M
>> >> ./simulator-4.5-singlerun 424K ./test-tabnanny-4.4 2.2G
>> >> ./build-master 124M ./cloudstack-4.4-systemvm 331M
>> >> ./package-centos7-4.4.4-noredist 804K ./cloudstack-4.4-maven-build
>> >> 3.4M ./cloudstack-4.3-maven-build 645M ./package-centos7-master 80M
>> >> ./test-fetch-marvin-master 180M ./parameterized-slowbuild 508K
>> >> ./cloudstack.apt-get.eu-noredis-rhel
>> >> 104K ./dockerbuild-master-marvin
>> >> 9.1M ./test-regression-matrix-master
>> >> 345M ./cloudstack-4.3-package-deb
>> >> 3.8G ./parameterized-sytemvm
>> >> 26M ./HyperVAgent4.3
>> >> 510M ./cloudstack-4.3-package-rpm
>> >> 1.5M ./test-packaging-master
>> >> 7.2M ./build-docs-admin-master
>> >> 8.3M ./cloudstack-marvin-master
>> >> 1013M ./cloudstack-4.3-systemvm
>> >> 131M ./run-checkin-tests
>> >> 2.5M ./simulator-gate
>> >> 145M ./cloudstack-master-twiceweekly-coverity
>> >> 3.2M ./cloudstack-4.4-hyperv-agent
>> >> 104K ./test-setup-advancedzone-master 1.1G ./master-slowbuild 44M
>> >> ./cloudstack-master 301M ./package-centos7-master-noredist 396K
>> >> ./test-tabnanny-master 816K ./simulator-build 20K ./test-libcloud
>> >> 236K ./test-smoke-matrix-4.3 4.9M ./dockerbuild-4.5-simulator 331M
>> >> ./package-centos7-4.4-noredist 632K
>> >> ./cloudstack.apt-get.eu-noredis-deb
>> >> 144K ./dockerbuild-master-management_centos6
>> >> 292M ./package-centos6-4.6
>> >> 9.4M ./stop-jetty
>> >> 326M ./cloudstack-4.4-package-rpm
>> >> 40K ./mgmt-build-reviewboard-requests 20K ./build-test-centos-builtin
>> >> 76K ./simulator-deploydb 20K ./Master - Python Static Code Analysis
>> >> 53M ./test-smoke-matrix-4.4 442M ./build-master-slowbuild 376K
>> >> ./build-docs-niciranvp-master 840K ./test-matrix-4.4 2.7M
>> >> ./dockerbuild-master-simulator 21M ./build-docs-install-master 50M
>> >> ./test-fetch-marvin-4.4 789M
>> >> ./cloudstack-rpm-packages-with-branch-parameter
>> >> 5.3G ./build-systemvm-4.5
>> >> 1.1G ./cloudstack-4.3-systemvm64
>> >> 3.2M ./build-docs-release-notes-master 8.0K
>> >> ./cloudstack-master-codecoverage 121M ./fastsimulatorbuild 4.3M
>> >> ./cloudstack-4.4-marvin 352M ./build-master-noredist 7.2M
>> >> ./test-setup-advancedzone-4.4 3.3M ./cloudstack-master-hyperv-agent
>> >> 1.9G ./build-systemvm64-master 29M ./start-jetty 3.4M
>> >> ./simulator-4.4-integration-tests 264M ./build-4.5-noredist 2.0G
>> >> ./package-deb-master 576K ./build-master-simulator 137M
>> >> ./mgmt-update-nonoss-libs 11M ./test-matrix-extended-master 235M
>> >> ./cloudstack-4.4-maven-build-noredist
>> >> 5.3M ./simulator-hotfix-trigger
>> >> 8.0K ./cloudstack-maven-build-with-branch-parameter
>> >> 2.7M ./test-yumrepo-refresh-master
>> >> 1.3G ./cloudstack-4.5-hyperv-agent
>> >> 1.6M ./Build-by-Branch
>> >> 3.5G ./build-systemvm-master
>> >> 369M ./package-deb-4.6
>> >> 172K ./test-maintenance-operations-4.4 48K
>> >> ./cloudstack-4.3-package-rpm-noredist
>> >> 532K ./build-docs-vxlan-master
>> >> 18M ./build-docs-devguide-master
>> >> 456K ./test-environment-refresh-master 876K ./test-matrix-master 2.1M
>> >> ./test-yumrepo-refresh-4.4 51M ./test-environment-refresh-4.4 6.3M
>> >> ./simulator-singlerun 724M ./package-deb-4.5 16M
>> >> ./docs-4.3-gsoc-guide 384K ./test-matrix-extended-4.4 20K
>> >> ./pull-requests 204K ./build-docs-midonet-master 358M
>> >> ./cloudstack-4.4-package-deb 506M ./build-master-jdk18 2.3M
>> >> ./cloudstack.apt-get.eu-noredis-centos7
>> >> 499M ./package-centos7-4.6
>> >> 119M ./build-4.5-simulator
>> >> 12K ./cloudstack-4.2-systemvm_test
>> >> 1.6M ./test-regression-matrix-4.4
>> >> 144K ./build-systemvm64-master-with-debian8
>> >> 12K ./cloudstack-4.2-systemvm
>> >> 12K ./cloudstack-4.2-systemvm64
>> >> 3.1M ./cloudstack-4.4-auto-marvin
>> >> 334M ./package-rhel63-master-noredist 6.8M
>> >> ./test-maintenance-operations-master
>> >> 36G .
>> >>
>> >> On Tue, Sep 29, 2015 at 9:04 AM, Pierre-Luc Dion <pdion891@apache.org>
>> wrote:
>> >>> David, Do you think you could do another cleanup on  the master of
>> j.bac.o ?
>> >>>
>> >>> Could you list the jobs that consume most of the spaces?
>> >>>
>> >>> Thanks,
>> >>>
>> >>>
>> >>> On Wed, Sep 23, 2015 at 12:22 AM, Rajani Karuturi
>> >>> <Rajani.Karuturi@citrix.com> wrote:
>> >>>>
>> >>>> Hi David,
>> >>>> disk space issue again. Can you please do a little more cleanup?
>> >>>>
>> >>>>
>> >>>> http://jenkins.buildacloud.org/computer/coohq-slave-01/executors/0/
>> >>>> causeOfDeath
>> >>>>
>> >>>> ~Rajani
>> >>>>
>> >>>>
>> >>>>
>> >>>> On 18-Sep-2015, at 11:59 pm, David Nalley <david@gnsa.us>
wrote:
>> >>>>
>> >>>>> I freed up a little bit of space, but we'll have to do something
>> >>>>> different in the long term.
>> >>>>>
>> >>>>> --David
>> >>>>>
>> >>>>> On Fri, Sep 18, 2015 at 7:37 AM, Pierre-Luc Dion
>> >>>>> <pdion@cloudops.com>
>> >>>>> wrote:
>> >>>>>> Hi,
>> >>>>>>
>> >>>>>> Srikanteswararao (or any one having access to jenkins server),
do
>> >>>>>> you think you could do some more magic here?  We ran out
of space
>> >>>>>> again on the master jenkins server for  j.bac.o.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> PL
>> >>>>>>
>> >>>>>> On Mon, Jun 1, 2015 at 4:26 PM, Pierre-Luc Dion
>> >>>>>> <pdion@cloudops.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> thanks  Srikanteswararao, it worked  and we have centos63
rpm
>> >>>>>>> now for master branch!
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> *Pierre-Luc DION*
>> >>>>>>> Architecte de Solution Cloud | Cloud Solutions Architect
t
>> >>>>>>> 855.652.5683
>> >>>>>>>
>> >>>>>>> *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions
>> >>>>>>> Experts
>> >>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >>>>>>> *|* tw @CloudOps_
>> >>>>>>>
>> >>>>>>> On Mon, Jun 1, 2015 at 3:09 PM, Srikanteswararao talluri
<
>> >>>>>>> talluri@apache.org> wrote:
>> >>>>>>>
>> >>>>>>>> There is no space .
>> >>>>>>>>
>> >>>>>>>> /dev/xvdf1       50G   47G     0 100% /var/lib/jenkins/workspace
>> >>>>>>>>
>> >>>>>>>> I have cleaned up builds for 'fastsimulatorbuild'
job which
>> >>>>>>>> were using 9G.
>> >>>>>>>> Modified the job to delete old builds and artefacts
after 30
>> >>>>>>>> builds/30days.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> ~Talluri
>> >>>>>>>>
>> >>>>>>>> On Mon, Jun 1, 2015 at 8:14 PM, Pierre-Luc Dion
>> >>>>>>>> <pdion@cloudops.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Can someone that have access to the principal
jenkins server
>> >>>>>>>>> to check
>> >>>>>>>> disk
>> >>>>>>>>> space left?
>> >>>>>>>>>
>> >>>>>>>>> getting this error on few slave, the one I've
check does not
>> >>>>>>>>> have disk
>> >>>>>>>> full
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Thread has died
>> >>>>>>>>>
>> >>>>>>>>> java.io.IOException: No space left on device
>> >>>>>>>>>       at java.io.FileOutputStream.writeBytes(Native
Method)
>> >>>>>>>>>       at
>> java.io.FileOutputStream.write(FileOutputStream.java:318)
>> >>>>>>>>>       at
>> >>>>>>>>> sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>> >>>>>>>>>       at
>> sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:316)
>> >>>>>>>>>       at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
>> >>>>>>>>>       at
>> >>>>>>>>> java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
>> >>>>>>>>>       at java.io.BufferedWriter.close(BufferedWriter.java:266)
>> >>>>>>>>>       at
>> >>>>>>>>> hudson.util.AtomicFileWriter.close(AtomicFileWriter.java:94)
>> >>>>>>>>>       at
>> >>>>>>>> hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:109)
>> >>>>>>>>>       at hudson.util.TextFile.write(TextFile.java:121)
>> >>>>>>>>>       at hudson.model.Job.saveNextBuildNumber(Job.java:274)
>> >>>>>>>>>       at hudson.model.Job.assignBuildNumber(Job.java:332)
>> >>>>>>>>>       at hudson.model.Run.<init>(Run.java:286)
>> >>>>>>>>>       at
>> hudson.model.AbstractBuild.<init>(AbstractBuild.java:167)
>> >>>>>>>>>       at hudson.model.Build.<init>(Build.java:92)
>> >>>>>>>>>       at
>> hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:34)
>> >>>>>>>>>       at
>> >>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> >>>>>>>>> Method)
>> >>>>>>>>>       at
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeCon
>> >>>>>>>> structorAccessorImpl.java:57)
>> >>>>>>>>>       at
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Deleg
>> >>>>>>>> atingConstructorAccessorImpl.java:45)
>> >>>>>>>>>       at
>> >>>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>> >>>>>>>>>       at
>> >>>>>>>>>
>> jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:186)
>> >>>>>>>>>       at
>> >>>>>>>> hudson.model.AbstractProject.newBuild(AbstractProject.java:1000
>> >>>>>>>> )
>> >>>>>>>>>       at
>> >>>>>>>>>
>> >>>>>>>>>
>> hudson.model.AbstractProject.createExecutable(AbstractProject.java:1193)
>> >>>>>>>>>       at
>> >>>>>>>>>
>> >>>>>>>>>
>> hudson.model.AbstractProject.createExecutable(AbstractProject.java:143)
>> >>>>>>>>>       at hudson.model.Executor.run(Executor.java:213)
>> >>>>>>>>>
>> >>>>>>>>> more info
>> >>>>>>>>> <http://wiki.jenkins-ci.org/display/JENKINS/Dead+Executor>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> ref:
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> http://jenkins.buildacloud.org/computer/cca-slave-02/executors/
>> >>>>>>>> 0/causeOfDeath
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Thanks!
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>
>> >>>
>> >
>>
>
>
>
> --
> Daan

Mime
View raw message