incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: [DISCUSS] Packaging in 4.1
Date Wed, 13 Feb 2013 19:41:39 GMT
CentOS is currently experiencing bugs around changing user cloud's
home to /var/cloudstack, from /var/lib/cloud. Is Debian changing
/var/lib/cloud to /var/cloudstack as well? I see references to it in:

debian/cloud-client.postinst
debian/cloud-client.install
debian/cloud-usage.postinst
packaging/debian/replace.properties


On Wed, Feb 13, 2013 at 10:52 AM, Marcus Sorensen <shadowsor@gmail.com> wrote:
> Yes, this line here in package.sh:
>
> VERSION=`(cd ../../; mvn
> org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate
> -Dexpression=project.version) | grep -v '^\['`
>
> if you don't have all of the maven stuff downloaded, instead of spitting out:
>
> 4.1.0-SNAPSHOT
>
> It spits out:
>
> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
> (2 KB at 1.1 KB/sec)
> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
> (135 KB at 1094.3 KB/sec)
> 4.1.0-SNAPSHOT
>
>
> On Wed, Feb 13, 2013 at 10:45 AM, Marcus Sorensen <shadowsor@gmail.com> wrote:
>> It works for me subsequent to the first run.
>>
>> I think it's because of the output of downloading maven stuff, the
>> script doesn't seem to like that.
>>
>> On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
>> <pradeep.soundararajan@citrix.com> wrote:
>>> I am following these steps:
>>>
>>> chmod 755 ./packaging/centos63/package.sh
>>> cd packaging/centos63
>>> ./package.sh
>>>
>>> cd $WORKSPACE
>>> tempdir=`mktemp -d`
>>> mkdir -p "$tempdir"
>>> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
>>> createrepo $tempdir/
>>>
>>> The above is working well for me.
>>>
>>> Thanks,
>>> Pradeep S
>>>
>>>
>>> -----Original Message-----
>>> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
>>> Sent: Wednesday, February 13, 2013 10:31 PM
>>> To: 'Marcus Sorensen'; Wido den Hollander
>>> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
>>> Subject: RE: [DISCUSS] Packaging in 4.1
>>>
>>> Hey Marcus,
>>>
>>> I haven't updated package.sh in some time as I do most of by build directly with
Jenkins.  This is procedure that I'm currently using:
>>>
>>> <from toplevel project dir>
>>> rm -rf dist
>>> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>>>
>>> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz
--exclude .git --exclude dist .
>>>
>>> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease
true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>>>
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Hugo
>>>> -----Original Message-----
>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>> Sent: Wednesday, February 13, 2013 3:08 PM
>>>> To: Wido den Hollander
>>>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>>>> Soundararajan; cloudstack-dev@incubator.apache.org
>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>
>>>> Hm, this package.sh is still doing weird things for me. If I pull a
>>>> fresh incubator-cloudstack and run this:
>>>>
>>>> cd packaging/centos63
>>>> chmod +x package.sh
>>>> ./package.sh
>>>>
>>>> I get this output (truncated, but you get the idea)
>>>>
>>>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>>>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>>>> metadata.xml:
>>>> Cannot stat: No such file or directory
>>>> tar: (22: Cannot stat: No such file or directory
>>>> tar: KB: Cannot stat: No such file or directory
>>>> tar: at: Cannot stat: No such file or directory
>>>> tar: 96.8: Cannot stat: No such file or directory
>>>> tar: KB/sec): Cannot stat: No such file or directory
>>>>
>>>> [root@devcloud-kvm centos63]# ls
>>>> ?      ?120    145    ?172    ?203    235.3  ?27     320.7   401.7
>>>>       ?54    ?72    92
>>>> 10     ?121    ?145   172.4   20.3    ?236   270     323     402
>>>>       ?55    728.8  ?92
>>>> (10    121.3   14.5   173     204     238    ?271    326.8   (402
>>>>       551    73     921.8
>>>> ?10    121.8   146    ?174    ?204    238.6  274     327     405.7
>>>>       (551   ?73    93
>>>> 100    122     ?146   175.9   20.4    239    ?275    (33     406.3
>>>>       55.7   739.4  ?93
>>>> ?100   122.1   148    176     ?205    ?239   275.6   ?33
>>>> 4.1.0-SNAPSHOT  557.5  74     93.0
>>>> 10.0   1227.3  ?148   ?176    206     24     278     330.2   42
>>>>       56     ?74    93.3
>>>> ?101   123     149    177     207     (24    278.3   331     ?42
>>>>       ?56    75     94
>>>> 101.0  ?123    ?149   ?177    20.7    ?24    ?279    33.4    424
>>>>       56.0   75.5   ?94
>>>> 102    124     15     ?178    ?208    ?240   28      335     (424
>>>>       ?57    755.1  94.2
>>>> ?102   ?124    (15    179.3   ?209    242    (28     33.5    429.8
>>>>       58     ?76    95
>>>> 103    ?125    ?15    18      209.0   243    ?28     3352.2  ?43
>>>>       ?58    762.9  954.6
>>>> 104    126     150    (18     209.4   ?243   280.9   339     437.4
>>>>       580.7  77     96
>>>> ?104   ?126    (150   ?18     210     243.6  282     34      44
>>>>       58.2   ?77    ?96
>>>> ?105   ?127    ?150   180     211     ?244   ?283    (34     (44
>>>>       582.0  78     96.1
>>>> 106    128     151    ?180    21.1    246    283.0   ?34     ?44
>>>>       582.9  ?78    96.8
>>>> ?106   ?128    15.1   181     211.4   247    286     34.1    443.3
>>>>       (59    79     ?97
>>>> 107    ?129    152    ?182    ?212    ?247   ?287    343     44.4
>>>>       ?59    8      97.5
>>>> 10.7   (13     ?152   ?184    ?213    ?248   (289    347     44.8
>>>>       598.6  (8     98
>>>> 108    ?13     153    184.0   214     25     ?289    35      455.7
>>>>       6      ?8     ?98
>>>> ?108   130     ?153   185     215     (25    (29     ?35     46
>>>>       (6     ?80    99
>>>> ?109   ?130    153.3  ?186    2157.1  ?25    ?29     351     ?46
>>>>       ?6     ?81    998
>>>> (11    ?131    153.9  1873.3  (216    250    290     355     468
>>>>       60     818.7  (998
>>>> ?11    131.2   ?154   188     ?216    251    29.1    35.7    (468
>>>>       (60    82     99.8
>>>> 110    132     154.4  ?188    ?217    ?251   29.3    359     469.1
>>>>       ?60    ?82    at
>>>> ?110   ?132    156    1883.0  218     251.6  294     36      47
>>>>       ?61    83     available
>>>> 11.0   132.5   ?156   189     219     ?252   295.3   ?36     (47
>>>>       618.0  83.3   B
>>>> 110.9  134     157    189.0   22      252.0  296.1   36.0    ?47
>>>>       62     84     cloud-agent.rc
>>>> 111    ?134    ?158   19      (22     254    29.7    363     471
>>>>       ?62    ?84    cloud-ipallocator.rc
>>>> 112    (135    158.3  ?19     ?22     254.8  2972.9  367     (471
>>>>       63     8.4    cloud-management.rc
>>>> ?112   ?135    16     ?190    ?220    255    298     367.6   47.9
>>>>       (63    ?85    cloud-management.sysconfig
>>>> ?113   1359.0  (16    ?192    ?221    ?255   3       368     48
>>>>       ?63    85.7   cloud.spec
>>>> 113.0  136     ?16    193     222     256    (3      3683.1  ?48
>>>>       636.5  859.6  cloud-usage.rc
>>>> 114    ?136    1.6    ?194    223     (256   ?3      (37     48.1
>>>>       64     86     dependency
>>>> ?114   137     160    ?196    ?223    ?256   30      ?37     483.2
>>>>       ?64    ?86    ?Downloaded:
>>>> 114.6  137.8   ?160   197     ?225    258    (30     370     483.5
>>>>       66     86.8   Downloading:
>>>> 115    138     161    ?197    226     ?258   ?30     374     488.4
>>>>       ?66    87     for
>>>> 11.5   ?138    16.1   198     227     258.0  302     378     (49
>>>>       67     ?87    http:
>>>> 116    ?139    ?162   ?198    ?227    ?259   30.2    38      ?49
>>>>       ?67    87.7   information
>>>> ?116   1396.8  164    2       ?228    26     305     ?38     491.0
>>>>       68     88     is
>>>> 116.0  14      ?164   (2      229.4   (26    306.3   382     492
>>>>       ?68    (88    KB
>>>> 116.7  (14     16.5   ?2      23      ?26    307     386     (492
>>>>       68.2   ?88    missing,
>>>> (117   ?14     166    20      (23     (260   (31     ?39     (5
>>>>       684.9  89     no
>>>> ?117   140     ?166   ?20     ?23     ?260   ?31     390     ?5
>>>>       69     (89    org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>>>> 117.7  ?140    167.5  ?200    2.3     26.0   310.2   394     50
>>>>       ?69    ?89    package.sh
>>>> 118    141     168    20.0    230     262    311     398     ?50
>>>>       (7     89.0   POM
>>>> 119    ?141    ?168   200.9   231     26.2   315     39.9    51
>>>>       ?7     (9     replace.properties
>>>> (119   142     169    201     ?231    ?263   31.8    399.5   (51
>>>>       70     ?9     The
>>>> ?119   ?142    (17    ?201    ?232    266    319     4       ?51
>>>>       ?70    90     ?[WARNING]
>>>> 12     142.5   ?17    202     23.3    266.8  31.9    (4      52
>>>>       70.2   ?90
>>>> (12    144     ?170   ?202    234     ?267   32      ?4      ?52
>>>>       71     901.6
>>>> ?12    ?144    170.4  202.9   235     27     (32     40      5.2
>>>>       ?71    91
>>>> 120    144.8   172    203     ?235    (27    ?32     ?40     54
>>>>       72     91.4
>>>>
>>>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>>>> wonder if it's setting an env variable that allows it to work the second
time.
>>>>
>>>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>>>> <shadowsor@gmail.com> wrote:
>>>> > The packaging/centos63/package.sh makes some assumptions about how
>>>> > it's being run that end up with some ugly results if it's not done
>>>> > exactly right. For example, I tried from the incubator-cloudstack
>>>> > directory:  "sh ./packaging/centos63/package.sh", which seemed to
>>>> > copy /proc into my current directory and attemped to tar it up. Then
>>>> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
>>>> > roughly the same result, although it died trying to run "Downloading:
>>>> > http://repo.maven..." as a bash command.
>>>> >
>>>> > Seems it didn't like being run as an 'sh' either way, even though
>>>> > it's not in the code as executable. After doing a chmod to make it
>>>> > executable, it seems to work but only if your cwd is
>>>> > incubator-cloudstack/packaging/centos63.
>>>> >
>>>> > Maybe we could change it to fail gracefully if your current path
>>>> > doesn't end in "packaging/centos63", and make it executable in git?
>>>> >
>>>> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wido@widodh.nl>
>>>> wrote:
>>>> >>
>>>> >>
>>>> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>>> >>>
>>>> >>> Hey guys,
>>>> >>>
>>>> >>> Just a quick note before the weekend with a status update on
RPM.
>>>> >>>
>>>> >>> The management server package is pretty much done and installation
>>>> >>> on a clean system works like a charm. This is actually tested
>>>> >>> every few hours with a Jenkins setup a colleague and I built.
We
>>>> >>> take the sources, compile and package. The packages are added
to a
>>>> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>>>> >>> One is configured as database server and another one as CloudStack
>>>> >>> management server by chef. After installation an ApiKey is created
>>>> >>> for the admin user. This proves that the package can be installed
>>>> >>> on a
>>>> clean system and that the management server starts.
>>>> >>>
>>>> >>> With this testing we have found several issues of which a few
>>>> >>> haven't been resolved yet (hopefully this weekend):
>>>> >>>
>>>> >>>   * 4.1-new-db-schema.sql is not loaded by
>>>> >>> cloudstack-setup-databases
>>>> >>> * userid is null in reponse to a login call with the admin user,
>>>> >>> expected
>>>> >>> 2
>>>> >>> * Excryption initialization is now done in Transaction, this
>>>> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail
is
>>>> >>> db.properties is not in the classpath
>>>> >>>
>>>> >>> Next week:
>>>> >>>   * we will continue with the setup and add some real tests
to
>>>> >>> create zones and add hypervisors.
>>>> >>>   *I will also start testing with the agent and usage package,
>>>> >>> they are created at the moment but not tested for functionality.
>>>> >>>   * Deploy fedora 18 image and extend the test to that
>>>> >>>   * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>>>> >>> with
>>>> >>> wido/noa)
>>>> >>>
>>>> >>
>>>> >> We'll sync next week! A lot of the .deb work is already done, but
>>>> >> we just have to make sure the RPM and DEB packages contain the same
files.
>>>> >>
>>>> >> Then it will just be tuning and some work in the pre and postinst
>>>> >> files, but that could be a pain, but we'll just see when we go along.
>>>> >>
>>>> >> Wido
>>>> >>
>>>> >>
>>>> >>> Cheers,
>>>> >>>
>>>> >>> Hugo
>>>> >>>
>>>> >>>> -----Original Message-----
>>>> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>> >>>> Sent: Thursday, February 07, 2013 2:39 AM
>>>> >>>> To: David Nalley
>>>> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>>>> >>>> cloudstack- dev@incubator.apache.org
>>>> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>> >>>>
>>>> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>> >>>>>
>>>> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>>>> <Alex.Huang@citrix.com>
>>>> >>>>
>>>> >>>> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Well, first, Apache CloudStack only releases
source code.
>>>> >>>>>>>
>>>> >>>>>>> But Wido is kind enough to also host RPM / DEB
package repos
>>>> >>>>>>> for users to take advantage of.  Our install
guide explains
>>>> >>>>>>> how to build from source, as well as how to
use Wido's repos.
>>>> >>>>>>>
>>>> >>>>>>> This was all true for 4.0.0-incubating, and
I think it still
>>>> >>>>>>> holds true for all future releases.
>>>> >>>>>>>
>>>> >>>>>> Chip,
>>>> >>>>>>
>>>> >>>>>> Can you refresh my memory as to why this is?  I
look at
>>>> >>>>>> something like cxf
>>>> >>>>
>>>> >>>> or tomcat, they all have binary downloads available.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> http://cxf.apache.org/download.html
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>> Because providing 'binaries' isn't necessarily problematic,
but
>>>> >>>>> making yum and apt repos work in the ASF mirror system
seems a
>>>> >>>>> bit more of an issue. Plus, Wido stepped up to do the
work, no
>>>> >>>>> one else has offered any other alternatives.
>>>> >>>>>
>>>> >>>>> --David
>>>> >>>>>
>>>> >>>>
>>>> >>>> Yup - exactly what David said.  We had discussed trying
to get
>>>> >>>> ASF Infra to help us host package repos somewhere, but I
don't
>>>> >>>> think that went anywhere.  And since Wido's doing it, it
avoided
>>>> >>>> all sorts of questions from the infra team around mirrors,
>>>> >>>> archiving, etc...
>>>> >>>>
>>>> >>>> -chip

Mime
View raw message