incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Noah Slater <nsla...@tumbolia.org>
Subject Re: [VOTE] Apache Cloudstack 4.0.0-incubating Release, first round
Date Fri, 12 Oct 2012 08:46:44 GMT
Okay, stepping through the test procedure.
*
*
*> Download DevCloud image from
http://download.cloud.com/templates/devcloud/DevCloud.ova into your machine.
*

We should add a warning that this is a 1.5GB image!

Who maintains this? Who WILL maintain this?

*> Open VirtualBox, File->Import Appliance->choose DevCloud.ova*

The license screen lists the license terms as:

"*Apache License 2.0*"

That's it. Should this include the license text instead?

*> Install your required development tools (if not already available)*

Where? On my local machine? Or into the running DevCloud image?

I am assuming my local machine for now.

*> sudo aptitude install openjdk-6-jdk ant*

I've Googled around, and I can't find any obvious way to install this on OS
X 10.8. I found a link to the official Java SE 6 JDK, but I don't know
enough about Java to know whether this would be fine, or whether I need the
OpenJDK.

Looking at the rest of the instructions, I think I'm going to have a hard
time installing all this without some guidance.

Is there anyone on the ML who has this set up on OS X 10.8? Can you add
instructions for getting set up?

Skipping this section because I cannot complete it without further
instruction.

*> mkdir -p ~/cloudstack; cd ~/cloudstack*

I would use something under /tmp for safety.

I would also include a step to blow away any previous dir. (You'd be
surprised how often people will just try to do a second round of voting
from the same dir, and files get mixed up and confused.)

*> Download the artifacts:*

We are exporting a `zip` file and a `tar.gz` file. Which means we have have
two source artefacts to vote on. I've not encountered this before. I think
this means that I have to test both of them separately, which is a burden.
Can we just pick one archive format and vote on that? Realistically, who is
going to be installing from source anyway? Do we really need the `zip`
format here?

The `zip` file is 16M, compared to 10M for the `tar.gz` file. Seems odd.

*> Import keys: *

We should mention that the user import from the KEYS file first, and
suggest importing from a public key server second.

*>  gpg --verify apache-cloudstack-4.0.0-incubating-src.tar.gz.asc*

Good sig.

(We perhaps need to indicate what output people are looking for here. This
is a crucial step.)

*> gpg --verify apache-cloudstack-4.0.0-incubating-src.zip.asc*

Good sig.

(We perhaps need to indicate what output people are looking for here. This
is a crucial step.)

*> gpg --print-md MD5 apache-cloudstack-4.0.0-incubating-src.tar.gz | diff
- apache-cloudstack-4.0.0-incubating-src.tar.gz.md5*
*> gpg --print-md MD5 apache-cloudstack-4.0.0-incubating-src.zip | diff -
apache-cloudstack-4.0.0-incubating-src.zip.md5*
*> gpg --print-md SHA512 apache-cloudstack-4.0.0-incubating-src.tar.gz |
diff - apache-cloudstack-4.0.0-incubating-src.tar.gz.sha*
*> gpg --print-md SHA512 apache-cloudstack-4.0.0-incubating-src.zip | diff
- apache-cloudstack-4.0.0-incubating-src.zip.sha*

No output. With my user hat on, I don't know whether this was successful or
not.

`md5sum` and `sha1sum` might be simpler ways of verifying the hashes, but
this would require the hashes to be generated with these tools as well.

Consider this example:

$ md5sum apache-cloudstack-4.0.0-incubating-src.tar.gz >
apache-cloudstack-4.0.0-incubating-src.tar.gz.md5.example
$ md5sum --check apache-cloudstack-4.0.0-incubating-src.tar.gz.md5.example
apache-cloudstack-4.0.0-incubating-src.tar.gz: OK

*> Extract the source code and verify the contents: [unzip OR tar command]*

This is a problem.

People testing are instructed to only test on artefact. Either we reduce
our artefacts to one instead of two, or people must unpack both and test
both. Or we vote on them both separately. I favour having one artefact.

At this point I made two directories, and ran my tests in parallel on both
artefacts.

I also ran this:

$ diff -r tar/apache-cloudstack-4.0.0-incubating-src
zip/apache-cloudstack-4.0.0-incubating-src

No output, so we're good there. That 6M difference must be compression
efficiency(!)

I also ran these commands:

$ mkdir git
$ mkdir tree
$ git clone https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.gitgit
$ cd git
$ git archive --prefix=../tree/ 5ee613fbf2da5f6212f2f166c980506e0c6cb5cb |
tar Pxf -
$ diff -r tar/apache-cloudstack-4.0.0-incubating-src tree/
$ diff -r zip/apache-cloudstack-4.0.0-incubating-src tree/

No output.

Which means that the artefacts do not differ from the tree-ish provided in
the vote email.

On the one hand this is great! But I find it surprising. I was expecting
there to be a build system that generated files as it prepared the
artefacts. But I guess I am coming from an Autoconf world, where much of it
is bootstrapped by the RM at release time.

Either way, this check should be part of the test procedure. It verifies
the artefact came from the tree-ish in the email.

*> mvn --projects='org.apache.cloudstack:cloudstack'
org.apache.rat:apache-rat-plugin:0.8:check*

This passes.

But it downloads a LOT of jars. So, of course, I checked every license.
*grin*

http://repo1.maven.org/maven2/org/apache/rat/apache-rat-core/0.8/apache-rat-core-0.8.jar(Apache)
http://repo1.maven.org/maven2/commons-collections/commons-collections/3.2/commons-collections-3.2.jar(Apache)
http://repo1.maven.org/maven2/commons-lang/commons-lang/2.1/commons-lang-2.1.jar(Apache)
http://repo1.maven.org/maven2/commons-io/commons-io/1.4/commons-io-1.4.jar(Apache)
http://repo1.maven.org/maven2/org/apache/commons/commons-compress/1.0/commons-compress-1.0.jar(Apache)
http://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar(Apache)
http://repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/1.4/plexus-utils-1.4.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-site-renderer/1.0-alpha-8/doxia-site-renderer-1.0-alpha-8.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-core/1.0-alpha-8/doxia-core-1.0-alpha-8.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-sink-api/1.0-alpha-8/doxia-sink-api-1.0-alpha-8.jar(Apache)
http://repo1.maven.org/maven2/org/codehaus/plexus/plexus-i18n/1.0-beta-6/plexus-i18n-1.0-beta-6.jar(Apache)
http://repo1.maven.org/maven2/org/codehaus/plexus/plexus-velocity/1.1.3/plexus-velocity-1.1.3.jar(Apache)
http://repo1.maven.org/maven2/velocity/velocity/1.4/velocity-1.4.jar(Apache)
http://repo1.maven.org/maven2/velocity/velocity-dep/1.4/velocity-dep-1.4.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-decoration-model/1.0-alpha-8/doxia-decoration-model-1.0-alpha-8.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/reporting/maven-reporting-impl/2.0.4/maven-reporting-impl-2.0.4.jar(Apache)
http://repo1.maven.org/maven2/commons-validator/commons-validator/1.2.0/commons-validator-1.2.0.jar(Apache)
http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar(Apache)
http://repo1.maven.org/maven2/commons-digester/commons-digester/1.6/commons-digester-1.6.jar(Apache)
http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar(Apache)
http://repo1.maven.org/maven2/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar(Apache)
http://repo1.maven.org/maven2/oro/oro/2.0.7/oro-2.0.7.jar (Apache)
http://repo1.maven.org/maven2/org/apache/maven/reporting/maven-reporting-api/2.0.4/maven-reporting-api-2.0.4.jar(Apache)
http://repo1.maven.org/maven2/org/apache/maven/shared/maven-plugin-testing-harness/1.1/maven-plugin-testing-harness-1.1.jar(Apache)
http://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
(Apache)
http://repo1.maven.org/maven2/org/codehaus/plexus/plexus-archiver/1.0-alpha-7/plexus-archiver-1.0-alpha-7.jar(Apache)

All of them, as far as I can tell, are available under the Apache License.

I then ran:

$ cat pom.xml | grep '<exclude>' | sed 's,.*<exclude>,^,' | sed
's,</exclude>,,' | sed 's,\*\*/\*,.*,' | sed 's,\*\*,.*,g' | sed
's,/\*,/.*,' > ../license.skip

$ grep -rL 'http://www.apache.org/licenses/LICENSE-2.0' * | grep -vEf
../license.skip

This produced a lot of files, so I filtered them by removing images.

I got the following list, with comments inline:

> DISCLAIMER

Why is this not picked up by RAT?

> KEYS

This should not be in the repository.

> awsapi/NOTICE.txt
> awsapi/README.txt

These files should probably not exist.

This information should be in the top level NOTICE and LICENSE files.

> debian/changelog
> debian/README

I am guessing RAT recognises these as files not needing a license.

> deps/XenServerJava/LICENSE.txt
> deps/XenServerJava/README.txt

These files should probably not exist.

This information should be in the top level NOTICE and LICENSE files.

> docs/publican-cloudstack/NOTICE

This file should probably not exist.

This information should be in the top level NOTICE and LICENSE files.

> patches/systemvm/debian/config/etc/logrotate.conf
> patches/systemvm/debian/config/etc/logrotate.d/apache2
> patches/systemvm/debian/config/etc/logrotate.d/dnsmasq
> patches/systemvm/debian/config/etc/logrotate.d/haproxy
> patches/systemvm/debian/config/etc/logrotate.d/ppp
> patches/systemvm/debian/config/etc/logrotate.d/rsyslog
> patches/systemvm/debian/config/etc/sysctl.conf
> patches/systemvm/debian/config/opt/cloud/bin/vpc_passwd_server
> patches/systemvm/debian/config.dat

Why are these files not picked up by RAT?

> patches/systemvm/debian/README

What is this doing here?

> patches/systemvm/debian/vpn/etc/ipsec.conf
> patches/systemvm/debian/vpn/etc/ipsec.secrets
> patches/systemvm/debian/vpn/etc/ppp/options.xl2tpd
> patches/systemvm/debian/vpn/etc/xl2tpd/xl2tpd.conf

Why are these files not picked up by RAT?

> test/conf/README
> test/integration/component/README
> test/integration/README
> test/integration/smoke/README
> tools/devcloud/veewee/README

Should we have this many README files?

Moving on to a more manual check of the files...

`waf` seems to be some sort of composite binary file. Can we ship this? My
guess is no.

LICENSE is ISO-8859 encoded. This should be changed to UTF-8 to match our
other text files.

I then ran this file (based on experimentation with `file` over the source
tree):

cat > ../okay_file_types.txt << EOF
ASCII C\+\+ program text
a python script text executable
Bourne-Again shell script text executable
a python script text executable
ASCII English text
exported SGML document text
XML  document text
a bash script text executable
PNG image data
ASCII c program text
SVG Scalable Vector Graphics image
HTML document text
POSIX shell script text executable
PNG image data
GIF image data
Lotus 1-2-3
RFC1421 Security Certificate Signing Request text
a /usr/bin/make -f script text executable
ASCII text
JPEG image data
UTF-8 Unicode
Netpbm PBM image text
a /usr/bin/python script text executable
ISO-8859 English text
a /usr/bin/python -u script text executable
DOS batch file text
a /usr/bin/python -W ignore::Depr script text executable
EOF

And then this:

$ find . -type f | xargs file | grep -vEf ../okay_file_types.txt | sed
's,:.*,,'

Which gave me this output:

./awsapi/resource/AmazonEC2/xes.keystore
./client/tomcatconf/cloudmanagementserver.keystore
./console-proxy/certs/realhostip.keystore
./utils/certs/cloud.keystore

These files are binary. Do we need to ship the source?

Then I ran:

$ grep -rsi "copyright" * | grep -v "regarding copyright ownership" | sed
's,:, ,' | awk '{print $1}' | sort | uniq > ../files_to_check.txt

I manually reconciled this list with the pom.xml file.

> awsapi/src/com/cloud/bridge/io/DimeDelimitedInputStream.java

This file has two license headers in. One of them is incorrect.

> deps/XenServerJava/BSD
> deps/XenServerJava/LICENSE.Apache-2.0.txt
> deps/XenServerJava/LICENSE.txt

These should not be here. They need to be moved to the top level LICENSE.

(And the exclusion in pom.xml needs to be removed.)

> docs/publican-cloudstack/LICENSE
> docs/publican-cloudstack/NOTICE

These files need to be removed, and merged into the top level LICENSE and
NOTICE.

> plugins/network-elements/nicira-nvp/README.NiciraIntegration

Seems like an odd file.

It has our standard header, but also a separate copyright statement. Which
is it?

The file is not mentioned in LICENSE.

> scripts/vm/hypervisor/xenserver/xcposs/NFSSR.py
> scripts/vm/hypervisor/xenserver/xcpserver/NFSSR.py
> scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py
> scripts/vm/hypervisor/xenserver/xenserver56fp1/NFSSR.py
> scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py

These files have our standard boilerplate, but claim they are "(C) 2008
Citrix Systems Inc".

> test/integration/component/test_allocation_states.py

This file claims "Copyright 2012 Citrix Systems, Inc." No mention in
LICENSE.

> tools/whisker/descriptor.xml

This file has our standard boiler plate, but is crammed full of other
licenses. Is this okay?

> ui/lib/excanvas.js

This file claims it is "Copyright 2006 Google Inc.". No mention in LICENSE.

Okay, back to the test procedure instructions...

*> Compile*

Section skipped because I cannot able to set up my build environment
without further instruction.

I am unable to complete the rest of the testing procedure for the same
reason.

One concern is that even though I got the image to boot, I could not access
it via localhost:8080.

That's it for now! :-)

-- 
NS

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message