hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: follow up Hadoop mavenization work
Date Tue, 02 Aug 2011 10:23:43 GMT
On 30/07/11 01:09, Rottinghuis, Joep wrote:
> Thanks for the replies.
> To elaborate on why I want to build on a server w/o Internet access:
> Build should not reach out to Internet and grab jars from unverified sources w/o md5
hash check etc.

automated hash checking is flawed for various reasons
  -older versions of M2 didn't do the check; you have to build with 
--strict-checksums to force that check in
  -some artifacts have crept into the repository with bad checksums (see 
below), which Ivy finds, as it does checksum
  -verifying checksums from the same HTTP server that served up the file 
doesn't prevent malicious attacks. Verifying against an HTTPS server 
managed by the ASF would

> The resulting code will run on a large production cluster with sensitive/private data.
From a compliance and risk perspective I want to be able to control which jars get pulled
in from where.
> Manual verification of ~/.m2, tar.gz and scp to build server is an acceptable workaround.

The way to verify the artifacts are valid is to through the release 
notes of every artifact you depend on, check the (signed) release notes 
of them and that the checksum you've got on the downloaded artifact 

Even then you are vulnerable to "the bad POM attack": POM checksums 
aren't included in release notes, so someone could put a POM up there 
that declares a dependency on a non-ASF artifact containing malicious 
code. Unless you know the exact dependency tree of your entire 
application, you are vulnerable here.


Internally I keep under SCM all our dependencies, set up Ivy to build 
offline only with a strict conflict manager, which halts the build if 
there are inconsistent versions, then tune the ivy.xml files to exclude 
the old versions. I do verify the checksums of ASF releases, and examine 
the dependency graph to see if there's anything in there I don't 
recognise, though I don't decompile every JAR for review.

---------- Forwarded message ----------
From: Steve Loughran <steve.loughran@gmail.com>
Date: 10 September 2010 13:09
Subject: bad checksums in activemq-protobuf-1.1.pom
To: repository@apache.org


says 255bd0c7703022d85da7416f87802a11053de120

but shasum activemq-protobuf-1.1.pom
c92f02aa8a96139ff4274e8c80701bb8f4bd7c1e  activemq-protobuf-1.1.pom

View raw message