www-repository mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <steve.lough...@gmail.com>
Subject Re: any issues related to the people.apache.org attack
Date Wed, 02 Sep 2009 17:31:25 GMT
>  The integrity protection of 524191 files requires an entirely
>  different mechanism than checking the integrity of some 500 files.

In hadoop every data server runs a thread continually doing CRC32
validation of blocks; this is how you protect your 4PB filesystem from
errors. Something could be done there
-store JARs in HAR archives to keep the file count low
-run MR jobs to generate the PGP keys
-compare with expected
-reduce: list all JARs that fail.

This only verifies the JARs in the Hadoop filesystem though...

View raw message