From general-return-1746-apmail-hadoop-general-archive=hadoop.apache.org@hadoop.apache.org Mon Jul 05 08:13:44 2010 Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 56485 invoked from network); 5 Jul 2010 08:13:43 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 5 Jul 2010 08:13:43 -0000 Received: (qmail 20149 invoked by uid 500); 5 Jul 2010 08:13:40 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 18659 invoked by uid 500); 5 Jul 2010 08:13:37 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 18651 invoked by uid 99); 5 Jul 2010 08:13:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jul 2010 08:13:35 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=CTYPE_001C_B,HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [145.100.8.50] (HELO sara-exch-fe2.ka.sara.nl) (145.100.8.50) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jul 2010 08:13:26 +0000 Received: from planck.ka.sara.nl (145.100.8.32) by sara-exch-fe2.ka.sara.nl (145.100.8.50) with Microsoft SMTP Server (TLS) id 14.0.694.0; Mon, 5 Jul 2010 10:12:44 +0200 Received: from planck.ka.sara.nl ([145.100.8.32]) by planck.ka.sara.nl ([145.100.8.32]) with mapi; Mon, 5 Jul 2010 10:12:44 +0200 From: Evert Lammerts To: "general@hadoop.apache.org" Date: Mon, 5 Jul 2010 10:12:43 +0200 Subject: Hadoop versions & distributions Thread-Topic: Hadoop versions & distributions Thread-Index: AcscGdXRU3173thcQcCGXH49/En1Yg== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_0000_01CB1C2A.9A605600" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org ------=_NextPart_000_0000_01CB1C2A.9A605600 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0001_01CB1C2A.9A605600" ------=_NextPart_001_0001_01CB1C2A.9A605600 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit There are a number of different versions and distributions of Hadoop which, as far as I understand, all differ from each other. I know that in the 0.20-append branch, files in HDFS can be appended, and that the Y! distribution (0.20.S) implements security features through Kerberos. And then there are the 0.20.3 and 0.22.0 branches. And trunk of course, which I guess is 0.20.2 nowadays? In addition to that there are distributions by Cloudera(CDH2 / 3beta) and IBM (IDAH). >From my perspective, setting up a pilot cluster for a small number of users from different institutes, security (0.20.S) is very attractive - scientists like the idea of shielding their data and logic from other users. But what will I miss if I choose Y!'s distribution over all of these other options? Regards, Evert Lammerts ------=_NextPart_001_0001_01CB1C2A.9A605600 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

There are a number of different versions and = distributions of Hadoop which, as far as I understand, all differ from each other. I know = that in the 0.20-append branch, files in HDFS can be appended, and that the = Y! distribution (0.20.S) implements security features through Kerberos. And then there = are the 0.20.3 and 0.22.0 branches. And trunk of course, which I guess is 0.20.2 = nowadays? In addition to that there are distributions by Cloudera(CDH2 / 3beta) and = IBM (IDAH).

From my perspective, setting up a pilot cluster for = a small number of users from different institutes, security (0.20.S) is very = attractive – scientists like the idea of shielding their data and logic from = other users. But what will I miss if I choose Y!’s distribution over all = of these other options?

Regards,

Evert Lammerts

------=_NextPart_001_0001_01CB1C2A.9A605600-- ------=_NextPart_000_0000_01CB1C2A.9A605600 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIPUDCCBDIw ggMaoAMCAQICAQEwDQYJKoZIhvcNAQEFBQAwezELMAkGA1UEBhMCR0IxGzAZBgNVBAgMEkdyZWF0 ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBwwHU2FsZm9yZDEaMBgGA1UECgwRQ29tb2RvIENBIExpbWl0 ZWQxITAfBgNVBAMMGEFBQSBDZXJ0aWZpY2F0ZSBTZXJ2aWNlczAeFw0wNDAxMDEwMDAwMDBaFw0y ODEyMzEyMzU5NTlaMHsxCzAJBgNVBAYTAkdCMRswGQYDVQQIDBJHcmVhdGVyIE1hbmNoZXN0ZXIx EDAOBgNVBAcMB1NhbGZvcmQxGjAYBgNVBAoMEUNvbW9kbyBDQSBMaW1pdGVkMSEwHwYDVQQDDBhB QUEgQ2VydGlmaWNhdGUgU2VydmljZXMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC+ QJ30buHqdoccTUVEjr5GyIMGncEq/hgfjuQC+vOrXVCKFjELmgbQxXAizUktVGPMtm5oRgtT6stM JMC8ck7q8RWu9FSaEgrDerIzYOLaiVXzIljz3tzP74OGooyUT59o8piQRoQnx3a/48w1LIteB2Rl gsBIsKiR+WGfdiBQqJHHZrXreGIDVvCKGhPqMaMeoJn9OPb2JzJYbwf1a7j7FCuvt6rM1mNfc4za BZmoOKjLF3g2UazpnvR4Oo3PD9lC4pgMqy+fDgHe75+ZSfEt36x0TRuYtUfF5SnR+ZAYx2KcvoPH Jns+iiXHwN2d5jVoECCdj9je0sOEnA1e6C/JAgMBAAGjgcAwgb0wHQYDVR0OBBYEFKARCiM+lvEH 7OKvKe+CpX/QMKS0MA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MHsGA1UdHwR0MHIw OKA2oDSGMmh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0FBQUNlcnRpZmljYXRlU2VydmljZXMuY3Js MDagNKAyhjBodHRwOi8vY3JsLmNvbW9kby5uZXQvQUFBQ2VydGlmaWNhdGVTZXJ2aWNlcy5jcmww DQYJKoZIhvcNAQEFBQADggEBAAhW/ALwm+j/pPrWe8ZEgM5PxMX2AFjMpra8FEloBHbo5u5d7AIP YNaNUBhPJk4B4+awpe6/vHRUQb/9/BK4x09a9IlgBX9gtwVK8/bxwr/EuXSGti19a8zS80bdL8bg asPDNAMsfZbdWsIOpwqZwQWLqwwv81w6z2w3VQmH3lNAbFjv/LarZW4E9hvcPOBaFcae2fFZSDAh ZQNs7Okhc+ybA6HgN62gFRiP+roCzqcsqRATLNTlCCarIpdg+JBedNSimlO98qlo4KJuwtdssaMP nr/raOdW8q7y4ys4OgmBtWuF174t7T8at7Jj4vViLILUagBBUPE5g5+V6TaWmG4wggTdMIIDxaAD AgECAhBxkvvmGV+sTRKFdHE0ohinMA0GCSqGSIb3DQEBBQUAMHsxCzAJBgNVBAYTAkdCMRswGQYD VQQIDBJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcMB1NhbGZvcmQxGjAYBgNVBAoMEUNvbW9k byBDQSBMaW1pdGVkMSEwHwYDVQQDDBhBQUEgQ2VydGlmaWNhdGUgU2VydmljZXMwHhcNMDQwMTAx MDAwMDAwWhcNMjgxMjMxMjM1OTU5WjCBrjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYD VQQHEw5TYWx0IExha2UgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYD VQQLExhodHRwOi8vd3d3LnVzZXJ0cnVzdC5jb20xNjA0BgNVBAMTLVVUTi1VU0VSRmlyc3QtQ2xp ZW50IEF1dGhlbnRpY2F0aW9uIGFuZCBFbWFpbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC ggEBALI5haTyfatBO2JGN67NwWB1vDll+UoaR6K5zEjMapjVTTUZuaRC5c5J4oovHnzSMQfHTrSD ZJ0uKdWiZMSFvYVRNXmkTmiQexx6pJKoF/KYFfKTzMmkMpW7DE8wvZigC4vlbhuiRvp4vKJvq1le pS/Pytptqi/rrKGzaqq3Lmc1i3nhHmmI4uZGzaCl6r4LznY6eg6b6vzaJ1s9cx8i5khhxkzzabGo Lhu21DEgLLyCio6kDqXXiUP8FlqvHXHXEVnauocNr/rz4cLwpMVnjNbWVDreCqS6A3ezZcj9HtN0 YqoYymiTHqGFfvVHZcv4TVcodNI0/zC27vZiMBSMLOsCAwEAAaOCAScwggEjMB8GA1UdIwQYMBaA FKARCiM+lvEH7OKvKe+CpX/QMKS0MB0GA1UdDgQWBBSJgmd9xJ0mcABLtFBIfN49rgRufTAOBgNV HQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUH AwQwEQYDVR0gBAowCDAGBgRVHSAAMHsGA1UdHwR0MHIwOKA2oDSGMmh0dHA6Ly9jcmwuY29tb2Rv Y2EuY29tL0FBQUNlcnRpZmljYXRlU2VydmljZXMuY3JsMDagNKAyhjBodHRwOi8vY3JsLmNvbW9k by5uZXQvQUFBQ2VydGlmaWNhdGVTZXJ2aWNlcy5jcmwwEQYJYIZIAYb4QgEBBAQDAgEGMA0GCSqG SIb3DQEBBQUAA4IBAQCdlcs8uH6lCcQevwvCx3aOOTyUxhCqTwzJ4KuEXYlU4GU7820cfDcsJVRf liH8N4SRnRXcFE+Bz1Qda2xFYMct+ZdRTPlmyjyggoymyPDi6dRK+ew/VsnddozDggFPbADzHhph dARHA6nGQFeRvGUixSdnT1fbZFrZjR+6hi/0Bq6cae3p9M8pF9jgSp8aIC+XTFG7RgfEijdOIOMJ MWjHnsSLneh+EbwyaBCWEZhE2CpRYE2I63Q630MGMsg5Vow6EVLTQaRDA/Tt7zMn2zngFE4mydj1 OeKJuJNdtykmQeqzm66D/Hd1yujKtf7iZUpjPkTE0MNeh3OpmByvfxV/MIIGNTCCBR2gAwIBAgIR APUHLR99vIaqc6YYFoh++L8wDQYJKoZIhvcNAQEFBQAwga4xCzAJBgNVBAYTAlVTMQswCQYDVQQI EwJVVDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0 d29yazEhMB8GA1UECxMYaHR0cDovL3d3dy51c2VydHJ1c3QuY29tMTYwNAYDVQQDEy1VVE4tVVNF UkZpcnN0LUNsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgRW1haWwwHhcNMTAwMjA4MDAwMDAwWhcN MTEwMjA4MjM1OTU5WjCB4DE1MDMGA1UECxMsQ29tb2RvIFRydXN0IE5ldHdvcmsgLSBQRVJTT05B IE5PVCBWQUxJREFURUQxRjBEBgNVBAsTPVRlcm1zIGFuZCBDb25kaXRpb25zIG9mIHVzZTogaHR0 cDovL3d3dy5jb21vZG8ubmV0L3JlcG9zaXRvcnkxHzAdBgNVBAsTFihjKTIwMDMgQ29tb2RvIExp bWl0ZWQxFzAVBgNVBAMTDkV2ZXJ0IExhbW1lcnRzMSUwIwYJKoZIhvcNAQkBFhZldmVydC5sYW1t ZXJ0c0BzYXJhLm5sMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtYAj4sXuBqEwPxYY BMkYLPGrnqT35USIERoWs26xtSY+dLVc7HqaHDZ1lU5m9wFx/Jqph1urlRuAcCj03kGSS23Ta6kt HTRMYRCHI3nOLfEEbH4POT+wDytORUHNeEKp/PrBn1kmKk+dk+bJI7MNdy2XkiEeO+OqKmLiBGkT glIxngOeqgqR2IvmVv2hLAyzq8Ax20inwveZsDMa7QdQLSVUaX1kSRSihwz4Jw9X/K+6oA3ZLGp9 pZYnFHBNTHM2DR/C7dNwQgbZS7TY/Jb1kObYNhwD2JzzJlkoe0blR17MeJkF7/+Y414j0AdPFB7I ggZ4Tm7Maheer9cIQSsvIQIDAQABo4ICGDCCAhQwHwYDVR0jBBgwFoAUiYJnfcSdJnAAS7RQSHze Pa4Ebn0wHQYDVR0OBBYEFNTyUAqBNg9JXuiPHa9jfLvRM3IyMA4GA1UdDwEB/wQEAwIFoDAMBgNV HRMBAf8EAjAAMCAGA1UdJQQZMBcGCCsGAQUFBwMEBgsrBgEEAbIxAQMFAjARBglghkgBhvhCAQEE BAMCBSAwRgYDVR0gBD8wPTA7BgwrBgEEAbIxAQIBAQEwKzApBggrBgEFBQcCARYdaHR0cHM6Ly9z ZWN1cmUuY29tb2RvLm5ldC9DUFMwgaUGA1UdHwSBnTCBmjBMoEqgSIZGaHR0cDovL2NybC5jb21v ZG9jYS5jb20vVVROLVVTRVJGaXJzdC1DbGllbnRBdXRoZW50aWNhdGlvbmFuZEVtYWlsLmNybDBK oEigRoZEaHR0cDovL2NybC5jb21vZG8ubmV0L1VUTi1VU0VSRmlyc3QtQ2xpZW50QXV0aGVudGlj YXRpb25hbmRFbWFpbC5jcmwwbAYIKwYBBQUHAQEEYDBeMDYGCCsGAQUFBzAChipodHRwOi8vY3J0 LmNvbW9kb2NhLmNvbS9VVE5BQUFDbGllbnRDQS5jcnQwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3Nw LmNvbW9kb2NhLmNvbTAhBgNVHREEGjAYgRZldmVydC5sYW1tZXJ0c0BzYXJhLm5sMA0GCSqGSIb3 DQEBBQUAA4IBAQBaJvuiBPgqdq9MEYH9dj2vW8hprAIBEnOOvfXdoP604kBhSbj3s/l0ZQyQY35W YVAltuJQ365uJKigPQHxS+slpUOAJPEVNmwPUIW0GAjxCPxgLdJgAc4jOV8f7oF3pnfI4ukCmjPN LpaylZzMhFt+t6zDWiMtUqyxK/4on62LoLp2D88lxwxI3q3i00bPlV0wZ+mjzS7vrQGcUgDPWEBL FD0I3pR+Z1EH1qjelBmBJV9paVzfzmoU93LIDJA7/MEUXd3JiMZGEoVd2qE5qIZO4fNJ+Cyo6tu1 ECSQxoL6qN+sBwekiFA9Q/zlUp0HfU6Jt8WmtZ44SOh67LcmCL4ZMYIEaDCCBGQCAQEwgcQwga4x CzAJBgNVBAYTAlVTMQswCQYDVQQIEwJVVDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHjAcBgNV BAoTFVRoZSBVU0VSVFJVU1QgTmV0d29yazEhMB8GA1UECxMYaHR0cDovL3d3dy51c2VydHJ1c3Qu Y29tMTYwNAYDVQQDEy1VVE4tVVNFUkZpcnN0LUNsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgRW1h aWwCEQD1By0ffbyGqnOmGBaIfvi/MAkGBSsOAwIaBQCgggJ4MBgGCSqGSIb3DQEJAzELBgkqhkiG 9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEwMDcwNTA4MTI0MVowIwYJKoZIhvcNAQkEMRYEFP54v6RG fT6L5qDh49dCUmQJS01UMGcGCSqGSIb3DQEJDzFaMFgwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMAcGBSsOAwIaMAoGCCqG SIb3DQIFMIHVBgkrBgEEAYI3EAQxgccwgcQwga4xCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJVVDEX MBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0d29yazEh MB8GA1UECxMYaHR0cDovL3d3dy51c2VydHJ1c3QuY29tMTYwNAYDVQQDEy1VVE4tVVNFUkZpcnN0 LUNsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgRW1haWwCEQD1By0ffbyGqnOmGBaIfvi/MIHXBgsq hkiG9w0BCRACCzGBx6CBxDCBrjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYDVQQHEw5T YWx0IExha2UgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYDVQQLExho dHRwOi8vd3d3LnVzZXJ0cnVzdC5jb20xNjA0BgNVBAMTLVVUTi1VU0VSRmlyc3QtQ2xpZW50IEF1 dGhlbnRpY2F0aW9uIGFuZCBFbWFpbAIRAPUHLR99vIaqc6YYFoh++L8wDQYJKoZIhvcNAQEBBQAE ggEAa4jDHbSsjjJ/eKiYH6OVmsDDgcEmZy/3wOqotzW74nEsBCM+lcw+01KIyNg5opVZ28ppx/7T +jJZ3gfg8U1sDPa8lc4MCku0NxGWdCYb1qoVy2j2p69fbgdKZYpJID6GWlkNqLKcsO1MAcIWcZUh pAptmpUZyk6Jr5qYbYakrL7edj72m9GZOWZW5g5PpUX3f1al+D7c3oRQxM7Bfv0TKGnZPQrb8+ZA +LRNk+p4te2v7wmqCDgemzEJrsMUTi1jLWgXsOY1SEoYfeyP947VBIj6gRgMxcQrSoEniDWandLD KgBg16aze9jVsePQ+fjZiAj2Wa6YRABG6Zp7822BoAAAAAAAAA== ------=_NextPart_000_0000_01CB1C2A.9A605600--