Return-Path: Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: (qmail 21535 invoked from network); 22 Feb 2011 15:59:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 22 Feb 2011 15:59:05 -0000 Received: (qmail 10531 invoked by uid 500); 22 Feb 2011 15:59:05 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 10138 invoked by uid 500); 22 Feb 2011 15:59:01 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Delivered-To: moderator for mapreduce-dev@hadoop.apache.org Received: (qmail 90721 invoked by uid 99); 22 Feb 2011 15:46:49 -0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=CTYPE_001C_B,HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) x-SBRS: None X-REMOTE-IP: 10.12.10.51 X-IronPort-AV: E=Sophos;i="4.62,207,1297054800"; d="p7s'?scan'208,217";a="272754729" From: "Cummins, Robert [USA]" To: "mapreduce-dev@hadoop.apache.org" Date: Tue, 22 Feb 2011 10:45:53 -0500 Subject: mapreduce passed parameter value ordering Thread-Topic: mapreduce passed parameter value ordering Thread-Index: AcvSBFlKYzigZBfgQ3epEmadjen20g== Message-ID: <3C5A6E2B4813AD4C9A63F513883AC4CA154B1AD1CC@ASHBMBX01.resource.ds.bah.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_0000_01CBD1DA.73089890" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org ------=_NextPart_000_0000_01CBD1DA.73089890 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0001_01CBD1DA.73089890" ------=_NextPart_001_0001_01CBD1DA.73089890 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Is the following acceptable or is an enhancement/bug fix needed? For mapper class: class Mapper1 extends TableMapper With reducer class: class Reducer1 extends TableReducer Iterable values are usually received by the reducer in the order the values are written to the context by the mapper. However in my testing about 5% of cases, the same order is not maintained, and the ability of the reducer to categorize a value by order lost. Suggestion: It would be a helpful and simple feature to have chronological order guaranteed as a facility for identification by the reducer. ------=_NextPart_001_0001_01CBD1DA.73089890 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Is the following = acceptable or is an enhancement/bug fix needed?

For mapper = class:

class Mapper1 extends = TableMapper<ImmutableBytesWritable, IntWritable> =

With reducer = class:

class Reducer1 extends = TableReducer<ImmutableBytesWritable, IntWritable, = ImmutableBytesWritable>

= Iterable<IntWritable> values are usually received by the reducer = in the order the values are written to the context by the mapper. = However in my testing about 5% of cases, the same order is not = maintained, and the ability of the reducer to categorize a value by = order lost.

Suggestion: It = would be a helpful and simple feature to have chronological order = guaranteed as a facility for identification by the = reducer.

------=_NextPart_001_0001_01CBD1DA.73089890-- ------=_NextPart_000_0000_01CBD1DA.73089890 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIPCzCCA14w ggJGoAMCAQICAj7tMA0GCSqGSIb3DQEBBQUAMFMxCzAJBgNVBAYTAlVTMQwwCgYDVQQKEwNCQUgx EzARBgNVBAsTCkNvbXBvbmVudHMxITAfBgNVBAMTGEJvb3ogQWxsZW4gSGFtaWx0b24gQ0EgMjAe Fw0xMDEyMTUxNDMyMjBaFw0xMzA2MTUxNDMyMjBaME0xCzAJBgNVBAYTAlVTMQwwCgYDVQQKEwNC QUgxDTALBgNVBAsTBFNNVFAxITAfBgNVBAMTGEN1bW1pbnMgSnIgUm9iZXJ0IDU0MTk3NDCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyzPfaNILX+OGznj+RFt5K0N/8boFwy4WOssceRDU75Pf ZpEbAbiKdZCRo5aRnKO0FxlMFaTdl7s91Vltt4WhBi0N3yi21gNu087/1nik6p66OQBfqI2FRtvD h8TrJWoo3ImCKPrXrq05EBFJrpoaQcQ1vtMys6m6OIZEUT+te5UCAwEAAaOBxTCBwjAOBgNVHQ8B Af8EBAMCBaAwHQYDVR0OBBYEFJ2UpHtlaWvPiWTc8oscHxUIHcjOMBgGA1UdIAQRMA8wDQYLKwYB BAHeAAEDAgEwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL21zcGtpLmJhaC5jb20vQ1JML2JhaENB LmNybDAhBgNVHREEGjAYgRZjdW1taW5zX3JvYmVydEBiYWguY29tMB8GA1UdIwQYMBaAFGJj1O58 C8RdxsCuFDBd/xyc5kjUMA0GCSqGSIb3DQEBBQUAA4IBAQALJm3ttzmj4LRjD/coO+n41tC1PCPk aJCFsFQiFGhdntoxPZ2wJzzYlHSLBpb9APG2eNC7WCStBbsdW6kK3Mr4CewC+9oaKYsHJ+t57PX8 fZH3P2HOFzcm+DdKNsG2w0DhmhVwqWzaUKGWmbYks9grZzJ9Pg8OTDogcICk4hpumzQ1RZ0WAnT0 TFiH/4jKT/kD39P1yfEwQ7DiSEtFq+BBcAcsHF7RsMrKfAf1hSC0YhW09pYU51MQf2c97cNaSpQg BwuQdctxtEFPf5D+x/lbpxgO2vLq0CK72ogAus2z3dml8RNbmD+lfpKT7SgvH6cPb90dS6KEcPV4 igD4/HkAMIIDdTCCAl2gAwIBAgILAgAAAAAA1ni3lAUwDQYJKoZIhvcNAQEEBQAwVzELMAkGA1UE BhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExEDAOBgNVBAsTB1Jvb3QgQ0ExGzAZBgNV BAMTEkdsb2JhbFNpZ24gUm9vdCBDQTAeFw05ODA5MDExMjAwMDBaFw0xNDAxMjgxMjAwMDBaMFcx CzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENB MRswGQYDVQQDExJHbG9iYWxTaWduIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQDaDuaZjc6j40+Kfvvxi4Mla+pIH/EqsLmVEQS98GPR4mdmzxzdzxtIK+6NiY6arymAZavp xy0Sy6scTHAHoT0KMM0VjU/43dSMUBUc71DuxC73/OlS8pF94G3VNTCOXkNz8kHp1Wrjsok6Vjk4 bwY8iGlbKk3Fp1S4bInMm/k8yuX9ifUSPJJ4ltbcdG6TRGHRjcdGsnUOhugZitVtbNV4FpWi6cgK OOvyJBNPc1STE4U6G7weNLWLBYy5d4ux2x8gkasJU26Qzns3dLlwR5EiUWMWea6xrkEmCMgZK9FG qkjWZCrXgzT/LCrBbBlDSgeF59N89iFo7+ryUp9/k5DPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIA BjAdBgNVHQ4EFgQUYHtmGkUNl8qJUC99BM00qP/8/UswDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG 9w0BAQQFAAOCAQEArqqf/LfSyx9fOSkoGJ40yWxPbxrwZKJwSk8ThptgKJ7ogUmYfQq75bCdPTbb jwVR/wkxKh/diXeeDy5slQTthsu0AD+EAk2AaioteAuubyuig0SDH81Qgkwkr733pbTIWg/050de SY43lv6aiAU62cDbKYfmGZZHpzqmjIs8d/5GY6dT2iHRrH5Jokvmw2dZL7OKDrssvamqQnw1wdh/ 1acxOk5jQzmvCLBhNIzTmKlDNPYPhyk7ncJWWJh3w/cbrPad+D6qp1RF8PX51TFl/mtYnHGzHtdS 6jIX/EBgHcl5JLL2bP2oZg6C3ZjL2sJETy6ge/L3ayx2EYRGinij4zCCA+cwggLPoAMCAQICCwQA AAAAARVLWsWnMA0GCSqGSIb3DQEBBQUAMFcxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxT aWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYDVQQDExJHbG9iYWxTaWduIFJvb3QgQ0Ew HhcNMDMxMjE2MTMwMDAwWhcNMjgwMTI4MTEwMDAwWjBxMSgwJgYDVQQDEx9HbG9iYWxTaWduIFJv b3RTaWduIFBhcnRuZXJzIENBMR0wGwYDVQQLExRSb290U2lnbiBQYXJ0bmVycyBDQTEZMBcGA1UE ChMQR2xvYmFsU2lnbiBudi1zYTELMAkGA1UEBhMCQkUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw ggEKAoIBAQC97zDxMPE0qYlld01Gp42Q/a5PjsooF7pZ46iSCkUDKoqP5QlQVVKB8KORsdkSKoH2 wgMcPILAcs3xpwDX9VScCkfumpVBko6grQk90+uidK2fGSAJtn2mXjWfTzlqA7WKrR+WYmsXuauH YNVdbdmSydATrtSI2VCoRJEEsOpH6l+y7QTB1wF8IfjEcSP8a0xlRDPDjR3m0mYcUilGxAbnCzXw WQFmAInPnON7eKpT4u6sNZXn/V3XQpSV0xpuMVVH1+utx0yfVHGDGhfI+efOWAH0Nr+uP1mfZXxA B1xzIDSiEsNJ9GhAaR6J4IXpOreXY7tHsDlrQQB+9Uu4f+MhAgMBAAGjgZkwgZYwDgYDVR0PAQH/ BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFFaE7LVxpedj2NtRBNb65vBIUknOMDMG A1UdHwQsMCowKKAmoCSGImh0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5uZXQvUm9vdC5jcmwwHwYDVR0j BBgwFoAUYHtmGkUNl8qJUC99BM00qP/8/UswDQYJKoZIhvcNAQEFBQADggEBAM3jdJbcl2yg+adk EQSrLfYrGT5f0cW4sOSI6/ViwwnyV82u29qNpFDsZCfYr8/wRPtAyXNYuRtNmmX+W1avwpbRZick u5U6EPnDt45j4QNfvhS3x1M/wtff+jYrG2DzaOO9d8dAg3KqToVruzPAnHRe4LtyU+ukWuJ6qFj2 SboFve4AZvHE4REw7p7mJ3eiNnwNxaDXTIbNQnYlZDstcRSiGUv2SiFdefUJsWvMYj+rk6ydvLeJ aka9BMH+gxac7g0pt7ekZuDrzCK3xNLg2iQ5eROT8eHUzIWw+pEaFhTKY/s+t44RcQT8Bdg2wSr+ VAJS7jlgkMJp42BGugMicwswggRBMIIDKaADAgECAgsEAAAAAAEeRIePPzANBgkqhkiG9w0BAQUF ADBxMSgwJgYDVQQDEx9HbG9iYWxTaWduIFJvb3RTaWduIFBhcnRuZXJzIENBMR0wGwYDVQQLExRS b290U2lnbiBQYXJ0bmVycyBDQTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTELMAkGA1UEBhMC QkUwHhcNMDgxMjE3MTIwMDAwWhcNMTgxMjE3MTIwMDAwWjBTMQswCQYDVQQGEwJVUzEMMAoGA1UE ChMDQkFIMRMwEQYDVQQLEwpDb21wb25lbnRzMSEwHwYDVQQDExhCb296IEFsbGVuIEhhbWlsdG9u IENBIDIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDWuHZcK1yegNrDgj6H2UQgpqfI Nbuy992wl+B/ISej95VTjwNAcfU8ImMBCuRSzhM1PvAkY1ZiU+cbRUhTly0QFOUs509Vg6RX86vs RAoJQAK03NxemowetuVX2/pvahkbJMhYg9A7crK/5lqBW/0daZg9hDZU2y6rFIrFdVvLjK6ylhzi OFU0glpbqIsZO+mGqBxZ38K59fes1+pgG0rKbEnC3seGy9j4Vc0FAc2oxc5eZ6KHQqonrmTAptYr Eg3djhh6BiYyDdOwsl2eaYpVXJddUT48r+Pq1XLavbgQYI65j0S/V77C3CxXpJJYe1feUYcW+ZxP TAdNfKAGVsPJAgMBAAGjgfcwgfQwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQEw HQYDVR0OBBYEFGJj1O58C8RdxsCuFDBd/xyc5kjUME0GA1UdIARGMEQwQgYJKwYBBAGgMgE8MDUw MwYIKwYBBQUHAgIwJxolaHR0cDovL3d3dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzA/BgNV HR8EODA2MDSgMqAwhi5odHRwOi8vY3JsLmdsb2JhbHNpZ24ubmV0L1Jvb3RTaWduUGFydG5lcnMu Y3JsMB8GA1UdIwQYMBaAFFaE7LVxpedj2NtRBNb65vBIUknOMA0GCSqGSIb3DQEBBQUAA4IBAQAz XP/7q/WKpRw59lKucyopiDDld2SW3uWiCozLQWPWxfwA0yJSmZRiHWDYARthLWK55sRb3m+NhSiv auXYUBZOrpCqDb5VqD/fxU8wJS1ol90vXUsB43to4S7pso76ABwbzVha2snHDQgzyB1+E1TrMiaz Zn/Xwgzk8vZhcfChYB1GhQKCDyij5A3dqC6UP2gDSufKD5KCS/T/vBNtgm+oVId22WkHpUiGxgob xEVVS6W+rgXQm15rTGPMSimsWuMzRwTNx1tNlx/fn0IrUX2BTM6QsUIhnCHP0x3GBo6oXmF4W4VA R4O4uV9R6IL7w2QzorhjE8PtSxYChxPBBrvkMYICnzCCApsCAQEwWTBTMQswCQYDVQQGEwJVUzEM MAoGA1UEChMDQkFIMRMwEQYDVQQLEwpDb21wb25lbnRzMSEwHwYDVQQDExhCb296IEFsbGVuIEhh bWlsdG9uIENBIDICAj7tMAkGBSsOAwIaBQCgggGcMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTExMDIyMTIwMTcyN1owIwYJKoZIhvcNAQkEMRYEFD2ELpGJn89P2jy1 9e33osp6kLQ5MGcGCSqGSIb3DQEJDzFaMFgwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0G CCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMAcGBSsOAwIaMAoGCCqGSIb3DQIF MGgGCSsGAQQBgjcQBDFbMFkwUzELMAkGA1UEBhMCVVMxDDAKBgNVBAoTA0JBSDETMBEGA1UECxMK Q29tcG9uZW50czEhMB8GA1UEAxMYQm9veiBBbGxlbiBIYW1pbHRvbiBDQSAyAgI+7TBqBgsqhkiG 9w0BCRACCzFboFkwUzELMAkGA1UEBhMCVVMxDDAKBgNVBAoTA0JBSDETMBEGA1UECxMKQ29tcG9u ZW50czEhMB8GA1UEAxMYQm9veiBBbGxlbiBIYW1pbHRvbiBDQSAyAgI+7TANBgkqhkiG9w0BAQEF AASBgAMIgTYgW88zP08O7rG4iQpc03YnoXTRFzrqREfdsHEAIKGWmv+a8GDZQEmogmf2o0k5KXMs 6hZ03SkE0H7m4qIU4PQApAO0GbMuVusWwi/jWHnFlqSUOgFRdEQbCAWIj7FIP8XxsJoZVRbz3Bhb +vRyM3GvxMgaMSeaMsV1VQ3RAAAAAAAA ------=_NextPart_000_0000_01CBD1DA.73089890--