Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B8D5618ED3 for ; Tue, 16 Jun 2015 12:25:53 +0000 (UTC) Received: (qmail 20579 invoked by uid 500); 16 Jun 2015 12:25:50 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 20516 invoked by uid 500); 16 Jun 2015 12:25:50 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 20504 invoked by uid 99); 16 Jun 2015 12:25:50 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jun 2015 12:25:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 6E30BC0953 for ; Tue, 16 Jun 2015 12:25:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.979 X-Spam-Level: ** X-Spam-Status: No, score=2.979 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id UXV4cDXQfw0t for ; Tue, 16 Jun 2015 12:25:49 +0000 (UTC) Received: from APAC01-SG1-obe.outbound.protection.outlook.com (mail-sg1on0089.outbound.protection.outlook.com [134.170.132.89]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id E4C1322F03 for ; Tue, 16 Jun 2015 12:25:48 +0000 (UTC) Received: from HK2PR0301MB1236.apcprd03.prod.outlook.com (10.162.245.23) by HK2PR0301MB1234.apcprd03.prod.outlook.com (10.162.245.21) with Microsoft SMTP Server (TLS) id 15.1.190.14; Tue, 16 Jun 2015 12:25:21 +0000 Received: from HK2PR0301MB1236.apcprd03.prod.outlook.com ([10.162.245.23]) by HK2PR0301MB1236.apcprd03.prod.outlook.com ([10.162.245.23]) with mapi id 15.01.0195.005; Tue, 16 Jun 2015 12:25:21 +0000 From: Hemanth Meka To: "user@hive.apache.org" Subject: Hive counters for records read and written Thread-Topic: Hive counters for records read and written Thread-Index: AQHQp9Rj2dipRYk8YUmD1qSbi2M+/52vDx9p Date: Tue, 16 Jun 2015 12:25:21 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: hive.apache.org; dkim=none (message not signed) header.d=none; x-originating-ip: [115.119.253.242] x-microsoft-exchange-diagnostics: 1;HK2PR0301MB1234;3:NbRqW7mVqCcFaX9s2fyXvhVzNQkk+B8t6JGcwJIqn8ZAH6nZLZuvL3N4z9+uLS5oEtNX+gKKpE218+sL/9cRl2G6PTPauWm9sR3CWdSV9ubmM6NbvpZUnOFGhJdNOZRYlW3vSZksJ0BVGAwg8bNvrA==;10:KdbaV1QgscYUAODaQCX59rmv8IFeCwf7rfItFlm8XWTc8VWY7UHse9H6LJt9/otESRZwM4utw4WvR+oNyV4d4d4pvJuCTm1TX9mCwGRxLQI=;6:hNAd55kf9qhA53/eqMiprGlMVD1+hCXXTZ7dvx/cU6bL2X2/UIsF9QIu9A408Kcj x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HK2PR0301MB1234; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(520003)(5005006)(3002001);SRVR:HK2PR0301MB1234;BCL:0;PCL:0;RULEID:;SRVR:HK2PR0301MB1234; x-forefront-prvs: 06098A2863 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(106116001)(33656002)(102836002)(62966003)(77156002)(86362001)(122556002)(40100003)(77096005)(19627405001)(76176999)(19625215002)(2501003)(92566002)(110136002)(107886002)(76576001)(5001960100002)(2351001)(5001920100001)(189998001)(229853001)(5003600100002)(50986999)(2950100001)(54356999)(16236675004)(2656002)(87936001)(450100001)(66066001)(46102003)(5002640100001)(2900100001)(74316001);DIR:OUT;SFP:1101;SCL:1;SRVR:HK2PR0301MB1234;H:HK2PR0301MB1236.apcprd03.prod.outlook.com;FPR:;SPF:None;MLV:sfv;LANG:en; Content-Type: multipart/alternative; boundary="_000_HK2PR0301MB123638D0EE1E20C56D77F9CBEDA70HK2PR0301MB1236_" MIME-Version: 1.0 X-OriginatorOrg: datametica.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jun 2015 12:25:21.2402 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 38a39216-dc07-4452-8246-db7086ac9972 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2PR0301MB1234 --_000_HK2PR0301MB123638D0EE1E20C56D77F9CBEDA70HK2PR0301MB1236_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, I can see that two new counters have been added for hive (RECORDS_IN/RECORD= S_OUT) in hive 0.14. Prior to this release which counters could be used to get the records read = by hive job and records written. Because i noticed that in hive 0.14 for a= few hive jobs i see map_input_records but the map_output_records counter i= s 0 but the job actually writes something to output table and the hive log = also gives that count correctly. In this case how else can we get records read and records written in relea= ses before 0.14. Regards Hemanth --_000_HK2PR0301MB123638D0EE1E20C56D77F9CBEDA70HK2PR0301MB1236_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable


Hi,


I can see that two new counters have been added for hive (RECORDS_IN/REC= ORDS_OUT) in hive 0.14. 


Prior to this release which counters could be used to get the records re= ad by hive job and records written.  Because i noticed that in hive 0.= 14 for a few hive jobs i see map_input_records but the map_output_records c= ounter is 0 but the job actually writes something to output table and the hive log also gives that count correctly= . 


In this case how else can we get records read and records written  = in releases before 0.14.


Regards

Hemanth

--_000_HK2PR0301MB123638D0EE1E20C56D77F9CBEDA70HK2PR0301MB1236_--