Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A3D1B18ABF for ; Wed, 17 Jun 2015 06:00:12 +0000 (UTC) Received: (qmail 43535 invoked by uid 500); 17 Jun 2015 06:00:12 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 43485 invoked by uid 500); 17 Jun 2015 06:00:12 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 43473 invoked by uid 99); 17 Jun 2015 06:00:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Jun 2015 06:00:12 +0000 Date: Wed, 17 Jun 2015 06:00:12 +0000 (UTC) From: "Joep Rottinghuis (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-3706) Generalize native HBase writer for additional tables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589354#comment-14589354 ] Joep Rottinghuis commented on YARN-3706: ---------------------------------------- Fixed code, now have successful run: {noformat} 15/06/16 22:57:15 INFO mapreduce.Job: Counters: 23 File System Counters FILE: Number of bytes read=1651635 FILE: Number of bytes written=1927484 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=0 HDFS: Number of bytes written=0 HDFS: Number of read operations=0 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Map-Reduce Framework Map input records=1 Map output records=0 Input split bytes=48 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=18 Total committed heap usage (bytes)=325058560 org.apache.hadoop.mapred.TimelineServicePerformanceV2$PerfCounters TIMELINE_SERVICE_WRITE_COUNTER=230 TIMELINE_SERVICE_WRITE_KBS=230 TIMELINE_SERVICE_WRITE_TIME=66 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=0 TRANSACTION RATE (per mapper): 3484.848484848485 ops/s IO RATE (per mapper): 3484.848484848485 KB/s TRANSACTION RATE (total): 3484.848484848485 ops/s IO RATE (total): 3484.848484848485 KB/s {noformat} and an individual history file: {noformat} 15/06/16 22:58:06 INFO mapreduce.Job: Job job_local1358267884_0001 completed successfully 15/06/16 22:58:06 INFO mapreduce.Job: Counters: 22 File System Counters FILE: Number of bytes read=1651635 FILE: Number of bytes written=1927113 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=141020 HDFS: Number of bytes written=0 HDFS: Number of read operations=3 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Map-Reduce Framework Map input records=1 Map output records=0 Input split bytes=48 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=63 Total committed heap usage (bytes)=460324864 org.apache.hadoop.mapred.TimelineServicePerformanceV2$PerfCounters TIMELINE_SERVICE_WRITE_COUNTER=25 TIMELINE_SERVICE_WRITE_TIME=145 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=0 TRANSACTION RATE (per mapper): 172.41379310344828 ops/s IO RATE (per mapper): 0.0 KB/s TRANSACTION RATE (total): 172.41379310344828 ops/s IO RATE (total): 0.0 KB/s {noformat} Will make change suggested by [~zjshen], run another test run and upload a new patch. > Generalize native HBase writer for additional tables > ---------------------------------------------------- > > Key: YARN-3706 > URL: https://issues.apache.org/jira/browse/YARN-3706 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver > Reporter: Joep Rottinghuis > Assignee: Joep Rottinghuis > Priority: Minor > Attachments: YARN-3706-YARN-2928.001.patch, YARN-3706-YARN-2928.010.patch, YARN-3706-YARN-2928.011.patch, YARN-3706-YARN-2928.012.patch, YARN-3706-YARN-2928.013.patch, YARN-3706-YARN-2928.014.patch, YARN-3726-YARN-2928.002.patch, YARN-3726-YARN-2928.003.patch, YARN-3726-YARN-2928.004.patch, YARN-3726-YARN-2928.005.patch, YARN-3726-YARN-2928.006.patch, YARN-3726-YARN-2928.007.patch, YARN-3726-YARN-2928.008.patch, YARN-3726-YARN-2928.009.patch > > > When reviewing YARN-3411 we noticed that we could change the class hierarchy a little in order to accommodate additional tables easily. > In order to get ready for benchmark testing we left the original layout in place, as performance would not be impacted by the code hierarchy. > Here is a separate jira to address the hierarchy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)