From common-issues-return-148118-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Wed Feb 14 16:32:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 63A42180656 for ; Wed, 14 Feb 2018 16:32:04 +0100 (CET) Received: (qmail 6000 invoked by uid 500); 14 Feb 2018 15:32:03 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 5989 invoked by uid 99); 14 Feb 2018 15:32:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Feb 2018 15:32:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id D5311C030C for ; Wed, 14 Feb 2018 15:32:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id JMZf-f1-z30r for ; Wed, 14 Feb 2018 15:32:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 7008A5F180 for ; Wed, 14 Feb 2018 15:32:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id AEDE7E0257 for ; Wed, 14 Feb 2018 15:32:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 42FB1240F6 for ; Wed, 14 Feb 2018 15:32:00 +0000 (UTC) Date: Wed, 14 Feb 2018 15:32:00 +0000 (UTC) From: "Jason Lowe (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-15227) add mapreduce.outputcommitter.factory.scheme.s3a to core-default MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364275#comment-16364275 ] Jason Lowe commented on HADOOP-15227: ------------------------------------- Yeah, mapred-default and mapred-site aren't loaded until the JobConf class is loaded. A common mistake for code is to create a plain Configuration early in {{main}} and try to lookup mapred properties (or even hdfs or yarn properties) expecting to get the default if they are not set by the user. The easy fix is to create a JobConf instead of a Configuration if the code knows it wants to do mapred stuff. > add mapreduce.outputcommitter.factory.scheme.s3a to core-default > ---------------------------------------------------------------- > > Key: HADOOP-15227 > URL: https://issues.apache.org/jira/browse/HADOOP-15227 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.1.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Blocker > > Need to add this property to core-default.xml. It's documented as being there, but it isn't. > {code} > > mapreduce.outputcommitter.factory.scheme.s3a > org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory > > The committer factory to use when writing data to S3A filesystems. > > > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org