Return-Path: Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: (qmail 28571 invoked from network); 2 Jul 2010 18:18:00 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Jul 2010 18:18:00 -0000 Received: (qmail 8340 invoked by uid 500); 2 Jul 2010 18:18:00 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 8320 invoked by uid 500); 2 Jul 2010 18:17:59 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 8313 invoked by uid 99); 2 Jul 2010 18:17:59 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jul 2010 18:17:59 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jul 2010 18:17:54 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id CCCAF23889B2; Fri, 2 Jul 2010 18:16:30 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r960079 - in /hadoop/common/trunk: CHANGES.txt src/java/core-default.xml src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java Date: Fri, 02 Jul 2010 18:16:30 -0000 To: common-commits@hadoop.apache.org From: shv@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20100702181630.CCCAF23889B2@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: shv Date: Fri Jul 2 18:16:30 2010 New Revision: 960079 URL: http://svn.apache.org/viewvc?rev=960079&view=rev Log: HADOOP-6756. Documentation for common configuration keys. Contributed by Erik Steffl. Added: hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java (with props) Modified: hadoop/common/trunk/CHANGES.txt hadoop/common/trunk/src/java/core-default.xml hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java Modified: hadoop/common/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/trunk/CHANGES.txt?rev=960079&r1=960078&r2=960079&view=diff ============================================================================== --- hadoop/common/trunk/CHANGES.txt (original) +++ hadoop/common/trunk/CHANGES.txt Fri Jul 2 18:16:30 2010 @@ -2,15 +2,19 @@ Hadoop Change Log Trunk (unreleased changes) + INCOMPATIBLE CHANGES + NEW FEATURES - HADOOP-6791. Refresh for proxy superuser config + + HADOOP-6791. Refresh for proxy superuser config (common part for HDFS-1096) (boryas) - HADOOP-6581. Add authenticated TokenIdentifiers to UGI so that - they can be used for authorization (Kan Zhang and Jitendra Pandey - via jghoman) + HADOOP-6581. Add authenticated TokenIdentifiers to UGI so that + they can be used for authorization (Kan Zhang and Jitendra Pandey + via jghoman) IMPROVEMENTS + HADOOP-6644. util.Shell getGROUPS_FOR_USER_COMMAND method name - should use common naming convention (boryas) @@ -47,7 +51,13 @@ Trunk (unreleased changes) HADOOP-6814. Adds an API in UserGroupInformation to get the real authentication method of a passed UGI. (Jitendra Pandey via ddas) + HADOOP-6756. Documentation for common configuration keys. + (Erik Steffl via shv) + + OPTIMIZATIONS + BUG FIXES + HADOOP-6638. try to relogin in a case of failed RPC connection (expired tgt) only in case the subject is loginUser or proxyUgi.realUser. (boryas) @@ -92,7 +102,7 @@ Trunk (unreleased changes) (ddas) HADOOP-6815. refreshSuperUserGroupsConfiguration should use server side -configuration for the refresh (boryas) + configuration for the refresh (boryas) Release 0.21.0 - Unreleased Modified: hadoop/common/trunk/src/java/core-default.xml URL: http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/core-default.xml?rev=960079&r1=960078&r2=960079&view=diff ============================================================================== --- hadoop/common/trunk/src/java/core-default.xml (original) +++ hadoop/common/trunk/src/java/core-default.xml Fri Jul 2 18:16:30 2010 @@ -53,6 +53,8 @@ ordering of the filters. + + hadoop.security.authorization false @@ -67,6 +69,35 @@ + hadoop.security.group.mapping + org.apache.hadoop.security.ShellBasedUnixGroupsMapping + + Class for user to group mapping (get groups for a given user) for ACL + + + + + hadoop.security.groups.cache.secs + 300 + + This is the config controlling the validity of the entries in the cache + containing the user->group mapping. When this duration has expired, + then the implementation of the group mapping provider is invoked to get + the groups of the user and then cached back. + + + + + hadoop.security.service.user.name.key + + + For those cases where the same RPC protocol is implemented by multiple + servers, this configuration is required for specifying the principal + name to use for the service when the client wishes to make an RPC call. + + + + hadoop.rpc.protection authentication This field sets the quality of protection for secured sasl @@ -148,6 +179,19 @@ facilitate opening large MapFiles using less memory. + + io.map.index.interval + 128 + + MapFile consist of two files - data file (tuples) and index file + (keys). For every io.map.index.interval records written in the + data file, an entry (record-key, data-file-position) is written + in the index file. This is to allow for doing binary search later + within the index file to look up records by their keys and get their + closest positions in the data file. + + + @@ -240,6 +284,20 @@ + fs.ftp.host + 0.0.0.0 + FTP filesystem connects to this server + + + + fs.ftp.host.port + 21 + + FTP filesystem connects to fs.ftp.host on this port + + + + fs.ramfs.impl org.apache.hadoop.fs.InMemoryFileSystem The FileSystem for ramfs: uris. @@ -717,5 +775,33 @@ Replication factor + + + + tfile.io.chunk.size + 1048576 + + Value chunk size in bytes. Default to + 1MB. Values of the length less than the chunk size is + guaranteed to have known value length in read time (See also + TFile.Reader.Scanner.Entry.isValueLengthKnown()). + + + + + tfile.fs.output.buffer.size + 262144 + + Buffer size used for FSDataOutputStream in bytes. + + + + + tfile.fs.input.buffer.size + 262144 + + Buffer size used for FSDataInputStream in bytes. + + Modified: hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java URL: http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java?rev=960079&r1=960078&r2=960079&view=diff ============================================================================== --- hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java (original) +++ hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeys.java Fri Jul 2 18:16:30 2010 @@ -25,135 +25,65 @@ import org.apache.hadoop.classification. * This class contains constants for configuration keys used * in the common code. * + * It inherits all the publicly documented configuration keys + * and adds unsupported keys. + * */ + @InterfaceAudience.Private @InterfaceStability.Unstable -public class CommonConfigurationKeys { - - // The Keys - public static final String IO_NATIVE_LIB_AVAILABLE_KEY = - "io.native.lib.available"; - public static final boolean IO_NATIVE_LIB_AVAILABLE_DEFAULT = true; - public static final String NET_TOPOLOGY_SCRIPT_NUMBER_ARGS_KEY = - "net.topology.script.number.args"; - public static final int NET_TOPOLOGY_SCRIPT_NUMBER_ARGS_DEFAULT = 100; +public class CommonConfigurationKeys extends CommonConfigurationKeysPublic { - //FS keys + /** Default location for user home directories */ public static final String FS_HOME_DIR_KEY = "fs.homeDir"; + /** Default value for FS_HOME_DIR_KEY */ public static final String FS_HOME_DIR_DEFAULT = "/user"; - public static final String FS_DEFAULT_NAME_KEY = "fs.defaultFS"; - public static final String FS_DEFAULT_NAME_DEFAULT = "file:///"; - public static final String FS_PERMISSIONS_UMASK_KEY = "fs.permissions.umask-mode"; + /** Default umask for files created in HDFS */ + public static final String FS_PERMISSIONS_UMASK_KEY = + "fs.permissions.umask-mode"; + /** Default value for FS_PERMISSIONS_UMASK_KEY */ public static final int FS_PERMISSIONS_UMASK_DEFAULT = 0022; - public static final String FS_DF_INTERVAL_KEY = "fs.df.interval"; - public static final long FS_DF_INTERVAL_DEFAULT = 60000; - - - //Defaults are not specified for following keys - public static final String NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY = - "net.topology.script.file.name"; - public static final String NET_TOPOLOGY_CONFIGURED_NODE_MAPPING_KEY = - "net.topology.configured.node.mapping"; - public static final String NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY = - "net.topology.node.switch.mapping.impl"; - + /** @deprecated not used, jira was created to remove this constant: + * https://issues.apache.org/jira/browse/HADOOP-6802 + */ public static final String FS_CLIENT_BUFFER_DIR_KEY = - "fs.client.buffer.dir"; - public static final String FS_TRASH_CHECKPOINT_INTERVAL_KEY = "fs.trash.checkpoint.interval"; - public static final long FS_TRASH_CHECKPOINT_INTERVAL_DEFAULT = 0; - - //TBD: Code is not updated to use following keys. - //These keys will be used in later versions - // - public static final long FS_LOCAL_BLOCK_SIZE_DEFAULT = 32*1024*1024; - public static final String FS_AUTOMATIC_CLOSE_KEY = "fs.automatic.close"; - public static final boolean FS_AUTOMATIC_CLOSE_DEFAULT = true; - public static final String FS_FILE_IMPL_KEY = "fs.file.impl"; - public static final String FS_FTP_HOST_KEY = "fs.ftp.host"; - public static final String FS_FTP_HOST_PORT_KEY = "fs.ftp.host.port"; - public static final String FS_TRASH_INTERVAL_KEY = "fs.trash.interval"; - public static final long FS_TRASH_INTERVAL_DEFAULT = 0; - - public static final String IO_MAPFILE_BLOOM_SIZE_KEY = "io.mapfile.bloom.size"; - public static final int IO_MAPFILE_BLOOM_SIZE_DEFAULT = 1024*1024; - public static final String IO_MAPFILE_BLOOM_ERROR_RATE_KEY = - "io.mapfile.bloom.error.rate" ; - public static final float IO_MAPFILE_BLOOM_ERROR_RATE_DEFAULT = 0.005f; - public static final String IO_COMPRESSION_CODEC_LZO_CLASS_KEY = "io.compression.codec.lzo.class"; - public static final String IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_KEY = - "io.compression.codec.lzo.buffersize"; - public static final int IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_DEFAULT = 64*1024; - public static final String IO_MAP_INDEX_INTERVAL_KEY = "io.map.index.interval"; - public static final int IO_MAP_INDEX_INTERVAL_DEFAULT = 128; - public static final String IO_MAP_INDEX_SKIP_KEY = "io.map.index.skip"; - public static final int IO_MAP_INDEX_SKIP_DEFAULT = 0; - public static final String IO_SEQFILE_COMPRESS_BLOCKSIZE_KEY = "io.seqfile.compress.blocksize"; - public static final int IO_SEQFILE_COMPRESS_BLOCKSIZE_DEFAULT = 1000000; - public static final String IO_SKIP_CHECKSUM_ERRORS_KEY = "io.skip.checksum.errors"; - public static final boolean IO_SKIP_CHECKSUM_ERRORS_DEFAULT = false; - public static final String IO_SORT_MB_KEY = "io.sort.mb"; - public static final int IO_SORT_MB_DEFAULT = 100; - public static final String IO_SORT_FACTOR_KEY = "io.sort.factor"; - public static final int IO_SORT_FACTOR_DEFAULT = 100; - public static final String IO_SERIALIZATIONS_KEY = "io.serializations"; - - public static final String TFILE_IO_CHUNK_SIZE_KEY = "tfile.io.chunk.size"; - public static final int TFILE_IO_CHUNK_SIZE_DEFAULT = 1024*1024; - public static final String TFILE_FS_INPUT_BUFFER_SIZE_KEY = "tfile.fs.input.buffer.size"; - public static final int TFILE_FS_INPUT_BUFFER_SIZE_DEFAULT = 256*1024; - public static final String TFILE_FS_OUTPUT_BUFFER_SIZE_KEY = "tfile.fs.output.buffer.size"; - public static final int TFILE_FS_OUTPUT_BUFFER_SIZE_DEFAULT = 256*1024; - + "fs.client.buffer.dir"; + /** How often does RPC client send pings to RPC server */ public static final String IPC_PING_INTERVAL_KEY = "ipc.ping.interval"; + /** Default value for IPC_PING_INTERVAL_KEY */ public static final int IPC_PING_INTERVAL_DEFAULT = 60000; + /** Enables pings from RPC client to the server */ public static final String IPC_CLIENT_PING_KEY = "ipc.client.ping"; + /** Default value of IPC_CLIENT_PING_KEY */ public static final boolean IPC_CLIENT_PING_DEFAULT = true; - public static final String IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY = - "ipc.client.connection.maxidletime"; - public static final int IPC_CLIENT_CONNECTION_MAXIDLETIME_DEFAULT = 10000; - public static final String IPC_CLIENT_CONNECT_MAX_RETRIES_KEY = - "ipc.client.connect.max.retries"; - public static final int IPC_CLIENT_CONNECT_MAX_RETRIES_DEFAULT = 10; - public static final String IPC_CLIENT_TCPNODELAY_KEY = "ipc.client.tcpnodelay"; - public static final boolean IPC_CLIENT_TCPNODELAY_DEFAULT = false; - public static final String IPC_SERVER_LISTEN_QUEUE_SIZE_KEY = - "ipc.server.listen.queue.size"; - public static final int IPC_SERVER_LISTEN_QUEUE_SIZE_DEFAULT = 128; - public static final String IPC_CLIENT_KILL_MAX_KEY = "ipc.client.kill.max"; - public static final int IPC_CLIENT_KILL_MAX_DEFAULT = 10; - public static final String IPC_CLIENT_IDLETHRESHOLD_KEY = "ipc.client.idlethreshold"; - public static final int IPC_CLIENT_IDLETHRESHOLD_DEFAULT = 4000; - public static final String IPC_SERVER_TCPNODELAY_KEY = "ipc.server.tcpnodelay"; - public static final boolean IPC_SERVER_TCPNODELAY_DEFAULT = false; - public static final String IPC_SERVER_RPC_MAX_RESPONSE_SIZE_KEY = - "ipc.server.max.response.size"; - public static final int IPC_SERVER_RPC_MAX_RESPONSE_SIZE_DEFAULT = - 1024*1024; - public static final String IPC_SERVER_RPC_READ_THREADS_KEY = - "ipc.server.read.threadpool.size"; - public static final int IPC_SERVER_RPC_READ_THREADS_DEFAULT = 1; - /** - * How many calls per handler are allowed in the queue. + /** Responses larger than this will be logged */ + public static final String IPC_SERVER_RPC_MAX_RESPONSE_SIZE_KEY = + "ipc.server.max.response.size"; + /** Default value for IPC_SERVER_RPC_MAX_RESPONSE_SIZE_KEY */ + public static final int IPC_SERVER_RPC_MAX_RESPONSE_SIZE_DEFAULT = + 1024*1024; + /** Number of threads in RPC server reading from the socket */ + public static final String IPC_SERVER_RPC_READ_THREADS_KEY = + "ipc.server.read.threadpool.size"; + /** Default value for IPC_SERVER_RPC_READ_THREADS_KEY */ + public static final int IPC_SERVER_RPC_READ_THREADS_DEFAULT = 1; + + /** How many calls per handler are allowed in the queue. */ + public static final String IPC_SERVER_HANDLER_QUEUE_SIZE_KEY = + "ipc.server.handler.queue.size"; + /** Default value for IPC_SERVER_HANDLER_QUEUE_SIZE_KEY */ + public static final int IPC_SERVER_HANDLER_QUEUE_SIZE_DEFAULT = 100; + + /** Internal buffer size for Lzo compressor/decompressors */ + public static final String IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_KEY = + "io.compression.codec.lzo.buffersize"; + /** Default value for IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_KEY */ + public static final int IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_DEFAULT = + 64*1024; + /** This is for specifying the implementation for the mappings from + * hostnames to the racks they belong to */ - public static final String IPC_SERVER_HANDLER_QUEUE_SIZE_KEY = - "ipc.server.handler.queue.size"; - /** - * The default number of calls per handler in the queue. - */ - public static final int IPC_SERVER_HANDLER_QUEUE_SIZE_DEFAULT = 100; - - public static final String HADOOP_RPC_SOCKET_FACTORY_CLASS_DEFAULT_KEY = - "hadoop.rpc.socket.factory.class.default"; - public static final String HADOOP_SOCKS_SERVER_KEY = "hadoop.socks.server"; - public static final String HADOOP_JOB_UGI_KEY = "hadoop.job.ugi"; - public static final String HADOOP_UTIL_HASH_TYPE_KEY = "hadoop.util.hash.type"; - public static final String HADOOP_UTIL_HASH_TYPE_DEFAULT = "murmur"; - public static final String HADOOP_SECURITY_GROUP_MAPPING = "hadoop.security.group.mapping"; - public static final String HADOOP_SECURITY_GROUPS_CACHE_SECS = "hadoop.security.groups.cache.secs"; - public static final String HADOOP_SECURITY_AUTHENTICATION = "hadoop.security.authentication"; - public static final String HADOOP_SECURITY_AUTHORIZATION = - "hadoop.security.authorization"; - public static final String HADOOP_SECURITY_SERVICE_USER_NAME_KEY = - "hadoop.security.service.user.name.key"; + public static final String NET_TOPOLOGY_CONFIGURED_NODE_MAPPING_KEY = + "net.topology.configured.node.mapping"; } Added: hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java URL: http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java?rev=960079&view=auto ============================================================================== --- hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java (added) +++ hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java Fri Jul 2 18:16:30 2010 @@ -0,0 +1,220 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs; + +import org.apache.hadoop.classification.InterfaceAudience; + +/** + * This class contains constants for configuration keys used + * in the common code. + * + * It includes all publicly documented configuration keys. In general + * this class should not be used directly (use CommonConfigurationKeys + * instead) + * + */ + +@InterfaceAudience.Public +public class CommonConfigurationKeysPublic { + + // The Keys + /** See core-default.xml */ + public static final String IO_NATIVE_LIB_AVAILABLE_KEY = + "io.native.lib.available"; + /** Default value for IO_NATIVE_LIB_AVAILABLE_KEY */ + public static final boolean IO_NATIVE_LIB_AVAILABLE_DEFAULT = true; + /** See core-default.xml */ + public static final String NET_TOPOLOGY_SCRIPT_NUMBER_ARGS_KEY = + "net.topology.script.number.args"; + /** Default value for NET_TOPOLOGY_SCRIPT_NUMBER_ARGS_KEY */ + public static final int NET_TOPOLOGY_SCRIPT_NUMBER_ARGS_DEFAULT = 100; + + //FS keys + /** See core-default.xml */ + public static final String FS_DEFAULT_NAME_KEY = "fs.defaultFS"; + /** Default value for FS_DEFAULT_NAME_KEY */ + public static final String FS_DEFAULT_NAME_DEFAULT = "file:///"; + /** See core-default.xml */ + public static final String FS_DF_INTERVAL_KEY = "fs.df.interval"; + /** Default value for FS_DF_INTERVAL_KEY */ + public static final long FS_DF_INTERVAL_DEFAULT = 60000; + + + //Defaults are not specified for following keys + /** See core-default.xml */ + public static final String NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY = + "net.topology.script.file.name"; + /** See core-default.xml */ + public static final String NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY = + "net.topology.node.switch.mapping.impl"; + + /** See core-default.xml */ + public static final String FS_TRASH_CHECKPOINT_INTERVAL_KEY = + "fs.trash.checkpoint.interval"; + /** Default value for FS_TRASH_CHECKPOINT_INTERVAL_KEY */ + public static final long FS_TRASH_CHECKPOINT_INTERVAL_DEFAULT = 0; + + // TBD: Code is still using hardcoded values (e.g. "fs.automatic.close") + // instead of constant (e.g. FS_AUTOMATIC_CLOSE_KEY) + // + /** Not used anywhere, looks like default value for FS_LOCAL_BLOCK_SIZE */ + public static final long FS_LOCAL_BLOCK_SIZE_DEFAULT = 32*1024*1024; + /** See core-default.xml */ + public static final String FS_AUTOMATIC_CLOSE_KEY = "fs.automatic.close"; + /** Default value for FS_AUTOMATIC_CLOSE_KEY */ + public static final boolean FS_AUTOMATIC_CLOSE_DEFAULT = true; + /** See core-default.xml */ + public static final String FS_FILE_IMPL_KEY = "fs.file.impl"; + /** See core-default.xml */ + public static final String FS_FTP_HOST_KEY = "fs.ftp.host"; + /** See core-default.xml */ + public static final String FS_FTP_HOST_PORT_KEY = "fs.ftp.host.port"; + /** See core-default.xml */ + public static final String FS_TRASH_INTERVAL_KEY = "fs.trash.interval"; + /** Default value for FS_TRASH_INTERVAL_KEY */ + public static final long FS_TRASH_INTERVAL_DEFAULT = 0; + + /** See core-default.xml */ + public static final String IO_MAPFILE_BLOOM_SIZE_KEY = + "io.mapfile.bloom.size"; + /** Default value for IO_MAPFILE_BLOOM_SIZE_KEY */ + public static final int IO_MAPFILE_BLOOM_SIZE_DEFAULT = 1024*1024; + /** See core-default.xml */ + public static final String IO_MAPFILE_BLOOM_ERROR_RATE_KEY = + "io.mapfile.bloom.error.rate" ; + /** Default value for IO_MAPFILE_BLOOM_ERROR_RATE_KEY */ + public static final float IO_MAPFILE_BLOOM_ERROR_RATE_DEFAULT = 0.005f; + /** Codec class that implements Lzo compression algorithm */ + public static final String IO_COMPRESSION_CODEC_LZO_CLASS_KEY = + "io.compression.codec.lzo.class"; + /** See core-default.xml */ + public static final String IO_MAP_INDEX_INTERVAL_KEY = + "io.map.index.interval"; + /** Default value for IO_MAP_INDEX_INTERVAL_DEFAULT */ + public static final int IO_MAP_INDEX_INTERVAL_DEFAULT = 128; + /** See core-default.xml */ + public static final String IO_MAP_INDEX_SKIP_KEY = "io.map.index.skip"; + /** Default value for IO_MAP_INDEX_SKIP_KEY */ + public static final int IO_MAP_INDEX_SKIP_DEFAULT = 0; + /** See core-default.xml */ + public static final String IO_SEQFILE_COMPRESS_BLOCKSIZE_KEY = + "io.seqfile.compress.blocksize"; + /** Default value for IO_SEQFILE_COMPRESS_BLOCKSIZE_KEY */ + public static final int IO_SEQFILE_COMPRESS_BLOCKSIZE_DEFAULT = 1000000; + /** See core-default.xml */ + public static final String IO_SKIP_CHECKSUM_ERRORS_KEY = + "io.skip.checksum.errors"; + /** Default value for IO_SKIP_CHECKSUM_ERRORS_KEY */ + public static final boolean IO_SKIP_CHECKSUM_ERRORS_DEFAULT = false; + /** + * @deprecated Moved to mapreduce, see mapreduce.task.io.sort.mb + * in mapred-default.xml + * See https://issues.apache.org/jira/browse/HADOOP-6801 + */ + public static final String IO_SORT_MB_KEY = "io.sort.mb"; + /** Default value for IO_SORT_MB_DEFAULT */ + public static final int IO_SORT_MB_DEFAULT = 100; + /** + * @deprecated Moved to mapreduce, see mapreduce.task.io.sort.factor + * in mapred-default.xml + * See https://issues.apache.org/jira/browse/HADOOP-6801 + */ + public static final String IO_SORT_FACTOR_KEY = "io.sort.factor"; + /** Default value for IO_SORT_FACTOR_DEFAULT */ + public static final int IO_SORT_FACTOR_DEFAULT = 100; + /** See core-default.xml */ + public static final String IO_SERIALIZATIONS_KEY = "io.serializations"; + + /** See core-default.xml */ + public static final String TFILE_IO_CHUNK_SIZE_KEY = "tfile.io.chunk.size"; + /** Default value for TFILE_IO_CHUNK_SIZE_DEFAULT */ + public static final int TFILE_IO_CHUNK_SIZE_DEFAULT = 1024*1024; + /** See core-default.xml */ + public static final String TFILE_FS_INPUT_BUFFER_SIZE_KEY = + "tfile.fs.input.buffer.size"; + /** Default value for TFILE_FS_INPUT_BUFFER_SIZE_KEY */ + public static final int TFILE_FS_INPUT_BUFFER_SIZE_DEFAULT = 256*1024; + /** See core-default.xml */ + public static final String TFILE_FS_OUTPUT_BUFFER_SIZE_KEY = + "tfile.fs.output.buffer.size"; + /** Default value for TFILE_FS_OUTPUT_BUFFER_SIZE_KEY */ + public static final int TFILE_FS_OUTPUT_BUFFER_SIZE_DEFAULT = 256*1024; + + /** See core-default.xml */ + public static final String IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY = + "ipc.client.connection.maxidletime"; + /** Default value for IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY */ + public static final int IPC_CLIENT_CONNECTION_MAXIDLETIME_DEFAULT = 10000; + /** See core-default.xml */ + public static final String IPC_CLIENT_CONNECT_MAX_RETRIES_KEY = + "ipc.client.connect.max.retries"; + /** Default value for IPC_CLIENT_CONNECT_MAX_RETRIES_KEY */ + public static final int IPC_CLIENT_CONNECT_MAX_RETRIES_DEFAULT = 10; + /** See core-default.xml */ + public static final String IPC_CLIENT_TCPNODELAY_KEY = + "ipc.client.tcpnodelay"; + /** Defalt value for IPC_CLIENT_TCPNODELAY_KEY */ + public static final boolean IPC_CLIENT_TCPNODELAY_DEFAULT = false; + /** See core-default.xml */ + public static final String IPC_SERVER_LISTEN_QUEUE_SIZE_KEY = + "ipc.server.listen.queue.size"; + /** Default value for IPC_SERVER_LISTEN_QUEUE_SIZE_KEY */ + public static final int IPC_SERVER_LISTEN_QUEUE_SIZE_DEFAULT = 128; + /** See core-default.xml */ + public static final String IPC_CLIENT_KILL_MAX_KEY = "ipc.client.kill.max"; + /** Default value for IPC_CLIENT_KILL_MAX_KEY */ + public static final int IPC_CLIENT_KILL_MAX_DEFAULT = 10; + /** See core-default.xml */ + public static final String IPC_CLIENT_IDLETHRESHOLD_KEY = + "ipc.client.idlethreshold"; + /** Default value for IPC_CLIENT_IDLETHRESHOLD_DEFAULT */ + public static final int IPC_CLIENT_IDLETHRESHOLD_DEFAULT = 4000; + /** See core-default.xml */ + public static final String IPC_SERVER_TCPNODELAY_KEY = + "ipc.server.tcpnodelay"; + /** Default value for IPC_SERVER_TCPNODELAY_KEY */ + public static final boolean IPC_SERVER_TCPNODELAY_DEFAULT = false; + + /** See core-default.xml */ + public static final String HADOOP_RPC_SOCKET_FACTORY_CLASS_DEFAULT_KEY = + "hadoop.rpc.socket.factory.class.default"; + /** See core-default.xml */ + public static final String HADOOP_SOCKS_SERVER_KEY = "hadoop.socks.server"; + /** See core-default.xml */ + public static final String HADOOP_UTIL_HASH_TYPE_KEY = + "hadoop.util.hash.type"; + /** Default value for HADOOP_UTIL_HASH_TYPE_KEY */ + public static final String HADOOP_UTIL_HASH_TYPE_DEFAULT = "murmur"; + /** See core-default.xml */ + public static final String HADOOP_SECURITY_GROUP_MAPPING = + "hadoop.security.group.mapping"; + /** See core-default.xml */ + public static final String HADOOP_SECURITY_GROUPS_CACHE_SECS = + "hadoop.security.groups.cache.secs"; + /** See core-default.xml */ + public static final String HADOOP_SECURITY_AUTHENTICATION = + "hadoop.security.authentication"; + /** See core-default.xml */ + public static final String HADOOP_SECURITY_AUTHORIZATION = + "hadoop.security.authorization"; + /** See core-default.xml */ + public static final String HADOOP_SECURITY_SERVICE_USER_NAME_KEY = + "hadoop.security.service.user.name.key"; +} + Propchange: hadoop/common/trunk/src/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java ------------------------------------------------------------------------------ svn:mime-type = text/plain