Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 00F64200B9B for ; Tue, 6 Sep 2016 21:43:37 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id F3D25160AD4; Tue, 6 Sep 2016 19:43:36 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2F864160AD3 for ; Tue, 6 Sep 2016 21:43:34 +0200 (CEST) Received: (qmail 12820 invoked by uid 500); 6 Sep 2016 19:43:33 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 12691 invoked by uid 99); 6 Sep 2016 19:43:33 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Sep 2016 19:43:33 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 1768AE058E; Tue, 6 Sep 2016 19:43:33 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: mjwall@apache.org To: commits@accumulo.apache.org Date: Tue, 06 Sep 2016 19:43:38 -0000 Message-Id: <1fa4baf61fe34940aa79c8f12ac68adf@git.apache.org> In-Reply-To: <8af9106e244c4ef580b9cc80df65975f@git.apache.org> References: <8af9106e244c4ef580b9cc80df65975f@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [07/51] [partial] accumulo git commit: Updates for 1.8.0 Release archived-at: Tue, 06 Sep 2016 19:43:37 -0000 http://git-wip-us.apache.org/repos/asf/accumulo/blob/97bc584b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.AbstractRecordReader.html ---------------------------------------------------------------------- diff --git a/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.AbstractRecordReader.html b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.AbstractRecordReader.html new file mode 100644 index 0000000..e16a72d --- /dev/null +++ b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.AbstractRecordReader.html @@ -0,0 +1,488 @@ + + + + + + +AbstractInputFormat.AbstractRecordReader (Apache Accumulo Project 1.8.0 API) + + + + + + + +
+ + + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ + +
+
org.apache.accumulo.core.client.mapred
+

Class AbstractInputFormat.AbstractRecordReader<K,V>

+
+
+
    +
  • java.lang.Object
    • +
  • +
      +
    • org.apache.accumulo.core.client.mapred.AbstractInputFormat.AbstractRecordReader<K,V>
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    org.apache.hadoop.mapred.RecordReader<K,V>
    +
    +
    +
    Direct Known Subclasses:
    +
    InputFormatBase.RecordReaderBase
    +
    +
    +
    Enclosing class:
    +
    AbstractInputFormat<K,V>
    +
    +
    +
    +
    protected abstract static class AbstractInputFormat.AbstractRecordReader<K,V>
+extends Object
+implements org.apache.hadoop.mapred.RecordReader<K,V>
    +
    An abstract base class to be used to create RecordReader instances that convert from Accumulo + Key/Value pairs to the user's K/V types. + + Subclasses must implement RecordReader.next(Object, Object) to update key and value, and also to update the following variables: +
    +
    • +
+
+
+ +
+
+
    +
  • + + + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        AbstractInputFormat.AbstractRecordReader

        +
        protected AbstractInputFormat.AbstractRecordReader()
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        jobIterators

        +
        protected abstract List<IteratorSetting> jobIterators(org.apache.hadoop.mapred.JobConf job,
+                                 String tableName)
        +
        Extracts Iterators settings from the context to be used by RecordReader.
        +
        Parameters:
        job - the Hadoop job configuration
        tableName - the table name for which the scanner is configured
        +
        Returns:
        List of iterator settings for given table
        Since:
        +
        1.7.0
        +
        • +
      + + + +
        +
      • +

        setupIterators

        +
        @Deprecated
+protected void setupIterators(org.apache.hadoop.mapred.JobConf job,
+                             Scanner scanner,
+                             String tableName,
+                             RangeInputSplit split)
        +
        Deprecated. since 1.7.0; Use jobIterators(org.apache.hadoop.mapred.JobConf, java.lang.String) instead.
        +
        Configures the iterators on a scanner for the given table name.
        +
        Parameters:
        job - the Hadoop job configuration
        scanner - the scanner for which to configure the iterators
        tableName - the table name for which the scanner is configured
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        initialize

        +
        public void initialize(org.apache.hadoop.mapred.InputSplit inSplit,
+              org.apache.hadoop.mapred.JobConf job)
+                throws IOException
        +
        Initialize a scanner over the given input split using this task attempt configuration.
        +
        Throws:
        +
        IOException
        +
        • +
      + + + +
        +
      • +

        close

        +
        public void close()
        +
        +
        Specified by:
        +
        close in interface org.apache.hadoop.mapred.RecordReader<K,V>
        +
        +
        • +
      + + + +
        +
      • +

        getPos

        +
        public long getPos()
+            throws IOException
        +
        +
        Specified by:
        +
        getPos in interface org.apache.hadoop.mapred.RecordReader<K,V>
        +
        Throws:
        +
        IOException
        +
        • +
      + + + +
        +
      • +

        getProgress

        +
        public float getProgress()
+                  throws IOException
        +
        +
        Specified by:
        +
        getProgress in interface org.apache.hadoop.mapred.RecordReader<K,V>
        +
        Throws:
        +
        IOException
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ +

Copyright © 2011–2016 The Apache Software Foundation. All rights reserved.

+ + http://git-wip-us.apache.org/repos/asf/accumulo/blob/97bc584b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.html ---------------------------------------------------------------------- diff --git a/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.html b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.html new file mode 100644 index 0000000..731ec96 --- /dev/null +++ b/1.8/apidocs/org/apache/accumulo/core/client/mapred/AbstractInputFormat.html @@ -0,0 +1,801 @@ + + + + + + +AbstractInputFormat (Apache Accumulo Project 1.8.0 API) + + + + + + + +
+ + + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ + +
+
org.apache.accumulo.core.client.mapred
+

Class AbstractInputFormat<K,V>

+
+
+
    +
  • java.lang.Object
    • +
  • +
      +
    • org.apache.accumulo.core.client.mapred.AbstractInputFormat<K,V>
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    org.apache.hadoop.mapred.InputFormat<K,V>
    +
    +
    +
    Direct Known Subclasses:
    +
    AccumuloMultiTableInputFormat, InputFormatBase
    +
    +
    +
    +
    public abstract class AbstractInputFormat<K,V>
+extends Object
+implements org.apache.hadoop.mapred.InputFormat<K,V>
    +
    An abstract input format to provide shared methods common to all other input format classes. At the very least, any classes inheriting from this class will + need to define their own RecordReader.
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Nested Class Summary

      + + + + + + + + + + +
      Nested Classes 
      Modifier and TypeClass and Description
      protected static class AbstractInputFormat.AbstractRecordReader<K,V> +
      An abstract base class to be used to create RecordReader instances that convert from Accumulo + Key/Value pairs to the user's K/V types.
      +
      +
      • +
    + +
      +
    • + + +

      Field Summary

      + + + + + + + + + + + + + + +
      Fields 
      Modifier and TypeField and Description
      protected static Class<?>CLASS 
      protected static org.apache.log4j.Loggerlog 
      +
      • +
    + + + +
      +
    • + + +

      Method Summary

      + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      Methods 
      Modifier and TypeMethod and Description
      protected static AuthenticationTokengetAuthenticationToken(org.apache.hadoop.mapred.JobConf job) +
      Gets the authenticated token from either the specified token file or directly from the configuration, whichever was used when the job was configured.
      +
      static StringgetClassLoaderContext(org.apache.hadoop.mapred.JobConf job) +
      Returns the name of the current classloader context set on this scanner
      +
      protected static ClientConfigurationgetClientConfiguration(org.apache.hadoop.mapred.JobConf job) +
      Fetch the client configuration from the job.
      +
      static InputTableConfiggetInputTableConfig(org.apache.hadoop.mapred.JobConf job, + String tableName) +
      Fetches a InputTableConfig that has been set on the configuration for a specific table.
      +
      static Map<String,InputTableConfig>getInputTableConfigs(org.apache.hadoop.mapred.JobConf job) +
      Fetches all InputTableConfigs that have been set on the given Hadoop job.
      +
      protected static InstancegetInstance(org.apache.hadoop.mapred.JobConf job) +
      Initializes an Accumulo Instance based on the configuration.
      +
      protected static org.apache.log4j.LevelgetLogLevel(org.apache.hadoop.mapred.JobConf job) +
      Gets the log level from this configuration.
      +
      protected static StringgetPrincipal(org.apache.hadoop.mapred.JobConf job) +
      Gets the user name from the configuration.
      +
      protected static AuthorizationsgetScanAuthorizations(org.apache.hadoop.mapred.JobConf job) +
      Gets the authorizations to set for the scans from the configuration.
      +
      org.apache.hadoop.mapred.InputSplit[]getSplits(org.apache.hadoop.mapred.JobConf job, + int numSplits) +
      Gets the splits of the tables that have been set on the job by reading the metadata table for the specified ranges.
      +
      protected static org.apache.accumulo.core.client.impl.TabletLocatorgetTabletLocator(org.apache.hadoop.mapred.JobConf job, + String tableId) +
      Deprecated.  +
      since 1.7.0 This method returns a type that is not part of the public API and is not guaranteed to be stable. The method was deprecated to + discourage its use.
      +
      +
      protected static BooleanisConnectorInfoSet(org.apache.hadoop.mapred.JobConf job) +
      Determines if the connector has been configured.
      +
      static voidsetClassLoaderContext(org.apache.hadoop.mapred.JobConf job, + String context) +
      Sets the name of the classloader context on this scanner
      +
      static voidsetConnectorInfo(org.apache.hadoop.mapred.JobConf job, + String principal, + AuthenticationToken token) +
      Sets the connector information needed to communicate with Accumulo in this job.
      +
      static voidsetConnectorInfo(org.apache.hadoop.mapred.JobConf job, + String principal, + String tokenFile) +
      Sets the connector information needed to communicate with Accumulo in this job.
      +
      static voidsetLogLevel(org.apache.hadoop.mapred.JobConf job, + org.apache.log4j.Level level) +
      Sets the log level for this job.
      +
      static voidsetMockInstance(org.apache.hadoop.mapred.JobConf job, + String instanceName) +
      Deprecated.  +
      since 1.8.0; use MiniAccumuloCluster or a standard mock framework
      +
      +
      static voidsetScanAuthorizations(org.apache.hadoop.mapred.JobConf job, + Authorizations auths) +
      Sets the Authorizations used to scan.
      +
      static voidsetZooKeeperInstance(org.apache.hadoop.mapred.JobConf job, + ClientConfiguration clientConfig) +
      Configures a ZooKeeperInstance for this job.
      +
      static voidsetZooKeeperInstance(org.apache.hadoop.mapred.JobConf job, + String instanceName, + String zooKeepers) +
      Deprecated.  +
      since 1.6.0; Use setZooKeeperInstance(JobConf, ClientConfiguration) instead.
      +
      +
      protected static voidvalidateOptions(org.apache.hadoop.mapred.JobConf job) +
      Check whether a configuration is fully configured to be used with an Accumulo InputFormat.
      +
      + +
        +
      • + + +

        Methods inherited from interface org.apache.hadoop.mapred.InputFormat

        +getRecordReader
        • +
      +
      • +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Field Detail

      + + + +
        +
      • +

        CLASS

        +
        protected static final Class<?> CLASS
        +
        • +
      + + + +
        +
      • +

        log

        +
        protected static final org.apache.log4j.Logger log
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        AbstractInputFormat

        +
        public AbstractInputFormat()
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        setClassLoaderContext

        +
        public static void setClassLoaderContext(org.apache.hadoop.mapred.JobConf job,
+                         String context)
        +
        Sets the name of the classloader context on this scanner
        +
        Parameters:
        job - the Hadoop job instance to be configured
        context - name of the classloader context
        Since:
        +
        1.8.0
        +
        • +
      + + + +
        +
      • +

        getClassLoaderContext

        +
        public static String getClassLoaderContext(org.apache.hadoop.mapred.JobConf job)
        +
        Returns the name of the current classloader context set on this scanner
        +
        Parameters:
        job - the Hadoop job instance to be configured
        +
        Returns:
        name of the current context
        Since:
        +
        1.8.0
        +
        • +
      + + + +
        +
      • +

        setConnectorInfo

        +
        public static void setConnectorInfo(org.apache.hadoop.mapred.JobConf job,
+                    String principal,
+                    AuthenticationToken token)
+                             throws AccumuloSecurityException
        +
        Sets the connector information needed to communicate with Accumulo in this job. + +

        + WARNING: Some tokens, when serialized, divulge sensitive information in the configuration as a means to pass the token to MapReduce tasks. This + information is BASE64 encoded to provide a charset safe conversion to a string, but this conversion is not intended to be secure. PasswordToken is + one example that is insecure in this way; however DelegationTokens, acquired using + SecurityOperations.getDelegationToken(DelegationTokenConfig), is not subject to this concern.

        +
        Parameters:
        job - the Hadoop job instance to be configured
        principal - a valid Accumulo user name (user must have Table.CREATE permission)
        token - the user's password
        +
        Throws:
        +
        AccumuloSecurityException
        Since:
        +
        1.5.0
        +
        • +
      + + + +
        +
      • +

        setConnectorInfo

        +
        public static void setConnectorInfo(org.apache.hadoop.mapred.JobConf job,
+                    String principal,
+                    String tokenFile)
+                             throws AccumuloSecurityException
        +
        Sets the connector information needed to communicate with Accumulo in this job. + +

        + Stores the password in a file in HDFS and pulls that into the Distributed Cache in an attempt to be more secure than storing it in the Configuration.

        +
        Parameters:
        job - the Hadoop job instance to be configured
        principal - a valid Accumulo user name (user must have Table.CREATE permission)
        tokenFile - the path to the token file
        +
        Throws:
        +
        AccumuloSecurityException
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        isConnectorInfoSet

        +
        protected static Boolean isConnectorInfoSet(org.apache.hadoop.mapred.JobConf job)
        +
        Determines if the connector has been configured.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Returns:
        true if the connector has been configured, false otherwise
        Since:
        +
        1.5.0
        +
        See Also:
        setConnectorInfo(JobConf, String, AuthenticationToken)
        +
        • +
      + + + +
        +
      • +

        getPrincipal

        +
        protected static String getPrincipal(org.apache.hadoop.mapred.JobConf job)
        +
        Gets the user name from the configuration.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Returns:
        the user name
        Since:
        +
        1.5.0
        +
        See Also:
        setConnectorInfo(JobConf, String, AuthenticationToken)
        +
        • +
      + + + + + + + +
        +
      • +

        setZooKeeperInstance

        +
        @Deprecated
+public static void setZooKeeperInstance(org.apache.hadoop.mapred.JobConf job,
+                                   String instanceName,
+                                   String zooKeepers)
        +
        Deprecated. since 1.6.0; Use setZooKeeperInstance(JobConf, ClientConfiguration) instead.
        +
        Configures a ZooKeeperInstance for this job.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        instanceName - the Accumulo instance name
        zooKeepers - a comma-separated list of zookeeper servers
        Since:
        +
        1.5.0
        +
        • +
      + + + +
        +
      • +

        setZooKeeperInstance

        +
        public static void setZooKeeperInstance(org.apache.hadoop.mapred.JobConf job,
+                        ClientConfiguration clientConfig)
        +
        Configures a ZooKeeperInstance for this job.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        clientConfig - client configuration containing connection options
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        setMockInstance

        +
        @Deprecated
+public static void setMockInstance(org.apache.hadoop.mapred.JobConf job,
+                              String instanceName)
        +
        Deprecated. since 1.8.0; use MiniAccumuloCluster or a standard mock framework
        +
        Configures a MockInstance for this job.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        instanceName - the Accumulo instance name
        Since:
        +
        1.5.0
        +
        • +
      + + + + + + + +
        +
      • +

        setLogLevel

        +
        public static void setLogLevel(org.apache.hadoop.mapred.JobConf job,
+               org.apache.log4j.Level level)
        +
        Sets the log level for this job.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        level - the logging level
        Since:
        +
        1.5.0
        +
        • +
      + + + +
        +
      • +

        getLogLevel

        +
        protected static org.apache.log4j.Level getLogLevel(org.apache.hadoop.mapred.JobConf job)
        +
        Gets the log level from this configuration.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Returns:
        the log level
        Since:
        +
        1.5.0
        +
        See Also:
        setLogLevel(JobConf, Level)
        +
        • +
      + + + +
        +
      • +

        setScanAuthorizations

        +
        public static void setScanAuthorizations(org.apache.hadoop.mapred.JobConf job,
+                         Authorizations auths)
        +
        Sets the Authorizations used to scan. Must be a subset of the user's authorization. Defaults to the empty set.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        auths - the user's authorizations
        Since:
        +
        1.5.0
        +
        • +
      + + + +
        +
      • +

        getScanAuthorizations

        +
        protected static Authorizations getScanAuthorizations(org.apache.hadoop.mapred.JobConf job)
        +
        Gets the authorizations to set for the scans from the configuration.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Returns:
        the Accumulo scan authorizations
        Since:
        +
        1.5.0
        +
        See Also:
        setScanAuthorizations(JobConf, Authorizations)
        +
        • +
      + + + +
        +
      • +

        getTabletLocator

        +
        @Deprecated
+protected static org.apache.accumulo.core.client.impl.TabletLocator getTabletLocator(org.apache.hadoop.mapred.JobConf job,
+                                                                             String tableId)
+                                                                              throws TableNotFoundException
        +
        Deprecated. since 1.7.0 This method returns a type that is not part of the public API and is not guaranteed to be stable. The method was deprecated to + discourage its use.
        +
        Initializes an Accumulo TabletLocator based on the configuration.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Returns:
        an Accumulo tablet locator
        +
        Throws:
        +
        TableNotFoundException - if the table name set on the configuration doesn't exist
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        getClientConfiguration

        +
        protected static ClientConfiguration getClientConfiguration(org.apache.hadoop.mapred.JobConf job)
        +
        Fetch the client configuration from the job.
        +
        Parameters:
        job - The job
        +
        Returns:
        The client configuration for the job
        Since:
        +
        1.7.0
        +
        • +
      + + + +
        +
      • +

        validateOptions

        +
        protected static void validateOptions(org.apache.hadoop.mapred.JobConf job)
+                               throws IOException
        +
        Check whether a configuration is fully configured to be used with an Accumulo InputFormat.
        +
        Parameters:
        job - the Hadoop context for the configured job
        +
        Throws:
        +
        IOException - if the context is improperly configured
        Since:
        +
        1.5.0
        +
        • +
      + + + +
        +
      • +

        getInputTableConfigs

        +
        public static Map<String,InputTableConfig> getInputTableConfigs(org.apache.hadoop.mapred.JobConf job)
        +
        Fetches all InputTableConfigs that have been set on the given Hadoop job.
        +
        Parameters:
        job - the Hadoop job instance to be configured
        +
        Returns:
        the InputTableConfig objects set on the job
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        getInputTableConfig

        +
        public static InputTableConfig getInputTableConfig(org.apache.hadoop.mapred.JobConf job,
+                                   String tableName)
        +
        Fetches a InputTableConfig that has been set on the configuration for a specific table. + +

        + null is returned in the event that the table doesn't exist.

        +
        Parameters:
        job - the Hadoop job instance to be configured
        tableName - the table name for which to grab the config object
        +
        Returns:
        the InputTableConfig for the given table
        Since:
        +
        1.6.0
        +
        • +
      + + + +
        +
      • +

        getSplits

        +
        public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
+                                              int numSplits)
+                                                throws IOException
        +
        Gets the splits of the tables that have been set on the job by reading the metadata table for the specified ranges.
        +
        +
        Specified by:
        +
        getSplits in interface org.apache.hadoop.mapred.InputFormat<K,V>
        +
        Returns:
        the splits from the tables based on the ranges.
        +
        Throws:
        +
        IOException - if a table set on the job doesn't exist or an error occurs initializing the tablet locator
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ +

Copyright © 2011–2016 The Apache Software Foundation. All rights reserved.

+ +