hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3479) Implement configuration items useful for Hadoop resource manager (v1)
Date Fri, 13 Jun 2008 06:10:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604745#action_12604745
] 

Hemanth Yamijala commented on HADOOP-3479:
------------------------------------------

One of the alternative designs that we considered is to see if the Configuration class can
be reused.

The basic difference, as explained in comments above, is in the nested nature of the configuration
- sets of properties are grouped under a queue configuration element, for e.g. We could possibly
enhance Configuration to support parsing and exposing such nested configuration also, without
changing the basic API. One approach could be to support nesting that occurs in the following
manner:
{code:xml}
<configuration>
  <level1 key="value">
    <level2>
      <property>
        <!-- usual key-value elements here -->
      </property>
    </level2>
  </level1>
</configuration>
{code}
We could then build a Map of an XPath type expression to a Properties object, where the XPatch
expression takes us till the parent of a list of *property* elements. The current configuration
is then basically a Properties object mapped for the element */configuration*. Queue related
properties could be looked up like /configuration/queues/queue[@name="default"]. We could
even modify current configuration to be stored as a hierarchy:
{code:xml}
<configuration>
  <hadoop>
    <mapred>
       <!-- mapred configuration -->
    </mapred>
    <hdfs>
       <!-- hdfs configuration -->
    </hdfs>
    <resource-manager>
       <!-- resource-manager configuration -->
    </resource-manager>
  </hadoop>
</configuration>
{code}
... and so on.

However, given this is the first attempt for building resource manager functionality, it seems
prudent to keep things simple, if at the cost of duplicating a little code between Configuration
and ResourceManagerConf, and refactor if things are working out fine. Also, Configuration
is a religious topic and it would take a while to converge of how and what to do. *smile*.
Only if people feel  very strongly about this approach, should we consider it for Hadoop 0.19
though.

> Implement configuration items useful for Hadoop resource manager (v1)
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-3479
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3479
>             Project: Hadoop Core
>          Issue Type: New Feature
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>         Attachments: 3479.patch
>
>
> HADOOP-3421 lists requirements for a new resource manager for Hadoop. Implementation
for these will require support for new configuration items in Hadoop. This JIRA is to define
such configuration, and track it's implementation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message