Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Date: Thu, 3 Mar 2011 16:56:37 +0000 (UTC)
From: "T Meyarivan (JIRA)" <jira@apache.org>
To: common-dev@hadoop.apache.org
Message-ID: 
 <519472433.11197.1299171397059.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] Created: (HADOOP-7160) Configurable initial buffersize for
 getGroupDetails()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Configurable initial buffersize for getGroupDetails()
-----------------------------------------------------

                 Key: HADOOP-7160
                 URL: https://issues.apache.org/jira/browse/HADOOP-7160
             Project: Hadoop Common
          Issue Type: Improvement
          Components: native, security
    Affects Versions: 0.22.0
            Reporter: T Meyarivan


trunk/src/native/src/org/apache/hadoop/security/getGroup.c

"""
int getGroupDetails(gid_t group, char **grpBuf) {
  struct group * grp = NULL;
  size_t currBufferSize = sysconf(_SC_GETGR_R_SIZE_MAX);
  if (currBufferSize < 1024) {
    currBufferSize = 1024;
  }
  *grpBuf = NULL; 
  char *buf = (char*)malloc(sizeof(char) * currBufferSize);

  if (!buf) {
    return ENOMEM;
  }
  int error;
  for (;;) {
    error = getgrgid_r(group, (struct group*)buf,
                       buf + sizeof(struct group),
                       currBufferSize - sizeof(struct group), &grp);
    if(error != ERANGE) {
       break;
    }
    free(buf);
    currBufferSize *= 2;
    buf = malloc(sizeof(char) * currBufferSize);
    if(!buf) {
      return ENOMEM;
    }
...
"""

For large groups, this implies at least 2 queries for the group (number of queries = math.ceil(math.log(response_size/1024, 2)))

In the case of a large cluster with central user/group databases (exposed via LDAP etc), this leads to unnecessary load on the central services. This can be alleviated to a large extent by changing the initial buffer size to a configurable parameter

--

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira