hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-7160) Configurable initial buffersize for getGroupDetails()
Date Thu, 03 Mar 2011 20:23:36 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002186#comment-13002186
] 

Allen Wittenauer commented on HADOOP-7160:
------------------------------------------

Even with tuning of negative-time-to-live, positive-time-to-live, etc?

> Configurable initial buffersize for getGroupDetails()
> -----------------------------------------------------
>
>                 Key: HADOOP-7160
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7160
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: native, security
>    Affects Versions: 0.22.0
>            Reporter: T Meyarivan
>
> {code:title=trunk/src/native/src/org/apache/hadoop/security/getGroup.c|borderStyle=solid}
> int getGroupDetails(gid_t group, char **grpBuf) {
>   struct group * grp = NULL;
>   size_t currBufferSize = sysconf(_SC_GETGR_R_SIZE_MAX);
>   if (currBufferSize < 1024) {
>     currBufferSize = 1024;
>   }
>   *grpBuf = NULL; 
>   char *buf = (char*)malloc(sizeof(char) * currBufferSize);
>   if (!buf) {
>     return ENOMEM;
>   }
>   int error;
>   for (;;) {
>     error = getgrgid_r(group, (struct group*)buf,
>                        buf + sizeof(struct group),
>                        currBufferSize - sizeof(struct group), &grp);
>     if(error != ERANGE) {
>        break;
>     }
>     free(buf);
>     currBufferSize *= 2;
>     buf = malloc(sizeof(char) * currBufferSize);
>     if(!buf) {
>       return ENOMEM;
>     }
> ...
> {code}
> For large groups, this implies at least 2 queries for the group (number of queries =
math.ceil(math.log(response_size/1024, 2)))
> In the case of a large cluster with central user/group databases (exposed via LDAP etc),
this leads to unnecessary load on the central services. This can be alleviated to a large
extent by changing the initial buffer size to a configurable parameter
> --

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message