hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-10077) o.a.h.s.Groups should refresh in the background
Date Wed, 30 Oct 2013 23:38:25 GMT
Colin Patrick McCabe created HADOOP-10077:

             Summary: o.a.h.s.Groups should refresh in the background
                 Key: HADOOP-10077
                 URL: https://issues.apache.org/jira/browse/HADOOP-10077
             Project: Hadoop Common
          Issue Type: Bug
    Affects Versions: 2.2.1
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe

{{org.apache.hadoop.security.Groups}} maintains a cache of mappings between user names and
sets of associated group names.  Periodically, the entries in this cache expire and must be
refetched from the operating system.

Currently, this is done in the context of whatever thread happens to try to access the group
mapping information right after the time period expires.  However, this is problematic, since
that thread may be holding the {{FSNamesystem}} lock.  This means that if the {{GroupMappingServiceProvider}}
is slow, the whole NameNode may grind to a halt until it finishes.  This can generate periodic
load spikes or even NameNode failovers.

Instead, we should allow the refreshing of the group mappings to be done asynchronously in
a background thread pool.

This message was sent by Atlassian JIRA

View raw message