hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5121) add RPCs for creating and manipulating cache pools
Date Tue, 03 Sep 2013 20:23:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757024#comment-13757024

Colin Patrick McCabe commented on HDFS-5121:

bq. Missing @throws annotations on some of the new entries in ClientProtocol.


bq. Not sure the client should be passing max # of entries to return in listPathCacheEntries
and listCachePools. Seems like the server should set the size of the results to return, and
to determine when there are no more entries the server can include the number of remaining
entries, as is done in ClientProtocol#getListing.

It's hard to unit test unless we can explicitly set the number of entries to return to a small
number.  Also, counting the total number of entries in the map could be time-consuming (since
we filter by pool, etc)

bq. Anywhere you take the FSNS lock in a user RPC which may need to be failover and be retried
to the other NN, you should check the operation category both before and after taking the


bq. The error message in the case of being in safe mode in removePathCacheEntries says "cannot
add path cache directive."


bq. I'm surprised you don't take the FSNS lock at all in FSNamesystem#listPathCacheEntries,
but even if you don't have to for some reason it seems like you should still be checking the
operation category so that client failover works properly in this case.

Fixed-- we now take readLock (and call checkOperation(OperationCategory.READ))

bq. You should make failed user RPCs still log audit events indicating that they failed, e.g.:

I now log in the "finally" section, so we see it either way.

bq. You should also use FSNamesystem#isAuditEnabled instead of directly calling auditLog.isInfoEnabled.


bq. Better to use GenericTestUtils#assertExceptionContains instead of just ignoring the expected
IOE in your tests:

> add RPCs for creating and manipulating cache pools
> --------------------------------------------------
>                 Key: HDFS-5121
>                 URL: https://issues.apache.org/jira/browse/HDFS-5121
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>    Affects Versions: HDFS-4949
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: hdfs-5121-3.patch, HDFS-5121-caching.001.patch, HDFS-5121-caching.002.patch
> We should add RPCs for creating and manipulating cache pools.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message