hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4103) Support O(1) snapshot creation
Date Wed, 26 Dec 2012 04:14:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539468#comment-13539468
] 

Tsz Wo (Nicholas), SZE commented on HDFS-4103:
----------------------------------------------

{quote}
    2. If the test fails, we may then enable the message and re-run the test. Is there anything
wrong?

What if the test only fails occasionally? In that case re-running the test with the logging
enabled will not be helpful. This just came up recently with an HDFS test failure that was
very difficult to debug for exactly this reason.
{quote}
First of all, such case does not seem common.  Do you agree?  Even if it occurs, we may first
enable the log and then debug.  BTW, the logs disabled are not closely related the test. 
That's why they are disabled.  Too many log messages also make the debug harder.

{quote}
The symbol's not what's important, ...
{quote}
However, you previously said "There's a few "System.out.println("XXX ...")" left throughout
the patch. ..."   Your comments seem not consistent with each other.

{quote}
I found that method difficult to follow, ...
{quote}
There are always some people who may find some code difficult.  It depends on one's knowledge
and skill.  I don't think we have to make the code easy to everyone.  Again, we are not teaching
Java or try to educate anyone.

{quote}
I consider unit tests to be very important pieces of code.
{quote}
I consider the correctness of the unit tests important but the style of tests is not as important
as the correctness.  I would rather spend our time and resource on the main code and the correctness
of the tests but not the style of the tests.  I think it is unwise to spend much time and
resource on the style of unit tests.  No?

{quote}
It was added by this patch, starting on line 328...
{quote}
You are right.  However, it is also only related to the style of unit tests.  I may not fix
it for the reason above.

{quote}
Just because a test is disabled doesn't mean the code shouldn't be improved.
{quote}
Since it is disabled, let's improve it when we enable it later on.

{quote}
It's true those calls were mostly in tests, but the point still remains.
{quote}
Same as before, this is also only related to the style of unit tests.  I may not fix it.

{quote}
... They're certainly not INode-specific, and they're already being (ab)used in other classes.
...
{quote}
They are currently inode-specific.

{quote}
Mostly because it makes it impossible to test the class in isolation, since one cannot instantiate
a non-static inner class without also instantiating the outer class. Also because it makes
it more difficult to keep track of what's in scope. In general if there's not a really good
reason to have an inner class non-static, I think it's better to make it static.
{quote}
I don't see such problems and I think non-static inner classes are very useful.  If you comment
is valid, all methods should be static since the non-static methods also require creating
objects.

                
> Support O(1) snapshot creation
> ------------------------------
>
>                 Key: HDFS-4103
>                 URL: https://issues.apache.org/jira/browse/HDFS-4103
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: Snapshot (HDFS-2802)
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>             Fix For: Snapshot (HDFS-2802)
>
>         Attachments: h4103_20121129.patch, h4103_20121202b.patch, h4103_20121202.patch,
h4103_20121209b.patch, h4103_20121209.patch, h4103_20121210b.patch, h4103_20121210.patch,
h4103_20121211.patch, h4103_20121212b.patch, h4103_20121212.patch, h4103_20121213.patch, h4103_20121215b.patch,
h4103_20121215.patch, h4103_20121216.patch, h4103_20121217b.patch, h4103_20121217.patch, h4103_20121218.patch,
h4103_20121219.patch, h4103_20121220.patch
>
>
> In our first snapshot implementation, snapshot creation runs in O(N) and occupies O(N)
memory space, where N = # files + # directories + # symlinks in the snapshot.  The advantages
of the implementation are that there is no additional cost for the modifications after snapshots
are created, and it leads to a simple implementation.
> In this JIRA, we optimize snapshot creation to O(1) although it introduces additional
cost in the modifications after snapshots are created.  Note that the INode is given as an
assumption, otherwise, there is a non-constant cost to find the INode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message