hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4045) SecondaryNameNode cannot read from QuorumJournal URI
Date Fri, 30 Nov 2012 01:49:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507032#comment-13507032

Todd Lipcon commented on HDFS-4045:

Took a look through the patch. I generally think this approach is better than the first. Here
are some more specific comments:

- I agree with Colin above that EditLogServletOutputStream doesn't seems like it should inherit
from EditLogFileOutputStream. There's no need to double-buffer in this case, so it really
reuses very little of its superclass. Instead, I think you could just operate directly on
the ServletOutputStream itself, pushing the raw bytes directly to the output. This means you
also wouldn't need the new {{writeRaw(FSEditLogOp)}} API -- that API doesn't make a lot of
sense to me either, since it seems kind of like a contradiction in terms.
- Instead of the trick in {{decodeOp()}} where you mark/reset/re-read to get the raw bytes,
did you consider adding a {{readRawOp()}} API? I agree conceptually with Colin that the EditLogInputStream
is a stream of operations, but if the API were still "give me the raw bytes corresponding
to the next op", that doesn't seem to break the abstraction as much as the previously proposed
API which had {{getInputStream}} in it.

With the above, serveEdits could be implemented something like:
os = response.getOutputStream();
byte[] op;
while ((op = es.readOpRaw()) != null) {
  os.write(op, 0, op.length);

(assuming that the "readOpRaw" includes both the txid and the checksum.

Even if the second suggestion above doesn't work out, I think avoiding the EditLogServletOutputStream
would make the code a little cleaner as well.

Colin, any thoughts on this?
> SecondaryNameNode cannot read from QuorumJournal URI
> ----------------------------------------------------
>                 Key: HDFS-4045
>                 URL: https://issues.apache.org/jira/browse/HDFS-4045
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Vinithra Varadharajan
>            Assignee: Andy Isaacson
>         Attachments: hdfs-4045-2.txt, hdfs4045-3.txt, hdfs4045-4.txt, hdfs-4045.txt
> If HDFS is set up in basic mode (non-HA) with QuorumJournal, and the dfs.namenode.edits.dir
is set to only the QuorumJournal URI and no local dir, the SecondaryNameNode is unable to
do a checkpoint.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message