hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7705) Add a log4j back end that can push out JSON data, one per line
Date Fri, 07 Oct 2011 10:28:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122681#comment-13122681
] 

Steve Loughran commented on HADOOP-7705:
----------------------------------------

This patch is independent of the rest of the Hadoop codebase; it's a new logger than can push
out JSON content that is easy for management tools to parse (indeed, the tests parse the data
to verify round-tripping. 

The test run shows what the output is, here are some examples from it

1. Simple log message, as it looks: one JSON node per line. All values (escaped) strings except
for time_t, which is a integer
{code}
{"name":"name","time":0,"level":"DEBUG","thread":"thread1","message":"hello, world"}
{code}
2. Log with a single nested exception, reformatted as multiline for readability. The exception
class is added (if provided), and the stack is included as an array. 
{code}
{
    "name":"testException",
    "time":1317980185037,
    "level":"INFO",
    "thread":"quoted\"",
    "message":"new line\n and {}",
    "exceptionclass":"java.net.NoRouteToHostException",
    "stack":[
        "java.net.NoRouteToHostException: that box caught fire 3 years ago",
        "\tat org.apache.hadoop.log.TestLog4Json.testException(TestLog4Json.java:49)",
        "\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
        "\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)",
        "\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)",
        "\tat java.lang.reflect.Method.invoke(Method.java:597)",
        "\tat junit.framework.TestCase.runTest(TestCase.java:168)",
        "\tat junit.framework.TestCase.runBare(TestCase.java:134)",
        "\tat junit.framework.TestResult$1.protect(TestResult.java:110)",
        "\tat junit.framework.TestResult.runProtected(TestResult.java:128)",
        "\tat junit.framework.TestResult.run(TestResult.java:113)",
        "\tat junit.framework.TestCase.run(TestCase.java:124)",
        "\tat junit.framework.TestSuite.runTest(TestSuite.java:232)",
        "\tat junit.framework.TestSuite.run(TestSuite.java:227)",
        "\tat org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)",
        "\tat org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)",
        "\tat org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)",
        "\tat org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)",
        "\tat org.apache.maven.surefire.Surefire.run(Surefire.java:104)",
        "\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
        "\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)",
        "\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)",
        "\tat java.lang.reflect.Method.invoke(Method.java:597)",
        "\tat org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:290)",
        "\tat org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1017)"
      ]
}
{code}

One potential improvement would be to include the timestamp as a human readable ISO time.
The reason I haven't done this is the cost of creating this data -if the target is a machine,
it's not needed.

This log could be included as a (commented out) example in the conf/log4 file. It must not
go in the src/test log4 file as if there is a bug in the code, that could break all tests.

                
> Add a log4j back end that can push out JSON data, one per line
> --------------------------------------------------------------
>
>                 Key: HADOOP-7705
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7705
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-7705-log4json.patch
>
>
> If we had a back end for Log4j that pushed out log events in single line JSON content,
we'd have something that is fairly straightforward to machine parse. If: it may be harder
to do than expected. Once working HADOOP-6244 could use it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message