chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CHUKWA-749) Redefine Solr facets for Hadoop log monitoring
Date Sun, 31 May 2015 00:39:17 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Yang updated CHUKWA-749:
-----------------------------
    Description: 
For improving the user experience of analyze big data environment log files, we can use Solr
to organize all log files in a logical order.  Log files of big data environment are generally
divided into application and system categories.

In Application perspective, the structure of an application look like this:
{code}
Application
  user
  coordinator-job
    date
    workflow
      action
        jobs
          scheduler (app master)
          tasks
            task-attempts
              container
                host
{code}

In system perspective,
{code}
System
  cluster
    service
      component
        host
          container
{code}

The two perspectives are join by host.  If we have Application perspective and System perspective
in Solr cores.  We can join the activities happened in Application with System using Solr.

  was:
For improving the user experience of analyze big data environment log files, we can use Solr
to organize all log files in a logical order.  Log files of big data environment are generally
divided into application and system categories.

In Application perspective, the structure of an application look like this:
{code}
Application
  user
  coordinator-job
    date
    workflow
      action
        jobs
          scheduler (app master)
          tasks
            task-attempts
              container
                host
{code}

In system perspective,
{code}
System
  cluster
    service
      component
        host
{code}

The two perspectives are join by host.  If we have Application perspective and System perspective
in Solr cores.  We can join the activities happened in Application with System using Solr.


> Redefine Solr facets for Hadoop log monitoring
> ----------------------------------------------
>
>                 Key: CHUKWA-749
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-749
>             Project: Chukwa
>          Issue Type: New Feature
>          Components: Data Collection
>    Affects Versions: 0.7.0
>         Environment: Linux, Java
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>
> For improving the user experience of analyze big data environment log files, we can use
Solr to organize all log files in a logical order.  Log files of big data environment are
generally divided into application and system categories.
> In Application perspective, the structure of an application look like this:
> {code}
> Application
>   user
>   coordinator-job
>     date
>     workflow
>       action
>         jobs
>           scheduler (app master)
>           tasks
>             task-attempts
>               container
>                 host
> {code}
> In system perspective,
> {code}
> System
>   cluster
>     service
>       component
>         host
>           container
> {code}
> The two perspectives are join by host.  If we have Application perspective and System
perspective in Solr cores.  We can join the activities happened in Application with System
using Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message