chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CHUKWA-472) TsProcessor: make date format configurable
Date Fri, 23 Apr 2010 21:39:49 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bill Graham updated CHUKWA-472:
-------------------------------

    Status: Open  (was: Patch Available)

Canceling this patch, since I want to add one more bit of functionality that I think will
be useful. The current implementation expects the date to be the first set of characters in
the record. In some cases (i.e. Apache logs) that's not the case. Adding the ability to optionally
specify a regular expression to locate where the date string exists in the record.

For a record like this for example, you could use configs like below:

{{10.10.182.49 [22/Apr/2010:15:07:27 -0700] "" 200 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac
OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" "some.site.com:8076"}}

{noformat}
  <property>
   <name>TsProcessor.time.regex.some_data_type</name>
   <value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2} [-+]\\d{4})\\]
.*</value>
  </property>
{noformat}
{noformat}
  <property>
   <name>TsProcessor.default.time.regex</name>
   <value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2} [-+]\\d{4})\\]
.*</value>
  </property>
{noformat}

> TsProcessor: make date format configurable
> ------------------------------------------
>
>                 Key: CHUKWA-472
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-472
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-472.1.patch
>
>
> The TsProcessor's default date format and it's date format for a given data type should
both be configurable.
> - To set time format for a given data type:
> {noformat}
> <property>
>  <name>TsProcessor.time.format.some_data_type</name>
>  <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}
> - To set the default time format:
> {noformat}
> <property>
>  <name>TsProcessor.default.time.format</name>
>  <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message