Return-Path: 
 <chukwa-dev-return-3827-apmail-hadoop-chukwa-dev-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-chukwa-dev-archive@minotaur.apache.org
Received: (qmail 9934 invoked from network); 23 Apr 2010 21:40:12 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 23 Apr 2010 21:40:12 -0000
Received: (qmail 18148 invoked by uid 500); 23 Apr 2010 21:40:12 -0000
Delivered-To: apmail-hadoop-chukwa-dev-archive@hadoop.apache.org
Received: (qmail 17988 invoked by uid 500); 23 Apr 2010 21:40:11 -0000
Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:chukwa-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:chukwa-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:chukwa-dev@hadoop.apache.org>
List-Id: <chukwa-dev.hadoop.apache.org>
Reply-To: chukwa-dev@hadoop.apache.org
Delivered-To: mailing list chukwa-dev@hadoop.apache.org
Received: (qmail 17980 invoked by uid 99); 23 Apr 2010 21:40:11 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Apr 2010 21:40:11 +0000
X-ASF-Spam-Status: No, hits=-1339.6 required=10.0
	tests=ALL_TRUSTED,AWL
X-Spam-Check-By: apache.org
Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Apr 2010 21:40:10 +0000
Received: from thor (localhost [127.0.0.1])
	by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o3NLdn0l014188
	for <chukwa-dev@hadoop.apache.org>; Fri, 23 Apr 2010 21:39:49 GMT
Message-ID: <31833871.168091272058789361.JavaMail.jira@thor>
Date: Fri, 23 Apr 2010 17:39:49 -0400 (EDT)
From: "Bill Graham (JIRA)" <jira@apache.org>
To: chukwa-dev@hadoop.apache.org
Subject: [jira] Updated: (CHUKWA-472) TsProcessor: make date format
 configurable
In-Reply-To: <1385409241.22181270579173755.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/CHUKWA-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham updated CHUKWA-472:
-------------------------------

    Status: Open  (was: Patch Available)

Canceling this patch, since I want to add one more bit of functionality that I think will be useful. The current implementation expects the date to be the first set of characters in the record. In some cases (i.e. Apache logs) that's not the case. Adding the ability to optionally specify a regular expression to locate where the date string exists in the record.

For a record like this for example, you could use configs like below:

{{10.10.182.49 [22/Apr/2010:15:07:27 -0700] "" 200 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" "some.site.com:8076"}}

{noformat}
  <property>
   <name>TsProcessor.time.regex.some_data_type</name>
   <value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2} [-+]\\d{4})\\] .*</value>
  </property>
{noformat}
{noformat}
  <property>
   <name>TsProcessor.default.time.regex</name>
   <value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2} [-+]\\d{4})\\] .*</value>
  </property>
{noformat}

> TsProcessor: make date format configurable
> ------------------------------------------
>
>                 Key: CHUKWA-472
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-472
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-472.1.patch
>
>
> The TsProcessor's default date format and it's date format for a given data type should both be configurable.
> - To set time format for a given data type:
> {noformat}
> <property>
>  <name>TsProcessor.time.format.some_data_type</name>
>  <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}
> - To set the default time format:
> {noformat}
> <property>
>  <name>TsProcessor.default.time.format</name>
>  <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.