From issues-return-40831-archive-asf-public=cust-asf.ponee.io@karaf.apache.org Fri Sep 7 09:01:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id D5C07180674 for ; Fri, 7 Sep 2018 09:01:05 +0200 (CEST) Received: (qmail 42967 invoked by uid 500); 7 Sep 2018 07:01:04 -0000 Mailing-List: contact issues-help@karaf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@karaf.apache.org Delivered-To: mailing list issues@karaf.apache.org Received: (qmail 42951 invoked by uid 99); 7 Sep 2018 07:01:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Sep 2018 07:01:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 39032C00C7 for ; Fri, 7 Sep 2018 07:01:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 0th3_sq9-eBF for ; Fri, 7 Sep 2018 07:01:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 501705F356 for ; Fri, 7 Sep 2018 07:01:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id BAF60E092E for ; Fri, 7 Sep 2018 07:01:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 74F7226B56 for ; Fri, 7 Sep 2018 07:01:00 +0000 (UTC) Date: Fri, 7 Sep 2018 07:01:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@karaf.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (KARAF-5888) Be able to define a line parser service in the file collector MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KARAF-5888?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1660= 6759#comment-16606759 ]=20 ASF GitHub Bot commented on KARAF-5888: --------------------------------------- jbonofre closed pull request #49: KARAF-5888 Be able to define a line parse= r service in the file collector URL: https://github.com/apache/karaf-decanter/pull/49 =20 =20 =20 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/api/pom.xml b/api/pom.xml index fcc7acf..1fdcb11 100644 --- a/api/pom.xml +++ b/api/pom.xml @@ -46,7 +46,8 @@ org.apache.karaf.decanter.api, - org.apache.karaf.decanter.api.marshaller + org.apache.karaf.decanter.api.marshaller, + org.apache.karaf.decanter.api.parser diff --git a/api/src/main/java/org/apache/karaf/decanter/api/parser/Parser.= java b/api/src/main/java/org/apache/karaf/decanter/api/parser/Parser.java new file mode 100644 index 0000000..c5567c3 --- /dev/null +++ b/api/src/main/java/org/apache/karaf/decanter/api/parser/Parser.java @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied= . + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.karaf.decanter.api.parser; + +import java.util.Map; + +public interface Parser { + + String SERVICE_KEY_ID =3D "parserId"; + + Map parse(String line); +} diff --git a/api/src/main/java/org/apache/karaf/decanter/api/parser/package= info b/api/src/main/java/org/apache/karaf/decanter/api/parser/packageinfo new file mode 100644 index 0000000..c72722a --- /dev/null +++ b/api/src/main/java/org/apache/karaf/decanter/api/parser/packageinfo @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +version 1.0.0 diff --git a/api/src/main/java/org/apache/karaf/decanter/impl/parser/Identi= tyParser.java b/api/src/main/java/org/apache/karaf/decanter/impl/parser/Ide= ntityParser.java new file mode 100644 index 0000000..f088d9a --- /dev/null +++ b/api/src/main/java/org/apache/karaf/decanter/impl/parser/IdentityParse= r.java @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied= . + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.karaf.decanter.impl.parser; + +import org.apache.karaf.decanter.api.parser.Parser; +import org.osgi.service.component.annotations.Component; + +import java.util.HashMap; +import java.util.Map; + +@Component(immediate =3D true, property =3D Parser.SERVICE_KEY_ID + "=3Did= entify") +public class IdentityParser implements Parser { + + @Override + public Map parse(String line) { + Map data =3D new HashMap<>(); + + try { + data.put("line", Integer.parseInt(line)); + return data; + } catch (Exception e) { + // nothing to do + } + + try { + data.put("line", Long.parseLong(line)); + return data; + } catch (Exception e) { + // nothing to do + } + + data.put("line", line); + return data; + } +} diff --git a/api/src/test/java/org/apache/karaf/decanter/impl/parser/TestId= entityParser.java b/api/src/test/java/org/apache/karaf/decanter/impl/parser= /TestIdentityParser.java new file mode 100644 index 0000000..fae39ed --- /dev/null +++ b/api/src/test/java/org/apache/karaf/decanter/impl/parser/TestIdentityP= arser.java @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied= . + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.karaf.decanter.impl.parser; + +import org.apache.karaf.decanter.api.parser.Parser; +import org.junit.Assert; +import org.junit.Test; + +import java.util.Map; + +public class TestIdentityParser { + + @Test + public void testParse() { + + Parser parser =3D new IdentityParser(); + Map data; + + // String type test + String lineString =3D "2018-09-07T08:40:41,768 | INFO | FelixStar= tLevel | core | 14 - " + + "org.apache.aries.jmx.core - 1.1.8 | Unregistering MBean w= ith ObjectName " + + "[osgi.compendium:service=3Dcm,version=3D1.3,framework=3Do= rg.apache.felix.framework," + + "uuid=3De75146c3-f73c-46e3-878b-1b88e58d76cf] for service = with service.id [16]"; + + System.out.println("line String :: " + lineString); + data =3D parser.parse(lineString); + Assert.assertNotNull("parser result is null", data); + Assert.assertEquals("parser size result is incorrect",1, data.size= ()); + + Assert.assertTrue("parser value is not a string",data.get("line") = instanceof String); + + // Integer type test + String lineInteger =3D "512"; + + System.out.println("line Integer :: " + lineInteger); + data =3D parser.parse(lineInteger); + Assert.assertNotNull("parser result is null", data); + Assert.assertEquals("parser size result is incorrect", 1, data.siz= e()); + + Assert.assertTrue("parser value is not an integer",data.get("line"= ) instanceof Integer); + + // Long type test + String lineLong =3D "9223372036854775806"; + + System.out.println("line Long :: " + lineLong); + data =3D parser.parse(lineLong); + Assert.assertNotNull("parser result is null", data); + Assert.assertEquals("parser size result is incorrect", 1, data.siz= e()); + + Assert.assertTrue("parser value is not a long",data.get("line") in= stanceof Long); + } +} diff --git a/collector/file/pom.xml b/collector/file/pom.xml index 37ab46a..6a253b9 100644 --- a/collector/file/pom.xml +++ b/collector/file/pom.xml @@ -43,6 +43,10 @@ org.apache.karaf.decanter.collector org.apache.karaf.decanter.collector.utils + + org.apache.karaf.decanter + org.apache.karaf.decanter.api + =20 diff --git a/collector/file/src/main/java/org/apache/karaf/decanter/collect= or/file/DecanterTailerListener.java b/collector/file/src/main/java/org/apac= he/karaf/decanter/collector/file/DecanterTailerListener.java index 0894b81..040953d 100644 --- a/collector/file/src/main/java/org/apache/karaf/decanter/collector/file= /DecanterTailerListener.java +++ b/collector/file/src/main/java/org/apache/karaf/decanter/collector/file= /DecanterTailerListener.java @@ -25,6 +25,7 @@ =20 import org.apache.commons.io.input.Tailer; import org.apache.commons.io.input.TailerListenerAdapter; +import org.apache.karaf.decanter.api.parser.Parser; import org.apache.karaf.decanter.collector.utils.PropertiesPreparator; import org.osgi.service.cm.ConfigurationException; import org.osgi.service.component.ComponentContext; @@ -48,6 +49,9 @@ @Reference public EventAdmin dispatcher; =20 + @Reference + public Parser parser; + private final static Logger LOGGER =3D LoggerFactory.getLogger(Decante= rTailerListener.class); =20 private String type; @@ -95,17 +99,16 @@ public void handle(String line) { data.put("path", path); data.put("regex", regex); =20 - // TODO: try some line parsing if (regex !=3D null) { Pattern pattern =3D Pattern.compile(regex); Matcher matcher =3D pattern.matcher(line); if (matcher.matches()) { - data.put("line", line); + data.putAll(this.parser.parse(line)); } else { return; } } else { - data.put("line", line); + data.putAll(this.parser.parse(line)); } =20 try { diff --git a/manual/src/main/asciidoc/user-guide/collectors.adoc b/manual/s= rc/main/asciidoc/user-guide/collectors.adoc index 3b03541..81304ba 100644 --- a/manual/src/main/asciidoc/user-guide/collectors.adoc +++ b/manual/src/main/asciidoc/user-guide/collectors.adoc @@ -149,6 +149,16 @@ my=3Dstuff =20 The file collector will tail on `karaf.log` file, and send any new line ma= tching the `regex` in this log file as collected data. =20 +By default, the collector use the `org.apache.karaf.decanter.impl.parser.I= dentityParser` parser to parse the line into +a typed Object (Long, Integer or String) before send it to the EventDispat= cher data map. + +You can write your own parser by implementing the `org.apache.karaf.decant= er.api.parser.Parser` interface and declare +it into the file collector configuration file: + +---- +parser.target=3D(parserId=3DmyParser) +---- + =3D=3D=3D=3D EventAdmin =20 The Decanter EventAdmin Collector is an event-driven collector, listening = for all internal events happening in =20 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. =20 For queries about this service, please contact Infrastructure at: users@infra.apache.org > Be able to define a line parser service in the file collector > ------------------------------------------------------------- > > Key: KARAF-5888 > URL: https://issues.apache.org/jira/browse/KARAF-5888 > Project: Karaf > Issue Type: Improvement > Components: decanter > Reporter: Jean-Baptiste Onofr=C3=A9 > Assignee: Francois Papon > Priority: Major > Fix For: decanter-2.1.0 > > > Now the file collector uses a tail that takes any new lines appended in a= file and send the "raw" line to the dispatcher. It would be great to be ab= le to use a line parser in the collector that send the extracted property f= rom the line. -- This message was sent by Atlassian JIRA (v7.6.3#76005)