From dev-return-13382-archive-asf-public=cust-asf.ponee.io@karaf.apache.org Fri Jan 18 06:19:02 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 07F4B18063F for ; Fri, 18 Jan 2019 06:19:01 +0100 (CET) Received: (qmail 43518 invoked by uid 500); 18 Jan 2019 05:19:01 -0000 Mailing-List: contact dev-help@karaf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@karaf.apache.org Delivered-To: mailing list dev@karaf.apache.org Received: (qmail 43505 invoked by uid 99); 18 Jan 2019 05:19:00 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Jan 2019 05:19:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D7E4418063B for ; Fri, 18 Jan 2019 05:18:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.3 X-Spam-Level: X-Spam-Status: No, score=0.3 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id fzny16p3vTiw for ; Fri, 18 Jan 2019 05:18:58 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 8A6935F23C for ; Fri, 18 Jan 2019 05:18:57 +0000 (UTC) X-Originating-IP: 82.238.224.4 Received: from [192.168.134.110] (bre91-1-82-238-224-4.fbx.proxad.net [82.238.224.4]) (Authenticated sender: jb@nanthrax.net) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 4459F40003 for ; Fri, 18 Jan 2019 05:18:56 +0000 (UTC) Subject: Re: SCR Bundle refreshes Pax Logging? To: dev@karaf.apache.org References: <162d9e32-1c0a-3f0e-b6c4-4e1810bb2068@nanthrax.net> <1655208d-a9ff-ba86-ea0b-81ce843897a3@nanthrax.net> From: =?UTF-8?Q?Jean-Baptiste_Onofr=c3=a9?= Message-ID: Date: Fri, 18 Jan 2019 06:18:55 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit A simple hack is to actually install the bundle causing the refresh as boot feature or startup.properties. If the optional imports are already resolved/provided, the refresh won't happen. In your case, just install commons-csv (via startup.properties for instance). Regards JB On 17/01/2019 21:12, Fabian Lange wrote: > Is there a hack with which I can prevent the bundle from refreshing? I > can of course monkeypatch the manifest :) > > Fabian > > On Thu, Jan 17, 2019 at 9:06 PM Jean-Baptiste Onofré wrote: >> >> Yes, the purpose was to support extra appenders easily. >> >> An alternative to optional import would be to use fragment but it's less >> flexible. A discover/locator service would be easier indeed. >> >> Regards >> JB >> >> On 17/01/2019 19:46, Grzegorz Grzybek wrote: >>> I understand. I don't remember (wasn't there when pax-logging was founded), >>> but it's about those exotic appenders you can use. >>> But in OSGi, it'd be probably better to rewrite/adjust the >>> discover/extensibility part in pax-logging-log4j2, to use our beloved TCCL >>> instead or kind of service discovery / locator. >>> >>> regards >>> Grzegorz Grzybek >>> >>> czw., 17 sty 2019 o 19:37 Fabian Lange napisał(a): >>> >>>> I will have the same problem with jackson as well ;) >>>> >>>> pax-logging-log4j2 has really broad optional imports. probably for the >>>> other formatters that can be plugged. >>>> >>>> thats really inconvenient in my case :( >>>> >>>> Fabian >>>> >>>> On Thu, Jan 17, 2019 at 7:08 PM Jean-Baptiste Onofré >>>> wrote: >>>>> >>>>> Yeah, I don't remember why pax-logging-log4j2 has this import. >>>>> >>>>> Let me check where the package could be used. >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> On 17/01/2019 18:42, Fabian Lange wrote: >>>>>> Well, that does look like a wrong dependency in pax-logging-log4j2, >>>> doesnt it? >>>>>> or can a logic for that be found ;) >>>>>> >>>>>> Fabian >>>>>> >>>>>> On Thu, Jan 17, 2019 at 6:34 PM Grzegorz Grzybek >>>> wrote: >>>>>>> >>>>>>> You don't have to find the source of WTF :) >>>>>>> >>>>>>> pax-logging-log4j2 has: Import-Package: >>>>>>> org.apache.commons.csv;resolution:=optional >>>>>>> >>>>>>> regards >>>>>>> Grzegorz Grzybek >>>>>>> >>>>>>> czw., 17 sty 2019 o 17:07 Fabian Lange >>>> napisał(a): >>>>>>> >>>>>>>> Hi, >>>>>>>> see its not a karaf problem. >>>>>>>> Grzegorz gave me really good hints off-list, and it turns out I do >>>>>>>> load a feature which contains this apache commons bundle: >>>>>>>> >>>>>>>> mvn:org.apache.commons/commons-csv/1.5 >>>>>>>> >>>>>>>> after I remove this bundle, the logging classes are no longer loaded >>>> twice. >>>>>>>> Now the next step is to find out WTF, but I leave that for another >>>> day >>>>>>>> >>>>>>>> Fabian >>>>>>>> >>>>>>>> On Thu, Jan 17, 2019 at 3:49 PM Grzegorz Grzybek < >>>> gr.grzybek@gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hello >>>>>>>>> >>>>>>>>> I suggest checking jansi - you have SL=8 for jansi bundle and SL=7 >>>> for >>>>>>>>> pax-logging-log4j2. >>>>>>>>> >>>>>>>>> pax-logging-log4j has optional import for org.fusesource.jansi - >>>> and this >>>>>>>>> may be the cause of refresh. >>>>>>>>> >>>>>>>>> In my custom distro, my etc/startup.properties has: >>>>>>>>> >>>>>>>>> mvn\:org.fusesource.jansi/jansi/1.17 = 8 >>>>>>>>> mvn\:org.ops4j.pax.logging/pax-logging-api/1.10.1 = 8 >>>>>>>>> mvn\:org.ops4j.pax.logging/pax-logging-log4j2/1.10.1 = 8 >>>>>>>>> >>>>>>>>> So I've already ensured that jansi starts/resolves before >>>>>>>>> pax-logging-log4j2. >>>>>>>>> >>>>>>>>> I hope this helps. >>>>>>>>> >>>>>>>>> regards >>>>>>>>> Grzegorz Grzybek >>>>>>>>> >>>>>>>>> czw., 17 sty 2019 o 15:15 Jean-Baptiste Onofré >>>>>>>> napisał(a): >>>>>>>>> >>>>>>>>>> Not a problem, Jira should be used when we "suspect" something. I >>>> think >>>>>>>>>> it's good for the tracking and also for the history of faced >>>> problems. >>>>>>>>>> >>>>>>>>>> Just my €0.01 ;) >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> JB >>>>>>>>>> >>>>>>>>>> On 17/01/2019 15:12, Fabian Lange wrote: >>>>>>>>>>> I already feel bad for asking such wide questions here. I usually >>>>>>>> only >>>>>>>>>>> file tickets once I kind-of know whats going on. >>>>>>>>>>> Could be a Felix or SCR issue as well ;) >>>>>>>>>>> >>>>>>>>>>> Fabian >>>>>>>>>>> >>>>>>>>>>> On Thu, Jan 17, 2019 at 3:10 PM Jean-Baptiste Onofré < >>>>>>>> jb@nanthrax.net> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Fabian, >>>>>>>>>>>> >>>>>>>>>>>> did you create a Jira about that ? It's for the tracking as I'm >>>>>>>>>>>> preparing Karaf 4.2.3 ;) >>>>>>>>>>>> >>>>>>>>>>>> Thanks ! >>>>>>>>>>>> Regards >>>>>>>>>>>> JB >>>>>>>>>>>> >>>>>>>>>>>> On 17/01/2019 15:08, Fabian Lange wrote: >>>>>>>>>>>>> Quick update, this apparently is still the case with Karaf 4.2.2 >>>>>>>>>>>>> >>>>>>>>>>>>> Would appreciate if somebody knows a workaround. I am able to >>>> play >>>>>>>>>>>>> around with startlevels, but I cant seem to avoid this. >>>>>>>>>>>>> >>>>>>>>>>>>> Fabian >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Nov 26, 2018 at 1:21 AM Fabian Lange < >>>>>>>> lange.fabian@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> currently debugging an issue. Maybe the bits I came up so far >>>> are >>>>>>>>>>>>>> already sufficient for you guys to fix it, or you help me how >>>> to >>>>>>>> debug >>>>>>>>>>>>>> this better. >>>>>>>>>>>>>> In our distribution, we have these features >>>>>>>>>>>>>> >>>>>>>>>>>>>> 0 │ Active │ 0 │ 5.6.10 │ System Bundle, >>>> Fragments: 1 >>>>>>>>>>>>>> 1 │ Resolved │ 1 │ 4.2.1 │ Apache Karaf :: Features >>>> :: >>>>>>>>>>>>>> Extension, Hosts: 0 >>>>>>>>>>>>>> 2 │ Active │ 5 │ 2.5.4 │ OPS4J Pax Url - aether: >>>>>>>>>>>>>> 3 │ Active │ 7 │ 1.10.1 │ OPS4J Pax Logging - >>>> Log4j v2 >>>>>>>>>>>>>> 4 │ Active │ 7 │ 1.10.1 │ OPS4J Pax Logging - API >>>>>>>>>>>>>> 5 │ Active │ 8 │ 1.17.1 │ jansi >>>>>>>>>>>>>> 6 │ Active │ 9 │ 1.0.2 │ Apache Felix Coordinator >>>>>>>> Service >>>>>>>>>>>>>> 7 │ Active │ 10 │ 1.9.4 │ Apache Felix >>>> Configuration >>>>>>>>>> Admin Service >>>>>>>>>>>>>> 8 │ Active │ 11 │ 3.6.4 │ Apache Felix File Install >>>>>>>>>>>>>> 9 │ Active │ 15 │ 4.2.1 │ Apache Karaf :: Features >>>> :: >>>>>>>> Core >>>>>>>>>>>>>> 10 │ Active │ 20 │ 2.2.11.1 │ Apache ServiceMix :: >>>>>>>> Bundles :: >>>>>>>>>> jaxb-impl >>>>>>>>>>>>>> 11 │ Active │ 30 │ 1.2.0 │ Apache Felix Metatype >>>>>>>> Service >>>>>>>>>>>>>> 12 │ Active │ 30 │ 2.1.2 │ Apache Felix Declarative >>>>>>>>>> Services >>>>>>>>>>>>>> 13 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Bundle :: >>>>>>>> Core >>>>>>>>>>>>>> 14 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: >>>> ConfigAdmin >>>>>>>> :: >>>>>>>>>> Core >>>>>>>>>>>>>> 15 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Features >>>> :: >>>>>>>>>> Command >>>>>>>>>>>>>> 16 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Log :: >>>> Core >>>>>>>>>>>>>> 17 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: SCR :: >>>>>>>> Bundle >>>>>>>>>> State >>>>>>>>>>>>>> 18 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Service >>>> :: >>>>>>>> Core >>>>>>>>>>>>>> 19 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Shell :: >>>>>>>>>> Various Commands >>>>>>>>>>>>>> 20 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: Shell :: >>>>>>>> Core >>>>>>>>>>>>>> 21 │ Active │ 30 │ 4.2.1 │ Apache Karaf :: System :: >>>>>>>> Core >>>>>>>>>>>>>> 22 │ Active │ 30 │ 3.9.0 │ JLine Builtins >>>>>>>>>>>>>> 23 │ Active │ 30 │ 3.9.0 │ JLine Reader >>>>>>>>>>>>>> 24 │ Active │ 30 │ 3.9.0 │ JLine Terminal, >>>> Fragments: >>>>>>>> 25 >>>>>>>>>>>>>> 25 │ Resolved │ 30 │ 3.9.0 │ JLine JANSI Terminal, >>>>>>>> Hosts: 24 >>>>>>>>>>>>>> >>>>>>>>>>>>>> What I noticed is that A LOT of apache LOG4J classes are loaded >>>>>>>> twice >>>>>>>>>>>>>> in the JVM. >>>>>>>>>>>>>> I turned on -verbose:class and saw this snippet: >>>>>>>>>>>>>> >>>>>>>>>>>>>> [5.580s][info][class,load] >>>>>>>>>>>>>> org.apache.felix.scr.impl.logger.StdOutLogger source: >>>>>>>>>>>>>> jar:bundle://12.0:0/!/ >>>>>>>>>>>>>> [5.626s][info][class,load] >>>>>>>>>>>>>> org.apache.felix.framework.util.ImmutableMap$1 source: >>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>> file:/Users/fabian/work/karaf-dist/system/org/apache/felix/org.apache.felix.framework/5.6.10/org.apache.felix.framework-5.6.10.jar >>>>>>>>>>>>>> [5.834s][info][class,load] >>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>> org.apache.karaf.features.internal.service.BundleInstallSupportImpl$$Lambda$412/0x00000007fecd0c40 >>>>>>>>>>>>>> source: >>>>>>>>>> org.apache.karaf.features.internal.service.BundleInstallSupportImpl >>>>>>>>>>>>>> [5.834s][info][class,load] >>>>>>>>>>>>>> org.apache.felix.framework.Felix$RefreshHelper source: >>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>> file:/Users/fabian/work/karaf-dist/system/org/apache/felix/org.apache.felix.framework/5.6.10/org.apache.felix.framework-5.6.10.jar >>>>>>>>>>>>>> [5.970s][info][class,load] >>>>>>>>>>>>>> org.ops4j.pax.logging.log4j2.internal.Activator source: >>>>>>>>>>>>>> jar:bundle://3.0:0/!/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> So here is my suspicion: Whatever SCR does, it causes the >>>> Log4j2 >>>>>>>>>>>>>> bundle to reload all classes and activate again. This leads to >>>> all >>>>>>>>>>>>>> bundles before the SCR to reference the first loaded log4j >>>>>>>> classes, >>>>>>>>>>>>>> and all afterwards the refreshed bundle. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can we prevent this somehow? Also curiously SCR uses its >>>>>>>> StdOutLogger, >>>>>>>>>>>>>> which it shouldnt do. >>>>>>>>>>>>>> Is this reload caused by the Service Tracker >>>>>>>>>>>>>> org.apache.felix.scr.impl.logger.LogServiceEnabledLogger uses? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ideas, suggestions how to prevent this refresh? I played with >>>> the >>>>>>>> load >>>>>>>>>>>>>> order but it does not seem possible to get it right >>>>>>>>>>>>>> >>>>>>>>>>>>>> Fabian >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Jean-Baptiste Onofré >>>>>>>>>>>> jbonofre@apache.org >>>>>>>>>>>> http://blog.nanthrax.net >>>>>>>>>>>> Talend - http://www.talend.com >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Jean-Baptiste Onofré >>>>>>>>>> jbonofre@apache.org >>>>>>>>>> http://blog.nanthrax.net >>>>>>>>>> Talend - http://www.talend.com >>>>>>>>>> >>>>>>>> >>>>> >>>>> -- >>>>> Jean-Baptiste Onofré >>>>> jbonofre@apache.org >>>>> http://blog.nanthrax.net >>>>> Talend - http://www.talend.com >>>> >>> >> >> -- >> Jean-Baptiste Onofré >> jbonofre@apache.org >> http://blog.nanthrax.net >> Talend - http://www.talend.com -- Jean-Baptiste Onofré jbonofre@apache.org http://blog.nanthrax.net Talend - http://www.talend.com