drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <prog...@mapr.com>
Subject Re: Dynamically change logging levels for loggers
Date Sat, 26 Aug 2017 20:23:09 GMT
See the “LogFixture” class for how I got this to work (for tests) using LogBack. If we
combine the LogBack details, with the prior Apex framework, we might have most of what we
need.

BTW: another huge help would be a way to “poke” Drill and have it dump the state of some
of its internal data structures, such as the memory allocator, the list of running fragments,
etc. We’ve seen cases where, after Drill runs for a long time, “something bad” happens.
But, it is hard to see that internal state to figure out what’s what.

Thanks,

- Paul

> On Aug 25, 2017, at 11:22 PM, Vlad Rozov <vrozov@apache.org> wrote:
> 
> +1. Even though it was done for log4j in Apache Apex, I am pretty sure that the same
can be done for logback. The only thing to consider is that all such functionality is specific
to a logging provider in use. I am quite familiar with how it was done in Apex and can help
if necessary.
> 
> Thank you,
> 
> Vlad
> 
> On 8/25/17 14:22, Timothy Farkas wrote:
>> +1 for exploring adding this feature. We had a feature to dynamically change log
levels at runtime through the rest API in Apache Apex and it was very helpful with debugging
things.
>> 
>> ________________________________
>> From: Paul Rogers <progers@mapr.com>
>> Sent: Friday, August 25, 2017 11:01:29 AM
>> To: dev@drill.apache.org
>> Subject: Re: Dynamically change logging levels for loggers
>> 
>> Hi Kunal,
>> 
>> Don’t know about rereading the config file, but I have had luck in the unit test
framework with adjusting log levels programmatically. (Tests turn on interesting log levels
for the duration of a single tests.) We might be able to use that capability (provided by
Logback) to make adjustments at run time.
>> 
>> - Paul
>> 
>>> On Aug 25, 2017, at 10:55 AM, Kunal Khatua <kkhatua@mapr.com> wrote:
>>> 
>>> I figured this is a rarely modified piece of code but most frequently used across
all components. Hoping that someone who might have worked on logging can share some insight
from their experience in general, if not within Drill.
>>> 
>>> I was wondering if changes to Drill's logback.xml can be picked up dynamically.
>>> 
>>> i.e. without restarting the Drillbit, change the logging level of specific classes
within the Drillbit.
>>> 
>>> I ask this because sometimes, a Drillbit needs to go through a warmup phase where
the JVM optimizes the functions frequently in use. Changing the logging from something like
an INFO to a DEBUG level would then allow me to correctly capture specific log messages without
having to lose all those optimizations due to a restart (for the DEBUG to take effect).
>>> 
>>> Is it something worth having ?
>>> 
>>> ~ Kunal
>> 
> 
> 
> Thank you,
> 
> Vlad

Mime
View raw message