storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stig Rohde Døssing <stigdoess...@gmail.com>
Subject Re: Removing old logs from supervisor machine
Date Thu, 04 Apr 2019 10:38:32 GMT
I believe this is handled by the Logviewer daemon. If you run "storm
logviewer" on your worker machines, the Logviewer daemon will periodically
delete old topology directories.

Take a look at
https://github.com/apache/storm/blob/9680928a0c6e26a8a2d3e5ceb9a632579546353d/storm-webapp/src/main/java/org/apache/storm/daemon/logviewer/utils/LogCleaner.java
to see how this works.

Den tor. 4. apr. 2019 kl. 10.53 skrev Mohit Goyal <itsmohitgoyal@gmail.com>:

> Hi,
>
> I'm using log4j to log all my topology logs in apache storm. Using the
> following standard format I'm generating appender files:
>
>
>> fileName="${sys:workers.artifacts}/${sys:storm.id
>> }/${sys:worker.port}/${sys:logfile.name}"
>> filePattern="${sys:workers.artifacts}/${sys:storm.id
>> }/${sys:worker.port}/${sys:logfile.name}.%i">
>
>  Now the problem I'm facing here is that if I deploy any topology again, it'll create
a separate folder and create log files in that folder since the new topology will have different
value of *sys:storm.id <http://storm.id/>* field. This way my supervisor machine will
still have logs from older version of topology, which over the time accumulates and cause
me some memory issues. Has anyone hare faced this issue and what can be the possible solution.
One solution can be that I remove *sys:storm.id <http://storm.id/>* field from filename
but since I've several topologies running, I don't want to remove topology name from my log
file path.
>
>
> Thanks and regards,
> Mohit Goyal
>
>

Mime
View raw message