1. A bit of logrotate config:
+ lastaction | ||
+ echo $* | sed -r 's/\.log/\.log\.1/g' | # convert .log to .log.1 - get current names of rotated files. input is original file names. | ||
+ lsof | awk '$1 ~ /python/ {print $2}' | # get ids of python (or ipython, or python3) processes that use rotated files | ||
+ sort | uniq | # as single process can write to multiple files, we make this list unique. | ||
+ xargs kill -SIGUSR1 # send a signal to processes | ||
+ endscript | ||
2. A function to add python SIGUSR1 handler, it reopens all file handlers:
+import logging | ||
+import signal | ||
+ | ||
+ | ||
+def sigusr1_react(): | ||
+ """ | ||
+ Run this function to make your process react on SIGUSR1 signal with | ||
+ reload (close and next open) of FileHandlers. Please, run it in main | ||
+ thread (see https://docs.python.org/2/library/signal.html#module-signal). | ||
+ | ||
+ It is a convenience function in case you are obliged to use FileHandler | ||
+ with rotating files. In our case it's scrapy + logrotate. | ||
+ """ | ||
+ def reload_filehandlers(signum, frame): | ||
+ handler_list = logging._handlerList | ||
+ file_handlers = [ | ||
+ x() for x in handler_list if isinstance(x(), logging.FileHandler) | ||
+ ] | ||
+ for fh in file_handlers: | ||
+ fh.stream.flush() | ||
+ fh.stream.close() | ||
+ fh.stream = None | ||
+ fh.stream = fh._open() | ||
+ | ||
+ signal.signal(signal.SIGUSR1, reload_filehandlers) |