Is Python's logging.getLogger scalable with many named loggers? -


i'm using logging.getlogging() in way not forbidden documentation not directly referred to.

my applications process data files , network connections, in threads. in order identify log lines each connection and/or data file, following:

data_file_name = "data_file_123.xml" logger = logging.getlogger(data_file_name) logger.info("this logged.")  2013-07-22 05:58:55,721 - data_file_123.xml - info - logged. 

this works well:

  • avoids passing logger instance around software.
  • ensures every line marked appropriate source identifier without having manually perform in each logging call.

my concern documentation @ http://docs.python.org/2/library/logging.html#logging.getlogger:

all calls function given name return same logger instance. means logger instances never need passed between different parts of application.

how the logger instances destroyed? destroyed? after processing million files there million named logger instances waiting in memory used? setting myself memory leak memory fills these old logging instances?

how the logger instances destroyed? destroyed? after processing million files there million named logger instances waiting in memory used? setting myself memory leak memory fills these old logging instances?

they aren't destroyed until interpreter exits. instances cached since behaviour want when logging. after processing million files there will 1 million logger instances alive.

as stated using logging module not part of aim of module, hence suboptimal solution.

there isn't public api rid of cached loggers, although can clear cache doing:

>>> root = logging.getlogger() >>> root.manager.loggerdict.clear() 

the loggerdict or manager attributes aren't described in public documentation, although aren't explicitly marked _private.

instead of having different logger each file processed i'd use different logger each thread, , insert name of file in required logging messages. can write simple function logging avoids having insert explicitly filename in every call logger.


Comments

Popular posts from this blog

html5 - What is breaking my page when printing? -

c# - must be a non-abstract type with a public parameterless constructor in redis -

ajax - PHP/JSON Login script (Twitter style) not setting sessions -