Add optimization option

[techtonik]

the built-in logging module isn't broken so don't reinvent the wheel

Hmm.. Do you have any performance benchmarks that prove that logging is really isn't broken in terms of performance?

It logging wasn't broken then I suppose there won't be such things as https://twistedmatrix.com/documents/15.2.1/core/howto/logger.html

Which also says that:

logging is a blocking API, and logging can be configured to block for long periods (eg. it may write to the network). No protection is provided to prevent blocking

As for don't reinvent the wheel here is a good example of "logging with superpowers". https://eliot.readthedocs.org/en/0.11.0/index.html =)

[reubano]

Hmm.. Do you have any performance benchmarks that prove that logging is really isn't broken in terms of performance?

I was referring to usability but performance is a good question. I haven't come across any complaints on the builtin loggers' performance. But if you know of any, please share.

It logging wasn't broken then I suppose there won't be such things as https://twistedmatrix.com/documents/15.2.1/core/howto/logger.html

Not a fair comparison. Twisted's logger is async whereas the builtin logger is sync. There is no builtin async logger to compare it with. I actually plan to provide support for async logging in the future. Likely I will just transparently defer to Twisted.

As for don't reinvent the wheel here is a good example of "logging with superpowers". https://eliot.readthedocs.org/en/0.11.0/index.html =)

Interesting project. The only issue I have is that it departs too far from the builtin logging API. My project aims to be fully compatible with the builtin logger, so no need to learn a completely new API.

[techtonik]

I was referring to usability but performance is a good question. I haven't come across any complaints on the builtin loggers' performance. But if you know of any, please share.

The first two relevant links of my Google query - http://dound.com/2010/02/python-logging-performance/ - 25% boost in performance after stripping logging calls. https://stackoverflow.com/questions/522730/how-can-i-strip-python-logging-calls-without-commenting-them-out

My project aims to be fully compatible with the builtin logger, so no need to learn a completely new API.

I can't say that I've learned the old API - every time I need it, I go into docs or copy/paste it from other project. But Eliot nesting makes it indeed a little bit obtrusive. I'd prefer more decoupled framework. Maybe something over https://github.com/lihaoyi/macropy

[reubano]

Your first link deals with creating expensive logging messages which is not necessarily a fault of the logging module. But it is quite common and has an easy fix: logging.Logger.isEnabledFor

if logger.isEnabledFor(logging.DEBUG):
    logger.debug('Message with %s, expensive_func())

That does bring up a good point though. It would be nice to add an option that will do this for you automatically.

The other link about tight loops is similarly answered:

Note

In some cases, isEnabledFor() can itself be more expensive than you’d like (e.g. for deeply nested loggers where an explicit level is only set high up in the logger hierarchy). In such cases (or if you want to avoid calling a method in tight loops), you can cache the result of a call to isEnabledFor() in a local or instance variable, and use that instead of calling the method each time. Such a cached value would only need to be recomputed when the logging configuration changes dynamically while the application is running (which is not all that common).

[techtonik]

But also it would be nice to have some performance benchmarks, so maybe in future macropy solution would be better.

[techtonik]

Also, a fair comparison to Twisted approach will add more points to the library as it will give people choice to either choose API that comes from standard library, or consider another approach, which in both turns allows them to write code better.

[reubano]

But also it would be nice to have some performance benchmarks, so maybe in future macropy solution would be better.

Just added a new issue to reflect this. I don' t have much experience writing perf tests so feel to send a PR :)

Also, a fair comparison to Twisted approach will add more points to the library as it will give people choice to either choose API that comes from standard library, or consider another approach, which in both turns allows them to write code better.

Agreed. When I get around to adding async, I will show how it compares to twisted. My plan is to stick to the standard API as closely as possible and transparently translate it to the async implementation. I'd also plan to make the async backend pluggable so a user could go with tornado or asyncio if he/she prefers.