Welcome to talisker’s documentation!¶
Contents:
Talisker - an opinionated WSGI app platform¶
Talisker is an enhanced runtime for your WSGI application that aims to provide a common operational platform for your python microservices.
It integrates with many standard python libraries to give you out-of-the-box logging, metrics, error reporting, status urls and more.
Python version support¶
Talisker 0.20.0 was the last to support Python 2.7. Talisker version >=0.21.0 only supports Python 3.5, 3.6, 3.8 and 3.10, as they come with Ubuntu LTS releases.
Quick Start¶
Simply install Talisker with Gunicorn via pip:
pip install talisker[gunicorn]
And then run your WSGI app with Talisker (as if it was regular gunicorn).:
talisker.gunicorn app:wsgi -c config.py ...
This gives you 80% of the benefits of Talisker: structured logging, metrics, sentry error handling, standardised status endpoints and more.
Note: right now, Talisker has extensive support for running with Gunicorn, with more WSGI server support planned.
Elevator Pitch¶
Talisker integrates and configures standard python libraries into a single tool, useful in both development and production. It provides:
structured logging for stdlib logging module (with grok filter)
gunicorn as a wsgi runner
request id tracing
standard status endpoints
statsd/prometheus metrics for incoming/outgoing http requests and more.
deep sentry integration
It also optionally supports the same level of logging/metrics/sentry integration for:
celery workers
general python scripts, like cron jobs or management tasks.
Talisker is opinionated, and designed to be simple to use. As such, it is not currently very configurable. However, PR’s are very welcome!
For more information, see The Documentation, which should be found at:
Overview¶
Goals¶
Talisker is designed with the following high level goals:
to provide a standard platform for running wsgi apps
to support both operational and developer workflows in the same tool
to be easy as possible to integrate with any wsgi app and framework
The original motivation was to standardise tooling across a number of different wsgi services, and improve the status quo along the way. Some of the inspiration came from Finagle, Twitter’s service framework for the JVM, particularly the operational standardisation it provides.
In particular, we wanted to standardise how we monitor applications in production, including logging, metrics, errors and alerts. Talisker provides a single library to do this for all our apps, that our operations tooling can configure easily, and means the application doesn’t need to care about it.
Also we wanted to provide best practice features for applications to use. So we added support for structured logging, integrated with the python stdlib logging. This allows developers to add custom tags to their logs, as well as add operational tags. We also provide easy to use helpers for best practice usage of things like statsd clients and requests sessions, which were used inconsistently across our projects, with repeated performance problems.
FAQ¶
Some questions that have actually been asked, if not particularly frequently.
Why does talisker use a custom entry point? Wouldn’t it be better to just be some WSGI middleware?
There are 3 reasons for using a talisker specific entry point
to configure stdlib logging early, before any loggers are created
to allow for easy configuration of gunicorn for logging, statsd and other things.
to do things like automatically wrap your wsgi in app in some simple standard middleware, to provide request id tracing and other things.
If it was just middleware, logging would get configured too late for gunicorn’s logs to be affected, and you would need to add explicit middleware and config to your app and its gunicorn config. Doing it as an alternate entry point means you literally just switch out gunicorn for talisker, and you are good to go.
Why just gunicorn? Why not twistd, or waitress, etc?
Simply because we use gunicorn currently. Integrating with other wsgi application runners is totally possible, twistd support is in the works, with uwsgi support on the road map.
Why is it called talisker?
‘WSGI’ sort of sounds like ‘whisky’ if you say it quick. One of my favourite whiskies is Talisker, I’ve even visited the distillery on the Isle of Skye. Also, Talisker is a heavily peated malt whisky, which is not to everyone’s taste, which seemed to fit thematically with a WSGI runtime that is also very opinionated and probably not to everyone’s taste. Also, it has 8 characters just like gunicorn, and it wasn’t taken on PyPI.
Development using Talisker¶
Talisker has been designed with the goal of working in development and production. This is to try and encourage the same tool used throughout.
Talisker’s default configuration is designed for production usage, e.g.:
only INFO level logs and above go to stderr
python’s warning system is disabled
DEVEL Mode¶
If the DEVEL env var is set, Talisker will run in DEVEL mode.
What this means varies on which tool you are using, but at a base level it enables python’s warning logs, as you generally want these in development.
For Gunicorn, DEVEL mode means a few more things:
sets timeout to 99999, to avoid timeouts when debugging
it enables auto reloading on code changes
Also, for developer convenience, if you manually set Gunicorn’s debug level to DEBUG, when in DEVEL mode, Talisker will actually log debug level messages to stderr.
Development Logging¶
Talisker logs have been designed to be readable in development.
This includes:
preserving the common first 4 fields in python logging for developer familiarity.
tags are rendered most-specific to least specific. This means that the tags a developer is interested in are likely first.
if stderr is an interactive tty, then logs are colorized, to aid human reading.
Colored Output¶
If in DEVEL mode, and stdout is a tty device, then Talisker will colorise log output.
To disable this, you can set the env var:
TALISKER_COLOR=no
or
TERM=dumb
The colorscheme looks best on dark terminal backgrounds, but should be readable on light terminals too.
If your terminal doesn’t support bold, dim, or italic text formatting, it might look unpleasent. In that case, you can try the simpler colors
TALISKER_COLOR=simple
Request Context¶
Talisker uses an implicit context to track requests during execution. It does this via the contextvars module from the Python standard library in Python 3.7+, falling back to the contextvars backport from PyPI. It also includes a minimal backport of ContextVar for use with Python 2.
Talisker creates a new context for each WSGI request or Celery job, and tracks the request id and other data in that context, such as timeout data. Asyncio is supported, either by the native support in python 3.7, or via the aiocontextvars package, which you can install by using the asyncio extra:
pip install talisker[asyncio]
Note: you need at least python 3.5.3+ to use asyncio with contextvars - aiocontextvars does not work on earlier versions.
Talisker also explicitly supports contexts when using the Gevent or Eventlet Gunicorn workers, by swapping the thread local storage out for the relative greenlet based storage. This support currently does not work in python 3.7 or above, as it is not possible to switch the underlying storage in the stdlib version of the contextvars library.
Request Id¶
One of the key elements of the context is to track the current request id. This id can be supplied via a the X-Request-Id header, or else a uuid4 is used.
This id is automatically attached to all log messages emitted during the request, as well as the detailed log message talisker emits for the request itself.
Talisker also support propagating this request id wherever possible. When using Talisker’s requests support, the current request id will be included in the outgoing request headers. When queuing celery jobs, the current request id will be passed as a header for that job, and then used by the job for all log messages when the job runs.
This allows deep tracing of a particular request id across multiple services boundaries, which is key to debugging complex issues in distributed systems.
Context API¶
Talisker exposes a public API for the current context.
from talisker import Context
Context.request_id # get/set current request id
Context.clear() # clear the current context
Context.new() # create a new context
# you can also add extras to the current logging context
Context.logging.push(foo=1)
# or
with Context.logging(bar=2):
...
Testing with Talisker¶
Talisker provides various tools to assist in unit testing application that use it’s features.
Test suite integration¶
In order to run your tests in the same configuration as production, you should initialise Talisker for testing as early as possible in your test suite:
talisker.testing.configure_testing()
This will set up Talisker logging with a logging.NullHandler on the root logger, as well as configure the global sentry client to point to dummy remote to capture sentry messages.
Talisker uses some module globals, thread locals, and request contexts to store state. For unit tests, it is a good idea to ensure this state is cleared between tests, or else tests can not be properly isolated. To do so, ensure the following is run before every test:
talisker.testing.clear_all()
Test Helpers¶
Talisker provides a context manager for testing that will capture every log message, sentry report and statsd metric generated while it is active. It produces a test context that can be used to assert against these artefacts:
with talisker.testing.TestContext() as ctx:
# code under test
The context object collects events that happened while active, and presents them for inspection:
# ctx.statsd is an array of statsd metrics as strings
self.assertEqual(ctx.statsd[0], 'some.metric:1.000000|ms')
# ctx.sentry is an array of sentry messages sent, as the JSON dict that was
# sent by the sentry client
self.assertEqual(ctx.sentry[0]['message'] == 'my message')
# ctx.logs is a talisker.testing.LogRecordList, which is essentially a list
# of logging.LogRecords
self.assertTrue(ctx.logs[0].msg == 'my msg')
Asserting against log messages is not simple, expecially with extra dicts, so the context provides some helpers.
For the most common cases of checking that something was logged:
# ctx.assert_log will assert that a log message exists, with a helpful
# AssertionError message if it does not
ctx.assert_log(
name='app.logger',
level='info',
msg='my msg',
extra={'foo': 'bar'},
)
The context.logs attribute also provides additional APIs: filter(), exists() and find():
my_logs = ctx.logs.filter(name='app.logger')
self.assertEqual(len(my_logs) == 2)
self.assertTrue(my_logs.exists(
level='info',
msg='my msg',
extra={'foo': 'bar'},
))
warning = my_logs.find(level='warning')
self.assertIn('baz', warning.extra)
These APIs will search logs that match the supplied keyword arguments, using the keyword to look up the attribute on the logging.LogRecord instances. A full list of such attributes can be found here:
https://docs.python.org/3/library/logging.html#logrecord-attributes
For all these APIs, the following applies:
The ‘level’ keyword can be a case insensitive string or an int (e.g. ‘info’ or logging.INFO), and the appropriate LogRecord attribute (levelname or levelno) will be used.
The ‘msg’ keyword is compared against the raw message. The ‘message’ keyword is compared against the interpolated msg % args.
Matching for strings is contains, not equality, i.e. needle in haystack, not just needle == haystack.
The extra dict is special cased: each supplied extra key/value is checked against the LogRecord.extra dict that Talisker adds.
assert_log(…) raises an AssertionError if the log does not exist. The error message includes how many matches each supplied term independently matched, to help narrow down issues.
filter(…) returns a LogRecordList of matching logs, so is chainable.
find(…) returns the first LogRecord found, or None.
exists(…) returns True if a matching LogRecord is found, else False
Configuration¶
Talisker can be configured by the environment variables listed below. These variables can also be supplied in a python file, although environment variables override any file configuration.
TALISKER_CONFIG¶
A path to a python file containing configuration variables.
Note: this will only have effect when set via environment variable, for obvious reasons.
DEVEL¶
Allows coloured logs, warnings, and other development convieniences.
DEVEL mode enables coloured log output, enables python warnings and, for gunicorn, it sets longer timeouts and auto reloads by default.
TALISKER_COLOUR¶
Controls the coloured output of logs. Defaults to on if stdin is a tty. Can be set to: 0 (off), 1 (on), or ‘simple’ for a simpler colourscheme. Requires DEVEL mode to be enabled.
Can also be disabled with TERM=dumb env var.
DEBUGLOG¶
Path to write debug level logs to, which is enabled if path is writable.
Debug logs are rotated every 24h to limit size, with only a single 24 hour archive is kept.
TALISKER_SLOWQUERY_THRESHOLD¶
Set the threshold (in ms) over which SQL queries will be logged. Defaults to -1 (off). The queries are sanitised, and thus safe to ship to a log aggregator.
Setting to 0 will log every query, which can be useful in development. The queries are sanitized by not including the bind parameter values.
TALISKER_EXPLAIN_SQL¶
Include EXPLAIN plans in sql sentry breadcrumbs. Defaults to false.
When enabled, this will issue extra queries to the db when generating sentry reports. The EXPLAIN queries will be quick to execute, as it doesn’t actually run the query, but still, caution is advised.
TALISKER_SOFT_REQUEST_TIMEOUT¶
Set the threshold (in ms) over which WSGI requests will report a soft time out to sentry. Defaults to -1 (off).
A soft timeout is simply a warning-level sentry report for the request. The aim is to provide early warning and context for when things exceed some limit.
Note: this can be set on a per-endpoint basis using the talisker.request_timeout decorator.
TALISKER_REQUEST_TIMEOUT¶
Set a deadline for all requests. Any network requests that talisker suports (requests, psycopg2) will have their timeouts set to this deadline.
Note: this can be set on a per-endpoint basis using the talisker.request_timeout decorator.
TALISKER_LOGSTATUS¶
Sets whether http requests to /_status/ endpoints are logged in the access log or not. Defaults to false.
These log lines can add a lot of noise, so they are turned off by default.
TALISKER_NETWORKS¶
Sets additional CIDR networks that are allowed to access restricted /_status/ endpoints. Comma separated list.
This protection is very basic, and can be easily spoofed by X-Forwarded-For headers. You should ensure that your HTTP front end server is configuring these correctly before passing on to gunicorn.
TALISKER_STATUS_INTERFACE¶
Which network interface to respond to /_status requests on.
TALISKER_REVISION_ID¶
Sets the explicit revision of the application. If not set, a best effort detection of VCS revision is used.
This is used to tag sentry reports, as well as returned as a header and from /_status/ping.
The default lookup will try find a version-info.txt file, or git, hg, or bzr revno, and finally a setup.py version.
TALISKER_UNIT¶
Sets the instance name for use with sentry reports.
TALISKER_ENV¶
Sets the deployed environment for use with sentry reports (e.g. production, staging).
TALISKER_DOMAIN¶
Sets the site domain name for use with sentry reports.
STATSD_DSN¶
Sets the Statsd DSN string, in the form: udp://host:port/my.prefix
You can also add the querystring parameter ?maxudpsize=N, to change from the default of 512.
SENTRY_DSN¶
Sets the sentry DSN, as per usual sentry client configuration.
See the sentry DSN documentation for more details.
TALISKER_SANITISE_KEYS¶
Comma separated list of keys with sensitive information. Data in these keys is masked in logging and sentry reports.
Note that sentry’s default set of password keys are already included - this is for extra keys.
TALISKER_ID_HEADER¶
Header containing request id. Defaults to X-Request-Id.
TALISKER_DEADLINE_HEADER¶
Header for request deadline. Defaults to X-Request-Deadline.
Logging¶
Talisker configures a specific carefully-designed logging set up. At a high level, it configures the following with the stdlib logging module:
logger class that can collect structured data
formatter that supports structured data via logfmt
root handler that logs everything at INFO level to stderr
Talisker also provides additional debugging options that can be used for more information.
The stdlib logging module is old, based on a Java API, and makes heavy use of global state. Some of the implementation details for talisker are work arounds for these limitations. But stdlib logging is everywhere, and is actually very flexible, so we can make it work how we want to.
Configuring Logging¶
If you use Talisker’s entry points, like talisker.gunicorn or talisker.celery, then logging will be configured by default.
You can also use the more generic entrypoints that talisker provides to run any script with Talisker logging enabled.
talisker.run script.py ...
or
python -m talisker script.py ...
If you are using your own entry point, you can configure Talisker with:
import talisker
talisker.initialise()
Note that this should be done before any loggers are created via logging.getLogger() or similar, as they will not be altered.
This will set up logging, taking into account environment variables like DEBUGLOG. If you want to configure those parameters explcitly, you can do:
config = {'devel': True, 'debuglog': '/path'}
talisker.logs.configure(config)
For testing, you can use special test configuration, which sets up talisker style logging, except adds a single NullHandler to the root logger:
talisker.logs.configure_test_logging()
This means logging will work as per talisker’s set up, but you will get no log output. You can use the helpers in talisker.testing to help testing log messages.
Logger Class¶
Talisker sets a custom base logger class via logging.setLoggerClass(). Its only difference from logger.Logger is that it supports more explicitly storing ‘extra’ arguments to the log call. This allows the StructuredFormatter class to append an arbitrary number of flags to the formatted message. Without this, there is no way to know which fields of a LogRecord are supposed to be added as tags.
It also supports pulling in additional extra data from the current context, which is primarily used for providing request_id data for the log message.
Root Handler¶
Talisker simply adds a handler to the root logger to log to stderr, at the INFO log level.
Simple, self-contained application, no log file config
No file permissions needed by app
System handles buffering, synchronisation, persistence, rotation and shipping
Works in development
PaaS friendly
Root Logger Limitations¶
If you log messages against the root logger object (i.e. logging.getLogger()), then Talisker’s structured logging enhancements will not work. This is due to the fact that the stdlib logging module instanciates the root logger object at import, and has no facility for safely switching it to be another instance or class. For libraries, it is not recommended practice to use the root logger, you should be specific about your library’s top level logger anyway.
Debug Logging¶
Talisker also supports adding an additional root handler that logs to disk at DEBUG level. The stderr logging output is unchanged.
To enable, just set the DEBUGLOG envvar to the path you want the log file to go to:
DEBUGLOG=/path/to/logfile talisker ...
If talisker can open that file, it will add a handler to log to it at DEBUG level, and log a message at the start of your log output to say it is doing do. If it cannot open that file, it will log a message saying so, but not fail. The handler is a TimedRotatingFileHandler, set to 24 hour period with no backup copies, i.e. logs last for 24 hours at most.
This is designed to support development and production use cases.
In development, typically usage of DEBUG logs is by grepping a file, rather than viewing in the console, given the verbosity. So we write to disk where the developer has told us to, and they can grep/view the file there.
In production, operators sometimes want to turn on more logging for limited period, to debug a specific problem. But we generally don’t want to ship that extra logging. This is in part due to scaling - debug logs can be 10x more verbose than INFO, this could lead to a 10x traffic spike on your log aggregation service. Additionally, debug logs often include details that are sensitive, and that you don’t want stored centrally. So this mechanism of writing to a temporary log file helps in that scenario too, as the INFO logging on stderr that is shipped is unchanged.
Log Format¶
Talisker uses a default format that is designed to be human readable in development, but still structured for richer data.
The talisker logging format is as follows:
format = '%(asctime)s.%(msecs)03dZ %(levelname)s %(name)s "%(message)s"'
datefmt = "%Y-%m-%d %H:%M:%S"
which should look like this:
2016-07-14 01:02:03.456Z INFO app "hello"
This provides:
the default data python logging usually has
a more ISOish timestamp (uses . for msecs rather than , but we omit the T for readability)
explicit UTC timestamps (logging module uses local time by default /o)
explicitly quoted message (embedded ” are escaped)
Talisker can also append an arbitrary number of ‘tags’ on the end of the log line, following the logfmt idea. e.g.:
2016-07-14 01:02:03.456Z INFO app "hello" foo=bar baz="some value"
These extra tags can be specified in 2 main ways:
By the developer at the call site:
logger.info('something happened', extra={'foo': 'bar', 'context': 'I can haz it'}) would output:: 2016-01-13 10:24:07.357Z INFO app "something happened" foo=bar, svc.context="I can haz it"For a specific context, e.g. for a request. Talisker uses this to add request_id to every log message for a specific request. e.g.:
logger.info('something happened')would output:
2016-01-13 10:24:07.357Z INFO app "something happened" request_id=<request id>You can add your own temporary context variables with a context manager:
with talisker.logs.logging_context(foo="bar"): logger.info('my important message')would output:
2016-01-13 10:24:07.357Z INFO app "my important message" foo=bar
Additionally, it would be expected that your log shipper should add additional tags, like hostname or service group, to the logfmt tags when shipping.
If there are any global or context keys, these will take precedence if there is a collision with developer supplied keys. The developer keys will be suffixed with a ‘_’ to preserve the info, with out stomping on the other keys.
Log Suppression¶
By default, talisker suppresses some loggers.
The python python py.warnings logger is set not to propagate, as these are just noise in production.
Additionally, talisker also configures the ‘requests’ logger to WARNING level. This is because the INFO level on requests is particularly verbose, and we use requests everywhere.
If you prefer to have full requests logs, you can simply set the level yourself.
e.g.:
logging.getLogger('requests').setLevel(logging.INFO)
Additional logging configuration¶
Talisker just sets a root handler with formatter. You are free to add your own additional loggers and handlers as needed via the normal methods, if you need to.
You can still benefit from the structured logging provided by talisker if you set your handler’s formatter to be an instance of talisker.logs.StructuredFormatter. This is a standard formatter, except it uses UTC for the time and adds the logfmt tags on the end. The default format is as specified in Log Format.
For example, suppose you want to enable debug logs for django’s db logger.
e.g:
handler = logging.FileHandler('db.log')
handler.setFormatter(talisker.logs.StructuredFormatter())
handler.setLevel(logging.DEBUG)
db = logging.getLogger('django.db.backends')
db.setLevel(logging.DEBUG)
db.setHandler(handler)
Gunicorn Logs and Configuration¶
When run via talisker.gunicorn, Gunicorn’s error logs use talisker’s logging.
Access logs are disabled by default, and are usually not needed when using Talisker, as it logs full structured log per request, which is a superset of the information in access logs. You can however still enable and configure gunicorn’s access logs as well if you wish to.
Talisker overrides some config options for gunicorn, mainly to do with logging. It issues warnings if the user specifies any of these configs, as they will not be applied. Specifically, the following gunicorn config items are ignored by the talisker.gunicorn runner:
–error-logfile, as talisker logs everything to stderr
–logger-class, talisker uses its custom class
–statsd-host and –statsd-port, as talisker uses the STATSD_DSN env var.
If you run talisker.gunicorn in devel mode, and specify –log-level=debug, it will output debug logs to the console.
Grok filters¶
Talisker includes a filter and patterns for parsing the logformat into logstash with grok. These are in the talisker/logstash/ directory of the source tree. They are also included in the python package as resources.
Gunicorn¶
Basic Usage¶
Gunicorn is the wsgi server used by Talisker. To use it, simply use the Talisker wrapper in place of regular gunicorn script.
$ talisker.gunicorn -c config.py myapp:wsgi_app
This wrapper simply initialises Talisker before passing control to Gunicorn’s entrypoint. As such, it takes exactly the same command line arguments, and behaves in the same way.
Talisker supports the sync, gevent, and eventlet workers. Others workers may work, but have not been tested. The only place it matters is in the context-local storage of Talsiker log tags and request ids. Talisker will use greenlet based contexts if it finds itself running in a greenlet context, or else a thread local object.
Python 3.6, Async and Requests¶
Due to changes in the SSL api in python 3.6, requests currently has a bug with https endpoints in monkeypatched async context. The details are at https://github.com/requests/requests/issues/3752, but basically the monkeypatching must be done before requests is imported. Normally, this would not affect gunicorn, as your app would only import requests in a worker process after the monkeypatch has been applied. However, because Talisker enables some integrations in the main process, before the gunicorn code is run, it triggers this bug. Specfically, we import the raven library to get early error handling, and raven imports requests.
We provide two special entrypoints to work around this problem, if you are using python 3.6 and eventlet or gevent workers with the requests library. They simply apply the appropriate monkeypatching first, before then just initialising talikser and running gunicorn as normal.
$ talisker.gunicorn.eventlet --worker-class eventlet -c config.py myapp:wsgi_app
$ talisker.gunicorn.gevent --worker-class gevent -c config.py myapp:wsgi_app
Sentry¶
Talisker provides out-of-the-box integration with sentry.
Specifically, Talisker adds:
some default configuration of the sentry client
handle WSGI errors
sentry error log handler (for logged exception messages)
log message, sql and http call breadcrumbs
sentry integration with flask, django, and celery
To get the current sentry client, simply use:
talisker.sentry.get_client()
Error Data¶
Talisker configures sentry breadcrumbs for log messages at INFO or higher level. It also adds the request id as a tag to the sentry message.
If you want to add some custom error context, you can use the client above as you would use the sentry client as normal. For example:
client = talisker.sentry.get_client()
client.context.merge({'tags': my_tags})
Sentry Configuration¶
Talisker uses the default SENTRY_DSN env var to configure sentry by default. Simply setting this will enable sentry for wsgi and logging.
In addition, Talisker configures the sentry client by default as follows:
sets install_logging_hook=False, as Talisker handles it
sets release to the current revision
sets hook_libraries=[], disabling breadcrumbs for request/httplib
sets environment to TALISKER_ENV envvar
sets name to TALISKER_UNIT envvar
sets site to TALISKER_DOMAIN envvar
ensures the RemovePostDataProcessor, SanitizePasswordsProcessor, and RemoveStackLocalsProcessor processors are always included, to be safe by default.
If you are using Talisker’s Flask or Django integration, you can configure your sentry client further via the usual config methods for those frameworks.
If you wish to manually configure the sentry client, use the following:
talisker.sentry.configure_client(**config)
This will reconfigure and reset the sentry client used by the wsgi middleware and logging handler that Talisker sets up.
If you want to set your own client instance, do:
talisker.sentry.set_client(client)
Whichever way you wish to configure sentry, talisker will honour your configuration except for 2 things
install_logging_hook will always be set to false, or else you’ll get duplicate exceptions logged to sentry.
the processors will always include the base 3, although you can add more. If you really need to remove the default processors, you can modify the list at talisker.sentry.default_processors and then call talisker.sentry.set_client().
Status Endpoints¶
Talisker provides a set of app-agnostic standard endpoints for your app for querying its status. This is designed so that in production you have standard ways to investigate problems, and to configure load balancer health checks and nagios checks.
/_status/pingA simple check designed for use with haproxy’s httpcheck option, returns 200, responds to GET, HEAD, or OPTIONS, the body content being the application’s revision
/_status/checkFor use with nagios check_http plugin, or similar.
It tries to hit
/_status/checkin your app. If that is not found, it just returns a 200, as a basic proxy for the application being up./_status/test/sentry(/_status/errorfor backwards compatibility)Raise a test error, designed to test sentry/raven integration.
/_status/test/statsdSend a test metric value. Designed to test statsd integration.
/_status/test/prometheusIncrement a test counter. Designed to test Prometheus integration.
/_status/metricsExposes prometheus metrics in Prometheus text format.
/_status/info/configShows the current Talisker configuration
/_status/info/packagesShows a list of installed python packages and their versions
/_status/info/workersShows a summary of master and worker processes (e.g CPU, memory, fd count) and other process information. Only available if psutil is installed.
/_status/info/objgraphShows the most common python objects in user for the worker that services the request. Only available if objgraph is installed.
/_status/info/logtreeDisplays the stdlib logging configuration using logging_tree. Only available if logging_tree is installed.
Request Timeouts¶
Talisker supports the idea of a request deadline, with the goal of failing early, especially when under load. This deadline can be specified as a timeout, either globaly or per-endpoint.
Talisker will try to use the remaining time left until the deadline as network timeout parameters. It supports HTTP and SQL requests out of the box, if you use talisker.requests.TaliskerAdapter and talisker.postgresql.TaliskerConnecton, respectivley. It also provides an API to get the remaining time left before the deadline, which you can use in other network operations.
timeout = Context.deadline_timeout()
Note: this will raise talisker.DeadlineExceeded if the deadline has been exceeded.
Talisker timeouts are not hard guarantees - Talisker will not cancel your request. They merely try to ensure that network operations will fail earlier rather than blocking for long periods.
Deadline Propagation¶
The deadline can be set via a the X-Request-Deadline request header, as an ISO 8601 datestring. This will override the configured endpoint deadline, if any. Talisker’s requests support will also send the current deadline as a header in any outgoing requests. This allows API gateway services to communicate top-level request deadlines ina calls to other services.
Configuring Timeouts¶
You can set a global timeout via the TALISKER_REQUEST_TIMEOUT config, or per endpoint with the talisker.request_timeout decorator.
@talisker.request_timeout(3000) # milliseconds
def view(request):
...
Soft Timeouts¶
Talisker supports the concept of a soft_timeout, which will send a sentry report if a request takes longer than the soft timeout threshold. This is useful to provide richer information for problematic requests.
You can set this global via the TALISKER_SOFT_REQUEST_TIMEOUT config or per endpoint via the talisker.request_timeout decorator.
@talisker.request_timeout(soft_timeout=3000) # milliseconds
def view(request):
...
Requests¶
Enhanced session¶
Talisker provides a way to upgrade a request.Session instance with a few extra features.
Firstly, the X-Request-Id header will be added to the outgoing request headers. This can be used by other services to track the originating request id. We usually append incoming request id to one generated for that request, e.g.:
X-Request-Id: <generated request id>-<incoming request id>.
This allows simple searches to uncover all related sub requests for a specific request, also known as fanout.
Secondly, we also collect metrics for outgoing requests. Specifically:
counter for all requests, broken down by host and view
counter for errors, broken down by host, type (http 5xx or connection error), view and error code (either POSIX error code or http status code, depending on type)
histogram for duration of http responses
In statsd, they would be named like so:
<prefix>.requests.count.<host>.<view>
<prefix>.requests.errors.<host>.<type>.<view>.<code>
<prefix>.requests.timeouts.<host>.<view>
<prefix>.requests.latency.<host>.<view>.<status>
Note: a view here is a human friendly name for the api/endpoint. If the upstream service returns an X-View-Name header in its response (e.g. is another talisker service), or if the user has given this call a name (see below), then this will be used.
You can customise the name of this metric if you wish, with some keyword arguments:
session.post(..., metric_api_name='myapi', metric_host_name='myservice')
will use these values in the resulting naming, in both prometheus and statsd.:
<prefix>.requests.count.myservice.myapi...
Session lifecycle¶
We found many of our services were not using session objects properly, often creating/destroying them per-request, thus not benefiting from the default connection pooling provided by requests. This is especially painful for latency when your upstream services are https, as nearly all ours are. But sessions are not thread-safe (see this issue for details), sadly, so a global session is risky.
So, talisker helps by providing a simple way to have thread local sessions.
Using a talisker session¶
To get a base requests.Session thread local session with metrics and request id tracing::
session = talisker.requests.get_session()
or use the wsgi environ:
session = environ['requests']
If you wish to use a custom subclass of Session rather than the default requests.Session, just pass the session class as an argument. Talisker will ensure there is one instance of this session subclass per thread.:
session = talisker.requests.get_session(MyCustomSessionClass)
This works because talisker does not subclass Session to add metrics or requests id tracing. Instead, it adds a response hook to the session object for metrics, and decorates the send method to inject the header (ugh, but I couldn’t find a better way).
If you wish to use talisker’s enhancements, but not the lifecycle management, you can do:
session = MySession()
talisker.requests.configure(session)
and session will now have metrics and id tracing.
Statsd¶
Talisker provides statsd integration and configuration.
Configuration¶
Statsd can be configured by the STATSD_DSN envvar, patterned after the SENTRY_DSN. This combines all statsd config into a single DSN url. For example:
.. code-block:: bash
# talk udp on port 1234 to host statsd, using a prefix of ‘my.prefix’ STATSD_DSN=udp://statsd:1234/my.prefix
# can also use / for prefix separators, the / converted to . STATSD_DSN=udp://statsd:1234/my/prefix
# ipv6 STATSD_DSN=udp6://statsd:1234/my.prefix
# custom max udp size of 1024 STATSD_DSN=udp://statsd:1234/my.prefix?maxudpsize=1024
Currently, only the udp statsd client is supported. If no config is provided, a dummy client is used that does nothing.
TODO: contribute this to upstream statsd module
Integration¶
If statsd is configured, talisker will configure gunicorn’s statsd functionality to use it. Additionally, it will enable statsd metrics for talisker’s requests sessions.
Your app code can get a statsd client by simply calling::
statsd = talisker.statsd.get_client()
Prometheus¶
Talisker provides optional prometheus_client integration and configuration.
Installation¶
The package supports extras args to install prometheus_client:
$ pip install talisker[prometheus]
Configuration¶
prometheus_client integration has extensive support for
multiprocessing with gunicorn.
If you are only using one worker process, then regular single process mode is used.
However, if you have multiple workers, then the
prometheus_multiproc_dir envvar is set to a tmpdir, as per
the prometheus_client multiprocessing docs.
This allows any worker being scraped to report metrics for all workers.
However, by default it leaks mmaped files when workers are killed, wasting disk space and slowing down metric collection. Talisker provides a non-trivial workaround for this, by having the gunicorn master merge left over metrics into a single file.
Note that in multiprocss mode, due to prometheus_client’s design, all registered metrics are exposed, regardless of registry.
The metrics are exposed at /_status/metrics
Celery¶
Talisker provides some optional integration with celery. If your project does not have celery installed, this will not be used.
To run a Celery worker with Talisker, use the provided Celery entrypoint:
$ talisker.celery worker -A myapp
Logging¶
Talisker configures Celery logging to use Talisker’s logging mechanisms. It ensures that the task_id and task_name will be logged with every log message for a Celery job.
Additionally, if the job is triggered by a Talisker process (e.g. a Talisker gunicorn worker) it will add the request_id to the logging tags for the celery job when it executes. This allows you to track tasks initiated by a specific request id.
Metrics¶
Talisker will enable basic celery task metrics by default.
Talisker sets up histogram metrics for
celery.<task_name>.enqueue (time to publish to queue)
celery.<task_name>.queue (time in queue)
celery.<task_name>.run (time to run task)
And counters for
celery.<task_name>.count (total tasks)
celery.<task_name>.retry (number of retried tasks)
celery.<task_name>.success (number of successful tasks)
celery.<task_name>.failure (number of failed tasks)
celery.<task_name>.revoked (number of revoked tasks)
Note: talisker supports celery>=3.1.0. If you need to be sure, the package supports extras args to install celery dependencies:
$ pip install talisker[celery]
Sentry¶
Talisker integrates Sentry with Celery, so Celery exceptions will be reported to Sentry. This uses the standard support in raven for integrating Celery and Sentry.
Flask¶
Usage¶
Talisker provides some opt-in support for flask apps. This does a few main things currently.
enable sentry flask support for your app. This means you will get more information in your sentry errors, as well as being able to configure sentry via your app config as normal.
disable flask default app logger configuration, and just use talisker’s configuration. This avoids double-logged exception messages.
Add X-View-Name header to each response, which helps give extra logging and metric info.
To enable, you can either use a special Talisker flask app:
app = talisker.flask.TaliskerApp(__name__)
or register your app with Talisker afterwards:
talisker.flask.register(app)
Sentry Details¶
Talisker integrates the flask support in raven.contrib.flask. See the raven flask documentation for more details.
The sentry flask extension is configured to work with talisker.
logging=False as Talisker has already set this up. This means the other possible logging config is ignored.
wrap_wsgi=False as Talisker has already set this up
register_signal=True, which is the default
If for some reason you wish to configure the flask sentry extension yourself:
talisker.flask.sentry(app, **config)
This has the same api as the default raven.contrib.flask.Sentry object, but with the above configuration.
Django¶
Talisker provides opt-in support for Django apps.
Logging¶
To integrate with Talisker, you should at a minimum disable Django’s default logging in your settings.py:
LOGGING_CONFIG = None
If you don’t, you’ll get Django’s default logging configuration in addition to Talisker’s, leading to some duplicated logs in development, and possibly emails of errors in production, which is often not a good idea.
If you have custom logging you want to add on top of Talisker’s, you can follow the Django documentation for configuring logging yourself, with something like:
LOGGING = {...}
LOGGING_CONFIG = None
import logging.config
logging.config.dictConfig(LOGGING)
which is exactly what Django does, but without the default logging.
Sentry¶
To integrate with Talisker’s sentry support, add raven to INSTALLED_APPS as normal, and also set SENTRY_CLIENT in your settings.py:
INSTALLED_APPS = [
'raven.contrib.django.raven_compat',
...
]
SENTRY_CLIENT = 'talisker.django.SentryClient'
This will ensure the extra info Talsiker adds to Sentry messages will be applied, and also that the WSGI and logging handlers will use your Sentry configuration in settings.py. It will also set install_sql_hook=False, as that leaks raw SQL to the sentry server for every query. This will hopefully be addressed in a future release.
Metadata¶
Talisker supports the use of an X-View-Name header for better introspection. This is used for metric and logging information, to help debugging.
To support this in Django, simply add the following middleware, in any order:
MIDDLEWARE = [
...
'talisker.django.middleware',
]
Management Tasks¶
If you use management tasks, and want them to run with Talisker logging, you can use the generic talisker runner:
talisker.run manage.py ...
or
python -m talisker manage.py ...
Postgresql¶
Talisker provides some optional integration with postgresql via pyscopg2, in the form of a custom connection/cursor implementation that integrates with logging and with sentry breadcrumbs.
Ensure you have the correct dependencies by using the pg extra:
pip install talisker[pg]
To use it in Django:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
...,
'OPTIONS': {
'connection_factory': talisker.postgresql.TaliskerConnection
}
}
}
To use with sqlalchemy:
engine = sqlalchemy.create_engine(
...,
connect_args={'connection_factory': talisker.postgresql.TaliskerConnection},
)
Query Security¶
Given the security sensitive nature of raw sql queries, Talisker is very cautious about what it might communicate externally, either via logs or via sentry.
Talisker will only ever log the query string with placeholders, and never the query parameters. This avoids leakage of sensitive information altogether, while still providing enough info to be useful to users trying to debug problems. If a query does not use query parameter, the query string is not sent, as there is no way to determine if it is sensitive or not.
One exception to this is stored procedures with parameters. The only access to the query is via the raw query that was actually run, which has already merged the query parameters, so we never send the raw query.
Note: in the future, we plan to add support for customised query sanitizing.
Slow query Logging¶
The connection logs slow queries to the talisker.slowqueries logger. The default timeout is -1, which disables slow query logging, but can be controlled with the TALISKER_SLOWQUERY_THRESHOLD env var. The value is measured in milliseconds:
export TALISKER_SLOWQUERY_THRESHOLD=100 # log queries over 100ms
Sentry Breadcrumbs¶
Talisker will capture all queries run during a request or other context as Sentry breadcrumbs. In the case of an error, they will include them in the report to sentry.
Contributing¶
Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.
Note: this project is copyrighted by Canonical, and the policy is that contributers must sign the CLA in order to contribute. More details here: https://www.ubuntu.com/legal/contributors
You can contribute in many ways:
Types of Contributions¶
Report Bugs¶
Report bugs at https://github.com/canonical-ols/talisker/issues.
If you are reporting a bug, please include:
Your operating system name and version.
Any details about your local setup that might be helpful in troubleshooting.
Detailed steps to reproduce the bug.
Fix Bugs¶
Look through the GitHub issues for bugs. Anything tagged with “bug” is open to whoever wants to implement it.
Implement Features¶
Look through the GitHub issues for features. Anything tagged with “feature” is open to whoever wants to implement it.
Write Documentation¶
talisker could always use more documentation, whether as part of the official talisker docs, in docstrings, or even on the web in blog posts, articles, and such.
Submit Feedback¶
The best way to send feedback is to file an issue at https://github.com/canonical-ols/talisker/issues.
If you are proposing a feature:
Explain in detail how it would work.
Keep the scope as narrow as possible, to make it easier to implement.
Remember that this is a volunteer-driven project, and that contributions are welcome :)
Get Started!¶
Ready to contribute? Here’s how to set up talisker for local development.
Fork the talisker repo on GitHub.
Clone your fork locally:
$ git clone git@github.com:your_name_here/talisker.git
run make with no arguments. This will create a virtualenv env in .env and install dependencies and run tests.
Create a branch for local development:
$ git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
When you’re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:
$ make # runs python3 tests and lint $ make tox # runs all python tests and lint
Commit your changes and push your branch to GitHub:
$ git add . $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature
Submit a pull request through the GitHub website.
Pull Request Guidelines¶
Before you submit a pull request, check that it meets these guidelines:
The pull request should include tests.
If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
The pull request should work for Python 2.7, 3.4 and 3.5, and for PyPy. Check https://travis-ci.org/canonical-ols/talisker/pull_requests and make sure that the tests pass for all supported Python versions.
Credits¶
Simon Davy <simon.davy@canonical.com>
Maximiliano Bertacchini <maximiliano.bertacchini@canonical.com>
Adam Collard <adam.collard@canonical.com>
Tom Wardill <tom.wardill@canonical.com>
Guillermo Gonzalez <guillermo.gonzalez@canonical.com>
Robin Winslow <robin.winslow@canonical.com>
Wouter van Bommel <wouter.bommel@canonical.com>
Release History¶
0.21.3 (2023-01-11)¶
Fix spelling of Python 3.5-specific requirements
Drop the future library
0.21.2 (2022-10-07)¶
Add support for python 3.10
Reduce test complexity by only supporting python versions that come with an ubuntu lts
Add support for newer Werkzeug
Pin jinja2 & markupsafe to sensible defaults for doc testing
0.21.1 (2021-11-10)¶
Add a fix to compare LogRecords that don’t contain the extra field i.e caused by log entries generated before talisker is initalized.
0.21.0 (2021-09-24)¶
Drop python 2.7
Support more django versions
Support more Celery versions
Add tests for python 3.9 and local tests for python 3.10
Remove travis tests
Remove the pg_wheel extra target
0.20.0 (2021-09-21)¶
Added support for python 3.7 and 3.8
Added support for celery => 5
Send more details (view-name) to sentry, for better grouping
Fix util.Local sorage thread safety, and include unittest
Last version to support python 2.7
0.19.0 (2020-09-07)¶
Fix proxy config application to backends in TalikerAdapter (#534)
Require contexts to be explicitly created. (#543)
Note: this change means that there is no default context. This fixes a number of issues with context existing in non-request/job code (e.g. startup code, gunicorn master process).
0.18.0 (2020-04-17)¶
Add support for Python 3.8 and gevent 1.5.0 (#515)
support latest prometheus-client, and raise minimum supported version to 0.5 (#516)
fix missing rogue import (#517)
Fix unbound local variable on request timeout. (#512)
0.17.0 (2020-04-08)¶
Fix the sdist builds so that they include setup.py /o
Sadly, many previous sdists are broken, and PyPI does not allow reuploading, so there is no real fix :(
0.16.1 (2020-04-02)¶
bumped supported version of werkzeug to <=1.2
0.16.0 (2020-03-04)¶
support gunicorn 20.x (#502)
Add EXPLAIN output to sql sentry breadcrumbs (#499)
Support X-Debug header that sends sentry report (#498)
Better gunicorn timeout handling. (#497)
Defer sentry reports until after request complete. (#496)
Sentry logging breadcrumb fixes (#493)
refactor wsgi request logging, so that we will have more information available (#492)
0.15.0 (2019-07-25)¶
support latest flask and werkzeug releases (#472)
Support per-request deadlines in network stack (#469)
Add TALISKER_STATUS_INTERFACE config to restrict bind to interface
Use contextvars for request context
0.14.3 (2019-04-24)¶
Fix lazy responses that have no content (#447)
Fix importing issues with talisker.run (#442)
0.14.2 (2019-03-29)¶
Change requests.TaliskerAdapter to require scheme (#435)
Do not assume requests Connection error always wraps a urllib3.Retry error. (#434)
0.14.1 (2019-03-21)¶
Fix no content responses by still trying to call start_response() early (#431)
0.14.0 (2019-03-18)¶
Make raven an optional dependency (#429)
properly support super lazy WSGI apps, like werkzeug’s debugger
0.13.0 (2019-03-08)¶
Make gunicorn an optional dependency
Fix use of werkzeug debugger
0.12.0 (2019-02-22)¶
This release includes a fair amount of internal refactoring, and some new features.
Logs and metrics for timeouts in gunicorn (#412)
Errors are now handled in WSGI, and return text/html/json per Accept header (#409)
Soft timeouts are sent after the request is finished (#411)
Add requests.TaliskerAdapter, a requests transport adapter with loadbalancing and retries (experimental, #405, #408)
Move logging from gunicorn into WSGI middleware (#402)
Don’t use raven’s WSGI middleware, do it in our middleware (#406)
All /_status/info/ endpoints support rendering JSON as well as text and HTML (#407)
support rendering json responses as well as text and html (#407)
move the request-id header name to config (#401)
fix requests latency buckets (#413)
0.11.1 (2019-01-09)¶
add additional sanitised keys by default, and add config for additional keys. Requires raven 6.4 (#392)
fix parsing of extra[‘start_time’] being a string in recent raven (#390)
0.11.0 (2018-12-18)¶
This release fixes an accidental hard dependency on celery that was added in 0.10.0.
The main feature is refactor of Talisker configuration, which now allows for using a config file as well as environment variables. It also provides consistant documentation for all configuration in a single place, and a new talisker.help cli tool, and a new /_status/info/config endpoint for introspection.
Add documentation for config, including new talisker.help command (#386)
Ensure optional extras are actually optional (#385)
Improve config error handling and logging of errors. (#382)
Add centralised config, with config file support (#380)
0.10.2 (2018-11-29)¶
Add companion TestContext.assert_not_log to match assert_log
0.10.1 (2018-11-26)¶
Add assert_log helper to TestContext (#377)
Fixed where some classes where accidentally old-style classes in python 2
0.10.0 (2018-11-20)¶
Add a public testing helper (testing.talisker.TestContext), to isolate and collect talisker data in tests (#329)
Expose new talisker.testing.clear_all() api to make it easier to reset talisker state in (#328)
Add a timeout for prometheus locks, and asupport non-blocking locking in gevent (#358)
Do not enable prometheus multiprocess mode if there is only one gunicorn worker (#358)
Remove username/email from any user data in sentry (#342)
Add a relative-to-request start timestamp to all sentry breadcrumbs. (#340)
Support prometheus 0.4+ (#334)
0.9.16 (2018-10-02)¶
Improve the ip->name mapping for requests metric names (#304)
improve sentry report metadata, add tags and also sql statement summary (#306)
Track per-request sql/http/log costs (#325)
add explicit context clears at various points (#305)
TERM=dumb turns off terminal colors
0.9.15 (2018-09-12)¶
Support statsd 3.3 (#290)
Properly serialize the prometheus worker cleanup (#296)
Gracefully fail if lock can’t be initialised (#303)
0.9.14 (2018-08-21)¶
A collection of minor improvements and fixes, and relicensing to Apache 2.
Relicense to Apache-2 (#258, #275) (thanks to Adam Collard)
Aggregate prometheus metrics into one file on worker death. (#281, #288)
Wrapping logging and metrics code with try/except, so that we never fail (#276)
Fail more gracefully in the case where talisker has not been setup properly (#264)
Add support for TALISKER_REVISION_ID variable (#262) (thanks to Robin Winslow and Adam Collard)
Return sentry id from capture (#261)
Set level=warning in soft req. timeout sentry messages (#255) (thanks to Guillermo Gonzalez)
0.9.13 (2018-07-04)¶
Fix X-Request-Id to be a native string on py2 (#247)
0.9.12 (2018-06-20)¶
only set prometheus envvar in gunicorn runner, not for all runners (#242)
0.9.11 (2018-06-18)¶
Fix a regression that broke prometheus metrics in multiprocess environments.
0.9.10 (2018-06-07)¶
Change default slow query threshold to -1 even in DEVEL (#226)
Move the clearing of various request contexts as late as possible (#233)
add soft request timeout support (#231) - thanks to Guillermo Gonzalez
support raven 6.9.0 (#232) - thanks to Guillermo Gonzalez
0.9.9 (2018-05-21)¶
This release collects a number of bugfixes
Make parsing the postgres dsn more robust (#224)
Protect django middleware view introspection (#220) (thanks to tomwardill)
Ensure that configured sentry processors is a set (#219) (thanks to tomwardill)
Fix link escaping in objgraph page (#217)
0.9.8 (2018-05-10)¶
The main feature in this release is new endpoints for debugging (#213):
/_status/info/packages: show a list of installed python packages and versions
/_status/info/workers: show resource usage of gunicorn workers, and general process infomation [requires psutil]
/_status/info/objgraph: show python object counts and potential leaks [requires objgraph]
These endpoints are IP restricted to TALISKER_NETWORKS, and can render as text via curl or html via browser.
Renamed all structured logging fields called “duration” to “duration_ms” to indicate units (#215)
Unknown /_status/ urls are passed through to app, to allow for 404 format control by the app (#212)
We only quote logfmt stings if needed, reduces visual noise on log lines (#173)
DEVEL colorscheme improved to support light terminals, and a simpler option that doesn’t use italics or bold (#188)
log request size/type in gunicorn logs (#174)
Added support for flask 1.0 and prometheus client 0.2, make django 1.10 the minimum supported version (#209, #198)
0.9.7 (2018-03-28)¶
The main work in this release has been improvments to metrics.
Collect prometheus metrics as well as statsd for gunicorn, requests, and celery (#172)
Support flask/django adding X-View-Name header to indicate view function (#151)
Control over naming requests metrics (#171)
Gunicorn logging enhancements (#165)
Gather better metadata from OSError exceptions
Fixed some small logging issues
0.9.6 (2018-02-21)¶
The logfmt output has been reworked to explictly quote strings, and test coverage much improved in the process. This allows for more robust parsing in logstash, such as allowing numeric fields.
New talisker.testing module, which has helpers for functionally testing talisker servers and related talisker tools.
Added a functional test suite using the new talisker.testing helpers
Custom ruby logstash filter to handle parsing of numeric values and escaped quotes.
0.9.5 (2017-05-23)¶
add support for postgresql via psycopg2 (#85). This will add breadcrumbs to sentry reports, as slow query logs. See http://talisker.readthedocs.io/en/latest/postgresql.html for more info
Access log cleanups (#94). We no longer include the querystring in the logmsg, just as a field.
Improved proxy handling in private endpoints. (#92). Fixes X-Forwarder-For handling.
Clear sentry context on request start (#90). This stops some breadcrumbs bleeding between requests.
Fix sentry default config when used with flask (#89). This was causing release tag to not be applied to sentry reports.
truncate long log messages (#86). This prevents DOSing logstash.
0.9.4 (2017-04-25)¶
require explicit DEVEL env var to enable colored output.
Add ability to force switch colors off with TALISKER_COLOR=no
Fix bug in grok filter to allow _ in logger name
Drop log tags that are empty, as logstash can’t cope with them
Truncate very long log messages and tags (at 10k/2k respectively) to avoid accidental DOS.
0.9.3 (2017-04-13)¶
Fix gunicorn logger metrics and logging, adding tests (#75)
0.9.2 (2017-04-11)¶
Bug fix release
Fix celery metrics with eager tasks (#70)
Fix statsd cli args and metric format (#71)
Also fix depencecies on recent setuptools
0.9.1 (2017-03-23)¶
This release has a couple of important bugfixes, upgrading is strongly encouraged.
Feature: Add a generic script runner to run any python script with talisker logging, primary usecase is django managment commands:
talisker.run myscript.py …
Improvement: DEVEL env var is no longer required (although still respected). Talisker will assume DEVEL mode when stderr is a tty.
Bugfix: re-add http metrics for gunicorn which were accidentaly dropped in a refactor, with regression tests
Bugfix: fix celery integration with 3.1.13+, with regression tests
Bugfix: Add missing request_id to new accesslogs
Bugfix: Fix issue #35, respect –log-level for gunicorn in DEVEL mode. This means you can do –log-devel=debug and get debug level logging to your console.
Improvement: support raven 6
Testing: now testing against pypy in CI, and also agains the minimum supported versions of various dependencies too, to help prevent further accidental dependencies on latest version apis (which is what broke celery 3.1.x integration)
0.9.0 (2017-01-24)¶
The major feature in this release is support for sentry, which is integrated with wsgi, logging, and celery. Also supports opt-in integration with flask and django, see the relevant docs for more info.
Other changes
refactor of how logging contexts were implemented. More flexible and reliable. Note talisker.logs.extra_logging and talisker.logs.set_logging_context are now deprecated, you should use talisker.logs.logging_context and talisker.logs.logging_context.push, respectively, as covered in the updated logging docs.
improved celery logging, tasks logs now have task_id and task_name automatically added to their logs.
improved logging messages when parsing TALISKER_NETWORKS at startup
0.8.0 (2016-12-13)¶
prometheus: add optinal support for promethues_client
celery: request id automatically sent and logged, and support for 4.0
docs: initial ‘talisker contract’
statsd: better client initialisation
internal: refactoring of global variables, better /_status/ url dispatch
0.7.1 (2016-11-09)¶
remove use of future’s import hooks, as they mess with raven’s vendored imports
slight tweak to logfmt serialisation, and update docs to match
0.7.0 (2016-11-03)¶
Upgrading
This release includes a couple of minor backwards incompatible changes:
access logs now use the talisker format, rather than CLF. See the docs for more info. If you are using access logs already, then the easiest upgrade path is to output the access logs to stderr (access_logfile=”-“), and delete your old log files.
talisker no longer prefixes developer supplied tags with ‘svc.’. This should only matter if you’ve already set up dashboards or similar with the old prefixed name, and you will need to remove the prefix
Changes:
access logs now in logfmt rather than CLF
dummy statsd client is now useful in testing
logs are colored in development, to aid reading
the ‘svc’ prefix for tags has been removed
0.6.7 (2016-10-05)¶
actually include the encoding fix for check endpoint
0.6.6 (2016-10-05)¶
add celery metrics
fix issue with encoding in check endpoint when iterable
0.6.5 (2016-09-26)¶
make celery runner actually work, wrt logging
0.6.4 (2016-09-23)¶
fix encoding issue with X-Request-Id header (again!)
0.6.3 (2016-09-21)¶
fix setuptools entry points, which were typoed into oblivion.
0.6.2 (2016-09-21)¶
make gunicorn use proper statsd client
log some extra warnings if we try to configure gunicorn things that talisker overides.
better documented public api via __all__
first take on some celery helpers
some packaging improvements
0.6.1 (2016-09-12)¶
actually do remove old DEBUGLOG backups, as backupCount=0 does not remove any. Of course.
0.6.0 (2016-09-09)¶
Propagate gunicorn.error log, and remove its default handler.
This allows consistant logging, making the choice in all cases that your gunicorn logs go to the same stream as your other application log, making the choice in all cases that your gunicorn logs go to the same stream as your other application logs.
We issue a warning if the user tries to configure errorlog manually, as it won’t work as expected.