CorrelationID: Django-Python

Suppose a user request comes in and after various aspects of the request has been handled by several services. And something goes wrong!!
Even though every service would have been logging along the way, it’s not easy to find all the relevant log messages for this request amongst other log messages. That is the problem which can be easily solved by Correlation IDs.

Correlation ID, or mostly used as request-id, is a unique identifier value attached to requests or messages linked to event chains. Correlation IDs are particularly powerful when it comes to log activity. Providing a Correlation ID with a log entry provides a key by which auditing can be implemented.

Theory

The idea is simple. Whenever a user-facing service receives a request

• It will create a correlation id or request-id.
• Include it in every log message
• Pass it along the HTTP header to every other service.

All other services part of the request journey performs the same steps except Request ID generation. Additionally, Useful contextual information about the request can also be pass by using the same method.

Django-Python Implementation

Fortunately, there are many python packages available implementing above mentioned steps. For Django specifically, I have worked with one such package, django-log-request-id. It is very simple to get started with.

• Install the package

pip3 install django-log-request-id

• Add the middleware class in Django settings.py.

MIDDLEWARE = (
    # other middleware
    'log_request_id.middleware.RequestIDMiddleware',
    # other middleware
)

• Change the logging configuration in settings.py. log_request_id.filters.RequestIDFilter is needed to add request_id variable in the logs.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'request_id': {
            '()': 'log_request_id.filters.RequestIDFilter'
        },
    },
    'formatters': {
        'standard': {
            'format': '[%(request_id)s] %(asctime)s [%(levelname)s] %(name)s: %(message)s'
        },
    },
    'handlers': {
        'debug_handler': {
            'class': 'logging.handlers.RotatingFileHandler',
            'level': 'DEBUG',
            'formatter': 'standard',
            'filename': 'debug.log',
            'maxBytes': 10485760,
            'backupCount': 20,
            'encoding': 'utf8',
            'filters': ['request_id'],
        },
    },
    'loggers': {
        'underwriting': {
            'handlers': ['debug_handler']
            'filters': ['request_id'],
            'propagate': True
        },
    },
}

• Add HTTP header settings

LOG_REQUEST_ID_HEADER = 'HTTP_X_CORRELATION_ID'
GENERATE_REQUEST_ID_IF_NOT_IN_HEADER = True
REQUEST_ID_RESPONSE_HEADER = 'X-Correlation-Id'

LOG_REQUEST_ID_HEADER variable defines the incoming http headerHTTP_X_CORRELATION_ID => X-Correlation-Id. This is the http header which should contain the Request ID.
GENERATE_REQUEST_ID_IF_NOT_IN_HEADER variable will generate a new Request ID if incoming http header is null or empty.
REQUEST_ID_RESPONSE_HEADER variable automatically attaches header with Request ID in API responses.

This is the end of the setup. All the log statements will now have a Request ID attached in beginning. Happy Coding!

Any suggestions or thoughts, let me know:
Insta + Twitter + LinkedIn + Medium | Shivam Aggarwal