Complete Python Logging Guide: From Beginner to Advanced

1. Complete Guide to Logging in Python | From Debugging to Production

2. What is Python logging? Basic concepts and why it’s needed

Benefits of logging

1. Easier error analysis It helps pinpoint where errors occurred and their causes.
2. Track program behavior You can see which parts are running correctly and where issues occur.
3. Enable operational monitoring It provides the foundation for continuously monitoring system health in production.

Differences from print()

• Cannot flexibly specify the output destination (e.g., files or external services).
• Cannot manage log levels (severity levels).

3. How to Start Logging with Python’s Built-in logging Module

Overview of Log Levels

• DEBUG: Debug information. Useful during development.
• INFO: Informational messages. Indicate the program’s progress.
• WARNING: Warnings. Indicate potential problems.
• ERROR: Errors. Indicate when the program is not functioning correctly.
• CRITICAL: Critical errors. Indicate system crashes.

Basic Usage

import logging

# Basic logging configuration
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

# Example output for each log level
logging.debug('This is debug information.')
logging.info('This is an info message.')
logging.warning('This is a warning message.')
logging.error('This is an error message.')
logging.critical('This is a critical error message.')

Code Explanation

• basicConfig: Configures logging output. The level argument lets you set the minimum log level to output.
• Format specification: The format argument allows you to customize the format of log messages.
• Example: %(asctime)s represents the timestamp, %(levelname)s the log level, and %(message)s the message content.

4. How to output logs to files and the console in Python

Logging to the console

import logging

logging.basicConfig(level=logging.INFO)
logging.info('Example of logging to the console')

Logging to a file

import logging

logging.basicConfig(filename='app.log', level=logging.INFO, format='%(asctime)s - %(message)s')
logging.info('This is an example of logging to a file.')

Logging to both the console and a file

import logging

# Create a logger
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)

# File handler
file_handler = logging.FileHandler('app.log')
file_handler.setLevel(logging.INFO)

# Configure the handler format
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

# Add handlers to the logger
logger.addHandler(console_handler)
logger.addHandler(file_handler)

logger.info('Example log that will be output to both the console and the file.')

5. Effective Logging Techniques in Python Projects

Consistent Logging Configuration

import logging

# Common logging configuration function
def setup_logger(name, log_file, level=logging.INFO):
    handler = logging.FileHandler(log_file)
    handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))

    logger = logging.getLogger(name)
    logger.setLevel(level)
    logger.addHandler(handler)
    return logger

# Create loggers for each module
app_logger = setup_logger('app_logger', 'app.log')
db_logger = setup_logger('db_logger', 'db.log')

# Example of log output
app_logger.info('Application log message')
db_logger.error('Database error message')

Combining Error Handling and Logging

import logging

logging.basicConfig(filename='error.log', level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s')

try:
    # Example code that raises an error
    1 / 0
except ZeroDivisionError as e:
    logging.error(f'A division-by-zero error occurred: {e}')

Including Context Information in Log Messages

import logging

logging.basicConfig(filename='context.log', level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

user_id = 12345
transaction_id = 'txn_001'

logging.info(f'User ID: {user_id}, Transaction ID: {transaction_id} - Operation succeeded.')

Proper Log Filtering

import logging

class DebugFilter(logging.Filter):
    def filter(self, record):
        return record.levelno == logging.DEBUG

logger = logging.getLogger('filtered_logger')
handler = logging.StreamHandler()
handler.addFilter(DebugFilter())

logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

logger.debug('This is a debug log.')
logger.info('This will not be displayed.')

6. Three Recommended Libraries to Enhance Python Logging

Loguru: A simple logging library for beginners

• Easy installation and setup.
• More intuitive syntax than the standard logging.

pip install loguru

from loguru import logger

logger.info('This is log output using Loguru.')
logger.error('Error messages can be logged easily, too.')

Logzero: A lightweight, easy-to-use logging library

• Easily customizable.
• Built-in log rotation.

pip install logzero

from logzero import logger

logger.info('Log output using Logzero')
logger.warning('This is a warning message.')

Structlog: A library specialized for structured logging

• Easily outputs logs in JSON format.
• Well-suited for distributed systems and cloud environments.

pip install structlog

import structlog

logger = structlog.get_logger()
logger.info('event', user='12345', action='login')

7. Resolving Questions About Python Logging!

Q1. Why aren’t logs being output?

Q2. What if the log file grows too large?

Q3. How should you choose third-party libraries?

8. Master Python logging to boost development efficiency