Coding Best Practices

# Backend Code Process: Black Code Formatter

This document outlines the best practices for formatting Python code in our backend services using Black, the uncompromising Python code formatter.

## What is Black?

Black is a PEP 8 compliant opinionated formatter with its own style. It formats code in a consistent, deterministic way, minimizing diffs and eliminating debates about code style.

## Why Black?

- **Consistency**: Ensures uniform code style across all projects
- **Minimal diffs**: Produces smaller git diffs by using consistent formatting rules
- **Readability**: Improves code readability through standardized formatting
- **No debates**: Eliminates style discussions in code reviews
- **PEP 8 compliant**: Follows Python's style guide recommendations

## Installation

```bash
pip install black

Add to your project's development dependencies:

pip install -r requirements-dev.txt

Or in your pyproject.toml:

[tool.poetry.dev-dependencies]
black = "^25.9.0"

Basic Usage

Format a file or directory:

black {source_file_or_directory}

Alternatively:

python -m black {source_file_or_directory}

Black Code Style Key Points

Line Length

Black uses a default line length of 88 characters, which is 10% over the traditional 80 character limit. This has been found to produce significantly shorter files while maintaining readability.

Wrapping and Indentation

Black ignores previous formatting and applies uniform horizontal and vertical whitespace
One full expression or simple statement per line when possible
When expressions don't fit on a line, Black will intelligently break them with proper indentation
Closing brackets are always dedented
Trailing commas are always added to expressions split by comma where each element is on its own line

String Formatting

Black will normalize string quotes, preferring double quotes (") over single quotes (') when both would work.

Empty Lines

Black avoids spurious vertical whitespace
Single empty lines inside functions are preserved
Two blank lines between top-level definitions, one blank line between method definitions
Black enforces single empty lines between a class-level docstring and the first field or method

Comments

Black enforces two spaces between code and a comment on the same line
A space before the comment text begins
Comment contents are not formatted

Project Configuration

Create a pyproject.toml file in your project root:

[tool.black]
line-length = 88
target-version = ['py38']
include = '\.pyi?$'
extend-exclude = '''/(\n  # Exclude directories
  \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | _build
  | buck-out
  | build
  | dist
)/
'''

Integration with Other Tools

isort

For import sorting, configure isort to be compatible with Black:

[tool.isort]
profile = "black"

Flake8

Configure flake8 to work with Black:

[flake8]
max-line-length = 88
extend-ignore = E203,E501,E701

Pre-commit Hook

Add Black to your pre-commit configuration:

# .pre-commit-config.yaml
repos:
-   repo: https://github.com/psf/black
    rev: 25.9.0
    hooks:
    -   id: black

CI Integration

Add Black to your CI pipeline:

# Example GitHub Actions workflow
name: Code Quality

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: pip install black
      - name: Check formatting
        run: black --check .

Editor Integration

VS Code

Install the "Python" extension and add to settings.json:

{
    "python.formatting.provider": "black",
    "editor.formatOnSave": true,
    "python.formatting.blackArgs": ["--line-length", "88"]
}

PyCharm

Install the "BlackConnect" plugin and configure it to run Black on save.

Best Practices

Always run Black before committing code
Don't fight the formatter - if you find yourself constantly undoing Black's changes, you're doing it wrong
Use the same Black version across your team to ensure consistent formatting
Add Black to your CI pipeline to enforce consistent formatting
Configure your editor to format on save for a seamless experience

Ignoring Formatting

In rare cases where you need to preserve specific formatting:

# fmt: off
matrix = [
    1, 0, 0,
    0, 1, 0,
    0, 0, 1,
]
# fmt: on

Advanced Configuration Options

Line Length Customization

While Black defaults to 88 characters, you can customize this in your pyproject.toml:

[tool.black]
line-length = 100  # Custom line length

Target Python Versions

Specify which Python versions your code should be compatible with:

[tool.black]
target-version = ['py38', 'py39', 'py310']

Skip String Normalization

If you want to preserve string quote styles:

[tool.black]
skip-string-normalization = true  # Preserves quote style

Fast Mode

For large codebases, you can use fast mode which skips some checks:

black --fast {source_file_or_directory}

Edge Cases and Special Formatting

Magic Comments

Black respects certain magic comments to control formatting:

# fmt: off  # Disable formatting for the next block
hard_coded_array = [
    1,    2,    3,
    4,    5,    6,
]
# fmt: on  # Re-enable formatting

# fmt: skip  # Skip formatting for this line only
my_dict = {"key": "value"}  # This line won't be formatted

Handling Long Strings

For long strings, consider using parentheses for better formatting:

long_string = (
    "This is a very long string that would exceed the line length limit "
    "so we break it into multiple lines using implicit string concatenation."
)

Handling Complex Expressions

For complex expressions, Black will format them with appropriate line breaks:

# Before formatting
result = some_function_call(argument1, argument2, some_other_function_call(argument3, argument4), yet_another_call())

# After Black formatting
result = some_function_call(
    argument1,
    argument2,
    some_other_function_call(argument3, argument4),
    yet_another_call(),
)

Troubleshooting Common Issues

Handling Syntax Errors

Black requires valid Python syntax. If your code has syntax errors, Black will fail with an error message. Fix the syntax errors first before running Black.

Conflicts with Other Formatters

If you're using multiple formatters, run Black last in the chain to ensure its formatting rules are applied.

Large Files Performance

For very large files, Black might be slow. Consider using --fast mode or formatting specific files instead of the entire codebase at once.

Handling Generated Code

For generated code that shouldn't be formatted, add it to the exclude patterns:

[tool.black]
extend-exclude = '/generated/'

Black Compatibility

Python Version Compatibility

Black supports Python 3.8 and newer. The latest version (25.9.0) is tested with Python 3.8 through 3.12.

Framework-Specific Considerations

Django

Black works well with Django projects. For Django templates, consider using djhtml for formatting.

FastAPI/Flask

Black works seamlessly with FastAPI and Flask codebases without special configuration.

Performance Considerations

Caching

Black uses a cache to speed up formatting of files that haven't changed. The cache is stored in:

Linux: ~/.cache/black/
macOS: ~/Library/Caches/black/
Windows: C:\Users\<username>\AppData\Local\black\Cache\

Formatting Large Codebases

For large codebases:

Use the --quiet flag to reduce output
Consider using parallel processing: black --worker 4 .
Format only changed files in CI pipelines

# Format only files changed in the current branch compared to main
git diff --name-only main | grep -E '\.py$' | xargs black

Team Adoption Strategies

Gradual Implementation

For existing codebases:

Start small: Begin with new files or a specific module
Use check mode: Run black --check to identify files that need formatting without changing them
Incremental adoption: Format files as you modify them

Onboarding New Team Members

Include Black setup in your onboarding documentation
Provide a script to set up the development environment with Black
Make sure editor integration is part of the setup process

Handling Resistance

Common concerns and solutions:

"Black's style doesn't match our preferences": Emphasize consistency and reduced bike-shedding
"It changes too much at once": Implement gradually on changed files
"It makes git blame less useful": Use git blame --ignore-rev with a commit that applies Black to the codebase

Continuous Integration Best Practices

GitHub Actions Workflow with Caching

name: Code Quality

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          cache: 'pip'
          
      - name: Cache Black
        uses: actions/cache@v3
        with:
          path: ~/.cache/black
          key: black-cache-${{ runner.os }}-${{ hashFiles('**/pyproject.toml') }}
          
      - name: Install dependencies
        run: pip install black
        
      - name: Check formatting
        run: black --check .

Automated PR Formatting

Consider using GitHub Actions to automatically format code in PRs:

name: Format Code

on:
  pull_request:
    paths:
      - '**.py'

jobs:
  format:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          ref: ${{ github.head_ref }}
          
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          
      - name: Install Black
        run: pip install black
        
      - name: Format with Black
        run: black .
        
      - name: Commit changes
        uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: "Apply Black formatting"

Django Query Optimization Best Practices

Database Query Optimization

Use `select_related` for ForeignKey Relationships

When you need to access a related object via a ForeignKey, use select_related to fetch the related object in the same query:

# Inefficient - causes N+1 query problem
authors = Author.objects.all()  # 1 query
for author in authors:
    print(author.profile.bio)  # N additional queries, one for each author

# Efficient - only 1 query
authors = Author.objects.select_related('profile').all()
for author in authors:
    print(author.profile.bio)  # No additional queries

Use `prefetch_related` for ManyToMany Relationships

For ManyToMany relationships or reverse ForeignKey relationships, use prefetch_related:

# Inefficient - causes N+1 query problem
books = Book.objects.all()  # 1 query
for book in books:
    print([author.name for author in book.authors.all()])  # N additional queries

# Efficient - only 2 queries (one for books, one for all authors)
books = Book.objects.prefetch_related('authors').all()
for book in books:
    print([author.name for author in book.authors.all()])  # No additional queries

Use `Prefetch` Objects for Complex Prefetching

For more control over prefetched querysets:

from django.db.models import Prefetch

# Prefetch only active authors for each book
books = Book.objects.prefetch_related(
    Prefetch('authors', queryset=Author.objects.filter(active=True))
).all()

Defer and Only

Use defer() to exclude fields you don't need, or only() to specify only the fields you need:

# Fetch only what you need
authors = Author.objects.only('name', 'email').all()

# Exclude large fields you don't need
books = Book.objects.defer('content', 'cover_image').all()

Use Database-Level Aggregation

Leverage Django's aggregation functions instead of Python-level aggregation:

from django.db.models import Count, Avg, Sum

# Inefficient
books = Book.objects.all()
avg_rating = sum(book.rating for book in books) / books.count()

# Efficient
avg_rating = Book.objects.aggregate(Avg('rating'))['rating__avg']

Annotate for Calculated Fields

from django.db.models import Count, F, ExpressionWrapper, DecimalField

# Add calculated fields to your queryset
books = Book.objects.annotate(
    num_authors=Count('authors'),
    revenue=ExpressionWrapper(F('price') * F('sales'), output_field=DecimalField())
)

Use `values` or `values_list` for Simple Data

When you only need specific fields and not model instances:

# Get just the titles and publication years
book_data = Book.objects.values('title', 'pub_year')

# Get a flat list of all book titles
titles = Book.objects.values_list('title', flat=True)

Bulk Operations

Use bulk operations for creating, updating, or deleting multiple objects:

# Inefficient - N queries
for title in new_titles:
    Book.objects.create(title=title)

# Efficient - 1 query
Book.objects.bulk_create([Book(title=title) for title in new_titles])

# Bulk update
Book.objects.filter(publisher=old_publisher).update(publisher=new_publisher)

# Bulk delete
Book.objects.filter(pub_year__lt=2000).delete()

Use `exists()` Instead of `count()` or `len()`

When you only need to check if records exist:

# Inefficient - retrieves all records
if Book.objects.filter(author=author).count() > 0:
    # Do something

# Efficient - stops after finding first match
if Book.objects.filter(author=author).exists():
    # Do something

Query Optimization with `Q` Objects

For complex queries with OR conditions:

from django.db.models import Q

# Find books that match either condition
books = Book.objects.filter(
    Q(title__icontains='django') | Q(tags__name='python')
).distinct()

Database Indexes

Add indexes to fields that are frequently used in filters, ordering, or joins:

class Book(models.Model):
    title = models.CharField(max_length=200)
    pub_date = models.DateField(db_index=True)  # Add index
    
    class Meta:
        indexes = [
            models.Index(fields=['title']),
            models.Index(fields=['author', 'pub_date']),  # Composite index
        ]

Use `iterator()` for Large Querysets

When dealing with very large querysets that don't fit in memory:

# Process millions of records without loading all into memory
for book in Book.objects.iterator():
    process_book(book)

Database Connection Pooling

Use connection pooling in production with packages like django-db-connection-pool:

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'dj_db_conn_pool.backends.postgresql',
        'NAME': 'mydatabase',
        'USER': 'myuser',
        'PASSWORD': 'mypassword',
        'HOST': 'localhost',
        'PORT': '5432',
        'POOL_OPTIONS': {
            'POOL_SIZE': 20,
            'MAX_OVERFLOW': 10,
            'RECYCLE': 300,  # Recycle connections after 5 minutes
        }
    }
}

Optimize `count()` Queries

For simple count queries, use the database's COUNT function directly:

from django.db import connection

def fast_count(table_name):
    cursor = connection.cursor()
    cursor.execute(f"SELECT COUNT(*) FROM {table_name}")
    return cursor.fetchone()[0]

Use Database-Specific Features When Appropriate

from django.contrib.postgres.search import SearchVector, SearchQuery, SearchRank

# PostgreSQL full-text search
books = Book.objects.annotate(
    search=SearchVector('title', 'description'),
).filter(search=SearchQuery('python programming')).annotate(
    rank=SearchRank('search', SearchQuery('python programming'))
).order_by('-rank')

Django REST Framework Production Best Practices

Performance Optimization

Pagination

Always implement pagination for list endpoints:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 100,
}

# For specific views with custom pagination
class LargeResultsSetPagination(PageNumberPagination):
    page_size = 1000
    page_size_query_param = 'page_size'
    max_page_size = 10000

class BookViewSet(viewsets.ModelViewSet):
    queryset = Book.objects.all()
    serializer_class = BookSerializer
    pagination_class = LargeResultsSetPagination

Serializer Optimization

Use different serializers for different actions:

class BookListSerializer(serializers.ModelSerializer):
    class Meta:
        model = Book
        fields = ['id', 'title', 'author_name', 'pub_year']

class BookDetailSerializer(serializers.ModelSerializer):
    class Meta:
        model = Book
        fields = '__all__'
        depth = 1

class BookViewSet(viewsets.ModelViewSet):
    queryset = Book.objects.all()
    
    def get_serializer_class(self):
        if self.action == 'list':
            return BookListSerializer
        return BookDetailSerializer

Use `SerializerMethodField` Carefully

Avoid N+1 query problems in serializer method fields:

# Inefficient - causes N+1 query problem
class BookSerializer(serializers.ModelSerializer):
    author_books_count = serializers.SerializerMethodField()
    
    class Meta:
        model = Book
        fields = ['id', 'title', 'author', 'author_books_count']
    
    def get_author_books_count(self, obj):
        return obj.author.books.count()  # One query per book

# Efficient - annotate the queryset before serialization
class BookViewSet(viewsets.ModelViewSet):
    serializer_class = BookSerializer
    
    def get_queryset(self):
        return Book.objects.annotate(author_books_count=Count('author__books'))

class BookSerializer(serializers.ModelSerializer):
    author_books_count = serializers.IntegerField(read_only=True)
    
    class Meta:
        model = Book
        fields = ['id', 'title', 'author', 'author_books_count']

Caching

Implement caching for expensive or frequently accessed endpoints:

from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page

class BookViewSet(viewsets.ReadOnlyModelViewSet):
    queryset = Book.objects.all()
    serializer_class = BookSerializer
    
    @method_decorator(cache_page(60 * 15))  # Cache for 15 minutes
    def list(self, request, *args, **kwargs):
        return super().list(request, *args, **kwargs)

Use Throttling

Protect your API from abuse with throttling:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.AnonRateThrottle',
        'rest_framework.throttling.UserRateThrottle'
    ],
    'DEFAULT_THROTTLE_RATES': {
        'anon': '100/day',
        'user': '1000/day'
    }
}

Optimize Filtering

Use django-filter for efficient filtering:

from django_filters.rest_framework import DjangoFilterBackend

class BookViewSet(viewsets.ModelViewSet):
    queryset = Book.objects.all()
    serializer_class = BookSerializer
    filter_backends = [DjangoFilterBackend]
    filterset_fields = ['title', 'author', 'genre', 'pub_year']

Versioning

Implement API versioning to maintain backward compatibility:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
    'DEFAULT_VERSION': 'v1',
    'ALLOWED_VERSIONS': ['v1', 'v2'],
}

# urls.py
urlpatterns = [
    path('api/v1/', include('api.v1.urls')),
    path('api/v2/', include('api.v2.urls')),
]

Authentication and Permissions

Implement proper authentication and permissions:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.TokenAuthentication',
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    ],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.IsAuthenticated',
    ],
}

Content Negotiation

Support multiple content types:

REST_FRAMEWORK = {
    'DEFAULT_RENDERER_CLASSES': [
        'rest_framework.renderers.JSONRenderer',
        'rest_framework.renderers.BrowsableAPIRenderer',
    ],
    'DEFAULT_PARSER_CLASSES': [
        'rest_framework.parsers.JSONParser',
        'rest_framework.parsers.FormParser',
        'rest_framework.parsers.MultiPartParser',
    ],
}

Asynchronous Tasks

Use Celery for long-running operations:

# tasks.py
from celery import shared_task

@shared_task
def generate_report(user_id):
    # Long-running task
    pass

# views.py
class ReportViewSet(viewsets.ViewSet):
    def create(self, request):
        task = generate_report.delay(request.user.id)
        return Response({'task_id': task.id}, status=202)  # Accepted

Error Handling

Implement consistent error handling:

# exceptions.py
from rest_framework.views import exception_handler
from rest_framework.response import Response

def custom_exception_handler(exc, context):
    response = exception_handler(exc, context)
    
    if response is not None:
        response.data = {
            'error': {
                'code': response.status_code,
                'message': response.data.get('detail', str(exc)),
                'errors': response.data if 'detail' not in response.data else None
            }
        }
    
    return response

# settings.py
REST_FRAMEWORK = {
    'EXCEPTION_HANDLER': 'myapp.exceptions.custom_exception_handler',
}

Documentation

Use drf-spectacular for OpenAPI documentation:

# settings.py
INSTALLED_APPS = [
    # ...
    'drf_spectacular',
]

REST_FRAMEWORK = {
    'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
}

SPECTACULAR_SETTINGS = {
    'TITLE': 'Your API',
    'DESCRIPTION': 'Your API description',
    'VERSION': '1.0.0',
}

# urls.py
from drf_spectacular.views import SpectacularAPIView, SpectacularSwaggerView

urlpatterns = [
    # ...
    path('api/schema/', SpectacularAPIView.as_view(), name='schema'),
    path('api/docs/', SpectacularSwaggerView.as_view(url_name='schema'), name='swagger-ui'),
]

Testing

Implement comprehensive tests:

from rest_framework.test import APITestCase

class BookAPITestCase(APITestCase):
    def setUp(self):
        self.user = User.objects.create_user(username='testuser', password='password')
        self.client.force_authenticate(user=self.user)
        
    def test_list_books(self):
        response = self.client.get('/api/books/')
        self.assertEqual(response.status_code, 200)

DRY (Don't Repeat Yourself) Principles

Understanding DRY

DRY is a software development principle aimed at reducing repetition of code. The principle states: "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system."

Benefits of DRY Code

Reduced code duplication
Easier maintenance
Improved readability
Reduced bugs
Easier testing

DRY in Django Applications

Use Abstract Base Models

class TimeStampedModel(models.Model):
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    
    class Meta:
        abstract = True

class Book(TimeStampedModel):
    title = models.CharField(max_length=200)
    # No need to redefine created_at and updated_at

Create Reusable Mixins

class OwnershipMixin:
    def get_queryset(self):
        return super().get_queryset().filter(owner=self.request.user)

class BookViewSet(OwnershipMixin, viewsets.ModelViewSet):
    queryset = Book.objects.all()
    serializer_class = BookSerializer

Use Template Inheritance

<!-- base.html -->
<!DOCTYPE html>
<html>
<head>
    <title>{% block title %}Default Title{% endblock %}</title>
    {% block extra_css %}{% endblock %}
</head>
<body>
    {% block content %}{% endblock %}
    {% block extra_js %}{% endblock %}


</body>
</html>

<!-- book_list.html -->

<div data-gb-custom-block data-tag="extends" data-0='base.html'></div>

<div data-gb-custom-block data-tag="block">Book List</div>

<div data-gb-custom-block data-tag="block">
    <h1>Books</h1>
    <ul>
        {% for book in books %}
            <li>{{ book.title }}</li>
        {% endfor %}
    </ul>
</div>

Create Utility Functions

# utils.py
def generate_unique_slug(instance, slug_field, new_slug=None):
    """Generate a unique slug for a model instance."""
    if new_slug is not None:
        slug = new_slug
    else:
        slug = slugify(getattr(instance, slug_field))
    
    Klass = instance.__class__
    qs = Klass.objects.filter(slug=slug).exclude(pk=instance.pk)
    if qs.exists():
        # Slug exists, generate new unique slug
        new_slug = f"{slug}-{qs.count() + 1}"
        return generate_unique_slug(instance, slug_field, new_slug=new_slug)
    return slug

# models.py
from .utils import generate_unique_slug

class Article(models.Model):
    title = models.CharField(max_length=200)
    slug = models.SlugField(unique=True)
    
    def save(self, *args, **kwargs):
        if not self.slug:
            self.slug = generate_unique_slug(self, 'title')
        super().save(*args, **kwargs)

Use Django's Built-in Generic Views

from django.views.generic import ListView, DetailView, CreateView, UpdateView, DeleteView

class BookListView(ListView):
    model = Book
    context_object_name = 'books'

class BookDetailView(DetailView):
    model = Book

class BookCreateView(CreateView):
    model = Book
    form_class = BookForm
    success_url = reverse_lazy('book-list')

Centralize URL Patterns

# urls.py
from django.urls import path, include
from rest_framework.routers import DefaultRouter
from . import views

app_name = 'books'

router = DefaultRouter()
router.register('books', views.BookViewSet)
router.register('authors', views.AuthorViewSet)

urlpatterns = [
    path('api/', include(router.urls)),
]

Use Settings for Configuration

# settings.py
BOOK_COVER_UPLOAD_PATH = 'covers/%Y/%m/'

# models.py
from django.conf import settings

class Book(models.Model):
    cover = models.ImageField(upload_to=settings.BOOK_COVER_UPLOAD_PATH)

Create Custom Template Tags and Filters

# templatetags/book_tags.py
from django import template

register = template.Library()

@register.filter
def author_books_count(author):
    return author.books.count()

@register.inclusion_tag('books/book_list_snippet.html')
def show_latest_books(count=5):
    latest_books = Book.objects.order_by('-pub_date')[:count]
    return {'books': latest_books}

Use Signals for Cross-Cutting Concerns

# signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
from .models import Book, BookIndex

@receiver(post_save, sender=Book)
def update_book_index(sender, instance, created, **kwargs):
    """Update search index when a book is saved."""
    BookIndex.objects.update_or_create(
        book=instance,
        defaults={
            'title': instance.title,
            'content': instance.content,
            'author': instance.author.name
        }
    )

Use Form Inheritance

class BaseBookForm(forms.ModelForm):
    def clean_title(self):
        title = self.cleaned_data['title']
        if len(title) < 5:
            raise forms.ValidationError("Title must be at least 5 characters long")
        return title

class CreateBookForm(BaseBookForm):
    class Meta:
        model = Book
        fields = ['title', 'author', 'content']

class UpdateBookForm(BaseBookForm):
    class Meta:
        model = Book
        fields = ['title', 'content']

Common DRY Pitfalls

Over-DRYing

Avoid excessive abstraction that makes code harder to understand:

# Too abstract and hard to follow
class GenericProcessor:
    def process(self, obj, action, **kwargs):
        method = getattr(self, f"process_{action}")
        return method(obj, **kwargs)
    
    def process_validate(self, obj, **kwargs):
        pass
    
    def process_transform(self, obj, **kwargs):
        pass
    
    def process_save(self, obj, **kwargs):
        pass

Premature Abstraction

Wait until you see a pattern repeated at least 2-3 times before abstracting:

# Don't create abstractions for one-off use cases
# Wait until you see the pattern repeated

Balancing DRY with Other Principles

Sometimes other principles like separation of concerns are more important:

# It might be better to have some duplication if it keeps
# modules decoupled and independently maintainable

Django Models Best Practices

Naming Conventions

Model Naming

Use singular nouns: Name models with singular nouns representing a single entity (e.g., Product not Products).
Favor short names: Use concise names (e.g., Feedback instead of CustomerFeedbackSurveyResponse).
Use upper camel case: Follow Python's Pascal case naming convention (e.g., ProductCategory).
Avoid abbreviations: Use complete terms (e.g., Address not Addr).
Avoid reserved words: Don't use terms reserved by Django or Python (e.g., object, class, get).
Align with app name: Model names should align with the application name without duplicating it.

Relationship Field Naming

Use plural for ManyToManyField: Name should reflect the related model in plural form.
Use singular for OneToOneField: Name should reflect the related model in singular form.
Use descriptive names: Avoid vague or ambiguous names for relationship fields.
Use lowercase with underscores: For field names (e.g., full_name not FullName).
Be cautious with related_name: Avoid using the same name for a related_name argument and a field within one class.

Field Configuration

Choices in Models

Limit fields to predefined values: Use choices to define valid values and human-readable names.
Prefer CharField and IntegerField: These work best with choices.
Use Constants or Enums: Define values as constants or enums for maintainability.
Ensure data integrity: Choices don't enforce constraints at the database level; use additional methods.
Be careful when removing choices: Removal might cause errors in existing records.

Blank vs. Null Values

null for database storage: null=True allows NULL values in the database column.
blank for form validation: blank=True allows empty values in forms.
Prefer empty strings: For CharField/TextField, use empty strings rather than NULL values.

Meta Class Configuration

Use verbose_name and verbose_name_plural: Provide human-readable names for models.
Set default ordering: Use ordering attribute for consistent object listing.
Define custom table names: Use db_table to avoid name clashes.
Use unique_together: Enforce uniqueness across multiple columns.
Use indexes for performance: Define indexes to speed up queries.
Apply constraints: Use the constraints attribute for database-level constraints.

Database Optimization

Indexing Best Practices

Index frequently queried fields: Add indexes to fields used often in filters or joins.
Use unique=True for uniqueness: Ensures values are one-of-a-kind in the table.
Apply db_index selectively: Use for fields with a wide range of unique values.
Consider indexing overhead: Indexes add update overhead and consume disk space.
Maintain indexes regularly: Indexes can become fragmented over time.

Custom Managers

Chain query sets: Use custom managers for tailored query sets.
Declare managers in child models: Custom managers aren't inherited automatically.
Be explicit with related objects: Specify related objects when using custom managers.
Order managers carefully: Django uses the first defined manager as default.
Define default manager only when necessary: Be mindful of overriding the default.

Exception Handling

Use transaction.atomic(): Ensure database operations are executed consistently.
Log database exceptions: Capture details like exception type and traceback.
Implement middleware: Catch exceptions centrally across views.
Write comprehensive tests: Use Django's test cases to detect issues early.
Validate data before saving: Use validators to catch inconsistencies.

Performance Optimization

Counting Records

Use ModelName.objects.count(): Leverages SQL COUNT without fetching data.
Avoid len(ModelName.objects.all()): This fetches all data before counting.

ForeignKey Optimization

Use _id for direct access: book.author_id is more efficient than book.author.id.
Check existence efficiently: Use exists() instead of filter() when checking if objects exist.

Model Inheritance

Inheritance Strategies

Use abstract base classes: For shared attributes across multiple models.
Consider proxy models: To customize behavior without altering the database schema.
Limit multi-table inheritance: Use only for specialized fields or methods.

Primary Keys

Define custom primary keys: Use primary_key=True to override the default ID.
Use non-predictable IDs: Consider UUIDs for improved security.

Relationship Security

Secure Relationships

Specify on_delete behavior: Define what happens when referenced objects are deleted.
Use database transactions: Ensure all or none of the operations are committed.
Secure many-to-many tables: Set permissions to limit unauthorized access.

Django Design Philosophies

Django's design philosophy emphasizes several key principles that guide its development and usage. Understanding these principles helps developers create more effective Django applications.

Core Philosophies

Loose Coupling

Django's stack is designed with loose coupling and tight cohesion in mind. Different layers of the framework operate independently:

The template system knows nothing about web requests
The database layer knows nothing about data display
The view system is agnostic about which template system is used

This independence allows components to be replaced or modified without affecting the entire system.

Less Code

Django aims to minimize boilerplate code, leveraging Python's dynamic capabilities like introspection to reduce the amount of code developers need to write.

Quick Development

The framework is designed to make web development fast and efficient, automating repetitive tasks and providing ready-made solutions for common problems.

Don't Repeat Yourself (DRY)

Every piece of data should exist in only one place. Django follows this principle by deducing as much as possible from minimal information, reducing redundancy and improving maintainability.

Explicit is Better Than Implicit

Following Python's philosophy (PEP 20), Django avoids excessive "magic." Features should be clear and understandable, with magic used only when it creates significant convenience without confusing developers.

Consistency

The framework maintains consistency at all levels, from coding style to the overall user experience, making Django applications more predictable and easier to maintain.

Model Design Philosophy

Explicit Field Behavior

Fields shouldn't assume behaviors based solely on their names. Instead, behaviors should be defined through keyword arguments and field types to make code more predictable.

Domain Logic Encapsulation

Models should encapsulate all aspects of an "object" following Martin Fowler's Active Record pattern. Both data and metadata (like human-readable names and default ordering) should be defined in the model class.

Additional Django Models Best Practices

Application Structure

Limit Models Per App

Keep the number of models in a single app to no more than 10. If you have more, consider splitting the app into smaller, more focused apps using Domain-Driven Design principles.

Model Inheritance

Prefer Abstract Base Classes

Use abstract base classes for shared attributes across multiple models. This approach doesn't create additional database tables and provides a clean way to reuse code.

class BaseModel(models.Model):
    created = models.DateTimeField(auto_now_add=True)
    updated = models.DateTimeField(auto_now=True)
    
    class Meta:
        abstract = True

class Product(BaseModel):
    name = models.CharField(max_length=100)
    # Other fields...

Avoid Multi-Table Inheritance

Multi-table inheritance creates separate tables with joins, which can lead to performance issues. Instead, use OneToOneFields or ForeignKeys for relationships between models.

Data Integrity

Consider Denormalization Last

Denormalization should be a last resort, not a first solution. Consider other techniques like caching before denormalizing your database schema.

Use Two Identifiers for Models

Implement both a private identifier (primary key) and a public UUID for each model. This approach enhances security by not exposing internal record counts while still providing unique identifiers for public use.

import uuid
from django.db import models

class Product(models.Model):
    id = models.AutoField(primary_key=True)  # Private ID
    uid = models.UUIDField(default=uuid.uuid4, editable=False, unique=True)  # Public ID
    name = models.CharField(max_length=100)

Don't Use UUID as Primary Key

While UUIDs provide security benefits, using them as primary keys can lead to performance issues due to their non-sequential nature, especially in large databases.

Query Optimization

Use Advanced Query Tools

Leverage Django's ORM features like F() expressions instead of processing records in Python:

# Inefficient - Python processing
students = []
for student in Student.objects.iterator():
    if student.math_score > student.english_score:
        students.append(student)

# Efficient - Database processing
from django.db.models import F
students = Student.objects.filter(math_score__gt=F('english_score'))

Use pk Instead of id

Prefer using pk over id in queries for better flexibility. This allows you to change the primary key field without modifying your code.

# Less flexible
student = Student.objects.get(id=42)

# More flexible
student = Student.objects.get(pk=42)

Use update_fields with save()

When updating specific fields, use the update_fields parameter to optimize database operations and prevent unintended overwrites during concurrent updates:

product = Product.objects.get(pk=1)
product.name = "new product name"
product.save(update_fields=['name'])

Avoid count() When Unnecessary

Use exists() instead of count() when you only need to check if records exist. This is more efficient as it doesn't require counting all matching records.

# Inefficient
if queryset.count() > 0:
    # Do something

# Efficient
if queryset.exists():
    # Do something

Use OneToOneField for One-to-One Relationships

Instead of ForeignKey(unique=True), use OneToOneField for one-to-one relationships, as it's more idiomatic and clearly expresses the relationship.

Order Queryset at the Model Level

Define default ordering in the model's Meta class to ensure consistent results across your application:

class Product(models.Model):
    name = models.CharField(max_length=100)
    
    class Meta:
        ordering = ['name']

Django Views Best Practices

Class-Based vs Function-Based Views

When to Use Class-Based Views

Use class-based views (CBVs) when:

You need to reuse view logic
The view implements multiple HTTP methods (GET, POST, etc.)
You want to use mixins for common functionality

from django.views.generic import ListView

class ProductListView(ListView):
    model = Product
    template_name = 'products/list.html'
    context_object_name = 'products'
    paginate_by = 20
    
    def get_queryset(self):
        return Product.objects.filter(is_active=True)

When to Use Function-Based Views

Use function-based views (FBVs) when:

The view logic is simple and straightforward
You need maximum flexibility
You want to avoid the complexity of CBVs for simple cases

from django.shortcuts import render

def product_list(request):
    products = Product.objects.filter(is_active=True)
    return render(request, 'products/list.html', {'products': products})

View Organization

Use Mixins for Reusable Functionality

Create mixins for common functionality to promote code reuse:

class StaffRequiredMixin:
    def dispatch(self, request, *args, **kwargs):
        if not request.user.is_staff:
            return redirect('login')
        return super().dispatch(request, *args, **kwargs)

class ProductAdminView(StaffRequiredMixin, UpdateView):
    model = Product
    # Other view attributes

Keep Views Focused

Each view should have a single responsibility. Split complex views into smaller, more focused ones.

Use get_queryset and get_context_data

Override these methods in CBVs to customize the queryset and add additional context data:

class ProductListView(ListView):
    model = Product
    
    def get_queryset(self):
        queryset = super().get_queryset()
        category = self.request.GET.get('category')
        if category:
            queryset = queryset.filter(category__slug=category)
        return queryset
    
    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context['categories'] = Category.objects.all()
        return context

Performance Optimization

Use select_related and prefetch_related

Use these methods in your views to reduce database queries:

class BookListView(ListView):
    def get_queryset(self):
        return Book.objects.select_related('publisher').prefetch_related('authors')

Paginate Large Result Sets

Always paginate large result sets to improve performance and user experience:

from django.core.paginator import Paginator

def product_list(request):
    products = Product.objects.all()
    paginator = Paginator(products, 25)  # 25 products per page
    page_number = request.GET.get('page')
    page_obj = paginator.get_page(page_number)
    return render(request, 'products/list.html', {'page_obj': page_obj})

Django Templates Best Practices

Template Organization

Use Template Inheritance

Create a base template with common elements and extend it in child templates:

<!-- base.html -->
<!DOCTYPE html>
<html>
<head>
    <title>{% block title %}Default Title{% endblock %}</title>
    {% block extra_css %}{% endblock %}
</head>
<body>
    {% block content %}{% endblock %}
    {% block extra_js %}{% endblock %}


</body>
</html>

<!-- product_list.html -->

<div data-gb-custom-block data-tag="extends" data-0='base.html'></div>

<div data-gb-custom-block data-tag="block">Product List</div>

<div data-gb-custom-block data-tag="block">

    <h1>Products</h1>
    <!-- Product list content -->

</div>

Use Template Tags and Filters

Create custom template tags and filters for complex logic:

# templatetags/product_tags.py
from django import template

register = template.Library()

@register.filter
def discount_price(price, discount):
    return price * (1 - discount / 100)

@register.inclusion_tag('products/featured.html')
def show_featured_products(count=5):
    products = Product.objects.filter(featured=True)[:count]
    return {'products': products}

Keep Logic Out of Templates

Minimize logic in templates. Move complex logic to views, models, or template tags.

Performance

Use Template Fragment Caching

Cache expensive template fragments to improve performance:


<div data-gb-custom-block data-tag="load"></div>

<div data-gb-custom-block data-tag="cache" data-0='600'>
    {% for product in products %}
        <!-- Product display code -->
    {% endfor %}
</div>

Avoid Expensive Operations in Templates

Don't perform expensive operations in templates, especially within loops:

<!-- Bad: Performs a database query for each product -->

<div data-gb-custom-block data-tag="for">

    <p>{{ product.name }} - {{ product.category.all.count }} related products</p>

</div>

<!-- Good: Prefetch related data in the view -->

<div data-gb-custom-block data-tag="for">

    <p>{{ product.name }} - {{ product.categories_count }} related products</p>

</div>

Django Security Best Practices

Cross-Site Scripting (XSS) Protection

Use Template Escaping

Django templates automatically escape variables, but be careful with the safe filter and {% autoescape off %} blocks.

Use Content Security Policy

Implement a Content Security Policy to prevent XSS attacks:

CSP_DEFAULT_SRC = ("'self'",)
CSP_SCRIPT_SRC = ("'self'", "trusted-cdn.com")
CSP_STYLE_SRC = ("'self'", "trusted-cdn.com")

Cross-Site Request Forgery (CSRF) Protection

Use CSRF Protection

Always use CSRF tokens in forms:

<form method="post">
    {% csrf_token %}
    <!-- Form fields -->
    <button type="submit">Submit</button>
</form>

Exempt Only When Necessary

Only exempt views from CSRF protection when absolutely necessary (e.g., for certain API endpoints).

SQL Injection Prevention

Use ORM Queries

Prefer Django's ORM over raw SQL queries to prevent SQL injection.

Parameterize Raw Queries

If you must use raw SQL, always use parameterized queries:

from django.db import connection

def get_products_by_category(category_id):
    with connection.cursor() as cursor:
        cursor.execute(
            "SELECT * FROM products WHERE category_id = %s",
            [category_id]
        )
        return cursor.fetchall()

Authentication and Authorization

Use Django's Authentication System

Leverage Django's built-in authentication system rather than building your own.

Implement Proper Permission Checks

Use Django's permission system or custom permission mixins:

from django.contrib.auth.mixins import PermissionRequiredMixin

class ProductDeleteView(PermissionRequiredMixin, DeleteView):
    model = Product
    permission_required = 'products.delete_product'
    success_url = reverse_lazy('product-list')

Implement login attempt limiting with django-axes to prevent brute force attacks:

# settings.py
INSTALLED_APPS = [
    # ...
    'axes',
]

MIDDLEWARE = [
    # ...
    'axes.middleware.AxesMiddleware',
]

AXES_FAILURE_LIMIT = 5  # Lock out after 5 failed attempts

Secure Deployment

Use HTTPS

Always use HTTPS in production. Configure your settings:

# settings.py
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

Set Secure Headers

Implement security headers:

# settings.py
SECURE_HSTS_SECONDS = 31536000  # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'

Django Project Architecture

Project Structure

Feature-Based Organization

Organize your project by features rather than by technical components:

project/
├── config/             # Project configuration
├── products/           # Product feature
│   ├── models.py
│   ├── views.py
│   ├── urls.py
│   ├── forms.py
│   ├── services.py     # Business logic
│   └── tests/
├── orders/             # Order feature
│   ├── models.py
│   ├── views.py
│   └── ...
└── users/              # User feature
    ├── models.py
    ├── views.py
    └── ...

Separate Business Logic

Keep business logic in service classes or managers, separate from views:

# services.py
class OrderService:
    @staticmethod
    def create_order(user, cart_items):
        order = Order.objects.create(user=user)
        for item in cart_items:
            OrderItem.objects.create(
                order=order,
                product=item.product,
                quantity=item.quantity,
                price=item.product.price
            )
        return order

# views.py
from .services import OrderService

def checkout(request):
    cart_items = Cart.objects.filter(user=request.user)
    order = OrderService.create_order(request.user, cart_items)
    return redirect('order-confirmation', order_id=order.id)

Design Patterns

Repository Pattern

Use repositories to abstract database access:

class ProductRepository:
    @staticmethod
    def get_active_products():
        return Product.objects.filter(is_active=True)
    
    @staticmethod
    def get_featured_products(limit=5):
        return Product.objects.filter(is_featured=True)[:limit]
    
    @staticmethod
    def get_by_category(category_slug):
        return Product.objects.filter(category__slug=category_slug)

Factory Pattern

Use factories to create complex objects:

class ReportFactory:
    @staticmethod
    def create_report(report_type, data):
        if report_type == 'sales':
            return SalesReport(data)
        elif report_type == 'inventory':
            return InventoryReport(data)
        else:
            raise ValueError(f"Unknown report type: {report_type}")

Observer Pattern

Implement the observer pattern using Django signals:

from django.db.models.signals import post_save
from django.dispatch import receiver

@receiver(post_save, sender=Order)
def order_created_handler(sender, instance, created, **kwargs):
    if created:
        # Send confirmation email
        send_order_confirmation_email(instance)
        
        # Update inventory
        update_inventory_for_order(instance)

Django Testing Best Practices

Test Organization

Structure Tests by Type

Organize tests by type (unit, integration, functional) and by app:

project/
├── app1/
│   └── tests/
│       ├── __init__.py
│       ├── test_models.py
│       ├── test_views.py
│       └── test_forms.py
└── app2/
    └── tests/
        ├── __init__.py
        └── ...

Use Test Classes

Group related tests in classes to share setup and teardown logic:

from django.test import TestCase

class ProductModelTest(TestCase):
    def setUp(self):
        self.product = Product.objects.create(
            name="Test Product",
            price=9.99,
            is_active=True
        )
    
    def test_product_str(self):
        self.assertEqual(str(self.product), "Test Product")
    
    def test_product_is_active(self):
        self.assertTrue(self.product.is_active)

Test Types

Unit Tests

Test individual components in isolation:

class DiscountCalculatorTest(TestCase):
    def test_calculate_discount(self):
        calculator = DiscountCalculator()
        self.assertEqual(calculator.calculate(100, 20), 80)

Integration Tests

Test how components work together:

class OrderProcessingTest(TestCase):
    def setUp(self):
        self.user = User.objects.create_user('testuser', '[email protected]', 'password')
        self.product = Product.objects.create(name="Test Product", price=10.00)
        self.cart = Cart.objects.create(user=self.user)
        CartItem.objects.create(cart=self.cart, product=self.product, quantity=2)
    
    def test_order_creation(self):
        order_service = OrderService()
        order = order_service.create_from_cart(self.cart)
        
        self.assertEqual(order.user, self.user)
        self.assertEqual(order.items.count(), 1)
        self.assertEqual(order.total, 20.00)

View Tests

Test views using Django's test client:

class ProductViewTest(TestCase):
    def setUp(self):
        self.client = Client()
        self.product = Product.objects.create(
            name="Test Product",
            price=9.99,
            slug="test-product"
        )
    
    def test_product_detail_view(self):
        response = self.client.get(f'/products/{self.product.slug}/')
        self.assertEqual(response.status_code, 200)
        self.assertContains(response, "Test Product")
        self.assertTemplateUsed(response, 'products/detail.html')

Testing Tools

Use pytest

Consider using pytest with pytest-django for more powerful testing features:

import pytest
from django.urls import reverse

@pytest.mark.django_db
def test_product_list_view(client):
    url = reverse('product-list')
    response = client.get(url)
    assert response.status_code == 200

Use Factories

Use factory_boy to create test objects:

import factory
from django.contrib.auth.models import User
from myapp.models import Product

class UserFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = User
    
    username = factory.Sequence(lambda n: f'user{n}')
    email = factory.LazyAttribute(lambda o: f'{o.username}@example.com')

class ProductFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Product
    
    name = factory.Sequence(lambda n: f'Product {n}')
    price = factory.Faker('pydecimal', left_digits=3, right_digits=2, positive=True)
    created_by = factory.SubFactory(UserFactory)

Mock External Services

Use unittest.mock to mock external services:

from unittest.mock import patch

class PaymentServiceTest(TestCase):
    @patch('myapp.services.payment.stripe.Charge.create')
    def test_process_payment(self, mock_charge_create):
        mock_charge_create.return_value = {'id': 'ch_test123', 'status': 'succeeded'}
        
        payment_service = PaymentService()
        result = payment_service.process_payment('tok_visa', 1000, 'usd')
        
        self.assertTrue(result['success'])
        mock_charge_create.assert_called_once_with(
            amount=1000,
            currency='usd',
            source='tok_visa',
            description='Payment for order'
        )

Test Coverage

Measure Coverage

Use coverage.py to measure test coverage:

# Run tests with coverage
coverage run --source='.' manage.py test

# Generate a report
coverage report

# Generate HTML report
coverage html

Continuous Integration

Integrate tests into CI/CD pipeline:

# .github/workflows/test.yml
name: Django Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
    
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    
    - name: Run tests
      run: |
        python manage.py test
      env:
        DATABASE_URL: postgres://postgres:postgres@localhost:5432/test_db

Django Deployment Best Practices

Environment Configuration

Use Environment Variables

Store configuration in environment variables, not in code:

# settings.py
import os
from pathlib import Path

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = os.environ.get('DJANGO_DEBUG', '') == 'True'

ALLOWED_HOSTS = os.environ.get('DJANGO_ALLOWED_HOSTS', '').split(',')

# Database
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT', '5432'),
    }
}

Use django-environ

Consider using django-environ to manage environment variables:

# settings.py
import environ

env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)

# Take environment variables from .env file
environ.Env.read_env(os.path.join(BASE_DIR, '.env'))

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env('DEBUG')

ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')

# Database
DATABASES = {
    'default': env.db(),
}

Static and Media Files

Use a CDN

Serve static and media files from a CDN in production:

# settings.py
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

if not DEBUG:
    # Use a CDN in production
    STATIC_URL = 'https://your-cdn.com/static/'
    
    # Configure AWS S3 for media files
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
    AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
    AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
    AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
    MEDIA_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/media/'

Compress and Minify

Compress and minify static files:

# settings.py
INSTALLED_APPS = [
    # ...
    'compressor',
]

STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'compressor.finders.CompressorFinder',
]

COMPRESSOR_ENABLED = not DEBUG

Performance Optimization

Use a Cache

Implement caching in production:

# settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': env('REDIS_URL'),
    }
}

# Cache middleware
MIDDLEWARE = [
    'django.middleware.cache.UpdateCacheMiddleware',
    # ... other middleware
    'django.middleware.cache.FetchFromCacheMiddleware',
]

# Cache settings
CACHE_MIDDLEWARE_SECONDS = 60 * 15  # 15 minutes

Use a Task Queue

Use Celery for background tasks:

# celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')

app = Celery('project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

# tasks.py
from celery import shared_task

@shared_task
def process_order(order_id):
    order = Order.objects.get(id=order_id)
    # Process the order
    order.status = 'processing'
    order.save()
    # Send confirmation email
    send_order_confirmation(order)
    return True

Monitoring and Logging

Configure Logging

Set up comprehensive logging:

# settings.py
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {message}',
            'style': '{',
        },
    },
    'handlers': {
        'file': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': os.path.join(BASE_DIR, 'logs/django.log'),
            'formatter': 'verbose',
        },
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file', 'console'],
            'level': 'INFO',
            'propagate': True,
        },
        'myapp': {
            'handlers': ['file', 'console'],
            'level': 'INFO',
            'propagate': True,
        },
    },
}

Use Monitoring Tools

Implement application monitoring:

# settings.py
INSTALLED_APPS = [
    # ...
    'django_prometheus',
]

MIDDLEWARE = [
    'django_prometheus.middleware.PrometheusBeforeMiddleware',
    # ... other middleware
    'django_prometheus.middleware.PrometheusAfterMiddleware',
]

# urls.py
urlpatterns = [
    # ...
    path('metrics/', include('django_prometheus.urls')),
]

Advanced Django Features

Custom Management Commands

Create custom management commands for administrative tasks:

# myapp/management/commands/import_products.py
from django.core.management.base import BaseCommand
from myapp.models import Product
import csv

class Command(BaseCommand):
    help = 'Import products from a CSV file'

    def add_arguments(self, parser):
        parser.add_argument('csv_file', type=str, help='Path to the CSV file')

    def handle(self, *args, **options):
        csv_file = options['csv_file']
        count = 0
        
        with open(csv_file, 'r') as file:
            reader = csv.DictReader(file)
            for row in reader:
                Product.objects.create(
                    name=row['name'],
                    price=float(row['price']),
                    description=row['description'],
                    is_active=row['is_active'].lower() == 'true'
                )
                count += 1
        
        self.stdout.write(self.style.SUCCESS(f'Successfully imported {count} products'))

Custom Template Tags

Create custom template tags for complex rendering logic:

# myapp/templatetags/pagination_tags.py
from django import template

register = template.Library()

@register.inclusion_tag('common/pagination.html')
def paginate(page_obj, url_name, **kwargs):
    """Render pagination for a page object."""
    return {
        'page_obj': page_obj,
        'url_name': url_name,
        'kwargs': kwargs,
    }

Middleware

Create custom middleware for request/response processing:

# myapp/middleware.py
import time
import logging

logger = logging.getLogger('myapp.middleware')

class RequestTimingMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        start_time = time.time()
        
        response = self.get_response(request)
        
        duration = time.time() - start_time
        logger.info(f'Request to {request.path} took {duration:.2f}s')
        
        # Add timing header to response
        response['X-Request-Duration'] = f'{duration:.2f}s'
        
        return response

Context Processors

Create context processors to add data to all templates:

# myapp/context_processors.py
from myapp.models import Category

def categories(request):
    """Add categories to all templates."""
    return {
        'categories': Category.objects.all(),
    }

# settings.py
TEMPLATES = [
    {
        # ...
        'OPTIONS': {
            'context_processors': [
                # ...
                'myapp.context_processors.categories',
            ],
        },
    },
]

Custom Model Managers

Create custom model managers for complex queries:

# myapp/models.py
from django.db import models

class ActiveProductManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(is_active=True)
    
    def featured(self):
        return self.get_queryset().filter(is_featured=True)

class Product(models.Model):
    name = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    is_active = models.BooleanField(default=True)
    is_featured = models.BooleanField(default=False)
    
    objects = models.Manager()  # Default manager
    active = ActiveProductManager()  # Custom manager
    
    # Usage:
    # Product.objects.all()  # All products
    # Product.active.all()  # Only active products
    # Product.active.featured()  # Only active and featured products

Django Asynchronous Programming

Async Features in Django

Django has been evolving to support asynchronous programming, which allows handling more concurrent connections with fewer resources. This is particularly important for applications that need to handle many simultaneous connections or perform I/O-bound operations.

Using Channels for Async Support

Django Channels extends Django to handle WebSockets, chat protocols, IoT protocols, and more, beyond just HTTP:

# Install Channels
pip install channels

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'channels',
]

# Configure the ASGI application in asgi.py
import os
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

application = ProtocolTypeRouter({
    'http': get_asgi_application(),
    'websocket': URLRouter([
        # Your WebSocket routes here
    ]),
})

Async Views

Django supports asynchronous views that can perform I/O operations without blocking the main thread:

async def async_view(request):
    # Perform async operations
    await asyncio.sleep(1)  # Non-blocking sleep
    data = await async_function()  # Call async function
    return JsonResponse(data)

Using ASGI Servers

To take full advantage of Django's async features, use an ASGI server like Uvicorn or Daphne:

# Install Uvicorn
pip install uvicorn

# Run with Uvicorn
uvicorn myproject.asgi:application

Recommended Async Packages

channels - Extends Django with WebSocket support and more
starlette - Lightweight ASGI framework/toolkit
uvicorn - Lightning-fast ASGI server
asyncio - Python's built-in async library

Django Project Structure Best Practices

Recommended Project Structure

A well-organized Django project structure enhances maintainability and scalability. Here's a recommended structure based on best practices:

.
├── apps/                      # All Django applications
│   ├── accounts/             # User accounts app
│   │   ├── api/              # API components
│   │   │   ├── v1/           # API version 1
│   │   │   │   ├── serializers.py
│   │   │   │   ├── urls.py
│   │   │   │   └── views.py
│   │   │   └── __init__.py
│   │   ├── migrations/
│   │   ├── templates/        # App-specific templates
│   │   ├── tests/            # App-specific tests
│   │   ├── __init__.py
│   │   ├── admin.py
│   │   ├── apps.py
│   │   ├── models.py
│   │   ├── services.py       # Business logic
│   │   ├── urls.py
│   │   └── views.py
│   └── __init__.py
├── config/                    # Project configuration
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── deployments/               # Deployment configurations
│   ├── docker/
│   └── nginx/
├── docs/                      # Project documentation
│   ├── CHANGELOG.md
│   └── deployment.md
├── requirements/              # Dependencies by environment
│   ├── common.txt
│   ├── development.txt
│   └── production.txt
├── static/                    # Static files
├── templates/                 # Project-level templates
├── .env.example               # Example environment variables
├── .gitignore
├── manage.py
└── README.md

Key Structure Components

Apps Folder

Place all your Django applications in an apps folder. Each app should be designed to be plug-able, meaning it can be dragged and dropped into any other project and work independently.

API Versioning

Organize API components within each app in an api folder with version subfolders (v1, v2, etc.). This makes it easier to maintain multiple API versions:

app/
├── api/
│   ├── v1/
│   │   ├── serializers.py
│   │   ├── urls.py
│   │   └── views.py
│   └── v2/
       ├── serializers.py
       ├── urls.py
       └── views.py

Services Layer

Implement a services layer for business logic instead of placing it in views or models:

# services.py
class OrderService:
    @staticmethod
    def create_order(user, items):
        # Business logic for creating an order
        order = Order.objects.create(user=user)
        # More logic...
        return order

This approach isolates business logic, making it easier to test and maintain.

Configuration

Keep all project configuration in a dedicated config folder, including settings, main URLs, and WSGI/ASGI configurations.

Deployments

Isolate deployment-specific files (Docker, nginx, etc.) in a deployments folder for better organization.

Python Code Quality Best Practices

Coding Style Guidelines

Python Style

Use 4 spaces for indentation in Python files and 2 spaces for HTML files
Follow PEP 8 style guidelines
Use single quotes for strings unless the string contains a single quote
Use underscores for variable, function, and method names (snake_case)
Use InitialCaps for class names (PascalCase)

Imports Organization

Organize imports into groups with a blank line between each group:

Future imports
Standard library imports
Third-party library imports
Django imports
Local application imports

# Future imports
from __future__ import annotations

# Standard library
import json
from datetime import datetime

# Third-party
import requests

# Django
from django.db import models
from django.conf import settings

# Local
from .models import Product

Model Style

Field names should be lowercase with underscores
Follow a consistent order for model components:
1. Database fields
2. Custom manager attributes
3. class Meta
4. def __str__()
5. def save()
6. def get_absolute_url()
7. Custom methods

class Product(models.Model):
    # 1. Fields
    name = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    created_at = models.DateTimeField(auto_now_add=True)
    
    # 2. Custom manager
    objects = ProductManager()
    
    # 3. Meta
    class Meta:
        ordering = ['-created_at']
        verbose_name_plural = 'products'
    
    # 4. String representation
    def __str__(self):
        return self.name
    
    # 5. Save method
    def save(self, *args, **kwargs):
        # Custom save logic
        super().save(*args, **kwargs)
    
    # 6. URL
    def get_absolute_url(self):
        return reverse('product-detail', kwargs={'pk': self.pk})
    
    # 7. Custom methods
    def is_on_sale(self):
        return self.price < self.regular_price

Code Quality Tools

Linters

Use linters to check your code for potential errors and style violations:

Flake8 - Combines PyFlakes, pycodestyle, and McCabe complexity checker
Pylint - Comprehensive linter with many checks
Ruff - Fast Python linter written in Rust

Static Type Checkers

Use static type checkers to catch type-related bugs:

mypy - Static type checker for Python
Pyright - Fast type checker by Microsoft

Code Formatters

Automate code formatting to ensure consistency:

Black - The uncompromising Python code formatter
isort - Sort imports alphabetically and automatically separated into sections

Documentation Tools

Generate documentation from docstrings:

Sphinx - Documentation generator
MkDocs - Project documentation with Markdown

References

Django Security Best Practices

Authentication and Authorization

Use Django's Built-in Authentication System

Django's authentication system is robust and well-tested. Avoid building custom authentication systems unless absolutely necessary.

# settings.py - Use strong password validators
AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
        'OPTIONS': {'min_length': 12},
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

Implement Multi-Factor Authentication

Add an extra layer of security with multi-factor authentication:

# Install django-two-factor-auth
pip install django-two-factor-auth

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'django_otp',
    'django_otp.plugins.otp_totp',
    'django_otp.plugins.otp_static',
    'two_factor',
]

# Add to MIDDLEWARE
MIDDLEWARE = [
    # ...
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django_otp.middleware.OTPMiddleware',
]

Use Permission-Based Authorization

Implement proper permission checks using Django's permission system:

# Using function decorators
from django.contrib.auth.decorators import permission_required

@permission_required('app.change_model')
def edit_model(request, model_id):
    # View logic here
    pass

# Using class-based view mixins
from django.contrib.auth.mixins import PermissionRequiredMixin

class EditModelView(PermissionRequiredMixin, UpdateView):
    permission_required = 'app.change_model'
    # View attributes

Protection Against Common Attacks

Cross-Site Scripting (XSS) Protection

Django templates automatically escape variables, but be careful with the safe filter and {% autoescape off %} blocks.

Implement Content Security Policy (CSP) headers:

# Install django-csp
pip install django-csp

# Add to MIDDLEWARE
MIDDLEWARE = [
    # ...
    'csp.middleware.CSPMiddleware',
]

# Configure CSP
CSP_DEFAULT_SRC = ("'self'",)
CSP_SCRIPT_SRC = ("'self'", "trusted-cdn.com")
CSP_STYLE_SRC = ("'self'", "trusted-cdn.com")
CSP_IMG_SRC = ("'self'", "trusted-cdn.com", "data:")

Cross-Site Request Forgery (CSRF) Protection

Always use CSRF protection in forms:

<form method="post">
    {% csrf_token %}
    <!-- Form fields -->
    <button type="submit">Submit</button>


</form>

For AJAX requests, include the CSRF token in headers:

const csrftoken = document.querySelector('[name=csrfmiddlewaretoken]').value;

fetch('/api/endpoint/', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        'X-CSRFToken': csrftoken,
    },
    body: JSON.stringify(data),
});

SQL Injection Prevention

Use Django's ORM and avoid raw SQL queries. If you must use raw SQL, always use parameterized queries:

# Bad - vulnerable to SQL injection
Product.objects.raw(f"SELECT * FROM products WHERE category = '{category}'")

# Good - parameterized query
Product.objects.raw("SELECT * FROM products WHERE category = %s", [category])

# Better - use the ORM
Product.objects.filter(category=category)

Secure File Uploads

Validate and sanitize file uploads:

def handle_uploaded_file(f):
    # Validate file type
    if not f.name.endswith(('.jpg', '.jpeg', '.png', '.gif')):
        raise ValidationError("Unsupported file type")
        
    # Validate file size
    if f.size > 5 * 1024 * 1024:  # 5MB limit
        raise ValidationError("File too large")
        
    # Generate safe filename
    import uuid
    safe_filename = f"{uuid.uuid4().hex}{os.path.splitext(f.name)[1]}"
    
    # Save to secure location outside web root
    with open(os.path.join(settings.MEDIA_ROOT, 'uploads', safe_filename), 'wb+') as destination:
        for chunk in f.chunks():
            destination.write(chunk)

Secure Deployment

Use HTTPS

Always use HTTPS in production:

# settings.py
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

Set Security Headers

Implement security headers to protect against various attacks:

# settings.py
SECURE_HSTS_SECONDS = 31536000  # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'

Protect Sensitive Data

Never store sensitive data like API keys or passwords in code. Use environment variables:

# settings.py
import os
from django.core.exceptions import ImproperlyConfigured

def get_env_variable(var_name):
    try:
        return os.environ[var_name]
    except KeyError:
        error_msg = f"Set the {var_name} environment variable"
        raise ImproperlyConfigured(error_msg)
        
SECRET_KEY = get_env_variable('DJANGO_SECRET_KEY')
DATABASE_PASSWORD = get_env_variable('DATABASE_PASSWORD')

Regular Security Updates

Regularly update Django and all dependencies to patch security vulnerabilities:

# Check for vulnerable packages
pip install safety
safety check

# Update packages
pip install -U django
pip install -U -r requirements.txt

Django Performance Optimization

Database Optimization

Optimize Database Queries

Use select_related and prefetch_related

Reduce the number of database queries with select_related for ForeignKey and OneToOneField relationships:

# Instead of this (causes N+1 queries)
authors = Author.objects.all()
for author in authors:
    print(author.user.email)  # Each access triggers a new query

# Do this (single query with join)
authors = Author.objects.select_related('user')
for author in authors:
    print(author.user.email)  # No additional queries

Use prefetch_related for ManyToMany and reverse ForeignKey relationships:

# Instead of this (causes N+1 queries)
books = Book.objects.all()
for book in books:
    print([author.name for author in book.authors.all()])  # New query for each book

# Do this (2 queries total)
books = Book.objects.prefetch_related('authors')
for book in books:
    print([author.name for author in book.authors.all()])  # No additional queries

Use Indexes

Add database indexes to fields that are frequently used in filters, ordering, or joins:

class Product(models.Model):
    name = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    category = models.ForeignKey(Category, on_delete=models.CASCADE)
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)  # Add index
    is_active = models.BooleanField(default=True, db_index=True)  # Add index
    
    class Meta:
        indexes = [
            models.Index(fields=['name', 'category']),  # Composite index
            models.Index(fields=['-created_at']),  # Index for sorting
        ]

Use Database Functions

Leverage database functions for operations that can be performed at the database level:

from django.db.models import F, Sum, Count, Avg

# Instead of this (inefficient)
total = 0
for order in Order.objects.all():
    total += order.price * order.quantity

# Do this (single efficient query)
total = Order.objects.aggregate(total=Sum(F('price') * F('quantity')))['total']

# Group by with aggregation
category_stats = Product.objects.values('category').annotate(
    count=Count('id'),
    avg_price=Avg('price')
).order_by('-count')

Use Database Connection Pooling

Implement connection pooling to reuse database connections:

# Install django-db-connection-pool
pip install django-db-connection-pool

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'dj_db_conn_pool.backends.postgresql',
        'NAME': 'mydatabase',
        'USER': 'myuser',
        'PASSWORD': 'mypassword',
        'HOST': 'localhost',
        'PORT': '5432',
        'POOL_OPTIONS': {
            'POOL_SIZE': 20,
            'MAX_OVERFLOW': 10,
            'RECYCLE': 300,  # Recycle connections after 5 minutes
        },
    }
}

Caching Strategies

Configure Cache Backend

Set up a cache backend for better performance:

# settings.py - Redis cache example
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': 'redis://127.0.0.1:6379/1',
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
        }
    }
}

Cache Views

Cache entire views for anonymous users:

from django.views.decorators.cache import cache_page

@cache_page(60 * 15)  # Cache for 15 minutes
def product_list(request):
    products = Product.objects.all()
    return render(request, 'products/list.html', {'products': products})

Cache Template Fragments

Cache expensive parts of templates:


<div data-gb-custom-block data-tag="load"></div>

<div data-gb-custom-block data-tag="cache" data-0='600'>
    {% for product in products %}
        <!-- Product display code -->
    {% endfor %}
</div>

Use Low-Level Cache API

Cache specific data with the low-level cache API:

from django.core.cache import cache

# Set cache
cache.set('product_count', Product.objects.count(), timeout=300)  # 5 minutes

# Get from cache
product_count = cache.get('product_count')
if product_count is None:
    product_count = Product.objects.count()
    cache.set('product_count', product_count, timeout=300)

Static Files Optimization

Use a CDN

Serve static files from a Content Delivery Network (CDN):

# settings.py
STATIC_URL = 'https://your-cdn.com/static/'
MEDIA_URL = 'https://your-cdn.com/media/'

Compress and Minify Static Files

Use django-compressor to compress and minify CSS and JavaScript:

# Install django-compressor
pip install django-compressor

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'compressor',
]

# Configure compressor
STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'compressor.finders.CompressorFinder',
]

COMPRESSOR_ENABLED = not DEBUG
COMPRESSOR_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter']
COMPRESSOR_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']

In templates:


<div data-gb-custom-block data-tag="load"></div>

<div data-gb-custom-block data-tag="compress">

<link rel="stylesheet" href="

<div data-gb-custom-block data-tag="static" data-0='css/style.css'></div>">
<link rel="stylesheet" href="<div data-gb-custom-block data-tag="static" data-0='css/responsive.css'></div>

">

</div>

<div data-gb-custom-block data-tag="compress">

<script src="

<div data-gb-custom-block data-tag="static" data-0='js/main.js'></div>"></script>
<script src="<div data-gb-custom-block data-tag="static" data-0='js/analytics.js'></div>

"></script>

</div>

Asynchronous Task Processing

Use Celery for Background Tasks

Offload heavy processing to background tasks with Celery:

# Install Celery and Redis
pip install celery redis

# celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')

app = Celery('project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

# tasks.py
from celery import shared_task
from django.core.mail import send_mail

@shared_task
def send_notification_email(user_id, subject, message):
    from django.contrib.auth.models import User
    user = User.objects.get(id=user_id)
    send_mail(
        subject,
        message,
        '[email protected]',
        [user.email],
        fail_silently=False,
    )
    return True

# views.py
from .tasks import send_notification_email

def register_user(request):
    # User registration logic
    user = User.objects.create_user(...)
    
    # Send welcome email asynchronously
    send_notification_email.delay(
        user.id,
        'Welcome to our site',
        'Thank you for registering!'
    )
    
    return redirect('home')

Use Django Channels for WebSockets

Implement real-time features with Django Channels:

# Install Channels and Redis
pip install channels channels-redis

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'channels',
]

# Configure Channels
ASGI_APPLICATION = 'myproject.asgi.application'
CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            'hosts': [('127.0.0.1', 6379)],
        },
    },
}

Load Testing and Profiling

Profile your application during development:

# Install Django Debug Toolbar
pip install django-debug-toolbar

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'debug_toolbar',
]

# Add to MIDDLEWARE
MIDDLEWARE = [
    # ...
    'debug_toolbar.middleware.DebugToolbarMiddleware',
]

# Configure
INTERNAL_IPS = ['127.0.0.1']

# Add to urls.py
if settings.DEBUG:
    import debug_toolbar
    urlpatterns = [path('__debug__/', include(debug_toolbar.urls))] + urlpatterns

Use django-silk for Request Profiling

Profile requests in production-like environments:

# Install django-silk
pip install django-silk

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'silk',
]

# Add to MIDDLEWARE
MIDDLEWARE = [
    # ...
    'silk.middleware.SilkyMiddleware',
]

# Add to urls.py
urlpatterns = [path('silk/', include('silk.urls'))] + urlpatterns

Load Testing with Locust

Perform load testing to identify bottlenecks:

# Install Locust
pip install locust

# locustfile.py
from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 5)  # Wait 1-5 seconds between tasks
    
    @task
    def index_page(self):
        self.client.get("/")
        
    @task(3)  # 3x more frequent than other tasks
    def view_products(self):
        self.client.get("/products/")
        
    @task
    def view_product(self):
        product_id = random.randint(1, 100)
        self.client.get(f"/products/{product_id}/")

Run with:

locust -f locustfile.py --host=http://localhost:8000

Django Testing Best Practices

Test Organization

Structure Tests by App and Type

Organize tests by app and test type for better maintainability:

app/
├── tests/
│   ├── __init__.py
│   ├── test_models.py
│   ├── test_views.py
│   ├── test_forms.py
│   ├── test_api.py
│   └── test_services.py

Use Descriptive Test Names

Name your tests descriptively to clearly indicate what they're testing:

def test_user_creation_with_valid_data_succeeds():
    # Test code

def test_user_creation_with_duplicate_email_fails():
    # Test code

Test Types and Strategies

Unit Tests

Test individual components in isolation:

from django.test import TestCase
from myapp.models import Product

class ProductModelTest(TestCase):
    def setUp(self):
        self.product = Product.objects.create(
            name="Test Product",
            price=9.99,
            is_active=True
        )
    
    def test_product_str_method(self):
        self.assertEqual(str(self.product), "Test Product")
    
    def test_is_on_sale_with_discount(self):
        self.product.regular_price = 19.99
        self.assertTrue(self.product.is_on_sale())
    
    def test_is_on_sale_without_discount(self):
        self.product.regular_price = 9.99
        self.assertFalse(self.product.is_on_sale())

Integration Tests

Test how components work together:

from django.test import TestCase
from django.urls import reverse
from myapp.models import Product, Category

class ProductViewsIntegrationTest(TestCase):
    def setUp(self):
        self.category = Category.objects.create(name="Electronics")
        self.product = Product.objects.create(
            name="Test Product",
            price=9.99,
            category=self.category,
            is_active=True
        )
    
    def test_product_list_view(self):
        url = reverse('product-list')
        response = self.client.get(url)
        self.assertEqual(response.status_code, 200)
        self.assertContains(response, "Test Product")
        self.assertTemplateUsed(response, 'products/list.html')
    
    def test_product_detail_view(self):
        url = reverse('product-detail', args=[self.product.id])
        response = self.client.get(url)
        self.assertEqual(response.status_code, 200)
        self.assertContains(response, "Test Product")
        self.assertContains(response, "$9.99")

API Tests

Test your API endpoints:

from rest_framework.test import APITestCase
from rest_framework import status
from django.urls import reverse
from myapp.models import Product

class ProductAPITest(APITestCase):
    def setUp(self):
        self.product = Product.objects.create(
            name="Test Product",
            price=9.99,
            is_active=True
        )
        self.url = reverse('api:product-detail', args=[self.product.id])
    
    def test_get_product(self):
        response = self.client.get(self.url)
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(response.data['name'], "Test Product")
        self.assertEqual(float(response.data['price']), 9.99)
    
    def test_update_product(self):
        data = {'name': 'Updated Product', 'price': 19.99}
        response = self.client.patch(self.url, data, format='json')
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.product.refresh_from_db()
        self.assertEqual(self.product.name, "Updated Product")
        self.assertEqual(float(self.product.price), 19.99)

Testing Tools and Techniques

Use pytest for More Powerful Testing

# Install pytest and plugins
pip install pytest pytest-django pytest-cov

# Create a pytest.ini file
# pytest.ini
[pytest]
DJANGO_SETTINGS_MODULE = myproject.settings
python_files = test_*.py

# Write tests with pytest
import pytest
from django.urls import reverse

@pytest.mark.django_db
def test_product_list_view(client):
    url = reverse('product-list')
    response = client.get(url)
    assert response.status_code == 200
    assert 'products' in response.context

Use Factories for Test Data

Use factory_boy to create test objects efficiently:

# Install factory_boy
pip install factory_boy

# Create factories
import factory
from django.contrib.auth.models import User
from myapp.models import Product, Category

class UserFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = User
    
    username = factory.Sequence(lambda n: f'user{n}')
    email = factory.LazyAttribute(lambda o: f'{o.username}@example.com')
    password = factory.PostGenerationMethodCall('set_password', 'password')

class CategoryFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Category
    
    name = factory.Sequence(lambda n: f'Category {n}')

class ProductFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Product
    
    name = factory.Sequence(lambda n: f'Product {n}')
    price = factory.Faker('pydecimal', left_digits=3, right_digits=2, positive=True)
    category = factory.SubFactory(CategoryFactory)
    created_by = factory.SubFactory(UserFactory)

Mock External Services

Use mocking to isolate tests from external dependencies:

from unittest.mock import patch, MagicMock
from django.test import TestCase
from myapp.services import PaymentService

class PaymentServiceTest(TestCase):
    @patch('myapp.services.stripe.Charge.create')
    def test_process_payment(self, mock_charge_create):
        # Configure the mock
        mock_charge_create.return_value = {
            'id': 'ch_test123',
            'status': 'succeeded',
            'amount': 1000
        }
        
        # Call the service
        payment_service = PaymentService()
        result = payment_service.process_payment('tok_visa', 10.00, 'usd')
        
        # Assert the result
        self.assertTrue(result['success'])
        self.assertEqual(result['charge_id'], 'ch_test123')
        
        # Verify the mock was called correctly
        mock_charge_create.assert_called_once_with(
            amount=1000,  # Amount in cents
            currency='usd',
            source='tok_visa',
            description='Payment for order'
        )

Test Coverage

Measure and maintain good test coverage:

# Run tests with coverage
python -m pytest --cov=myapp

# Generate HTML report
python -m pytest --cov=myapp --cov-report=html

Continuous Integration Testing

Set up automated testing in your CI pipeline:

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/development.txt
    
    - name: Run tests
      run: |
        pytest --cov=myapp --cov-report=xml
      env:
        DATABASE_URL: postgres://postgres:postgres@localhost:5432/test_db
    
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage.xml

Testing Best Practices

Test Isolation

Ensure each test is independent and doesn't rely on the state from other tests:

# Bad - tests depend on each other
def test_create_user():
    User.objects.create_user('testuser', '[email protected]', 'password')
    assert User.objects.count() == 1

def test_authenticate_user():
    # This assumes test_create_user ran first
    user = authenticate(username='testuser', password='password')
    assert user is not None

# Good - tests are isolated
def test_create_user():
    User.objects.create_user('testuser', '[email protected]', 'password')
    assert User.objects.count() == 1

def test_authenticate_user():
    User.objects.create_user('testuser', '[email protected]', 'password')
    user = authenticate(username='testuser', password='password')
    assert user is not None

Test Edge Cases

Test not just the happy path, but also edge cases and error conditions:

def test_divide_numbers():
    # Test normal case
    assert divide(10, 2) == 5
    
    # Test edge cases
    assert divide(0, 5) == 0
    
    # Test error conditions
    with pytest.raises(ValueError):
        divide(10, 0)  # Division by zero should raise ValueError

Test Security

Write tests for security-related functionality:

def test_user_cannot_access_another_users_data():
    # Create two users
    user1 = UserFactory()
    user2 = UserFactory()
    
    # Create data for user1
    order = OrderFactory(user=user1)
    
    # Log in as user2
    self.client.force_login(user2)
    
    # Try to access user1's order
    url = reverse('order-detail', args=[order.id])
    response = self.client.get(url)
    
    # Should be forbidden
    self.assertEqual(response.status_code, 403)

Django Deployment Best Practices

Deployment Checklist

Pre-Deployment Checks

Before deploying to production, run Django's built-in checks:

# Check for common problems
python manage.py check --deploy

# Check for database migrations
python manage.py makemigrations --check

# Test with production-like settings
DJANGO_SETTINGS_MODULE=myproject.settings.production python manage.py test

Production Settings

Use separate settings for production:

# settings/base.py - Common settings

# settings/production.py
from .base import *

DEBUG = False
ALLOWED_HOSTS = ['example.com', 'www.example.com']

# Security settings
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

# Database settings
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT', '5432'),
    }
}

# Cache settings
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL'),
    }
}

Deployment Strategies

Docker-Based Deployment

Use Docker for consistent deployments:

# Dockerfile
FROM python:3.10-slim

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements /app/requirements
RUN pip install --no-cache-dir -r requirements/production.txt

COPY . /app/

RUN python manage.py collectstatic --noinput

CMD gunicorn myproject.wsgi:application --bind 0.0.0.0:8000

# docker-compose.yml
version: '3.8'

services:
  web:
    build: .
    restart: always
    env_file: .env
    depends_on:
      - db
      - redis
    ports:
      - "8000:8000"
  
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    env_file: .env.db
  
  redis:
    image: redis:6
    volumes:
      - redis_data:/data
  
  nginx:
    image: nginx:1.21
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d
      - ./staticfiles:/app/staticfiles
      - ./media:/app/media
      - ./certbot/conf:/etc/letsencrypt
      - ./certbot/www:/var/www/certbot
    depends_on:
      - web

volumes:
  postgres_data:
  redis_data:

Serverless Deployment

Consider serverless options for certain applications:

# serverless.yml for AWS Lambda with Zappa
service: django-app

provider:
  name: aws
  runtime: python3.9
  region: us-east-1

functions:
  app:
    handler: wsgi_handler.handler
    events:
      - http:
          path: /{proxy+}
          method: ANY
    environment:
      DJANGO_SETTINGS_MODULE: myproject.settings.production

Continuous Deployment

Implement continuous deployment with GitHub Actions:

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

jobs:
  test:
    # Test job from previous example
    # ...
  
  deploy:
    needs: test
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/production.txt
    
    - name: Deploy to production
      run: |
        echo "$SSH_PRIVATE_KEY" > deploy_key
        chmod 600 deploy_key
        ssh -i deploy_key -o StrictHostKeyChecking=no $SSH_USER@$SSH_HOST \
          "cd $DEPLOY_PATH && git pull && docker-compose up -d --build"
      env:
        SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
        SSH_USER: ${{ secrets.SSH_USER }}
        SSH_HOST: ${{ secrets.SSH_HOST }}
        DEPLOY_PATH: ${{ secrets.DEPLOY_PATH }}

Monitoring and Maintenance

Application Monitoring

Set up monitoring for your Django application:

# Install Sentry for error tracking
pip install sentry-sdk

# settings.py
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

sentry_sdk.init(
    dsn="https://your-sentry-dsn.ingest.sentry.io/project",
    integrations=[DjangoIntegration()],
    traces_sample_rate=0.5,
    send_default_pii=True,
    environment="production",
)

Database Backups

Set up automated database backups:

#!/bin/bash
# backup.sh

DATE=$(date +%Y-%m-%d_%H-%M-%S)
BACKUP_DIR="/path/to/backups"
DB_NAME="mydb"
DB_USER="myuser"

# Create backup directory if it doesn't exist
mkdir -p $BACKUP_DIR

# Create database backup
pg_dump -U $DB_USER $DB_NAME | gzip > "$BACKUP_DIR/$DB_NAME-$DATE.sql.gz"

# Remove backups older than 30 days
find $BACKUP_DIR -name "$DB_NAME-*.sql.gz" -mtime +30 -delete

Add to crontab:

0 2 * * * /path/to/backup.sh

Health Checks

Implement health checks to monitor your application:

# Install django-health-check
pip install django-health-check

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'health_check',
    'health_check.db',
    'health_check.cache',
    'health_check.storage',
    'health_check.contrib.migrations',
]

# Add to urls.py
urlpatterns = [
    # ...
    path('health/', include('health_check.urls')),
]

Automated Security Updates

Set up automated security updates for your dependencies:

# .github/workflows/security.yml
name: Security Updates

on:
  schedule:
    - cron: '0 0 * * 0'  # Run weekly

jobs:
  security:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install safety
    
    - name: Check for vulnerabilities
      run: |
        safety check -r requirements/production.txt
    
    - name: Create pull request for updates
      uses: peter-evans/create-pull-request@v4
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
        commit-message: 'chore: update dependencies for security'
        title: 'Security: Update dependencies'
        body: 'Automated security update of dependencies'
        branch: 'security-updates'

Django Internationalization (i18n) Best Practices

Setting Up Internationalization

Configure Settings

Enable internationalization in your Django project:

# settings.py
USE_I18N = True  # Enable internationalization
USE_L10N = True  # Enable localization
USE_TZ = True    # Enable timezone support

# Available languages
from django.utils.translation import gettext_lazy as _
LANGUAGES = [
    ('en', _('English')),
    ('es', _('Spanish')),
    ('fr', _('French')),
    ('de', _('German')),
    # Add more languages as needed
]

# Default language
LANGUAGE_CODE = 'en'

# Locale paths
import os
LOCALE_PATHS = [os.path.join(BASE_DIR, 'locale')]

# Middleware for language selection
MIDDLEWARE = [
    # ...
    'django.middleware.locale.LocaleMiddleware',  # Add after SessionMiddleware
    # ...
]

Create Translation Files

Generate and compile translation files:

# Create locale directories
mkdir -p locale

# Extract messages from Python code and templates
python manage.py makemessages -l es  # Spanish
python manage.py makemessages -l fr  # French
python manage.py makemessages -l de  # German

# Compile message files after translation
python manage.py compilemessages

Marking Strings for Translation

In Python Code

Mark strings for translation in Python code:

from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy as _  # For model fields

# In views
def my_view(request):
    message = _('Welcome to our website')  # Mark for translation
    return render(request, 'template.html', {'message': message})

# In models
class Product(models.Model):
    name = models.CharField(_('product name'), max_length=100)
    description = models.TextField(_('description'), blank=True)
    
    class Meta:
        verbose_name = _('product')
        verbose_name_plural = _('products')

In Templates

Mark strings for translation in templates:


<div data-gb-custom-block data-tag="load" data-0='18' data-1='8'></div>

<h1>

<div data-gb-custom-block data-tag="trans" data-0='Welcome to our website'></div>

</h1>

<div data-gb-custom-block data-tag="blocktrans">

    Hello, {{ username }}! How are you today?

</div>

<div data-gb-custom-block data-tag="blocktrans">

    There is {{ counter }} item in your cart.

<div data-gb-custom-block data-tag="plural"></div>

    There are {{ counter }} items in your cart.

</div>

In JavaScript

Translate strings in JavaScript using Django's JavaScript catalog:

# urls.py
from django.urls import path
from django.views.i18n import JavaScriptCatalog

urlpatterns = [
    # ...
    path('jsi18n/', JavaScriptCatalog.as_view(), name='javascript-catalog'),
]

<!-- In your template -->
<script src="

<div data-gb-custom-block data-tag="url" data-0='javascript-catalog'></div>

"></script>
<script>
    // Use the translation function
    alert(gettext('Welcome to our website'));
</script>

Language Switching

URL-Based Language Switching

Implement language switching in URLs:

# urls.py
from django.conf.urls.i18n import i18n_patterns
from django.urls import path, include

urlpatterns = [
    # Non-translated URLs
    path('admin/', admin.site.urls),
    path('i18n/', include('django.conf.urls.i18n')),  # Language switch view
]

# Translated URLs
urlpatterns += i18n_patterns(
    path('', include('myapp.urls')),
    # More URL patterns...
)

Language Selector Template

Create a language selector in your templates:


<div data-gb-custom-block data-tag="load" data-0='18' data-1='8'></div>

<form action="

<div data-gb-custom-block data-tag="url" data-0='set_language'></div>" method="post">
    {% csrf_token %}
    <input name="next" type="hidden" value="{{ request.path }}">
    
    <select name="language" onchange="this.form.submit()">
        {% get_current_language as CURRENT_LANGUAGE %}
        {% get_available_languages as LANGUAGES %}
        
        {% for lang_code, lang_name in LANGUAGES %}
            <option value="{{ lang_code }}" {% if lang_code == CURRENT_LANGUAGE %}selected{% endif %}>
                {{ lang_name }}
            </option>
        {% endfor %}
    </select>
</form>

Best Practices for Internationalization

Use Context with Translations

Provide context for ambiguous translations:

from django.utils.translation import pgettext

# 'month' provides context for 'May'
month = pgettext('month name', 'May')

# 'verb' provides context for 'May'
permission = pgettext('verb', 'May')

Handle Pluralization

Use proper pluralization in translations:

from django.utils.translation import ngettext

def display_items_count(count):
    return ngettext(
        'There is %(count)d item in your cart.',
        'There are %(count)d items in your cart.',
        count
    ) % {'count': count}

Format Dates and Numbers

Use localized formatting for dates and numbers:

from django.utils import formats
from django.utils import numberformat

# Format date according to current locale
formatted_date = formats.date_format(some_date, 'SHORT_DATE_FORMAT')

# Format number according to current locale
formatted_number = numberformat.format(1234.56, decimal_pos=2)

Translation Management

Keep translations organized and up-to-date:

Use translation comments to provide context for translators:

# Translators: This is a greeting message
message = _('Welcome to our website')

Use lazy translation for strings that are evaluated at runtime:

from django.utils.translation import gettext_lazy as _

class MyModel(models.Model):
    title = models.CharField(_('title'), max_length=100)

Keep translation files updated when making changes:

# Update existing translation files with new strings
python manage.py makemessages --all

# Compile after translations are updated
python manage.py compilemessages

Django Accessibility Best Practices

HTML Structure and Semantics

Use Semantic HTML

Use appropriate HTML elements for their intended purpose:

<!-- Bad: Not semantic -->
<div class="header">
    <div class="nav-item">Home</div>
</div>

<!-- Good: Semantic HTML -->
<header>
    <nav>
        <ul>
            <li><a href="/">Home</a></li>
        </ul>
    </nav>
</header>

Implement Proper Heading Structure

Maintain a logical heading hierarchy (h1-h6):

<!-- Good heading structure -->
<h1>Page Title</h1>
<section>
    <h2>Section Title</h2>
    <article>
        <h3>Article Title</h3>
        <h4>Subsection</h4>
    </article>
</section>

Forms Accessibility

Label Form Controls

Ensure all form controls have associated labels:

<!-- Bad: No label -->
<input type="text" name="username">

<!-- Good: With label -->
<label for="username">Username</label>
<input type="text" id="username" name="username">

In Django forms:

# forms.py
from django import forms

class MyForm(forms.Form):
    username = forms.CharField(label='Username')
    
    # Set label and help text for accessibility
    email = forms.EmailField(
        label='Email Address',
        help_text='Enter a valid email address',
        widget=forms.EmailInput(attrs={'aria-describedby': 'email-help'})
    )

Form Error Accessibility

Make form errors accessible:

# In your view
def my_form_view(request):
    if request.method == 'POST':
        form = MyForm(request.POST)
        if not form.is_valid():
            # Add non-field errors to template context
            return render(request, 'form.html', {
                'form': form,
                'has_errors': True,
            })
    else:
        form = MyForm()
    return render(request, 'form.html', {'form': form})

<!-- In your template -->
<form method="post">
    {% csrf_token %}
    
    {% if has_errors %}
        <div role="alert" class="error-summary">
            <h2>Please fix the errors below</h2>
            <ul>
                {% for field in form %}
                    {% for error in field.errors %}
                        <li><a href="#id_{{ field.name }}">{{ field.label }}: {{ error }}</a></li>
                    {% endfor %}
                {% endfor %}
                
                {% for error in form.non_field_errors %}
                    <li>{{ error }}</li>
                {% endfor %}
            </ul>
        </div>
    {% endif %}
    
    {% for field in form %}
        <div class="form-group {% if field.errors %}has-error{% endif %}">
            {{ field.label_tag }}
            
            {% if field.errors %}
                <span id="{{ field.name }}-error" class="error-message">
                    {% for error in field.errors %}{{ error }}{% endfor %}
                </span>
                {{ field|attr:"aria-describedby:{{ field.name }}-error" }}
            {% else %}
                {{ field }}
            {% endif %}
            
            {% if field.help_text %}
                <span id="{{ field.name }}-help" class="help-text">{{ field.help_text }}</span>
            {% endif %}
        </div>
    {% endfor %}
    
    <button type="submit">Submit</button>


</form>

Images and Media

Provide Alternative Text for Images

Ensure all images have appropriate alt text:

<!-- Informative image -->
<img src="logo.png" alt="Company Logo">

<!-- Decorative image -->
<img src="decorative-line.png" alt="">

In Django templates:


<div data-gb-custom-block data-tag="if" data-expression='product.image'>

    <img src="{{ product.image.url }}" alt="{{ product.image_alt_text|default:product.name }}">

</div>

Make Videos Accessible

Provide captions and transcripts for videos:

<video controls>
    <source src="video.mp4" type="video/mp4">
    <track kind="captions" src="captions.vtt" srclang="en" label="English">
    <track kind="descriptions" src="descriptions.vtt" srclang="en" label="English">
    Your browser does not support the video tag.
</video>

ARIA Attributes

Use ARIA Roles and Properties

Enhance accessibility with ARIA attributes:

<!-- Navigation menu -->
<nav aria-label="Main Navigation">
    <ul role="menubar">
        <li role="menuitem"><a href="/">Home</a></li>
        <li role="menuitem"><a href="/products">Products</a></li>
    </ul>
</nav>

<!-- Modal dialog -->
<div role="dialog" aria-labelledby="dialog-title" aria-modal="true">
    <h2 id="dialog-title">Confirmation</h2>
    <p>Are you sure you want to delete this item?</p>
    <button>Yes</button>
    <button>No</button>
</div>

Implement Skip Links

Add skip links for keyboard navigation:

<!-- At the top of your base template -->
<a href="#main-content" class="skip-link">Skip to main content</a>

<!-- Later in the document -->
<main id="main-content">
    <!-- Main content here -->
</main>

/* CSS for skip link */
.skip-link {
    position: absolute;
    top: -40px;
    left: 0;
    padding: 8px;
    background-color: #fff;
    z-index: 100;
}

.skip-link:focus {
    top: 0;
}

Testing Accessibility

Automated Testing

Implement automated accessibility testing:

# Install axe-selenium-python
pip install axe-selenium-python

# In your tests
from django.test import LiveServerTestCase
from selenium import webdriver
from axe_selenium_python import Axe

class AccessibilityTests(LiveServerTestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.maximize_window()
        self.axe = Axe(self.driver)
    
    def tearDown(self):
        self.driver.quit()
    
    def test_home_page_accessibility(self):
        self.driver.get(self.live_server_url)
        self.axe.inject()
        results = self.axe.run()
        
        # Assert no violations
        violations = results['violations']
        self.assertEqual(len(violations), 0, f"Accessibility violations found: {violations}")

Manual Testing

Perform manual accessibility testing:

Keyboard navigation: Ensure all interactive elements are accessible via keyboard
Screen reader testing: Test with screen readers like NVDA, JAWS, or VoiceOver
Color contrast: Verify sufficient contrast between text and background
Text resizing: Ensure content remains usable when text is resized up to 200%

Django Accessibility Packages

Recommended Packages

django-a11y: Tools for accessibility testing in Django
django-axes: Limit login attempts to improve security
django-crispy-forms: Create accessible forms with minimal markup

# Install django-crispy-forms
pip install django-crispy-forms

# Add to INSTALLED_APPS
INSTALLED_APPS = [
    # ...
    'crispy_forms',
]

# Set template pack
CRISPY_TEMPLATE_PACK = 'bootstrap4'

# forms.py
from django import forms
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Layout, Submit, Field

class AccessibleForm(forms.Form):
    name = forms.CharField()
    email = forms.EmailField()
    message = forms.CharField(widget=forms.Textarea())
    
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.helper = FormHelper()
        self.helper.layout = Layout(
            Field('name', aria_describedby='name-help'),
            Field('email'),
            Field('message'),
            Submit('submit', 'Send Message', css_class='btn btn-primary')
        )
        self.helper.form_id = 'contact-form'
        self.helper.form_method = 'post'

Django REST Framework Advanced Patterns

API Design Principles

RESTful Resource Naming

Use clear, resource-focused naming for your API endpoints:

# Good URL patterns
GET /api/articles/                # List articles
POST /api/articles/               # Create article
GET /api/articles/{id}/           # Retrieve article
PUT /api/articles/{id}/           # Update article
DELECH /api/articles/{id}/        # Delete article
GET /api/articles/{id}/comments/  # List article comments

# Avoid
GET /api/get_all_articles/
POST /api/create_article/
GET /api/article_details/{id}/

Versioning Your API

Implement API versioning to maintain backward compatibility:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
    'DEFAULT_VERSION': 'v1',
    'ALLOWED_VERSIONS': ['v1', 'v2'],
    'VERSION_PARAM': 'version',
}

# urls.py
from django.urls import path, include
from rest_framework import routers

router_v1 = routers.DefaultRouter()
router_v1.register('articles', ArticleViewSetV1)

router_v2 = routers.DefaultRouter()
router_v2.register('articles', ArticleViewSetV2)

urlpatterns = [
    path('api/v1/', include(router_v1.urls)),
    path('api/v2/', include(router_v2.urls)),
]

Serializer Patterns

Nested Serializers

Handle related objects with nested serializers:

from rest_framework import serializers
from .models import Article, Comment, Author

class CommentSerializer(serializers.ModelSerializer):
    class Meta:
        model = Comment
        fields = ['id', 'content', 'created_at']

class AuthorSerializer(serializers.ModelSerializer):
    class Meta:
        model = Author
        fields = ['id', 'name', 'bio']

class ArticleSerializer(serializers.ModelSerializer):
    comments = CommentSerializer(many=True, read_only=True)
    author = AuthorSerializer(read_only=True)
    author_id = serializers.PrimaryKeyRelatedField(
        queryset=Author.objects.all(),
        write_only=True,
        source='author'
    )
    
    class Meta:
        model = Article
        fields = ['id', 'title', 'content', 'author', 'author_id', 'comments', 'created_at']

Custom Field Serialization

Implement custom field serialization for complex data:

class ArticleSerializer(serializers.ModelSerializer):
    reading_time = serializers.SerializerMethodField()
    tags = serializers.ListField(child=serializers.CharField(), required=False)
    
    class Meta:
        model = Article
        fields = ['id', 'title', 'content', 'reading_time', 'tags']
    
    def get_reading_time(self, obj):
        # Calculate reading time based on content length
        word_count = len(obj.content.split())
        minutes = max(1, round(word_count / 200))  # Average reading speed
        return f"{minutes} min read"
    
    def to_representation(self, instance):
        # Customize the output representation
        representation = super().to_representation(instance)
        representation['url'] = self.context['request'].build_absolute_uri(
            reverse('article-detail', args=[instance.pk])
        )
        return representation
    
    def to_internal_value(self, data):
        # Custom input processing
        if 'tags' in data and isinstance(data['tags'], str):
            data['tags'] = [tag.strip() for tag in data['tags'].split(',')]
        return super().to_internal_value(data)

Serializer Inheritance

Use serializer inheritance to reuse code:

class BaseArticleSerializer(serializers.ModelSerializer):
    class Meta:
        model = Article
        fields = ['id', 'title', 'content', 'created_at']

class ArticleListSerializer(BaseArticleSerializer):
    # Simplified serializer for list views
    class Meta(BaseArticleSerializer.Meta):
        fields = ['id', 'title', 'created_at']

class ArticleDetailSerializer(BaseArticleSerializer):
    # Extended serializer for detail views
    comments = CommentSerializer(many=True, read_only=True)
    author = AuthorSerializer(read_only=True)
    
    class Meta(BaseArticleSerializer.Meta):
        fields = BaseArticleSerializer.Meta.fields + ['author', 'comments', 'updated_at']

ViewSet Patterns

Custom Actions

Add custom actions to your ViewSets:

from rest_framework import viewsets, status
from rest_framework.decorators import action
from rest_framework.response import Response

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    
    @action(detail=True, methods=['post'])
    def publish(self, request, pk=None):
        article = self.get_object()
        article.status = 'published'
        article.published_at = timezone.now()
        article.save()
        return Response({'status': 'article published'})
    
    @action(detail=True, methods=['get'])
    def comments(self, request, pk=None):
        article = self.get_object()
        comments = article.comments.all()
        serializer = CommentSerializer(comments, many=True)
        return Response(serializer.data)
    
    @action(detail=False, methods=['get'])
    def featured(self, request):
        featured_articles = Article.objects.filter(is_featured=True)
        serializer = self.get_serializer(featured_articles, many=True)
        return Response(serializer.data)

Dynamic Serializer Selection

Use different serializers based on the action:

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    
    def get_serializer_class(self):
        if self.action == 'list':
            return ArticleListSerializer
        elif self.action == 'create':
            return ArticleCreateSerializer
        return ArticleDetailSerializer
    
    def get_queryset(self):
        queryset = Article.objects.all()
        
        # Apply filters based on query parameters
        category = self.request.query_params.get('category')
        if category:
            queryset = queryset.filter(category__slug=category)
            
        # Order by most recent by default
        return queryset.order_by('-created_at')

Authentication and Permissions

Token Authentication

Implement token-based authentication:

# settings.py
INSTALLED_APPS = [
    # ...
    'rest_framework',
    'rest_framework.authtoken',
]

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.TokenAuthentication',
        'rest_framework.authentication.SessionAuthentication',
    ],
}

# views.py
from rest_framework.authtoken.views import ObtainAuthToken
from rest_framework.authtoken.models import Token
from rest_framework.response import Response

class CustomAuthToken(ObtainAuthToken):
    def post(self, request, *args, **kwargs):
        serializer = self.serializer_class(data=request.data,
                                           context={'request': request})
        serializer.is_valid(raise_exception=True)
        user = serializer.validated_data['user']
        token, created = Token.objects.get_or_create(user=user)
        return Response({
            'token': token.key,
            'user_id': user.pk,
            'email': user.email
        })

JWT Authentication

Implement JWT authentication for more secure token handling:

# Install djangorestframework-simplejwt
pip install djangorestframework-simplejwt

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    ],
}

SIMPLE_JWT = {
    'ACCESS_TOKEN_LIFETIME': timedelta(minutes=60),
    'REFRESH_TOKEN_LIFETIME': timedelta(days=1),
    'ROTATE_REFRESH_TOKENS': False,
    'BLACKLIST_AFTER_ROTATION': True,
}

# urls.py
from rest_framework_simplejwt.views import (
    TokenObtainPairView,
    TokenRefreshView,
)

urlpatterns = [
    # ...
    path('api/token/', TokenObtainPairView.as_view(), name='token_obtain_pair'),
    path('api/token/refresh/', TokenRefreshView.as_view(), name='token_refresh'),
]

Custom Permissions

Implement custom permissions for fine-grained access control:

from rest_framework import permissions

class IsAuthorOrReadOnly(permissions.BasePermission):
    """Custom permission to only allow authors to edit their own content."""
    
    def has_object_permission(self, request, view, obj):
        # Read permissions are allowed for any request
        if request.method in permissions.SAFE_METHODS:
            return True
            
        # Write permissions are only allowed to the author
        return obj.author == request.user

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    permission_classes = [permissions.IsAuthenticatedOrReadOnly, 
                          IsAuthorOrReadOnly]
                          
    def perform_create(self, serializer):
        serializer.save(author=self.request.user)

Performance Optimization

Pagination

Implement pagination for large result sets:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 20,
}

# Custom pagination classes
from rest_framework.pagination import PageNumberPagination, CursorPagination

class StandardResultsSetPagination(PageNumberPagination):
    page_size = 20
    page_size_query_param = 'page_size'
    max_page_size = 100

class CursorResultsSetPagination(CursorPagination):
    page_size = 20
    cursor_query_param = 'cursor'
    ordering = '-created_at'

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    pagination_class = StandardResultsSetPagination

Filtering and Searching

Implement filtering, searching, and ordering:

# Install django-filter
pip install django-filter

# settings.py
INSTALLED_APPS = [
    # ...
    'django_filters',
]

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
}

# views.py
from django_filters.rest_framework import DjangoFilterBackend
from rest_framework.filters import SearchFilter, OrderingFilter

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    filter_backends = [DjangoFilterBackend, SearchFilter, OrderingFilter]
    filterset_fields = ['category', 'status', 'author']
    search_fields = ['title', 'content', 'author__name']
    ordering_fields = ['created_at', 'title', 'views_count']
    ordering = ['-created_at']

Django GraphQL Integration

Setting Up GraphQL with Graphene

Installation and Configuration

Set up GraphQL with Graphene-Django:

# Install Graphene-Django
pip install graphene-django

# settings.py
INSTALLED_APPS = [
    # ...
    'graphene_django',
]

GRAPHENE = {
    'SCHEMA': 'myproject.schema.schema',
    'MIDDLEWARE': [
        'graphql_jwt.middleware.JSONWebTokenMiddleware',
    ],
}

AUTHENTICATION_BACKENDS = [
    'graphql_jwt.backends.JSONWebTokenBackend',
    'django.contrib.auth.backends.ModelBackend',
]

# urls.py
from django.urls import path
from graphene_django.views import GraphQLView
from django.views.decorators.csrf import csrf_exempt

urlpatterns = [
    # ...
    path('graphql/', csrf_exempt(GraphQLView.as_view(graphiql=True))),
]

Define Schema

Create GraphQL schema with types and queries:

# schema.py
import graphene
from graphene_django import DjangoObjectType
from .models import Article, Author

class AuthorType(DjangoObjectType):
    class Meta:
        model = Author
        fields = ('id', 'name', 'bio', 'articles')

class ArticleType(DjangoObjectType):
    class Meta:
        model = Article
        fields = ('id', 'title', 'content', 'author', 'created_at')

class Query(graphene.ObjectType):
    all_articles = graphene.List(ArticleType)
    article = graphene.Field(ArticleType, id=graphene.Int())
    all_authors = graphene.List(AuthorType)
    author = graphene.Field(AuthorType, id=graphene.Int())
    
    def resolve_all_articles(self, info, **kwargs):
        return Article.objects.all()
    
    def resolve_article(self, info, id):
        return Article.objects.get(pk=id)
    
    def resolve_all_authors(self, info, **kwargs):
        return Author.objects.all()
    
    def resolve_author(self, info, id):
        return Author.objects.get(pk=id)

schema = graphene.Schema(query=Query)

Mutations

Implement mutations for creating and updating data:

class CreateArticleMutation(graphene.Mutation):
    class Arguments:
        title = graphene.String(required=True)
        content = graphene.String(required=True)
        author_id = graphene.ID(required=True)
    
    article = graphene.Field(ArticleType)
    
    @classmethod
    def mutate(cls, root, info, title, content, author_id):
        author = Author.objects.get(pk=author_id)
        article = Article.objects.create(
            title=title,
            content=content,
            author=author
        )
        return CreateArticleMutation(article=article)

class UpdateArticleMutation(graphene.Mutation):
    class Arguments:
        id = graphene.ID(required=True)
        title = graphene.String()
        content = graphene.String()
    
    article = graphene.Field(ArticleType)
    
    @classmethod
    def mutate(cls, root, info, id, **kwargs):
        article = Article.objects.get(pk=id)
        
        for attr, value in kwargs.items():
            setattr(article, attr, value)
        
        article.save()
        return UpdateArticleMutation(article=article)

class Mutation(graphene.ObjectType):
    create_article = CreateArticleMutation.Field()
    update_article = UpdateArticleMutation.Field()

schema = graphene.Schema(query=Query, mutation=Mutation)

Authentication with GraphQL

Implement JWT authentication for GraphQL:

# Install django-graphql-jwt
pip install django-graphql-jwt

# schema.py
import graphene
import graphql_jwt

class Mutation(graphene.ObjectType):
    token_auth = graphql_jwt.ObtainJSONWebToken.Field()
    verify_token = graphql_jwt.Verify.Field()
    refresh_token = graphql_jwt.Refresh.Field()
    # Other mutations...

# Example of a protected query
class Query(graphene.ObjectType):
    me = graphene.Field(UserType)
    
    def resolve_me(self, info):
        user = info.context.user
        if user.is_anonymous:
            raise Exception('Not authenticated')
        return user

Relay Integration

Use Relay for more advanced GraphQL features:

from graphene_django.filter import DjangoFilterConnectionField
from graphene_django.types import DjangoObjectType
import graphene
from graphene import relay

class ArticleNode(DjangoObjectType):
    class Meta:
        model = Article
        filter_fields = {
            'title': ['exact', 'icontains', 'istartswith'],
            'content': ['icontains'],
            'created_at': ['gt', 'lt', 'gte', 'lte'],
            'author__name': ['exact', 'icontains'],
        }
        interfaces = (relay.Node, )

class Query(graphene.ObjectType):
    article = relay.Node.Field(ArticleNode)
    all_articles = DjangoFilterConnectionField(ArticleNode)

class CreateArticleMutation(relay.ClientIDMutation):
    class Input:
        title = graphene.String(required=True)
        content = graphene.String(required=True)
        author_id = graphene.ID(required=True)
    
    article = graphene.Field(ArticleNode)
    
    @classmethod
    def mutate_and_get_payload(cls, root, info, **input):
        author = Author.objects.get(pk=input.get('author_id'))
        article = Article.objects.create(
            title=input.get('title'),
            content=input.get('content'),
            author=author
        )
        return CreateArticleMutation(article=article)

class Mutation(graphene.ObjectType):
    create_article = CreateArticleMutation.Field()

GraphQL Best Practices

Optimize Database Queries

Use select_related and prefetch_related to avoid N+1 queries:

class Query(graphene.ObjectType):
    all_articles = graphene.List(ArticleType)
    
    def resolve_all_articles(self, info):
        # Prefetch related author data to avoid N+1 queries
        return Article.objects.select_related('author').all()

Use DataLoader for Batch Loading

Implement DataLoader for efficient batch loading:

# Install promise and graphql-core
pip install promise graphql-core

from promise import Promise
from promise.dataloader import DataLoader

class AuthorLoader(DataLoader):
    def batch_load_fn(self, author_ids):
        # Get all authors in a single query
        authors = {author.id: author for author in Author.objects.filter(id__in=author_ids)}
        return Promise.resolve([authors.get(author_id) for author_id in author_ids])

# In your schema
class ArticleType(DjangoObjectType):
    class Meta:
        model = Article
        fields = ('id', 'title', 'content', 'author', 'created_at')
    
    def resolve_author(self, info):
        author_loader = info.context.author_loader
        return author_loader.load(self.author_id)

# In your view
from graphene_django.views import GraphQLView

class CustomGraphQLView(GraphQLView):
    def get_context(self, request):
        context = super().get_context(request)
        context.author_loader = AuthorLoader()
        return context

Implement Pagination

Use cursor-based pagination for large result sets:

from graphene import relay
from graphene_django.filter import DjangoFilterConnectionField

class ArticleNode(DjangoObjectType):
    class Meta:
        model = Article
        filter_fields = ['title', 'author']
        interfaces = (relay.Node, )

class Query(graphene.ObjectType):
    all_articles = DjangoFilterConnectionField(ArticleNode)

Example GraphQL query with pagination:

query {
  allArticles(first: 10, after: "YXJyYXljb25uZWN0aW9uOjEw") {
    edges {
      node {
        id
        title
        content
      }
      cursor
    }
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
  }
}

Django Debugging and Troubleshooting

Debugging Tools

The Django Debug Toolbar is an essential tool for debugging Django applications:

# Install Django Debug Toolbar
pip install django-debug-toolbar

# settings.py
INSTALLED_APPS = [
    # ...
    'debug_toolbar',
]

MIDDLEWARE = [
    # ...
    'debug_toolbar.middleware.DebugToolbarMiddleware',  # As early as possible
]

# Only enable for internal IPs
INTERNAL_IPS = [
    '127.0.0.1',
]

# urls.py
if settings.DEBUG:
    import debug_toolbar
    urlpatterns = [path('__debug__/', include(debug_toolbar.urls))] + urlpatterns

Django Extensions

Django Extensions provides additional management commands and debugging tools:

# Install Django Extensions
pip install django-extensions

# settings.py
INSTALLED_APPS = [
    # ...
    'django_extensions',
]

Useful commands:

# Shell Plus - Django shell with auto-imported models
python manage.py shell_plus

# Show URLs - List all URLs in your project
python manage.py show_urls

# Graph models - Generate model diagrams
python manage.py graph_models -a -o myapp_models.png

# Validate templates - Check template syntax
python manage.py validate_templates

Logging Configuration

Set up comprehensive logging for better debugging:

# settings.py
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
            'style': '{',
        },
        'simple': {
            'format': '{levelname} {message}',
            'style': '{',
        },
    },
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': os.path.join(BASE_DIR, 'logs/django.log'),
            'formatter': 'verbose'
        },
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
        }
    },
    'loggers': {
        'django': {
            'handlers': ['console', 'file'],
            'propagate': True,
        },
        'django.request': {
            'handlers': ['mail_admins', 'file'],
            'level': 'ERROR',
            'propagate': False,
        },
        'myapp': {  # Your app's logger
            'handlers': ['console', 'file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    }
}

Usage in your code:

import logging

# Get a logger for the current module
logger = logging.getLogger(__name__)

def my_view(request):
    logger.debug('Debug message')
    logger.info('Info message')
    logger.warning('Warning message')
    logger.error('Error message')
    logger.critical('Critical message')
    
    # Log exceptions
    try:
        # Some code that might raise an exception
        result = 1 / 0
    except Exception as e:
        logger.exception('An error occurred: %s', str(e))
        # The exception() method logs a message with ERROR level and adds exception info

Common Issues and Solutions

Database Connection Issues

# Check database connection
from django.db import connections
from django.db.utils import OperationalError

db_conn = connections['default']
try:
    c = db_conn.cursor()
except OperationalError:
    print("Database connection failed")
else:
    print("Database connection successful")

Migration Problems

# Reset migrations (development only)
python manage.py showmigrations  # Check current state
python manage.py migrate app_name zero  # Revert all migrations for app
python manage.py makemigrations app_name  # Create fresh migration
python manage.py migrate app_name  # Apply fresh migration

# Fake migrations if tables already exist
python manage.py migrate --fake app_name

# Squash migrations
python manage.py squashmigrations app_name 0001 0010

Static Files Not Loading

# Check static files configuration
# settings.py
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'static'),
]

# Run collectstatic
python manage.py collectstatic --noinput

# Check if static files are being served
python manage.py findstatic css/main.css

Performance Issues

# Use the Django Debug Toolbar to identify slow queries

# Use select_related and prefetch_related for related objects
authors = Author.objects.select_related('user').prefetch_related('books')

# Use only() and defer() to fetch only needed fields
users = User.objects.only('username', 'email')
users = User.objects.defer('biography', 'preferences')

# Use database indexes for frequently queried fields
class Book(models.Model):
    title = models.CharField(max_length=100, db_index=True)
    publication_date = models.DateField(db_index=True)

Debugging Production Issues

Error Reporting

Integrate with error reporting services like Sentry:

# Install Sentry SDK
pip install sentry-sdk

# settings.py
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

sentry_sdk.init(
    dsn="https://your-sentry-dsn.ingest.sentry.io/project",
    integrations=[
        DjangoIntegration(),
    ],
    traces_sample_rate=0.5,  # Adjust sampling rate as needed
    send_default_pii=True,  # Send personally identifiable information
    environment=os.environ.get('DJANGO_ENVIRONMENT', 'development'),
)

Remote Debugging

Set up remote debugging for production-like environments:

# Install django-remote-debug-toolbar
pip install django-remote-debug-toolbar

# settings.py
INSTALLED_APPS = [
    # ...
    'debug_toolbar',
    'remote_debug',
]

MIDDLEWARE = [
    # ...
    'remote_debug.middleware.RemoteDebugMiddleware',
]

REMOTE_DEBUG_HOSTS = ['your-ip-address']

Django Containerization Best Practices

Docker Setup

Dockerfile

Create an optimized Dockerfile for Django:

# Use an official Python runtime as a parent image
FROM python:3.10-slim as builder

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# Set work directory
WORKDIR /app

# Install system dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc libpq-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Install Python dependencies
COPY requirements/ /app/requirements/
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements/production.txt

# Final stage
FROM python:3.10-slim

# Create a non-root user
RUN useradd -m django

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV HOME=/home/django
ENV APP_HOME=/home/django/app

# Create directories
RUN mkdir -p $APP_HOME $APP_HOME/staticfiles $APP_HOME/media
WORKDIR $APP_HOME

# Install dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends libpq5 && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Copy wheels from builder stage
COPY --from=builder /app/wheels /wheels
RUN pip install --no-cache /wheels/*

# Copy project files
COPY --chown=django:django . $APP_HOME

# Change ownership
RUN chown -R django:django $APP_HOME

# Switch to non-root user
USER django

# Run entrypoint script
ENTRYPOINT ["./docker-entrypoint.sh"]

# Run gunicorn
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "3", "myproject.wsgi:application"]

Docker Entrypoint

Create a robust entrypoint script:

#!/bin/bash

set -o errexit
set -o pipefail
set -o nounset

# Wait for PostgreSQL to be available
python << END
import sys
import time
import psycopg2

for _ in range(30):
    try:
        psycopg2.connect(
            dbname="${DB_NAME}",
            user="${DB_USER}",
            password="${DB_PASSWORD}",
            host="${DB_HOST}",
            port="${DB_PORT}",
        )
        break
    except psycopg2.OperationalError:
        sys.stderr.write("Waiting for PostgreSQL to become available...\n")
        time.sleep(1)
else:
    sys.stderr.write("Failed to connect to PostgreSQL\n")
    sys.exit(1)
END

# Apply database migrations
echo "Applying database migrations..."
python manage.py migrate --noinput

# Collect static files
echo "Collecting static files..."
python manage.py collectstatic --noinput

# Create superuser if needed
if [ "${DJANGO_SUPERUSER_USERNAME:-}" ]; then
    python manage.py createsuperuser \
        --noinput \
        --username $DJANGO_SUPERUSER_USERNAME \
        --email $DJANGO_SUPERUSER_EMAIL || true
fi

# Execute the command passed to docker-entrypoint.sh
exec "$@"

Docker Compose

Set up a complete development environment with Docker Compose:

# docker-compose.yml
version: '3.8'

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    image: myproject:latest
    restart: always
    volumes:
      - static_volume:/home/django/app/staticfiles
      - media_volume:/home/django/app/media
    env_file:
      - ./.env
    depends_on:
      - db
      - redis
    networks:
      - app-network

  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    env_file:
      - ./.env.db
    networks:
      - app-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:6
    volumes:
      - redis_data:/data
    networks:
      - app-network
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

  celery:
    build:
      context: .
      dockerfile: Dockerfile
    command: celery -A myproject worker -l INFO
    volumes:
      - ./:/home/django/app/
    env_file:
      - ./.env
    depends_on:
      - web
      - redis
    networks:
      - app-network

  nginx:
    image: nginx:1.21
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d
      - static_volume:/home/django/app/staticfiles
      - media_volume:/home/django/app/media
      - ./nginx/certbot/conf:/etc/letsencrypt
      - ./nginx/certbot/www:/var/www/certbot
    depends_on:
      - web
    networks:
      - app-network

volumes:
  postgres_data:
  redis_data:
  static_volume:
  media_volume:

networks:
  app-network:
    driver: bridge

Environment Configuration

Environment Variables

Manage environment variables properly:

# .env example
DJANGO_SETTINGS_MODULE=myproject.settings.production
DEBUG=False
SECRET_KEY=your-secret-key-here
ALLOWED_HOSTS=example.com,www.example.com
DB_NAME=mydatabase
DB_USER=myuser
DB_PASSWORD=mypassword
DB_HOST=db
DB_PORT=5432
REDIS_URL=redis://redis:6379/1
EMAIL_HOST=smtp.example.com
EMAIL_PORT=587
[email protected]
EMAIL_HOST_PASSWORD=your-email-password

# settings.py
import os
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent

# Load environment variables from .env file in development
if os.path.exists(os.path.join(BASE_DIR, '.env')):
    from dotenv import load_dotenv
    load_dotenv(os.path.join(BASE_DIR, '.env'))

# Security settings
SECRET_KEY = os.environ.get('SECRET_KEY')
DEBUG = os.environ.get('DEBUG', 'False') == 'True'
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(',')

# Database settings
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT', '5432'),
    }
}

Container Security

Security Best Practices

Use non-root users:

# Create a non-root user
RUN useradd -m django
USER django

Scan for vulnerabilities:

# Install and use trivy for vulnerability scanning
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image myproject:latest

Minimize image size:

# Use multi-stage builds
FROM python:3.10-slim as builder
# ... build dependencies ...

FROM python:3.10-slim
# ... copy only what's needed ...

Pin dependency versions:

# requirements.txt
Django==4.2.3
psycopg2-binary==2.9.6
redis==4.5.5

CI/CD Pipeline

GitHub Actions

Implement CI/CD with GitHub Actions:

# .github/workflows/ci-cd.yml
name: Django CI/CD

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
      
      redis:
        image: redis:6
        ports:
          - 6379:6379
        options: --health-cmd "redis-cli ping" --health-interval 10s --health-timeout 5s --health-retries 5
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'
        cache: 'pip'
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements/development.txt
    
    - name: Run tests
      run: |
        python manage.py test
      env:
        DEBUG: 'True'
        SECRET_KEY: 'test-secret-key'
        DB_NAME: test_db
        DB_USER: postgres
        DB_PASSWORD: postgres
        DB_HOST: localhost
        DB_PORT: 5432
        REDIS_URL: redis://localhost:6379/1
    
    - name: Run linting
      run: |
        flake8 .
        black --check .
        isort --check .
  
  build-and-push:
    needs: test
    if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop')
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v2
    
    - name: Login to DockerHub
      uses: docker/login-action@v2
      with:
        username: ${{ secrets.DOCKERHUB_USERNAME }}
        password: ${{ secrets.DOCKERHUB_TOKEN }}
    
    - name: Build and push
      uses: docker/build-push-action@v4
      with:
        context: .
        push: true
        tags: |
          myusername/myproject:latest
          myusername/myproject:${{ github.sha }}
        cache-from: type=registry,ref=myusername/myproject:buildcache
        cache-to: type=registry,ref=myusername/myproject:buildcache,mode=max
  
  deploy:
    needs: build-and-push
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Install kubectl
      uses: azure/setup-kubectl@v3
      with:
        version: 'v1.25.0'
    
    - name: Configure kubectl
      run: |
        mkdir -p $HOME/.kube
        echo "${{ secrets.KUBE_CONFIG }}" > $HOME/.kube/config
        chmod 600 $HOME/.kube/config
    
    - name: Deploy to Kubernetes
      run: |
        kubectl set image deployment/django-app django=myusername/myproject:${{ github.sha }}
        kubectl rollout status deployment/django-app

Django Middleware Customization

Creating Custom Middleware

Middleware allows you to process requests and responses globally across your Django application. Here's how to create and use custom middleware:

Basic Middleware Structure

# middleware.py
class SimpleMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        # One-time configuration and initialization

    def __call__(self, request):
        # Code to be executed for each request before the view is called
        print(f"Request path: {request.path}")
        
        response = self.get_response(request)
        
        # Code to be executed for each response after the view is called
        print(f"Response status: {response.status_code}")
        
        return response

Process View/Template/Exception Methods

class CustomMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        return self.get_response(request)
    
    def process_view(self, request, view_func, view_args, view_kwargs):
        # Called just before Django calls the view
        print(f"View function: {view_func.__name__}")
        return None  # Continue processing
    
    def process_template_response(self, request, response):
        # Called after the view has been called, if response has render() method
        print("Processing template response")
        return response  # Must return response object
    
    def process_exception(self, request, exception):
        # Called when view raises an exception
        print(f"Exception: {exception}")
        return None  # Continue default exception handling

Registering Middleware

# settings.py
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'myapp.middleware.SimpleMiddleware',  # Your custom middleware
    'myapp.middleware.CustomMiddleware',  # Another custom middleware
]

Useful Middleware Examples

Request Timing Middleware

import time
import logging

logger = logging.getLogger(__name__)

class RequestTimingMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        start_time = time.time()
        
        response = self.get_response(request)
        
        duration = time.time() - start_time
        logger.info(f"Request to {request.path} took {duration:.2f}s")
        
        # Add timing header to response
        response['X-Request-Duration'] = f"{duration:.2f}s"
        
        return response

User Activity Tracking Middleware

from django.utils import timezone
from myapp.models import UserActivity

class UserActivityMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)
        
        # Track authenticated user activity
        if request.user.is_authenticated:
            UserActivity.objects.create(
                user=request.user,
                path=request.path,
                method=request.method,
                status_code=response.status_code,
                timestamp=timezone.now()
            )
        
        return response

IP Restriction Middleware

from django.http import HttpResponseForbidden

class IPRestrictionMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        # Define allowed IPs
        self.allowed_ips = ['127.0.0.1', '192.168.1.1']

    def __call__(self, request):
        # Get client IP
        ip = self.get_client_ip(request)
        
        # Check if IP is allowed
        if ip not in self.allowed_ips:
            return HttpResponseForbidden("Access denied")
        
        return self.get_response(request)
    
    def get_client_ip(self, request):
        x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
        if x_forwarded_for:
            ip = x_forwarded_for.split(',')[0]
        else:
            ip = request.META.get('REMOTE_ADDR')
        return ip

JWT Authentication Middleware

import jwt
from django.conf import settings
from django.contrib.auth.models import User
from django.http import JsonResponse

class JWTAuthenticationMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        # Skip authentication for certain paths
        if request.path.startswith('/api/token/') or request.path.startswith('/admin/'):
            return self.get_response(request)
        
        # Get token from Authorization header
        auth_header = request.META.get('HTTP_AUTHORIZATION', '')
        if not auth_header.startswith('Bearer '):
            return JsonResponse({'error': 'Invalid or missing token'}, status=401)
        
        token = auth_header.split(' ')[1]
        
        try:
            # Decode token
            payload = jwt.decode(
                token, 
                settings.SECRET_KEY, 
                algorithms=['HS256']
            )
            
            # Get user
            user_id = payload.get('user_id')
            request.user = User.objects.get(id=user_id)
            
        except (jwt.DecodeError, jwt.ExpiredSignatureError, User.DoesNotExist):
            return JsonResponse({'error': 'Invalid or expired token'}, status=401)
        
        return self.get_response(request)

Django Asynchronous Programming

Async Views

Django supports asynchronous views, allowing you to handle requests without blocking the main thread. This is particularly useful for handling long-running operations, external API calls, or streaming responses.

Basic Async View

async def index(request):
    # Perform async operations
    await asyncio.sleep(1)  # Non-blocking sleep
    return HttpResponse("Hello, async world!")

Class-Based Async Views

class AsyncView(View):
    async def get(self, request):
        # Async logic here
        data = await self.fetch_data()
        return JsonResponse(data)
        
    async def fetch_data(self):
        await asyncio.sleep(0.5)
        return {"message": "Data fetched asynchronously"}

Deployment Considerations

Under WSGI servers, async views run in their own one-off event loop
To get full benefits of async, deploy using ASGI (e.g., Daphne, Uvicorn, or Hypercorn)
All middleware must be async-compatible for full async benefits

# Example ASGI server startup with Uvicorn
uvicorn myproject.asgi:application --host 0.0.0.0 --port 8000

Async ORM Operations

Django 4.1+ supports asynchronous database operations through the ORM:

Async Queries

# Async query with iteration
async def get_authors():
    async for author in Author.objects.filter(active=True):
        books = await author.books.all()
        yield (author, books)

# Async query with aggregation
async def count_books():
    return await Book.objects.acount()
    
# Async creation and updates
async def create_book(title, author_id):
    author = await Author.objects.aget(id=author_id)
    book = Book(title=title, author=author)
    await book.asave()
    return book

Available Async ORM Methods

All QuerySet methods that cause database queries have async versions with an a prefix:

acount() - Count objects asynchronously
aget() - Get a single object asynchronously
afirst() / alast() - Get first/last object asynchronously
acreate() - Create object asynchronously
abulk_create() - Bulk create objects asynchronously
aupdate() - Update objects asynchronously
adelete() - Delete objects asynchronously
aexists() - Check existence asynchronously
aiterator() - Iterate through results asynchronously

Model instances also have async methods:

asave() - Save model instance asynchronously
arefresh_from_db() - Refresh from database asynchronously
adelete() - Delete model instance asynchronously

Handling Async and Sync Code Together

Adapter Functions

Django provides adapter functions to transition between sync and async code:

from asgiref.sync import sync_to_async, async_to_sync

# Call sync code from async context
async def async_view(request):
    # Run a synchronous function in a thread pool
    result = await sync_to_async(some_sync_function, thread_sensitive=True)()
    return HttpResponse(result)

# Call async code from sync context
def sync_view(request):
    # Run an async function in a new event loop
    result = async_to_sync(some_async_function)()
    return HttpResponse(result)

Thread Sensitivity

When using sync_to_async, the thread_sensitive parameter is important for database operations:

# Correct way to use ORM from async code
async def get_user_data(user_id):
    user = await sync_to_async(
        lambda: User.objects.select_related('profile').get(id=user_id),
        thread_sensitive=True
    )()
    return user

Performance Considerations

Context Switching

Each switch between sync and async contexts adds a small overhead (~1ms)
Minimize context switches for optimal performance
Avoid mixing sync middleware with async views

Connection Pooling

# settings.py - Don't use persistent connections with async code
DATABASES = {
    'default': {
        # ... other settings ...
        'CONN_MAX_AGE': 0,  # Disable persistent connections for async code
    }
}

Handling Client Disconnects

For long-running async views, handle client disconnects gracefully:

async def streaming_view(request):
    response = StreamingHttpResponse(streaming_content())
    return response

async def streaming_content():
    try:
        for i in range(100):
            yield f"Data chunk {i}\n"
            await asyncio.sleep(0.5)
    except asyncio.CancelledError:
        # Client disconnected, perform cleanup
        logger.info("Client disconnected")
        raise  # Re-raise to let Django know the connection is closed

Async Safety

Some Django components are not async-safe and will raise SynchronousOnlyOperation if accessed from an async context:

# Wrong - will raise SynchronousOnlyOperation
async def bad_view(request):
    users = User.objects.all()  # Direct ORM access in async context
    return HttpResponse(str(users))

# Correct - use async ORM methods or sync_to_async
async def good_view(request):
    users = await User.objects.all().aiterator()
    return HttpResponse(str(list(users)))

Async Middleware

Create middleware that supports both sync and async contexts:

class SimpleMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        
    def __call__(self, request):
        # Sync path
        return self.get_response(request)
        
    async def __acall__(self, request):
        # Async path
        return await self.get_response(request)

Django Production Hardening and Optimization

Security Hardening

Web Server Configuration

Implement proper web server security headers:

# Example Nginx configuration with security headers
server {
    listen 80;
    server_name example.com;
    
    # Redirect all HTTP traffic to HTTPS
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name example.com;
    
    # SSL configuration
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';
    
    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Content-Security-Policy "default-src 'self'; script-src 'self'; img-src 'self' data:; style-src 'self'; font-src 'self'; connect-src 'self';" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
    
    # Proxy configuration
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Django Settings Hardening

# settings.py for production

# Security settings
DEBUG = False
SECRET_KEY = os.environ.get('SECRET_KEY')  # Never hardcode
ALLOWED_HOSTS = ['example.com', 'www.example.com']

# HTTPS settings
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000  # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'

# Cookie settings
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_HTTPONLY = True
SESSION_COOKIE_SAMESITE = 'Lax'  # Or 'Strict' for more security
CSRF_COOKIE_SAMESITE = 'Lax'

# Content Security Policy
CSP_DEFAULT_SRC = ("'self'",)
CSP_SCRIPT_SRC = ("'self'",)
CSP_STYLE_SRC = ("'self'",)
CSP_IMG_SRC = ("'self'", "data:")
CSP_FONT_SRC = ("'self'",)

Rate Limiting and Throttling

Implement rate limiting to prevent abuse:

# settings.py
INSTALLED_APPS = [
    # ...
    'django_ratelimit',
]

# views.py
from django_ratelimit.decorators import ratelimit

@ratelimit(key='ip', rate='5/m', method='POST', block=True)
def login_view(request):
    # Login logic here
    pass

For DRF APIs:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.AnonRateThrottle',
        'rest_framework.throttling.UserRateThrottle'
    ],
    'DEFAULT_THROTTLE_RATES': {
        'anon': '100/day',
        'user': '1000/day'
    }
}

Performance Optimization

Database Optimization

Connection Pooling

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT', '5432'),
        'CONN_MAX_AGE': 60,  # Keep connections alive for 60 seconds
        'OPTIONS': {
            # For PostgreSQL connection pooling with pgbouncer
            'application_name': 'myapp',
        },
    }
}

For more advanced connection pooling, use django-db-connection-pool:

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'dj_db_conn_pool.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        # ... other settings ...
        'POOL_OPTIONS': {
            'POOL_SIZE': 20,
            'MAX_OVERFLOW': 10,
            'RECYCLE': 300,  # Recycle connections after 5 minutes
        },
    }
}

Database Indexes

Create appropriate indexes for frequently queried fields:

# models.py
class Article(models.Model):
    title = models.CharField(max_length=200, db_index=True)
    slug = models.SlugField(max_length=200, unique=True, db_index=True)
    author = models.ForeignKey(User, on_delete=models.CASCADE)
    created_at = models.DateTimeField(auto_now_add=True, db_index=True)
    
    class Meta:
        indexes = [
            models.Index(fields=['author', 'created_at']),
            models.Index(fields=['created_at', '-title']),
        ]

Caching Strategy

Cache Configuration

# settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': 'redis://redis:6379/1',
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'PARSER_CLASS': 'redis.connection.HiredisParser',
            'SOCKET_CONNECT_TIMEOUT': 5,
            'SOCKET_TIMEOUT': 5,
            'COMPRESSOR': 'django_redis.compressors.zlib.ZlibCompressor',
        },
        'KEY_PREFIX': 'myapp',
        'TIMEOUT': 300,  # 5 minutes default timeout
    }
}

# Cache middleware
MIDDLEWARE = [
    # ...
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',
    # ...
]

# Cache settings
CACHE_MIDDLEWARE_SECONDS = 300
CACHE_MIDDLEWARE_KEY_PREFIX = 'myapp'

Selective Caching

# views.py
from django.views.decorators.cache import cache_page
from django.utils.decorators import method_decorator

@cache_page(60 * 15)  # Cache for 15 minutes
def article_list(request):
    articles = Article.objects.all()
    return render(request, 'articles/list.html', {'articles': articles})

class ArticleDetailView(DetailView):
    model = Article
    
    @method_decorator(cache_page(60 * 30))  # Cache for 30 minutes
    def dispatch(self, *args, **kwargs):
        return super().dispatch(*args, **kwargs)

Template Fragment Caching


<div data-gb-custom-block data-tag="load"></div>

<div>
  

<div data-gb-custom-block data-tag="cache" data-0='300'>
    <!-- Expensive sidebar content -->
    {% include "includes/sidebar.html" %}
  </div>

</div>

<div>
  <!-- Main content that changes frequently -->
  

<div data-gb-custom-block data-tag="include" data-0='includes/main_content.html'></div>

</div>

Static and Media Files

Using a CDN

# settings.py
STATIC_URL = 'https://cdn.example.com/static/'
MEDIA_URL = 'https://cdn.example.com/media/'

# For S3 storage
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',  # 1 day cache
}

Optimizing Static Files

# settings.py
INSTALLED_APPS = [
    # ...
    'compressor',
]

STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'compressor.finders.CompressorFinder',
]

COMPRESSOR_ENABLED = True
COMPRESSOR_OFFLINE = True


<div data-gb-custom-block data-tag="load"></div>

<div data-gb-custom-block data-tag="compress">

<link rel="stylesheet" href="

<div data-gb-custom-block data-tag="static" data-0='css/main.css'></div>">
<link rel="stylesheet" href="<div data-gb-custom-block data-tag="static" data-0='css/extra.css'></div>

">

</div>

<div data-gb-custom-block data-tag="compress">

<script src="

<div data-gb-custom-block data-tag="static" data-0='js/main.js'></div>"></script>
<script src="<div data-gb-custom-block data-tag="static" data-0='js/plugins.js'></div>

"></script>

</div>

Monitoring and Observability

Application Performance Monitoring

# settings.py
# New Relic configuration
import newrelic.agent
newrelic.agent.initialize('newrelic.ini')

# Sentry configuration
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

sentry_sdk.init(
    dsn=os.environ.get('SENTRY_DSN'),
    integrations=[
        DjangoIntegration(),
    ],
    traces_sample_rate=0.2,  # Capture 20% of transactions for performance monitoring
    send_default_pii=True,
    environment=os.environ.get('ENVIRONMENT', 'production'),
    release=os.environ.get('RELEASE_VERSION', '1.0.0'),
)

Structured Logging

# settings.py
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'json': {
            '()': 'pythonjsonlogger.jsonlogger.JsonFormatter',
            'format': '%(asctime)s %(levelname)s %(name)s %(message)s',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'json',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': 'INFO',
        },
        'django.request': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': False,
        },
        'myapp': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}

Health Checks

# settings.py
INSTALLED_APPS = [
    # ...
    'health_check',
    'health_check.db',
    'health_check.cache',
    'health_check.storage',
    'health_check.contrib.celery',
    'health_check.contrib.redis',
]

# urls.py
urlpatterns = [
    # ...
    path('health/', include('health_check.urls')),
]

Scalability Patterns

Horizontal Scaling

Stateless Application Design

Store session data in Redis or database, not in local memory
Use external storage for user uploads and generated files
Implement proper locking mechanisms for distributed systems

# settings.py
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
SESSION_CACHE_ALIAS = 'default'

Distributed Task Processing

# settings.py
CELERY_BROKER_URL = os.environ.get('REDIS_URL', 'redis://redis:6379/0')
CELERY_RESULT_BACKEND = os.environ.get('REDIS_URL', 'redis://redis:6379/0')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'
CELERY_TASK_ACKS_LATE = True
CELERY_WORKER_PREFETCH_MULTIPLIER = 1  # Disable prefetching for fair task distribution
CELERY_TASK_TIME_LIMIT = 60 * 60  # 1 hour max task execution time
CELERY_TASK_SOFT_TIME_LIMIT = 50 * 60  # 50 minutes soft limit

# For task routing
CELERY_TASK_ROUTES = {
    'myapp.tasks.high_priority': {'queue': 'high_priority'},
    'myapp.tasks.low_priority': {'queue': 'low_priority'},
}

Database Scaling

Read Replicas

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        # ... other settings ...
    },
    'replica': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_REPLICA_USER'),
        'PASSWORD': os.environ.get('DB_REPLICA_PASSWORD'),
        'HOST': os.environ.get('DB_REPLICA_HOST'),
        'PORT': os.environ.get('DB_REPLICA_PORT', '5432'),
    }
}

DATABASE_ROUTERS = ['myapp.routers.PrimaryReplicaRouter']

# myapp/routers.py
class PrimaryReplicaRouter:
    def db_for_read(self, model, **hints):
        """Send read queries to replicas."""
        return 'replica'

    def db_for_write(self, model, **hints):
        """Send write queries to primary."""
        return 'default'

    def allow_relation(self, obj1, obj2, **hints):
        """Allow relations between objects in any database."""
        return True

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """Only run migrations on the primary database."""
        return db == 'default'

Disaster Recovery

Automated Backups

# settings.py
INSTALLED_APPS = [
    # ...
    'django_dbbackup',
]

DBBACKUP_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DBBACKUP_STORAGE_OPTIONS = {
    'access_key': os.environ.get('AWS_ACCESS_KEY_ID'),
    'secret_key': os.environ.get('AWS_SECRET_ACCESS_KEY'),
    'bucket_name': os.environ.get('AWS_BACKUP_BUCKET_NAME'),
}
DBBACKUP_CLEANUP_KEEP = 7  # Keep last 7 backups
DBBACKUP_FILENAME_TEMPLATE = '{datetime}.{extension}'
DBBACKUP_MEDIA_FILENAME_TEMPLATE = '{datetime}.{extension}'

Cron job for automated backups:

# /etc/cron.d/django-backups
0 2 * * * www-data cd /path/to/project && python manage.py dbbackup -z && python manage.py mediabackup -z

High Availability Setup

Use multiple application servers behind a load balancer
Configure database replication with automatic failover
Implement redundant caching servers
Set up multiple availability zones or regions

# docker-compose.ha.yml example
version: '3.8'

services:
  web:
    image: myapp:latest
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s
        order: start-first
      restart_policy:
        condition: any
        max_attempts: 3
        window: 120s
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health/"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Django and DRF Settings Optimization

Django Settings for Production

Comprehensive Django Settings

# settings.py - Comprehensive production settings

# Core Django settings
DJANGO_SETTINGS_MODULE = 'myproject.settings.production'
DEBUG = False
SECRET_KEY = os.environ.get('SECRET_KEY')
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(',')

# Internationalization
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True

# Static and media files
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

# File upload settings
FILE_UPLOAD_MAX_MEMORY_SIZE = 5 * 1024 * 1024  # 5 MB
DATA_UPLOAD_MAX_MEMORY_SIZE = 10 * 1024 * 1024  # 10 MB
FILE_UPLOAD_PERMISSIONS = 0o644
FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o755

# Email settings
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = os.environ.get('EMAIL_HOST')
EMAIL_PORT = int(os.environ.get('EMAIL_PORT', 587))
EMAIL_USE_TLS = True
EMAIL_HOST_USER = os.environ.get('EMAIL_HOST_USER')
EMAIL_HOST_PASSWORD = os.environ.get('EMAIL_HOST_PASSWORD')
DEFAULT_FROM_EMAIL = os.environ.get('DEFAULT_FROM_EMAIL')
SERVER_EMAIL = os.environ.get('SERVER_EMAIL')

# Security settings
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000  # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'

# Session and cookie settings
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600  # 2 weeks
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'myapp_sessionid'
SESSION_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_HTTPONLY = True
CSRF_COOKIE_NAME = 'myapp_csrftoken'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_TRUSTED_ORIGINS = [f'https://{host}' for host in ALLOWED_HOSTS]

# Authentication settings
AUTHENTICATION_BACKENDS = [
    'django.contrib.auth.backends.ModelBackend',
]
AUTH_PASSWORD_VALIDATORS = [
    {'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator'},
    {'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 'OPTIONS': {'min_length': 12}},
    {'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator'},
    {'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator'},
]
PASSWORD_HASHERS = [
    'django.contrib.auth.hashers.Argon2PasswordHasher',
    'django.contrib.auth.hashers.PBKDF2PasswordHasher',
    'django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher',
]

# Admin settings
ADMINS = [(os.environ.get('ADMIN_NAME', 'Admin'), os.environ.get('ADMIN_EMAIL', '[email protected]'))]
MANAGERS = ADMINS

# Logging settings
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
            'style': '{',
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
        },
        'file': {
            'level': 'INFO',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.path.join(BASE_DIR, 'logs/django.log'),
            'maxBytes': 10 * 1024 * 1024,  # 10 MB
            'backupCount': 10,
            'formatter': 'verbose',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console', 'file'],
            'level': 'INFO',
            'propagate': True,
        },
        'django.request': {
            'handlers': ['console', 'file'],
            'level': 'WARNING',
            'propagate': False,
        },
        'django.security': {
            'handlers': ['console', 'file'],
            'level': 'WARNING',
            'propagate': False,
        },
        'myapp': {
            'handlers': ['console', 'file'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}

# Cache settings
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL', 'redis://redis:6379/1'),
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'PARSER_CLASS': 'redis.connection.HiredisParser',
            'SOCKET_CONNECT_TIMEOUT': 5,
            'SOCKET_TIMEOUT': 5,
            'CONNECTION_POOL_KWARGS': {'max_connections': 100},
            'COMPRESSOR': 'django_redis.compressors.zlib.ZlibCompressor',
        },
        'KEY_PREFIX': 'myapp',
        'TIMEOUT': 300,  # 5 minutes
    },
    'sessions': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL', 'redis://redis:6379/2'),
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'PARSER_CLASS': 'redis.connection.HiredisParser',
        },
        'KEY_PREFIX': 'session',
        'TIMEOUT': 1209600,  # 2 weeks
    },
}

# Database settings
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT', '5432'),
        'CONN_MAX_AGE': 60,
        'OPTIONS': {
            'connect_timeout': 10,
            'sslmode': 'require',
        },
        'ATOMIC_REQUESTS': False,  # Set to True if you want all views to be in transactions
        'AUTOCOMMIT': True,
        'TIME_ZONE': 'UTC',
    }
}

# Template settings
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
            'debug': False,
            'loaders': [
                ('django.template.loaders.cached.Loader', [
                    'django.template.loaders.filesystem.Loader',
                    'django.template.loaders.app_directories.Loader',
                ]),
            ],
        },
    },
]

Django REST Framework Settings

Comprehensive DRF Settings

# settings.py - DRF settings

REST_FRAMEWORK = {
    # Authentication
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework_simplejwt.authentication.JWTAuthentication',
        'rest_framework.authentication.SessionAuthentication',
    ],
    
    # Permission policies
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.IsAuthenticated',
    ],
    
    # Content negotiation
    'DEFAULT_CONTENT_NEGOTIATION_CLASS': 'rest_framework.negotiation.DefaultContentNegotiation',
    'DEFAULT_PARSER_CLASSES': [
        'rest_framework.parsers.JSONParser',
        'rest_framework.parsers.FormParser',
        'rest_framework.parsers.MultiPartParser',
    ],
    'DEFAULT_RENDERER_CLASSES': [
        'rest_framework.renderers.JSONRenderer',
        'rest_framework.renderers.BrowsableAPIRenderer',
    ],
    
    # Throttling
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.AnonRateThrottle',
        'rest_framework.throttling.UserRateThrottle',
    ],
    'DEFAULT_THROTTLE_RATES': {
        'anon': '100/day',
        'user': '1000/day',
    },
    
    # Pagination
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 20,
    
    # Filtering
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
    
    # Versioning
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
    'DEFAULT_VERSION': 'v1',
    'ALLOWED_VERSIONS': ['v1', 'v2'],
    'VERSION_PARAM': 'version',
    
    # Schema
    'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
    
    # Exception handling
    'EXCEPTION_HANDLER': 'myapp.api.exception_handlers.custom_exception_handler',
    'NON_FIELD_ERRORS_KEY': 'error',
    
    # Testing
    'TEST_REQUEST_DEFAULT_FORMAT': 'json',
    'TEST_REQUEST_RENDERER_CLASSES': [
        'rest_framework.renderers.JSONRenderer',
    ],
    
    # Caching
    'DEFAULT_CACHE_RESPONSE_TIMEOUT': 60 * 15,  # 15 minutes
    
    # Validators
    'DEFAULT_VALIDATOR_CLASS': 'rest_framework.validators.UniqueValidator',
    
    # Metadata
    'DEFAULT_METADATA_CLASS': 'rest_framework.metadata.SimpleMetadata',
    
    # Date and time formats
    'DATETIME_FORMAT': '%Y-%m-%dT%H:%M:%S.%fZ',
    'DATE_FORMAT': '%Y-%m-%d',
    'TIME_FORMAT': '%H:%M:%S',
    
    # Miscellaneous
    'UNICODE_JSON': True,
    'COMPACT_JSON': True,
    'STRICT_JSON': True,
    'COERCE_DECIMAL_TO_STRING': True,
    'URL_FIELD_NAME': 'url',
}

# JWT settings
SIMPLE_JWT = {
    'ACCESS_TOKEN_LIFETIME': timedelta(minutes=60),
    'REFRESH_TOKEN_LIFETIME': timedelta(days=1),
    'ROTATE_REFRESH_TOKENS': False,
    'BLACKLIST_AFTER_ROTATION': True,
    'UPDATE_LAST_LOGIN': False,
    'ALGORITHM': 'HS256',
    'SIGNING_KEY': SECRET_KEY,
    'VERIFYING_KEY': None,
    'AUDIENCE': None,
    'ISSUER': None,
    'JWK_URL': None,
    'LEEWAY': 0,
    'AUTH_HEADER_TYPES': ('Bearer',),
    'AUTH_HEADER_NAME': 'HTTP_AUTHORIZATION',
    'USER_ID_FIELD': 'id',
    'USER_ID_CLAIM': 'user_id',
    'USER_AUTHENTICATION_RULE': 'rest_framework_simplejwt.authentication.default_user_authentication_rule',
    'AUTH_TOKEN_CLASSES': ('rest_framework_simplejwt.tokens.AccessToken',),
    'TOKEN_TYPE_CLAIM': 'token_type',
    'TOKEN_USER_CLASS': 'rest_framework_simplejwt.models.TokenUser',
    'JTI_CLAIM': 'jti',
    'SLIDING_TOKEN_REFRESH_EXP_CLAIM': 'refresh_exp',
    'SLIDING_TOKEN_LIFETIME': timedelta(minutes=5),
    'SLIDING_TOKEN_REFRESH_LIFETIME': timedelta(days=1),
}

# Spectacular settings for OpenAPI schema
SPECTACULAR_SETTINGS = {
    'TITLE': 'My API',
    'DESCRIPTION': 'My API description',
    'VERSION': '1.0.0',
    'SERVE_INCLUDE_SCHEMA': False,
    'SCHEMA_PATH_PREFIX': r'/api/v[0-9]',
    'COMPONENT_SPLIT_REQUEST': True,
    'COMPONENT_SPLIT_PATCH': False,
    'AUTHENTICATION_WHITELIST': [
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    ],
    'SWAGGER_UI_SETTINGS': {
        'deepLinking': True,
        'persistAuthorization': True,
        'displayOperationId': True,
    },
    'APPEND_COMPONENTS': {
        'securitySchemes': {
            'bearerAuth': {
                'type': 'http',
                'scheme': 'bearer',
                'bearerFormat': 'JWT',
            }
        }
    },
    'SECURITY': [{'bearerAuth': []}],
}

API Query Optimization

Custom Model Managers and QuerySets

# models.py
from django.db import models

class ArticleQuerySet(models.QuerySet):
    def published(self):
        return self.filter(status='published')
    
    def featured(self):
        return self.filter(is_featured=True)
    
    def by_category(self, category_slug):
        return self.filter(category__slug=category_slug)
    
    def with_related(self):
        return self.select_related('author', 'category').prefetch_related('tags')
    
    def with_counts(self):
        return self.annotate(
            comment_count=models.Count('comments', distinct=True),
            like_count=models.Count('likes', distinct=True)
        )

class ArticleManager(models.Manager):
    def get_queryset(self):
        return ArticleQuerySet(self.model, using=self._db)
    
    def published(self):
        return self.get_queryset().published()
    
    def featured(self):
        return self.get_queryset().featured()
    
    def with_related(self):
        return self.get_queryset().with_related()
    
    def with_counts(self):
        return self.get_queryset().with_counts()

class Article(models.Model):
    title = models.CharField(max_length=200)
    slug = models.SlugField(max_length=200, unique=True)
    content = models.TextField()
    author = models.ForeignKey('auth.User', on_delete=models.CASCADE)
    category = models.ForeignKey('Category', on_delete=models.CASCADE)
    tags = models.ManyToManyField('Tag')
    status = models.CharField(max_length=10, choices=[
        ('draft', 'Draft'),
        ('published', 'Published'),
    ], default='draft')
    is_featured = models.BooleanField(default=False)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    
    objects = ArticleManager()
    
    class Meta:
        indexes = [
            models.Index(fields=['status', 'created_at']),
            models.Index(fields=['is_featured']),
            models.Index(fields=['author', 'status']),
            models.Index(fields=['category', 'status']),
        ]

Optimized Filtering with Django-Filter

# filters.py
import django_filters
from django.db.models import Q, Count
from .models import Article

class ArticleFilter(django_filters.FilterSet):
    search = django_filters.CharFilter(method='filter_search')
    category = django_filters.CharFilter(field_name='category__slug')
    tags = django_filters.CharFilter(method='filter_tags')
    author = django_filters.CharFilter(field_name='author__username')
    created_after = django_filters.DateFilter(field_name='created_at', lookup_expr='gte')
    created_before = django_filters.DateFilter(field_name='created_at', lookup_expr='lte')
    min_comments = django_filters.NumberFilter(method='filter_min_comments')
    
    class Meta:
        model = Article
        fields = ['status', 'is_featured', 'category', 'author']
    
    def filter_search(self, queryset, name, value):
        if not value:
            return queryset
        return queryset.filter(
            Q(title__icontains=value) | 
            Q(content__icontains=value) | 
            Q(author__username__icontains=value) |
            Q(category__name__icontains=value)
        )
    
    def filter_tags(self, queryset, name, value):
        if not value:
            return queryset
        tag_slugs = value.split(',')
        # Efficient filtering for multiple tags
        for tag_slug in tag_slugs:
            queryset = queryset.filter(tags__slug=tag_slug)
        return queryset
    
    def filter_min_comments(self, queryset, name, value):
        return queryset.annotate(num_comments=Count('comments')).filter(num_comments__gte=value)

Optimized ViewSets

# views.py
from rest_framework import viewsets
from rest_framework.decorators import action
from rest_framework.response import Response
from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page
from .models import Article
from .serializers import ArticleSerializer, ArticleDetailSerializer
from .filters import ArticleFilter

class ArticleViewSet(viewsets.ModelViewSet):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    filterset_class = ArticleFilter
    search_fields = ['title', 'content', 'author__username', 'category__name']
    ordering_fields = ['created_at', 'updated_at', 'title']
    ordering = ['-created_at']
    
    def get_queryset(self):
        # Base queryset with proper select_related/prefetch_related
        queryset = Article.objects.with_related()
        
        # Add annotations if needed
        if self.action in ['list', 'featured']:
            queryset = queryset.with_counts()
        
        # Filter by published status for non-staff users
        if not self.request.user.is_staff:
            queryset = queryset.published()
            
        return queryset
    
    def get_serializer_class(self):
        if self.action in ['retrieve', 'create', 'update', 'partial_update']:
            return ArticleDetailSerializer
        return ArticleSerializer
    
    @method_decorator(cache_page(60 * 15))  # Cache for 15 minutes
    @action(detail=False)
    def featured(self, request):
        featured = self.get_queryset().featured()
        page = self.paginate_queryset(featured)
        if page is not None:
            serializer = self.get_serializer(page, many=True)
            return self.get_paginated_response(serializer.data)
        serializer = self.get_serializer(featured, many=True)
        return Response(serializer.data)
    
    @action(detail=True)
    def related(self, request, pk=None):
        article = self.get_object()
        # Get articles with same category or tags, excluding current article
        related = Article.objects.published().filter(
            Q(category=article.category) | Q(tags__in=article.tags.all())
        ).exclude(id=article.id).distinct()[:5]
        serializer = self.get_serializer(related, many=True)
        return Response(serializer.data)

Advanced Query Optimization

Using Subqueries and Annotations

from django.db.models import OuterRef, Subquery, Count, Exists

# Get articles with their latest comment
latest_comment = Comment.objects.filter(
    article=OuterRef('pk')
).order_by('-created_at').values('content')[:1]

articles = Article.objects.annotate(
    latest_comment=Subquery(latest_comment),
    comment_count=Count('comments'),
    has_featured_image=Exists(Image.objects.filter(article=OuterRef('pk'), is_featured=True))
)

Conditional Aggregation

from django.db.models import Sum, Case, When, IntegerField

# Count different types of reactions
articles = Article.objects.annotate(
    likes=Sum(Case(
        When(reactions__type='like', then=1),
        default=0,
        output_field=IntegerField()
    )),
    dislikes=Sum(Case(
        When(reactions__type='dislike', then=1),
        default=0,
        output_field=IntegerField()
    ))
)

Bulk Operations

# Bulk create
Article.objects.bulk_create([
    Article(title='Article 1', content='Content 1', author=author),
    Article(title='Article 2', content='Content 2', author=author),
    # ... more articles
], batch_size=100)

# Bulk update
articles = list(Article.objects.filter(status='draft'))
for article in articles:
    article.status = 'published'
Article.objects.bulk_update(articles, ['status'], batch_size=100)

Optimized Filtering for Complex Queries

# Efficient filtering for articles with specific tags and category
from django.db.models import Count

# Get articles that have ALL specified tags (using aggregation)
def get_articles_with_all_tags(tag_slugs):
    return Article.objects.filter(tags__slug__in=tag_slugs)\
        .annotate(tag_count=Count('tags'))\
        .filter(tag_count=len(tag_slugs))

# Get trending articles based on recent activity
def get_trending_articles():
    one_week_ago = timezone.now() - timedelta(days=7)
    return Article.objects.annotate(
        recent_comments=Count('comments', filter=Q(comments__created_at__gte=one_week_ago)),
        recent_views=Count('views', filter=Q(views__timestamp__gte=one_week_ago))
    ).filter(
        Q(recent_comments__gt=0) | Q(recent_views__gt=10)
    ).order_by('-recent_views', '-recent_comments')

Custom Pagination for Large Datasets

from rest_framework.pagination import CursorPagination

class ArticleCursorPagination(CursorPagination):
    page_size = 20
    ordering = '-created_at'
    cursor_query_param = 'cursor'
    
    # For better performance with large datasets
    def paginate_queryset(self, queryset, request, view=None):
        # Optimize queryset before pagination
        if hasattr(queryset, 'with_related'):
            queryset = queryset.with_related()
        return super().paginate_queryset(queryset, request, view)

Using Database Functions

from django.db.models.functions import TruncMonth, ExtractYear

# Group articles by month
articles_by_month = Article.objects.annotate(
    month=TruncMonth('created_at')
).values('month').annotate(
    count=Count('id')
).order_by('month')

# Search using trigram similarity (PostgreSQL)
from django.contrib.postgres.search import TrigramSimilarity

def search_articles(query):
    return Article.objects.annotate(
        similarity=TrigramSimilarity('title', query) + 
                 TrigramSimilarity('content', query) * 0.8
    ).filter(similarity__gt=0.3).order_by('-similarity')

Comprehensive Backend Development Guide Summary

This README provides a comprehensive guide to Django backend development best practices, covering all aspects from code formatting to deployment. Here's a summary of the key sections:

Code Quality and Style

Black Code Formatter: Configuration, usage, and integration with other tools
Python Code Quality: Style guides, linting tools, and best practices
Django Coding Style: Conventions for models, views, templates, and imports

Django Core Concepts

Models: Best practices for design, optimization, and inheritance
Views: Class-based vs. function-based views, organization, and performance
Templates: Organization, inheritance, and performance optimization
URLs: Naming conventions and structure
Forms: Validation, rendering, and accessibility

Database Optimization

Query Optimization: Using select_related, prefetch_related, and other techniques
Indexing: When and how to use database indexes
Database Functions: Leveraging database-level operations
Connection Pooling: Managing database connections efficiently

Security

Authentication: Built-in systems, multi-factor authentication
Authorization: Permission-based access control
Protection Against Attacks: XSS, CSRF, SQL injection prevention
Secure Deployment: HTTPS, security headers, environment variables

API Development

REST Framework: Serializers, viewsets, authentication, permissions
GraphQL: Schema definition, queries, mutations, optimization
API Versioning: Maintaining backward compatibility
Performance: Pagination, filtering, caching

Internationalization

Setting Up i18n: Configuration, translation files
Marking Strings: In Python code, templates, and JavaScript
Language Switching: URL-based switching, language selectors

Accessibility

HTML Structure: Semantic HTML, heading hierarchy
Forms: Labels, error messages, ARIA attributes
Testing: Automated and manual accessibility testing

Testing

Test Organization: Structure and naming conventions
Test Types: Unit, integration, API testing
Tools: pytest, factories, mocking
Coverage: Measuring and maintaining good test coverage

GitBook Compatibility and Formatting

When publishing Django documentation to GitBook, you may encounter formatting issues. Here are common problems and solutions:

Common GitBook Formatting Issues

Code Block Rendering

GitBook may have issues with certain code block formats. To ensure proper rendering:

# Correct format for GitBook code blocks

```python
def example_function():
    return "This will render correctly"


Avoid using indentation for code blocks as GitBook may not recognize them correctly.

### Heading Hierarchy

GitBook is strict about heading hierarchy. Always follow a logical structure:

```markdown
# Main Title (H1)
## Section (H2)
### Subsection (H3)
#### Smaller section (H4)

Never skip levels (like going from H2 directly to H4).

Special Characters

Some special characters may cause rendering issues in GitBook:

Use HTML entities for special characters: < for <, > for >, & for &
Escape backticks in inline code: `\`example\ ``
For dollar signs in code blocks (common in template variables), use \$ instead of $

Table Formatting

Ensure tables have the correct format with proper alignment indicators:

| Column 1 | Column 2 | Column 3 |
|----------|:--------:|---------:|
| Left     | Center   | Right    |
| aligned  | aligned  | aligned  |

Image Paths

GitBook handles image paths differently than GitHub:

<!-- For GitBook compatibility -->
![Alt text](./images/diagram.png)

<!-- Or use absolute URLs -->
![Alt text](https://example.com/images/diagram.png)

GitBook-Specific Syntax

Hints and Callouts

GitBook supports special hint blocks:


<div data-gb-custom-block data-tag="hint" data-style='info'>

This is an information block that will be properly styled in GitBook.

</div>

<div data-gb-custom-block data-tag="hint" data-style='warning'>

This is a warning block.

</div>

<div data-gb-custom-block data-tag="hint" data-style='danger'>

This is a danger/error block.

</div>

<div data-gb-custom-block data-tag="hint" data-style='success'>

This is a success/tip block.

</div>

Tabs

For content that should be displayed in tabs:


<div data-gb-custom-block data-tag="tabs">

<div data-gb-custom-block data-tag="tab" data-title='Python'>

```python
print("Hello from Python")

console.log("Hello from JavaScript");

```

Page Links

GitBook has a specific format for internal page links:

[Link to another page](./path/to/page.md)

Automatic Formatting Tools

GitBook CLI

Use the GitBook CLI to validate your markdown before publishing:

# Install GitBook CLI
npm install -g gitbook-cli

# Validate a book structure
gitbook build ./my-book --log=debug

Markdown Linters

Use markdown linters to catch formatting issues:

# Install markdownlint
npm install -g markdownlint-cli

# Lint markdown files
markdownlint *.md

Best Practices for Django Documentation in GitBook

Consistent Code Formatting: Use triple backticks with language specifiers for all code blocks
Proper Escaping: Escape Django template tags properly with \{\{ and \}\}
Image Management: Store images in a dedicated directory and use relative paths
Table of Contents: Create a proper SUMMARY.md file for GitBook navigation
Metadata: Include a book.json file with proper metadata
Testing: Preview your documentation locally before publishing
Plugins: Consider using GitBook plugins for Django-specific formatting needs

// book.json example
{
  "title": "Django Backend Best Practices",
  "description": "A comprehensive guide to Django backend development",
  "plugins": [
    "prism", 
    "code-block-syntax", 
    "hints", 
    "github"
  ],
  "pluginsConfig": {
    "github": {
      "url": "https://github.com/yourusername/your-repo"
    },
    "theme-default": {
      "showLevel": true
    }
  }
}

Deployment

Containerization: Docker setup, environment configuration
CI/CD: GitHub Actions pipelines
Monitoring: Error reporting, health checks
Security: Container security best practices

Debugging and Troubleshooting

Debugging Tools: Django Debug Toolbar, Django Extensions
Logging: Configuration and usage
Common Issues: Database, migrations, static files

Advanced Features

Asynchronous Programming: Channels, ASGI
Task Processing: Celery for background tasks
Caching: Strategies and implementation
Middleware: Custom middleware development

This guide serves as a comprehensive reference for Django backend developers, from beginners to experts, covering all aspects of building high-quality, maintainable, and performant web applications with Django.

Last updated 2 months ago

Basic Usage

Black Code Style Key Points

Line Length

Wrapping and Indentation

String Formatting

Empty Lines

Comments

Project Configuration

Integration with Other Tools

isort

Flake8

Pre-commit Hook

CI Integration

Editor Integration

VS Code

PyCharm

Best Practices

Ignoring Formatting

Advanced Configuration Options

Line Length Customization

Target Python Versions

Skip String Normalization

Fast Mode

Edge Cases and Special Formatting

Magic Comments

Handling Long Strings

Handling Complex Expressions

Troubleshooting Common Issues

Handling Syntax Errors

Conflicts with Other Formatters

Large Files Performance

Handling Generated Code

Black Compatibility

Python Version Compatibility

Framework-Specific Considerations

Performance Considerations

Caching

Formatting Large Codebases

Team Adoption Strategies

Gradual Implementation

Onboarding New Team Members

Handling Resistance

Continuous Integration Best Practices

GitHub Actions Workflow with Caching

Automated PR Formatting

Django Query Optimization Best Practices

Database Query Optimization

Use select_related for ForeignKey Relationships

Use prefetch_related for ManyToMany Relationships

Use Prefetch Objects for Complex Prefetching

Defer and Only

Use Database-Level Aggregation

Annotate for Calculated Fields

Use values or values_list for Simple Data

Bulk Operations

Use exists() Instead of count() or len()

Query Optimization with Q Objects

Database Indexes

Use iterator() for Large Querysets

Database Connection Pooling

Optimize count() Queries

Use Database-Specific Features When Appropriate

Django REST Framework Production Best Practices

Performance Optimization

Pagination

Serializer Optimization

Use SerializerMethodField Carefully

Caching

Use Throttling

Optimize Filtering

Versioning

Authentication and Permissions

Content Negotiation

Asynchronous Tasks

Error Handling

Documentation

Testing

DRY (Don't Repeat Yourself) Principles

Understanding DRY

Benefits of DRY Code

Use `select_related` for ForeignKey Relationships

Use `prefetch_related` for ManyToMany Relationships

Use `Prefetch` Objects for Complex Prefetching

Use `values` or `values_list` for Simple Data

Use `exists()` Instead of `count()` or `len()`

Query Optimization with `Q` Objects

Use `iterator()` for Large Querysets

Optimize `count()` Queries

Use `SerializerMethodField` Carefully