Docker deployment

Deploy Gradio apps using Docker containers for consistency and portability

Docker provides a consistent way to package and deploy Gradio apps across different environments. This guide shows you how to containerize your Gradio applications and deploy them anywhere.

Why use Docker?

Deploying Gradio apps with Docker offers several advantages:

• Consistency: Your app runs identically across development, staging, and production
• Portability: Move containers between local machines, cloud providers, and servers
• Scalability: Use orchestration tools like Kubernetes to scale horizontally
• Isolation: Dependencies are contained within the image
• Reproducibility: Anyone can run your app with a single command

Prerequisites

Before you begin:

• Install Docker Desktop (includes Docker Engine and Docker Compose)
• Basic familiarity with Docker concepts
• A working Gradio application

Quick start

Create a simple Gradio app and containerize it:

1. Create your Gradio app

   Create app.py:

   import gradio as gr
   
   def greet(name):
       return f"Hello {name}!"
   
   demo = gr.Interface(
       fn=greet,
       inputs="text",
       outputs="text"
   )
   
   demo.launch()
   
2. Create a Dockerfile

   Create Dockerfile in the same directory:

   FROM python:3.10-slim
   
   WORKDIR /app
   
   COPY requirements.txt .
   RUN pip install --no-cache-dir -r requirements.txt
   
   COPY . .
   
   EXPOSE 7860
   
   ENV GRADIO_SERVER_NAME="0.0.0.0"
   
   CMD ["python", "app.py"]
   
3. Create requirements.txt

   List your Python dependencies:

   gradio>=4.0.0
   
4. Build and run

   Build the Docker image:

   docker build -t gradio-app .
   

   Run the container:

   docker run -p 7860:7860 gradio-app
   

   Access your app at http://localhost:7860

Dockerfile explained

Let’s break down the Dockerfile:

# Use official Python runtime as base image
FROM python:3.10-slim

# Set working directory in container
WORKDIR /app

# Copy requirements and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Expose Gradio's default port
EXPOSE 7860

# Set environment variable for Gradio
ENV GRADIO_SERVER_NAME="0.0.0.0"

# Run the application
CMD ["python", "app.py"]

Key elements

• FROM: Base image (use python:3.10-slim for smaller size)
• WORKDIR: Sets /app as the working directory
• COPY requirements.txt: Copies dependencies first (better caching)
• RUN pip install: Installs Python packages
• COPY .: Copies all app files
• EXPOSE 7860: Documents that the app listens on port 7860
• ENV GRADIO_SERVER_NAME: Makes Gradio accept external connections
• CMD: Command to run when container starts

Setting GRADIO_SERVER_NAME="0.0.0.0" is crucial - it allows connections from outside the container. Without this, you won’t be able to access the app.

Production-ready Dockerfile

For production deployments, use this enhanced Dockerfile:

FROM python:3.10-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Create non-root user
RUN useradd -m -u 1000 gradio && \
    mkdir -p /app && \
    chown -R gradio:gradio /app

WORKDIR /app

# Copy and install dependencies
COPY --chown=gradio:gradio requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application files
COPY --chown=gradio:gradio . .

# Switch to non-root user
USER gradio

# Expose port
EXPOSE 7860

# Set environment variables
ENV GRADIO_SERVER_NAME="0.0.0.0" \
    GRADIO_SERVER_PORT=7860

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:7860/ || exit 1

# Run application
CMD ["python", "app.py"]

Improvements:

• Non-root user for security
• System dependencies handling
• Health check for monitoring
• Proper file permissions

Multi-stage builds

Reduce image size with multi-stage builds:

# Build stage
FROM python:3.10 AS builder

WORKDIR /app

COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.10-slim

WORKDIR /app

# Copy installed packages from builder
COPY --from=builder /root/.local /root/.local

# Copy application
COPY . .

# Update PATH
ENV PATH=/root/.local/bin:$PATH

EXPOSE 7860
ENV GRADIO_SERVER_NAME="0.0.0.0"

CMD ["python", "app.py"]

Docker Compose

For apps with multiple services (database, cache, etc.):

version: '3.8'

services:
  gradio:
    build: .
    ports:
      - "7860:7860"
    environment:
      - GRADIO_SERVER_NAME=0.0.0.0
    volumes:
      - ./data:/app/data
    restart: unless-stopped
    
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    restart: unless-stopped

Run with:

docker-compose up -d

Environment variables

Manage configuration with environment variables:

In Dockerfile

ENV MODEL_NAME="gpt2" \
    MAX_LENGTH=100 \
    TEMPERATURE=0.7

At runtime

docker run -p 7860:7860 \
  -e MODEL_NAME="gpt2" \
  -e MAX_LENGTH=100 \
  gradio-app

Using .env file

Create .env:

MODEL_NAME=gpt2
MAX_LENGTH=100
TEMPERATURE=0.7

Run with:

docker run -p 7860:7860 --env-file .env gradio-app

In your app

import os
import gradio as gr

MODEL_NAME = os.getenv("MODEL_NAME", "gpt2")
MAX_LENGTH = int(os.getenv("MAX_LENGTH", "100"))

def generate(prompt):
    # Use MODEL_NAME and MAX_LENGTH
    return f"Generated with {MODEL_NAME}"

demo = gr.Interface(
    fn=generate,
    inputs="text",
    outputs="text"
)

demo.launch()

Volume mounting

Persist data between container restarts:

Mount data directory

docker run -p 7860:7860 \
  -v $(pwd)/data:/app/data \
  gradio-app

Mount model cache

docker run -p 7860:7860 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  gradio-app

GPU support

For apps using GPU models:

Dockerfile with CUDA

FROM nvidia/cuda:11.8.0-runtime-ubuntu22.04

RUN apt-get update && apt-get install -y python3 python3-pip

WORKDIR /app

COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 7860
ENV GRADIO_SERVER_NAME="0.0.0.0"

CMD ["python3", "app.py"]

Run with GPU

docker run -p 7860:7860 --gpus all gradio-app

Requires NVIDIA Container Toolkit installed on the host.

Deployment scenarios

AWS ECS

1. Push image to ECR:

   aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <account-id>.dkr.ecr.us-east-1.amazonaws.com
   
   docker tag gradio-app:latest <account-id>.dkr.ecr.us-east-1.amazonaws.com/gradio-app:latest
   
   docker push <account-id>.dkr.ecr.us-east-1.amazonaws.com/gradio-app:latest
   

2. Create ECS task definition with your image

3. Create ECS service

4. Configure load balancer with session stickiness

Enable session stickiness on your load balancer! Gradio requires multiple connections from the same client to route to the same instance. Set sessionAffinity: ClientIP or equivalent.

Google Cloud Run

1. Build and push:

   gcloud builds submit --tag gcr.io/PROJECT-ID/gradio-app
   

2. Deploy:

   gcloud run deploy gradio-app \
     --image gcr.io/PROJECT-ID/gradio-app \
     --platform managed \
     --port 7860 \
     --allow-unauthenticated
   

Azure Container Instances

az container create \
  --resource-group myResourceGroup \
  --name gradio-app \
  --image myregistry.azurecr.io/gradio-app:latest \
  --dns-name-label gradio-app \
  --ports 7860

DigitalOcean App Platform

1. Push to Docker Hub or DigitalOcean Container Registry
2. Create new app in App Platform
3. Select Docker Hub as source
4. Configure HTTP port: 7860
5. Deploy

Behind a reverse proxy

When deploying behind Nginx or similar:

Nginx configuration

server {
    listen 80;
    server_name demo.example.com;
    
    location / {
        proxy_pass http://localhost:7860;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Docker Compose with Nginx

version: '3.8'

services:
  gradio:
    build: .
    expose:
      - 7860
    environment:
      - GRADIO_SERVER_NAME=0.0.0.0
    
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    depends_on:
      - gradio

Best practices

1. Use .dockerignore

   Create .dockerignore to exclude unnecessary files:

   __pycache__
   *.pyc
   .git
   .env
   .DS_Store
   venv/
   *.md
   
2. Pin dependency versions

   In requirements.txt:

   gradio==4.44.0
   transformers==4.35.0
   torch==2.1.0
   
3. Use specific base image tags

   Instead of python:3.10, use python:3.10.13-slim

4. Minimize layers

   Combine RUN commands:

   RUN apt-get update && \
       apt-get install -y package1 package2 && \
       rm -rf /var/lib/apt/lists/*
   
5. Add health checks

   Monitor container health:

   HEALTHCHECK CMD curl -f http://localhost:7860/ || exit 1
   

Troubleshooting

Can’t access app from browser

• Ensure GRADIO_SERVER_NAME="0.0.0.0" is set
• Verify port mapping: -p 7860:7860
• Check firewall rules

Out of memory

• Increase Docker memory limit in Docker Desktop settings
• Use smaller base image (-slim or -alpine)
• Reduce model size or use quantization

Slow builds

• Use .dockerignore
• Order Dockerfile commands from least to most frequently changed
• Use multi-stage builds
• Enable BuildKit: DOCKER_BUILDKIT=1 docker build

Container exits immediately

• Check logs: docker logs <container-id>
• Verify app.py runs locally
• Ensure all dependencies are in requirements.txt

Next steps

Hugging Face Spaces

Deploy to managed infrastructure

Sharing apps

Add authentication and embedding