tools
/
drakkenheim-rag
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
drakkenheim-rag/DEPLOYMENT.md

6.8 KiB
Raw Permalink Blame History

Docker Deployment Guide

This guide explains how to deploy the Drakkenheim RAG application using Docker Compose with persistent storage.

Prerequisites

  • Docker and Docker Compose installed on your server
  • OpenAI API key

Project Structure

The project follows Python best practices with a src/ layout:

drakkenheim/
├── src/
│   └── drakkenheim/          # Main package
│       ├── __init__.py       # Package initialization
│       ├── auth.py           # Authentication & session management
│       ├── llm.py            # LLM, embeddings & vector operations
│       ├── document.py       # Document processing
│       ├── storage.py        # File persistence
│       └── ui.py             # User interface components
├── app.py                    # Main application entry point
├── generate_password.py      # Password generation utility
├── docker-compose.yml        # Docker orchestration
├── Dockerfile               # Container definition
└── requirements/            # Dependencies

Benefits of the src/ layout:

  • Clean separation between source code and project files
  • Prevents import issues during development and testing
  • Industry standard for Python projects
  • Better packaging and distribution support
  • Easier testing and CI/CD integration

Quick Start

  1. Clone and navigate to the project:

    git clone <your-repo-url>
cd drakkenheim

  2. Set up environment variables:

    # Copy the example environment file
cp env.example .env

# Generate secure password hash
python3 generate_password.py

# Edit .env and add your OpenAI API key and generated auth config
nano .env

  3. Build and start the application:

    docker compose up -d

  4. Access the application:

    • Open your browser to http://your-server-ip:8501
    • The application will be available and ready to use

Persistent Storage

The application uses Docker volumes to persist data between restarts:

  • ChromaDB Database: chroma_data volume
  • Processed Files Metadata: processed_files volume
  • Uploaded Files: uploaded_files volume

Management Commands

Start the application:

docker compose up -d

Stop the application:

docker compose down

View logs:

docker compose logs -f rag-app

Restart the application:

docker compose restart rag-app

Update the application:

# Pull latest changes
git pull

# Rebuild and restart
docker compose up -d --build

Backup data:

# Create backup directory
mkdir -p backups

# Backup ChromaDB data
docker run --rm -v drakkenheim_chroma_data:/data -v $(pwd)/backups:/backup alpine tar czf /backup/chroma_data_$(date +%Y%m%d_%H%M%S).tar.gz -C /data .

# Backup processed files
docker run --rm -v drakkenheim_processed_files:/data -v $(pwd)/backups:/backup alpine tar czf /backup/processed_files_$(date +%Y%m%d_%H%M%S).tar.gz -C /data .

Restore data:

# Restore ChromaDB data
docker run --rm -v drakkenheim_chroma_data:/data -v $(pwd)/backups:/backup alpine tar xzf /backup/chroma_data_YYYYMMDD_HHMMSS.tar.gz -C /data

# Restore processed files
docker run --rm -v drakkenheim_processed_files:/data -v $(pwd)/backups:/backup alpine tar xzf /backup/processed_files_YYYYMMDD_HHMMSS.tar.gz -C /data

Configuration

Environment Variables

Edit .env file to customize:

# Required
OPENAI_API_KEY=your_api_key_here

# Authentication (REQUIRED for production)
APP_PASSWORD_HASH=your_hashed_password_here
PASSWORD_SALT=your_random_salt_here
SESSION_TIMEOUT=3600

# Optional overrides
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
OPENAI_LLM_MODEL=gpt-4o-mini
STREAMLIT_SERVER_PORT=8501

Authentication Setup

Generate secure credentials:

# Run the password generator
python3 generate_password.py

# Follow the prompts to set your password
# Copy the generated values to your .env file

Security Features:

  • Password hashing: SHA-256 with salt
  • Session timeout: Automatic logout after inactivity
  • Secure storage: Passwords never stored in plain text
  • Development mode: No auth required if no password is set

Port Configuration

To change the port, modify docker-compose.yml:

ports:
  - "8080:8501"  # Change 8080 to your desired port

Monitoring

Health Check

The application includes a health check that monitors:

  • Streamlit server availability
  • Automatic restart on failure

Logs

# View all logs
docker-compose logs

# Follow logs in real-time
docker-compose logs -f

# View specific service logs
docker-compose logs rag-app

Troubleshooting

Common Issues

  1. Port already in use:

    # Check what's using the port
sudo netstat -tulpn | grep :8501

# Kill the process or change the port in docker-compose.yml

  2. Permission issues:

    # Fix volume permissions
sudo chown -R 1000:1000 /var/lib/docker/volumes/drakkenheim_*/

  3. Out of disk space:

    # Clean up unused Docker resources
docker system prune -a

# Check volume sizes
docker system df -v

  4. API key issues:

    • Verify your .env file has the correct API key
    • Check the logs for authentication errors
    • Ensure the API key has sufficient credits

Reset Application

# Stop and remove containers
docker compose down

# Remove volumes (WARNING: This deletes all data)
docker volume rm drakkenheim_chroma_data drakkenheim_processed_files drakkenheim_uploaded_files

# Start fresh
docker compose up -d

Security Considerations

  1. API Key Security:

    • Never commit .env files to version control
    • Use Docker secrets for production deployments
    • Rotate API keys regularly

  2. Network Security:

    • Use a reverse proxy (nginx) for production
    • Enable HTTPS with SSL certificates
    • Restrict access with firewall rules

  3. Data Security:

    • Regular backups of persistent volumes
    • Encrypt sensitive data at rest
    • Monitor access logs

Production Deployment

For production deployment, consider:

  1. Reverse Proxy Setup:

    server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:8501;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

  2. SSL Certificate:

    # Using Let's Encrypt
sudo certbot --nginx -d your-domain.com

  3. Monitoring:

    • Set up log aggregation
    • Monitor resource usage
    • Set up alerts for failures

Support

If you encounter issues:

  1. Check the logs: docker compose logs -f
  2. Verify your environment variables
  3. Ensure Docker has sufficient resources
  4. Check the troubleshooting section above