290 lines
6.8 KiB
Markdown
290 lines
6.8 KiB
Markdown
# Docker Deployment Guide
|
|
|
|
This guide explains how to deploy the Drakkenheim RAG application using Docker Compose with persistent storage.
|
|
|
|
## Prerequisites
|
|
|
|
- Docker and Docker Compose installed on your server
|
|
- OpenAI API key
|
|
|
|
## Project Structure
|
|
|
|
The project follows Python best practices with a `src/` layout:
|
|
|
|
```
|
|
drakkenheim/
|
|
├── src/
|
|
│ └── drakkenheim/ # Main package
|
|
│ ├── __init__.py # Package initialization
|
|
│ ├── auth.py # Authentication & session management
|
|
│ ├── llm.py # LLM, embeddings & vector operations
|
|
│ ├── document.py # Document processing
|
|
│ ├── storage.py # File persistence
|
|
│ └── ui.py # User interface components
|
|
├── app.py # Main application entry point
|
|
├── generate_password.py # Password generation utility
|
|
├── docker-compose.yml # Docker orchestration
|
|
├── Dockerfile # Container definition
|
|
└── requirements/ # Dependencies
|
|
```
|
|
|
|
**Benefits of the `src/` layout:**
|
|
- **Clean separation** between source code and project files
|
|
- **Prevents import issues** during development and testing
|
|
- **Industry standard** for Python projects
|
|
- **Better packaging** and distribution support
|
|
- **Easier testing** and CI/CD integration
|
|
|
|
## Quick Start
|
|
|
|
1. **Clone and navigate to the project:**
|
|
```bash
|
|
git clone <your-repo-url>
|
|
cd drakkenheim
|
|
```
|
|
|
|
2. **Set up environment variables:**
|
|
```bash
|
|
# Copy the example environment file
|
|
cp env.example .env
|
|
|
|
# Generate secure password hash
|
|
python3 generate_password.py
|
|
|
|
# Edit .env and add your OpenAI API key and generated auth config
|
|
nano .env
|
|
```
|
|
|
|
3. **Build and start the application:**
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
4. **Access the application:**
|
|
- Open your browser to `http://your-server-ip:8501`
|
|
- The application will be available and ready to use
|
|
|
|
## Persistent Storage
|
|
|
|
The application uses Docker volumes to persist data between restarts:
|
|
|
|
- **ChromaDB Database**: `chroma_data` volume
|
|
- **Processed Files Metadata**: `processed_files` volume
|
|
- **Uploaded Files**: `uploaded_files` volume
|
|
|
|
## Management Commands
|
|
|
|
### Start the application:
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
### Stop the application:
|
|
```bash
|
|
docker compose down
|
|
```
|
|
|
|
### View logs:
|
|
```bash
|
|
docker compose logs -f rag-app
|
|
```
|
|
|
|
### Restart the application:
|
|
```bash
|
|
docker compose restart rag-app
|
|
```
|
|
|
|
### Update the application:
|
|
```bash
|
|
# Pull latest changes
|
|
git pull
|
|
|
|
# Rebuild and restart
|
|
docker compose up -d --build
|
|
```
|
|
|
|
### Backup data:
|
|
```bash
|
|
# Create backup directory
|
|
mkdir -p backups
|
|
|
|
# Backup ChromaDB data
|
|
docker run --rm -v drakkenheim_chroma_data:/data -v $(pwd)/backups:/backup alpine tar czf /backup/chroma_data_$(date +%Y%m%d_%H%M%S).tar.gz -C /data .
|
|
|
|
# Backup processed files
|
|
docker run --rm -v drakkenheim_processed_files:/data -v $(pwd)/backups:/backup alpine tar czf /backup/processed_files_$(date +%Y%m%d_%H%M%S).tar.gz -C /data .
|
|
```
|
|
|
|
### Restore data:
|
|
```bash
|
|
# Restore ChromaDB data
|
|
docker run --rm -v drakkenheim_chroma_data:/data -v $(pwd)/backups:/backup alpine tar xzf /backup/chroma_data_YYYYMMDD_HHMMSS.tar.gz -C /data
|
|
|
|
# Restore processed files
|
|
docker run --rm -v drakkenheim_processed_files:/data -v $(pwd)/backups:/backup alpine tar xzf /backup/processed_files_YYYYMMDD_HHMMSS.tar.gz -C /data
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
Edit `.env` file to customize:
|
|
|
|
```bash
|
|
# Required
|
|
OPENAI_API_KEY=your_api_key_here
|
|
|
|
# Authentication (REQUIRED for production)
|
|
APP_PASSWORD_HASH=your_hashed_password_here
|
|
PASSWORD_SALT=your_random_salt_here
|
|
SESSION_TIMEOUT=3600
|
|
|
|
# Optional overrides
|
|
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
|
|
OPENAI_LLM_MODEL=gpt-4o-mini
|
|
STREAMLIT_SERVER_PORT=8501
|
|
```
|
|
|
|
### Authentication Setup
|
|
|
|
**Generate secure credentials:**
|
|
```bash
|
|
# Run the password generator
|
|
python3 generate_password.py
|
|
|
|
# Follow the prompts to set your password
|
|
# Copy the generated values to your .env file
|
|
```
|
|
|
|
**Security Features:**
|
|
- **Password hashing**: SHA-256 with salt
|
|
- **Session timeout**: Automatic logout after inactivity
|
|
- **Secure storage**: Passwords never stored in plain text
|
|
- **Development mode**: No auth required if no password is set
|
|
|
|
### Port Configuration
|
|
|
|
To change the port, modify `docker-compose.yml`:
|
|
|
|
```yaml
|
|
ports:
|
|
- "8080:8501" # Change 8080 to your desired port
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
### Health Check
|
|
The application includes a health check that monitors:
|
|
- Streamlit server availability
|
|
- Automatic restart on failure
|
|
|
|
### Logs
|
|
```bash
|
|
# View all logs
|
|
docker-compose logs
|
|
|
|
# Follow logs in real-time
|
|
docker-compose logs -f
|
|
|
|
# View specific service logs
|
|
docker-compose logs rag-app
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **Port already in use:**
|
|
```bash
|
|
# Check what's using the port
|
|
sudo netstat -tulpn | grep :8501
|
|
|
|
# Kill the process or change the port in docker-compose.yml
|
|
```
|
|
|
|
2. **Permission issues:**
|
|
```bash
|
|
# Fix volume permissions
|
|
sudo chown -R 1000:1000 /var/lib/docker/volumes/drakkenheim_*/
|
|
```
|
|
|
|
3. **Out of disk space:**
|
|
```bash
|
|
# Clean up unused Docker resources
|
|
docker system prune -a
|
|
|
|
# Check volume sizes
|
|
docker system df -v
|
|
```
|
|
|
|
4. **API key issues:**
|
|
- Verify your `.env` file has the correct API key
|
|
- Check the logs for authentication errors
|
|
- Ensure the API key has sufficient credits
|
|
|
|
### Reset Application
|
|
```bash
|
|
# Stop and remove containers
|
|
docker compose down
|
|
|
|
# Remove volumes (WARNING: This deletes all data)
|
|
docker volume rm drakkenheim_chroma_data drakkenheim_processed_files drakkenheim_uploaded_files
|
|
|
|
# Start fresh
|
|
docker compose up -d
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
1. **API Key Security:**
|
|
- Never commit `.env` files to version control
|
|
- Use Docker secrets for production deployments
|
|
- Rotate API keys regularly
|
|
|
|
2. **Network Security:**
|
|
- Use a reverse proxy (nginx) for production
|
|
- Enable HTTPS with SSL certificates
|
|
- Restrict access with firewall rules
|
|
|
|
3. **Data Security:**
|
|
- Regular backups of persistent volumes
|
|
- Encrypt sensitive data at rest
|
|
- Monitor access logs
|
|
|
|
## Production Deployment
|
|
|
|
For production deployment, consider:
|
|
|
|
1. **Reverse Proxy Setup:**
|
|
```nginx
|
|
server {
|
|
listen 80;
|
|
server_name your-domain.com;
|
|
|
|
location / {
|
|
proxy_pass http://localhost:8501;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
}
|
|
}
|
|
```
|
|
|
|
2. **SSL Certificate:**
|
|
```bash
|
|
# Using Let's Encrypt
|
|
sudo certbot --nginx -d your-domain.com
|
|
```
|
|
|
|
3. **Monitoring:**
|
|
- Set up log aggregation
|
|
- Monitor resource usage
|
|
- Set up alerts for failures
|
|
|
|
## Support
|
|
|
|
If you encounter issues:
|
|
1. Check the logs: `docker compose logs -f`
|
|
2. Verify your environment variables
|
|
3. Ensure Docker has sufficient resources
|
|
4. Check the troubleshooting section above
|