gnx/ETB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Updates

Browse Source
This commit is contained in:
Iliyan Angelov
2025-09-19 11:58:53 +03:00
parent 306b20e24a
commit 6b247e5b9f
11423 changed files with 1500615 additions and 778 deletions
Download Patch File Download Diff File Expand all files Collapse all files

958
ETB-API/ENTERPRISE_DEPLOYMENT_GUIDE.md Normal file
View File

@@ -0,0 +1,958 @@
# ETB-API Enterprise Deployment Guide
## 🚀 **Enterprise-Grade Deployment for Production**
This comprehensive guide provides step-by-step instructions for deploying the ETB-API platform in an enterprise environment with high availability, security, and scalability.
## 📋 **Table of Contents**
1. [Prerequisites](#prerequisites)
2. [Infrastructure Setup](#infrastructure-setup)
3. [Database Configuration](#database-configuration)
4. [Application Deployment](#application-deployment)
5. [Security Configuration](#security-configuration)
6. [Monitoring & Observability](#monitoring--observability)
7. [Backup & Recovery](#backup--recovery)
8. [High Availability](#high-availability)
9. [Performance Optimization](#performance-optimization)
10. [Maintenance & Operations](#maintenance--operations)
## 🔧 **Prerequisites**
### System Requirements
- **Operating System**: Ubuntu 20.04 LTS or CentOS 8+
- **CPU**: 8+ cores (16+ recommended for production)
- **RAM**: 32GB+ (64GB+ recommended for production)
- **Storage**: 500GB+ SSD (1TB+ recommended for production)
- **Network**: 1Gbps+ bandwidth
### Software Requirements
- **Python**: 3.9+
- **PostgreSQL**: 13+
- **Redis**: 6.2+
- **Nginx**: 1.18+
- **Docker**: 20.10+ (optional)
- **Kubernetes**: 1.21+ (optional)
### Dependencies
```bash
# Install system packages
sudo apt-get update
sudo apt-get install -y python3.9 python3.9-dev python3-pip
sudo apt-get install -y postgresql-13 postgresql-client-13
sudo apt-get install -y redis-server nginx
sudo apt-get install -y git curl wget unzip
# Install Python dependencies
pip3 install -r requirements.txt
```
## 🏗️ **Infrastructure Setup**
### 1. Database Cluster Setup
#### PostgreSQL Primary-Replica Configuration
```bash
# Primary Database Server
sudo -u postgres psql
CREATE DATABASE etb_incident_management;
CREATE USER etb_user WITH PASSWORD 'secure_password';
GRANT ALL PRIVILEGES ON DATABASE etb_incident_management TO etb_user;
# Configure replication
sudo nano /etc/postgresql/13/main/postgresql.conf
```
```conf
# postgresql.conf
listen_addresses = '*'
wal_level = replica
max_wal_senders = 3
max_replication_slots = 3
hot_standby = on
```
```bash
# Configure authentication
sudo nano /etc/postgresql/13/main/pg_hba.conf
```
```conf
# pg_hba.conf
host replication replicator 10.0.0.0/8 md5
host all all 10.0.0.0/8 md5
```
#### Redis Cluster Setup
```bash
# Redis Master
sudo nano /etc/redis/redis.conf
```
```conf
# redis.conf
bind 0.0.0.0
port 6379
requirepass secure_redis_password
maxmemory 2gb
maxmemory-policy allkeys-lru
save 900 1
save 300 10
save 60 10000
```
### 2. Load Balancer Configuration
#### Nginx Load Balancer
```bash
sudo nano /etc/nginx/sites-available/etb-api
```
```nginx
upstream etb_api_backend {
least_conn;
server 10.0.1.10:8000 max_fails=3 fail_timeout=30s;
server 10.0.1.11:8000 max_fails=3 fail_timeout=30s;
server 10.0.1.12:8000 max_fails=3 fail_timeout=30s;
}
upstream etb_api_websocket {
ip_hash;
server 10.0.1.10:8001;
server 10.0.1.11:8001;
server 10.0.1.12:8001;
}
server {
listen 80;
server_name api.yourcompany.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name api.yourcompany.com;
# SSL Configuration
ssl_certificate /etc/ssl/certs/etb-api.crt;
ssl_certificate_key /etc/ssl/private/etb-api.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512;
ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# Security Headers
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options DENY;
add_header X-XSS-Protection "1; mode=block";
add_header Content-Security-Policy "default-src 'self'";
# Rate Limiting
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
limit_req zone=api burst=20 nodelay;
# API Routes
location /api/ {
proxy_pass http://etb_api_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 30s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
}
# WebSocket Routes
location /ws/ {
proxy_pass http://etb_api_websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Static Files
location /static/ {
alias /var/www/etb-api/static/;
expires 1y;
add_header Cache-Control "public, immutable";
}
# Media Files
location /media/ {
alias /var/www/etb-api/media/;
expires 1y;
add_header Cache-Control "public";
}
# Health Checks
location /health/ {
proxy_pass http://etb_api_backend;
access_log off;
}
}
```
## 🗄️ **Database Configuration**
### 1. Database Optimization
```sql
-- PostgreSQL Performance Tuning
ALTER SYSTEM SET shared_buffers = '4GB';
ALTER SYSTEM SET effective_cache_size = '12GB';
ALTER SYSTEM SET maintenance_work_mem = '1GB';
ALTER SYSTEM SET checkpoint_completion_target = 0.9;
ALTER SYSTEM SET wal_buffers = '16MB';
ALTER SYSTEM SET default_statistics_target = 100;
ALTER SYSTEM SET random_page_cost = 1.1;
ALTER SYSTEM SET effective_io_concurrency = 200;
-- Restart PostgreSQL
sudo systemctl restart postgresql
```
### 2. Database Indexes
```sql
-- Create performance indexes
CREATE INDEX CONCURRENTLY idx_incident_status_priority ON incident_intelligence_incident(status, priority);
CREATE INDEX CONCURRENTLY idx_incident_created_at ON incident_intelligence_incident(created_at);
CREATE INDEX CONCURRENTLY idx_sla_instance_status ON sla_oncall_slainstance(status);
CREATE INDEX CONCURRENTLY idx_security_event_timestamp ON security_securityevent(timestamp);
CREATE INDEX CONCURRENTLY idx_monitoring_metric_timestamp ON monitoring_metric(timestamp);
-- Create partial indexes for active records
CREATE INDEX CONCURRENTLY idx_incident_active ON incident_intelligence_incident(id) WHERE status = 'active';
CREATE INDEX CONCURRENTLY idx_sla_active ON sla_oncall_slainstance(id) WHERE status = 'active';
```
## 🚀 **Application Deployment**
### 1. Environment Configuration
```bash
# Create environment file
sudo nano /etc/etb-api/.env
```
```env
# Database Configuration
DB_NAME=etb_incident_management
DB_USER=etb_user
DB_PASSWORD=secure_password
DB_HOST=10.0.1.5
DB_PORT=5432
# Redis Configuration
REDIS_URL=redis://:secure_redis_password@10.0.1.6:6379/0
CELERY_BROKER_URL=redis://:secure_redis_password@10.0.1.6:6379/0
CELERY_RESULT_BACKEND=redis://:secure_redis_password@10.0.1.6:6379/0
# Security
SECRET_KEY=your-super-secret-key-here
DEBUG=False
ALLOWED_HOSTS=api.yourcompany.com,10.0.1.10,10.0.1.11,10.0.1.12
# Email Configuration
EMAIL_HOST=smtp.yourcompany.com
EMAIL_PORT=587
EMAIL_USE_TLS=True
EMAIL_HOST_USER=noreply@yourcompany.com
EMAIL_HOST_PASSWORD=email_password
DEFAULT_FROM_EMAIL=noreply@yourcompany.com
# Monitoring
PROMETHEUS_ENABLED=True
GRAFANA_ENABLED=True
ELASTICSEARCH_URL=http://10.0.1.8:9200
# Backup
BACKUP_ENABLED=True
BACKUP_RETENTION_DAYS=30
AWS_S3_BACKUP_BUCKET=etb-api-backups
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1
# Security
SIEM_WEBHOOK_URL=https://siem.yourcompany.com/webhook
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK
ALERT_WEBHOOK_URL=https://alerts.yourcompany.com/webhook
```
### 2. Application Deployment Script
```bash
#!/bin/bash
# deploy.sh - Enterprise deployment script
set -e
# Configuration
APP_NAME="etb-api"
APP_USER="etb"
APP_DIR="/var/www/etb-api"
VENV_DIR="/var/www/etb-api/venv"
LOG_DIR="/var/log/etb-api"
BACKUP_DIR="/backups/etb-api"
# Create application user
sudo useradd -r -s /bin/false $APP_USER || true
# Create directories
sudo mkdir -p $APP_DIR $LOG_DIR $BACKUP_DIR
sudo chown -R $APP_USER:$APP_USER $APP_DIR $LOG_DIR $BACKUP_DIR
# Clone repository
cd $APP_DIR
sudo -u $APP_USER git clone https://github.com/yourcompany/etb-api.git .
# Create virtual environment
sudo -u $APP_USER python3 -m venv $VENV_DIR
sudo -u $APP_USER $VENV_DIR/bin/pip install --upgrade pip
sudo -u $APP_USER $VENV_DIR/bin/pip install -r requirements.txt
# Set up environment
sudo -u $APP_USER cp .env.example .env
sudo -u $APP_USER nano .env # Configure environment variables
# Run migrations
sudo -u $APP_USER $VENV_DIR/bin/python manage.py migrate
# Collect static files
sudo -u $APP_USER $VENV_DIR/bin/python manage.py collectstatic --noinput
# Create superuser
sudo -u $APP_USER $VENV_DIR/bin/python manage.py createsuperuser
# Set up systemd services
sudo cp deployment/systemd/*.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable etb-api etb-celery etb-celery-beat
sudo systemctl start etb-api etb-celery etb-celery-beat
# Set up log rotation
sudo cp deployment/logrotate/etb-api /etc/logrotate.d/
sudo chmod 644 /etc/logrotate.d/etb-api
# Set up monitoring
sudo -u $APP_USER $VENV_DIR/bin/python manage.py setup_monitoring
echo "Deployment completed successfully!"
```
### 3. Systemd Services
#### ETB-API Service
```ini
# /etc/systemd/system/etb-api.service
[Unit]
Description=ETB-API Django Application
After=network.target postgresql.service redis.service
Requires=postgresql.service redis.service
[Service]
Type=exec
User=etb
Group=etb
WorkingDirectory=/var/www/etb-api
Environment=PATH=/var/www/etb-api/venv/bin
EnvironmentFile=/etc/etb-api/.env
ExecStart=/var/www/etb-api/venv/bin/gunicorn --bind 0.0.0.0:8000 --workers 4 --worker-class gevent --worker-connections 1000 --max-requests 1000 --max-requests-jitter 100 --timeout 30 --keep-alive 2 --preload core.wsgi:application
ExecReload=/bin/kill -s HUP $MAINPID
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=etb-api
[Install]
WantedBy=multi-user.target
```
#### Celery Worker Service
```ini
# /etc/systemd/system/etb-celery.service
[Unit]
Description=ETB-API Celery Worker
After=network.target redis.service
Requires=redis.service
[Service]
Type=exec
User=etb
Group=etb
WorkingDirectory=/var/www/etb-api
Environment=PATH=/var/www/etb-api/venv/bin
EnvironmentFile=/etc/etb-api/.env
ExecStart=/var/www/etb-api/venv/bin/celery -A core worker -l info --concurrency=4 --max-tasks-per-child=1000
ExecReload=/bin/kill -s HUP $MAINPID
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=etb-celery
[Install]
WantedBy=multi-user.target
```
#### Celery Beat Service
```ini
# /etc/systemd/system/etb-celery-beat.service
[Unit]
Description=ETB-API Celery Beat Scheduler
After=network.target redis.service
Requires=redis.service
[Service]
Type=exec
User=etb
Group=etb
WorkingDirectory=/var/www/etb-api
Environment=PATH=/var/www/etb-api/venv/bin
EnvironmentFile=/etc/etb-api/.env
ExecStart=/var/www/etb-api/venv/bin/celery -A core beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
ExecReload=/bin/kill -s HUP $MAINPID
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=etb-celery-beat
[Install]
WantedBy=multi-user.target
```
## 🔒 **Security Configuration**
### 1. Firewall Configuration
```bash
# UFW Firewall Rules
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow from 10.0.0.0/8 to any port 5432 # PostgreSQL
sudo ufw allow from 10.0.0.0/8 to any port 6379 # Redis
sudo ufw enable
```
### 2. SSL/TLS Configuration
```bash
# Generate SSL certificate
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/ssl/private/etb-api.key \
-out /etc/ssl/certs/etb-api.crt \
-subj "/C=US/ST=State/L=City/O=Organization/CN=api.yourcompany.com"
# Set permissions
sudo chmod 600 /etc/ssl/private/etb-api.key
sudo chmod 644 /etc/ssl/certs/etb-api.crt
```
### 3. Security Hardening
```bash
# Disable unnecessary services
sudo systemctl disable apache2
sudo systemctl disable mysql
# Configure fail2ban
sudo apt-get install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
# Add custom jail for ETB-API
sudo nano /etc/fail2ban/jail.d/etb-api.conf
```
```ini
[etb-api]
enabled = true
port = 80,443
filter = etb-api
logpath = /var/log/etb-api/application.log
maxretry = 5
bantime = 3600
findtime = 600
```
## 📊 **Monitoring & Observability**
### 1. Prometheus Configuration
```yaml
# /etc/prometheus/prometheus.yml
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- "etb-api-rules.yml"
scrape_configs:
- job_name: 'etb-api'
static_configs:
- targets: ['10.0.1.10:8000', '10.0.1.11:8000', '10.0.1.12:8000']
metrics_path: '/api/monitoring/metrics/'
scrape_interval: 30s
- job_name: 'postgresql'
static_configs:
- targets: ['10.0.1.5:9187']
- job_name: 'redis'
static_configs:
- targets: ['10.0.1.6:9121']
- job_name: 'nginx'
static_configs:
- targets: ['10.0.1.7:9113']
```
### 2. Grafana Dashboard
```json
{
"dashboard": {
"title": "ETB-API Enterprise Dashboard",
"panels": [
{
"title": "Request Rate",
"type": "graph",
"targets": [
{
"expr": "rate(http_requests_total[5m])",
"legendFormat": "{{method}} {{endpoint}}"
}
]
},
{
"title": "Response Time",
"type": "graph",
"targets": [
{
"expr": "histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))",
"legendFormat": "95th percentile"
}
]
},
{
"title": "System Resources",
"type": "graph",
"targets": [
{
"expr": "system_cpu_usage_percent",
"legendFormat": "CPU Usage"
},
{
"expr": "system_memory_usage_percent",
"legendFormat": "Memory Usage"
}
]
}
]
}
}
```
### 3. Log Aggregation
```yaml
# /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/etb-api/*.log
fields:
service: etb-api
fields_under_root: true
output.elasticsearch:
hosts: ["10.0.1.8:9200"]
index: "etb-api-%{+yyyy.MM.dd}"
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
```
## 💾 **Backup & Recovery**
### 1. Automated Backup Script
```bash
#!/bin/bash
# backup.sh - Automated backup script
set -e
BACKUP_DIR="/backups/etb-api"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_NAME="etb-api-backup-$DATE"
# Create backup directory
mkdir -p $BACKUP_DIR/$BACKUP_NAME
# Database backup
pg_dump -h 10.0.1.5 -U etb_user etb_incident_management > $BACKUP_DIR/$BACKUP_NAME/database.sql
# Application backup
tar -czf $BACKUP_DIR/$BACKUP_NAME/application.tar.gz -C /var/www etb-api
# Configuration backup
tar -czf $BACKUP_DIR/$BACKUP_NAME/config.tar.gz /etc/etb-api /etc/nginx/sites-available/etb-api
# Create backup manifest
cat > $BACKUP_DIR/$BACKUP_NAME/manifest.json << EOF
{
"backup_name": "$BACKUP_NAME",
"created_at": "$(date -Iseconds)",
"components": {
"database": "database.sql",
"application": "application.tar.gz",
"configuration": "config.tar.gz"
},
"size": "$(du -sh $BACKUP_DIR/$BACKUP_NAME | cut -f1)"
}
EOF
# Compress backup
tar -czf $BACKUP_DIR/$BACKUP_NAME.tar.gz -C $BACKUP_DIR $BACKUP_NAME
rm -rf $BACKUP_DIR/$BACKUP_NAME
# Upload to S3
aws s3 cp $BACKUP_DIR/$BACKUP_NAME.tar.gz s3://etb-api-backups/
# Cleanup old backups
find $BACKUP_DIR -name "*.tar.gz" -mtime +30 -delete
echo "Backup completed: $BACKUP_NAME"
```
### 2. Recovery Script
```bash
#!/bin/bash
# restore.sh - Recovery script
set -e
BACKUP_NAME=$1
BACKUP_DIR="/backups/etb-api"
if [ -z "$BACKUP_NAME" ]; then
echo "Usage: $0 <backup_name>"
exit 1
fi
# Download from S3
aws s3 cp s3://etb-api-backups/$BACKUP_NAME.tar.gz $BACKUP_DIR/
# Extract backup
tar -xzf $BACKUP_DIR/$BACKUP_NAME.tar.gz -C $BACKUP_DIR
# Stop services
sudo systemctl stop etb-api etb-celery etb-celery-beat
# Restore database
psql -h 10.0.1.5 -U etb_user etb_incident_management < $BACKUP_DIR/$BACKUP_NAME/database.sql
# Restore application
tar -xzf $BACKUP_DIR/$BACKUP_NAME/application.tar.gz -C /var/www
# Restore configuration
tar -xzf $BACKUP_DIR/$BACKUP_NAME/config.tar.gz -C /
# Start services
sudo systemctl start etb-api etb-celery etb-celery-beat
echo "Recovery completed: $BACKUP_NAME"
```
## 🔄 **High Availability**
### 1. Load Balancer Health Checks
```nginx
# Health check configuration
upstream etb_api_backend {
least_conn;
server 10.0.1.10:8000 max_fails=3 fail_timeout=30s;
server 10.0.1.11:8000 max_fails=3 fail_timeout=30s;
server 10.0.1.12:8000 max_fails=3 fail_timeout=30s;
}
# Health check endpoint
location /health/ {
proxy_pass http://etb_api_backend;
proxy_connect_timeout 5s;
proxy_send_timeout 5s;
proxy_read_timeout 5s;
access_log off;
}
```
### 2. Database Failover
```bash
# PostgreSQL failover script
#!/bin/bash
# failover.sh - Database failover script
PRIMARY_HOST="10.0.1.5"
STANDBY_HOST="10.0.1.6"
# Check primary health
if ! pg_isready -h $PRIMARY_HOST -p 5432; then
echo "Primary database is down, initiating failover..."
# Promote standby
ssh $STANDBY_HOST "sudo -u postgres pg_ctl promote -D /var/lib/postgresql/13/main"
# Update application configuration
sed -i "s/DB_HOST=$PRIMARY_HOST/DB_HOST=$STANDBY_HOST/" /etc/etb-api/.env
# Restart application
sudo systemctl restart etb-api
echo "Failover completed to $STANDBY_HOST"
fi
```
## ⚡ **Performance Optimization**
### 1. Application Optimization
```python
# settings.py optimizations
CACHES = {
'default': {
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': 'redis://10.0.1.6:6379/1',
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
'CONNECTION_POOL_KWARGS': {
'max_connections': 50,
'retry_on_timeout': True,
},
'COMPRESSOR': 'django_redis.compressors.zlib.ZlibCompressor',
}
}
}
# Database connection pooling
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'etb_incident_management',
'USER': 'etb_user',
'PASSWORD': 'secure_password',
'HOST': '10.0.1.5',
'PORT': '5432',
'CONN_MAX_AGE': 600,
'CONN_HEALTH_CHECKS': True,
'OPTIONS': {
'MAX_CONNS': 20,
'MIN_CONNS': 5,
}
}
}
```
### 2. Nginx Optimization
```nginx
# nginx.conf optimizations
worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 65535;
events {
worker_connections 4096;
use epoll;
multi_accept on;
}
http {
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Buffer sizes
client_body_buffer_size 128k;
client_max_body_size 10m;
client_header_buffer_size 1k;
large_client_header_buffers 4 4k;
# Timeouts
client_body_timeout 12;
client_header_timeout 12;
keepalive_timeout 15;
send_timeout 10;
}
```
## 🔧 **Maintenance & Operations**
### 1. Monitoring Scripts
```bash
#!/bin/bash
# health_check.sh - Comprehensive health check
# Check application health
curl -f http://localhost:8000/health/ || exit 1
# Check database connectivity
pg_isready -h 10.0.1.5 -p 5432 || exit 1
# Check Redis connectivity
redis-cli -h 10.0.1.6 -p 6379 ping || exit 1
# Check disk space
df -h | awk '$5 > 90 {print $0; exit 1}'
# Check memory usage
free | awk 'NR==2{printf "%.2f%%\n", $3*100/$2}' | awk '{if($1 > 90) exit 1}'
echo "All health checks passed"
```
### 2. Log Rotation
```bash
# /etc/logrotate.d/etb-api
/var/log/etb-api/*.log {
daily
missingok
rotate 30
compress
delaycompress
notifempty
create 644 etb etb
postrotate
systemctl reload etb-api
endscript
}
```
### 3. Update Script
```bash
#!/bin/bash
# update.sh - Application update script
set -e
# Backup current version
./backup.sh
# Pull latest changes
cd /var/www/etb-api
sudo -u etb git pull origin main
# Update dependencies
sudo -u etb /var/www/etb-api/venv/bin/pip install -r requirements.txt
# Run migrations
sudo -u etb /var/www/etb-api/venv/bin/python manage.py migrate
# Collect static files
sudo -u etb /var/www/etb-api/venv/bin/python manage.py collectstatic --noinput
# Restart services
sudo systemctl restart etb-api etb-celery etb-celery-beat
echo "Update completed successfully"
```
## 📈 **Scaling Guidelines**
### 1. Horizontal Scaling
- **Application Servers**: Add more instances behind load balancer
- **Database**: Implement read replicas for read-heavy workloads
- **Cache**: Use Redis Cluster for distributed caching
- **Storage**: Implement distributed file storage (S3, Ceph)
### 2. Vertical Scaling
- **CPU**: Increase cores for compute-intensive operations
- **Memory**: Add RAM for caching and in-memory operations
- **Storage**: Use SSD for better I/O performance
- **Network**: Upgrade bandwidth for high-traffic scenarios
## 🚨 **Troubleshooting**
### Common Issues
1. **High Memory Usage**
```bash
# Check memory usage
free -h
ps aux --sort=-%mem | head
# Restart services if needed
sudo systemctl restart etb-api
```
2. **Database Connection Issues**
```bash
# Check database status
sudo systemctl status postgresql
pg_isready -h 10.0.1.5 -p 5432
# Check connection pool
psql -h 10.0.1.5 -U etb_user -c "SELECT * FROM pg_stat_activity;"
```
3. **Cache Issues**
```bash
# Check Redis status
sudo systemctl status redis
redis-cli -h 10.0.1.6 -p 6379 ping
# Clear cache if needed
redis-cli -h 10.0.1.6 -p 6379 FLUSHALL
```
## 📞 **Support & Maintenance**
### Regular Tasks
- **Daily**: Monitor system health and alerts
- **Weekly**: Review performance metrics and logs
- **Monthly**: Update dependencies and security patches
- **Quarterly**: Review and optimize performance
### Emergency Procedures
1. **Service Outage**: Check health endpoints and restart services
2. **Database Issues**: Check connectivity and failover if needed
3. **Security Incident**: Review logs and implement containment
4. **Performance Degradation**: Analyze metrics and scale resources
This comprehensive deployment guide provides enterprise-grade setup for the ETB-API platform. Adjust configurations based on your specific requirements and infrastructure.