Migrate data pipeline from SQLite to PostgreSQL + Docker setup

- Add db.py PostgreSQL abstraction layer (connection, upsert, table mgmt)
- Replace sqlite3 with psycopg2 in: live_candle_fetcher, resampler,
  data_fetcher, fetch_history, import_csv, indicators, base_strategy
- Sanitize table names (colons -> underscores) for PostgreSQL compat
- Replace INSERT OR REPLACE with ON CONFLICT upserts
- Replace pandas to_sql() with batch upsert_candles()
- Add scripts: resampler_loop, gap_detector, backup_runner, cron_scheduler
- Add migrate_sqlite_to_pg.py for one-time data migration
- Add Dockerfile, docker-compose.yml, supervisord.conf
- Add postgres/postgresql.conf tuned for 4GB RAM (Synology DS1513+)
- Add .dockerignore, .env.docker.example, secrets template
- Update requirements.txt (psycopg2-binary), .gitignore
- Add MIGRATION_PLAN.md with full plan and todo list
This commit is contained in:
DiTus
2026-07-30 22:14:31 +02:00
parent ade9b708a2
commit 7d702e9cbd
24 changed files with 1013 additions and 222 deletions

6
.gitignore vendored
View File

@ -42,4 +42,8 @@ agents/
.idea/
.DS_Store
Thumbs.db
.opencode/
.opencode/
# --- Docker ---
secrets/
.env.docker