Migrate data pipeline from SQLite to PostgreSQL + Docker setup
- Add db.py PostgreSQL abstraction layer (connection, upsert, table mgmt) - Replace sqlite3 with psycopg2 in: live_candle_fetcher, resampler, data_fetcher, fetch_history, import_csv, indicators, base_strategy - Sanitize table names (colons -> underscores) for PostgreSQL compat - Replace INSERT OR REPLACE with ON CONFLICT upserts - Replace pandas to_sql() with batch upsert_candles() - Add scripts: resampler_loop, gap_detector, backup_runner, cron_scheduler - Add migrate_sqlite_to_pg.py for one-time data migration - Add Dockerfile, docker-compose.yml, supervisord.conf - Add postgres/postgresql.conf tuned for 4GB RAM (Synology DS1513+) - Add .dockerignore, .env.docker.example, secrets template - Update requirements.txt (psycopg2-binary), .gitignore - Add MIGRATION_PLAN.md with full plan and todo list
This commit is contained in:
26
postgres/postgresql.conf
Normal file
26
postgres/postgresql.conf
Normal file
@ -0,0 +1,26 @@
|
||||
# PostgreSQL configuration tuned for Synology DS1513+ (4GB RAM)
|
||||
# Place this file at postgres/postgresql.conf and mount it into the container.
|
||||
|
||||
# --- Memory ---
|
||||
shared_buffers = 128MB
|
||||
effective_cache_size = 512MB
|
||||
work_mem = 8MB
|
||||
maintenance_work_mem = 64MB
|
||||
|
||||
# --- Connections ---
|
||||
max_connections = 10
|
||||
max_worker_processes = 2
|
||||
|
||||
# --- WAL / Checkpointing ---
|
||||
wal_buffers = 4MB
|
||||
checkpoint_completion_target = 0.9
|
||||
max_wal_senders = 3
|
||||
|
||||
# --- Network ---
|
||||
listen_addresses = '*'
|
||||
|
||||
# --- Logging ---
|
||||
log_statement = 'none'
|
||||
log_duration = off
|
||||
log_min_duration_statement = 0
|
||||
log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
|
||||
Reference in New Issue
Block a user