hyper/review.md

# Project Review and Recommendations

This review provides an analysis of the current state of the automated trading bot project, proposes specific code improvements, and identifies files that appear to be unused or are one-off utilities that could be reorganized.

The project is a well-structured, multi-process Python application for crypto trading. It has a clear separation of concerns between data fetching, strategy execution, and trade management. The use of `multiprocessing` and a centralized `main_app.py` orchestrator is a solid architectural choice.

The following sections detail recommendations for improving configuration management, code structure, and robustness, along with a list of files recommended for cleanup.

---

## Cleanup Status

The following cleanup actions have been completed:

### Deleted (Obsolete / Old Versions):
- `data_fetcher_old.py`
- `market_old.py`
- `base_strategy.py` (root; `strategies/base_strategy.py` is used)
- `strategy_sma_cross.py` (standalone old version; `strategies/ma_cross_strategy.py` is used)

### Deleted (Old Architecture Remnants):
- `address_monitor.py`
- `position_monitor.py`
- `trade_log.py`
- `wallet_data.py`
- `whale_tracker.py`

### Deleted (Zero-byte Docker Artifacts):
- `1a749d1ce7c2`, `37e7cf58e0c3`, `466c0182639b`, `65740cddd0af`, `6d00d75e1dce`, `851f9bf4c3cc`, `9dd58c972c63`, `d1611986dd76`, `d22d67dc5558`
- `Running`, `Using`

### Deleted (Runtime Artifacts):
- `clp_hedger.log`
- `clp_hedger/hedge_status.json`

### Moved to `scripts/`:
- `!migrate_to_sqlite.py` → `scripts/migrate_to_sqlite.py`
- `import_csv.py` → `scripts/import_csv.py`
- `del_market_cap_tables.py` → `scripts/del_market_cap_tables.py`
- `fix_timestamps.py` → `scripts/fix_timestamps.py`
- `list_coins.py` → `scripts/list_coins.py`
- `create_agent.py` → `scripts/create_agent.py`
- `check_wtioil.py` → `scripts/check_wtioil.py`

### Moved to `.temp/`:
- `strategy_template.py` → `.temp/strategy_template.py`
- `basic_ws.py` → `.temp/basic_ws.py`
- `backtester.py` → `.temp/backtester.py`

### `.gitignore` Updated:
- Added entries for `clp_hedger.log`, `clp_hedger/hedge_status.json`, `Using`, `Running`, and Docker layer hash files (`/[0-9a-f]{12}`)

### Example Config Files Created:
- `_data/strategies.json.example`
- `_data/backtesting_conf.json.example`
- `_data/coin_precision.json.example`

---

## Remaining Proposed Code Changes

### 1. Centralize Configuration

- **Issue:** Key configuration variables like `WATCHED_COINS` and `required_timeframes` are hardcoded in `main_app.py`. This makes them difficult to change without modifying the source code.
- **Proposal:**
    - Create a central configuration file, e.g., `_data/config.json`.
    - Move `WATCHED_COINS` and `required_timeframes` into this new file.
    - Load this configuration in `main_app.py` at startup.
- **Benefit:** Decouples configuration from code, making the application more flexible and easier to manage.

### 2. Refactor `main_app.py` for Clarity

- **Issue:** `main_app.py` is long and handles multiple responsibilities: process orchestration, dashboard rendering, and data reading.
- **Proposal:**
    - **Abstract Process Management:** The functions for running subprocesses (e.g., `run_live_candle_fetcher`, `run_resampler_job`) contain repetitive logic for logging, shutdown handling, and process looping. This could be abstracted into a generic `ProcessRunner` class.
    - **Create a Dashboard Class:** The complex dashboard rendering logic could be moved into a separate `Dashboard` class to improve separation of concerns and make the main application loop cleaner.
- **Benefit:** Improves code readability, reduces duplication, and makes the application easier to maintain and extend.

### 3. Improve Project Structure

- **Issue:** The root directory is still somewhat cluttered with Python scripts.
- **Proposal:**
    - Consider creating a `src/` or `app/` directory to house the core application source code (`main_app.py`, `trade_executor.py`, etc.), separating it clearly from configuration, data, and documentation.
- **Benefit:** A cleaner, more organized project structure that is easier for new developers to understand.

### 4. Enhance Robustness and Error Handling

- **Issue:** The agent loading in `trade_executor.py` relies on discovering environment variables by a naming convention (`_AGENT_PK`). This is clever but can be brittle if environment variables are named incorrectly.
- **Proposal:**
    - Explicitly define the agent names and their corresponding environment variable keys in the proposed `_data/config.json` file. The `trade_executor` would then load only the agents specified in the configuration.
- **Benefit:** Makes agent configuration more explicit and less prone to errors from stray environment variables.