Clean up unused files, organize structure, update docs
- Delete obsolete files: data_fetcher_old.py, market_old.py, base_strategy.py (root), strategy_sma_cross.py, and old architecture remnants (address_monitor.py, position_monitor.py, trade_log.py, wallet_data.py, whale_tracker.py) - Delete zero-byte Docker artifacts and runtime files (clp_hedger.log, clp_hedger/hedge_status.json) - Move one-off utility scripts to scripts/ directory - Move example/template files to .temp/ directory - Update .gitignore: add entries for clp_hedger.log, clp_hedger/hedge_status.json, Docker layer hash files, Using, Running, and backups/ - Update .dockerignore: add clp_hedger.log, clp_hedger/hedge_status.json, backups/ - Create example config files: _data/strategies.json.example, _data/backtesting_conf.json.example, _data/coin_precision.json.example - Update GEMINI.md: remove outdated session summaries and duplicate review section - Update review.md: add cleanup status section, update remaining recommendations - Update MIGRATION_PLAN.md: mark completed phases, update file references - Update DOCKER_MIGRATION_GUIDE.md: update import_csv.py path reference
This commit is contained in:
DiTus
198
GEMINI.md
198
GEMINI.md
@ -46,201 +46,3 @@ python main_app.py
|
||||
* **Strategies:** Custom strategies should inherit from the `BaseStrategy` class (defined in `strategies/base_strategy.py`) and implement the `calculate_signals` method.
|
||||
* **Documentation:** The `WIKI/` directory contains detailed documentation for the project. Start with `WIKI/SUMMARY.md`.
|
||||
|
||||
## Session Summary
|
||||
|
||||
**Date:** 2025-11-10
|
||||
|
||||
**Objective(s):**
|
||||
Fix urllib3 SSL compatibility warning and create sessionsummary agent following OpenCode.ai guidelines
|
||||
|
||||
**Key Accomplishments:**
|
||||
* Resolved NotOpenSSLWarning by downgrading urllib3 from 2.5.0 to 1.26.20
|
||||
* Updated requirements.txt to prevent future SSL compatibility issues
|
||||
* Created sessionsummary agent in .opencode/agent/ following OpenCode.ai specifications
|
||||
* Removed incorrect Python implementation and created proper markdown agent configuration
|
||||
|
||||
**Decisions Made:**
|
||||
* Chose to downgrade urllib3 instead of upgrading SSL environment for stability
|
||||
* Followed OpenCode.ai agent guidelines instead of creating custom Python implementation
|
||||
* Configured sessionsummary as subagent with proper permissions and tools
|
||||
|
||||
**Key Files Modified:**
|
||||
* `requirements.txt`
|
||||
* `GEMINI.md`
|
||||
* `.opencode/agent/sessionsummary.md`
|
||||
|
||||
**Next Steps/Open Questions:**
|
||||
* Test trading bot functionality after SSL fix to ensure no regressions
|
||||
* Integrate sessionsummary agent into regular development workflow
|
||||
* Add .opencode/ to .gitignore if not already present
|
||||
|
||||
## Session Summary
|
||||
|
||||
**Date:** 2025-11-11
|
||||
|
||||
**Objective(s):**
|
||||
Start new Gemini session and organize project files by creating .temp folder for examples and temporary files
|
||||
|
||||
**Key Accomplishments:**
|
||||
* Created .temp folder for organizing examples and temporary files
|
||||
* Updated .gitignore to include .temp/ directory
|
||||
* Moved model_comparison_examples.md to .temp folder for better organization
|
||||
* Established file management practices for future development
|
||||
|
||||
**Decisions Made:**
|
||||
* Chose to use .temp folder instead of mixing examples with main project files
|
||||
* Added .temp to .gitignore to prevent accidental commits of temporary files
|
||||
* Followed user instruction to organize project structure for better maintainability
|
||||
|
||||
**Key Files Modified:**
|
||||
* `.gitignore`
|
||||
* `.temp/` (created)
|
||||
* `model_comparison_examples.md` (moved to .temp/)
|
||||
|
||||
**Next Steps/Open Questions:**
|
||||
* Continue organizing any other example or temporary files into .temp folder
|
||||
* Maintain consistent file organization practices in future development
|
||||
* Consider creating additional organizational directories if needed
|
||||
|
||||
## Session Summary
|
||||
|
||||
**Date:** 2025-11-11
|
||||
|
||||
**Objective(s):**
|
||||
Fix DashboardDataFetcher path resolution error causing file operation failures
|
||||
|
||||
**Key Accomplishments:**
|
||||
* Identified root cause of file path error in dashboard_data_fetcher.py subprocess execution
|
||||
* Fixed path resolution by using absolute paths instead of relative paths
|
||||
* Added os.makedirs() call to ensure _logs directory exists before file operations
|
||||
* Tested fix and confirmed DashboardDataFetcher now works correctly
|
||||
* Committed and pushed fix to remote repository
|
||||
|
||||
**Decisions Made:**
|
||||
* Used os.path.dirname(os.path.abspath(__file__)) to get correct project root
|
||||
* Ensured backward compatibility while fixing the path resolution issue
|
||||
* Maintained atomic file write pattern for data integrity
|
||||
|
||||
**Key Files Modified:**
|
||||
* `dashboard_data_fetcher.py`
|
||||
* `GEMINI.md`
|
||||
|
||||
**Next Steps/Open Questions:**
|
||||
* Monitor DashboardDataFetcher to ensure no further path-related errors occur
|
||||
* Consider reviewing other subprocess scripts for similar path resolution issues
|
||||
* Test main_app.py to ensure dashboard displays data correctly
|
||||
|
||||
## Session Summary
|
||||
|
||||
**Date:** 2025-11-11
|
||||
|
||||
**Objective(s):**
|
||||
Debug and fix DashboardDataFetcher path resolution error causing file operation failures
|
||||
|
||||
**Key Accomplishments:**
|
||||
* Identified root cause of file path error in dashboard_data_fetcher.py subprocess execution
|
||||
* Fixed path resolution by using absolute paths instead of relative paths
|
||||
* Added os.makedirs() call to ensure _logs directory exists before file operations
|
||||
* Tested fix and confirmed DashboardDataFetcher now works correctly
|
||||
* Committed and pushed fix to remote repository
|
||||
* Organized project files with .temp folder for better structure
|
||||
|
||||
**Decisions Made:**
|
||||
* Used os.path.dirname(os.path.abspath(__file__)) to get correct project root
|
||||
* Ensured backward compatibility while fixing path resolution issue
|
||||
* Maintained atomic file write pattern for data integrity
|
||||
* Added proper directory existence checks to prevent runtime errors
|
||||
|
||||
**Key Files Modified:**
|
||||
* `dashboard_data_fetcher.py`
|
||||
* `GEMINI.md`
|
||||
* `.gitignore`
|
||||
* `.temp/` (created)
|
||||
|
||||
**Next Steps/Open Questions:**
|
||||
* Monitor DashboardDataFetcher to ensure no further path-related errors occur
|
||||
* Consider reviewing other subprocess scripts for similar path resolution issues
|
||||
* Test main_app.py to ensure dashboard displays data correctly
|
||||
* Continue improving project organization and file management practices
|
||||
|
||||
---
|
||||
|
||||
# Project Review and Recommendations
|
||||
|
||||
This review provides an analysis of the current state of the automated trading bot project, proposes specific code improvements, and identifies files that appear to be unused or are one-off utilities that could be reorganized.
|
||||
|
||||
The project is a well-structured, multi-process Python application for crypto trading. It has a clear separation of concerns between data fetching, strategy execution, and trade management. The use of `multiprocessing` and a centralized `main_app.py` orchestrator is a solid architectural choice.
|
||||
|
||||
The following sections detail recommendations for improving configuration management, code structure, and robustness, along with a list of files recommended for cleanup.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Code Changes
|
||||
|
||||
### 1. Centralize Configuration
|
||||
|
||||
- **Issue:** Key configuration variables like `WATCHED_COINS` and `required_timeframes` are hardcoded in `main_app.py`. This makes them difficult to change without modifying the source code.
|
||||
- **Proposal:**
|
||||
- Create a central configuration file, e.g., `_data/config.json`.
|
||||
- Move `WATCHED_COINS` and `required_timeframes` into this new file.
|
||||
- Load this configuration in `main_app.py` at startup.
|
||||
- **Benefit:** Decouples configuration from code, making the application more flexible and easier to manage.
|
||||
|
||||
### 2. Refactor `main_app.py` for Clarity
|
||||
|
||||
- **Issue:** `main_app.py` is long and handles multiple responsibilities: process orchestration, dashboard rendering, and data reading.
|
||||
- **Proposal:**
|
||||
- **Abstract Process Management:** The functions for running subprocesses (e.g., `run_live_candle_fetcher`, `run_resampler_job`) contain repetitive logic for logging, shutdown handling, and process looping. This could be abstracted into a generic `ProcessRunner` class.
|
||||
- **Create a Dashboard Class:** The complex dashboard rendering logic could be moved into a separate `Dashboard` class to improve separation of concerns and make the main application loop cleaner.
|
||||
- **Benefit:** Improves code readability, reduces duplication, and makes the application easier to maintain and extend.
|
||||
|
||||
### 3. Improve Project Structure
|
||||
|
||||
- **Issue:** The root directory is cluttered with numerous Python scripts, making it difficult to distinguish between core application files, utility scripts, and old/example files.
|
||||
- **Proposal:**
|
||||
- Create a `scripts/` directory and move all one-off utility and maintenance scripts into it.
|
||||
- Consider creating a `src/` or `app/` directory to house the core application source code (`main_app.py`, `trade_executor.py`, etc.), separating it clearly from configuration, data, and documentation.
|
||||
- **Benefit:** A cleaner, more organized project structure that is easier for new developers to understand.
|
||||
|
||||
### 4. Enhance Robustness and Error Handling
|
||||
|
||||
- **Issue:** The agent loading in `trade_executor.py` relies on discovering environment variables by a naming convention (`_AGENT_PK`). This is clever but can be brittle if environment variables are named incorrectly.
|
||||
- **Proposal:**
|
||||
- Explicitly define the agent names and their corresponding environment variable keys in the proposed `_data/config.json` file. The `trade_executor` would then load only the agents specified in the configuration.
|
||||
- **Benefit:** Makes agent configuration more explicit and less prone to errors from stray environment variables.
|
||||
|
||||
---
|
||||
|
||||
## Identified Unused/Utility Files
|
||||
|
||||
The following files were identified as likely being unused by the core application, being obsolete, or serving as one-off utilities. It is recommended to **move them to a `scripts/` directory** or **delete them** if they are obsolete.
|
||||
|
||||
### Obsolete / Old Versions:
|
||||
- `data_fetcher_old.py`
|
||||
- `market_old.py`
|
||||
- `base_strategy.py` (The one in the root directory; the one in `strategies/` is used).
|
||||
|
||||
### One-Off Utility Scripts (Recommend moving to `scripts/`):
|
||||
- `!migrate_to_sqlite.py`
|
||||
- `import_csv.py`
|
||||
- `del_market_cap_tables.py`
|
||||
- `fix_timestamps.py`
|
||||
- `list_coins.py`
|
||||
- `create_agent.py`
|
||||
|
||||
### Examples / Unused Code:
|
||||
- `basic_ws.py` (Appears to be an example file).
|
||||
- `backtester.py`
|
||||
- `strategy_sma_cross.py` (A strategy file in the root, not in the `strategies` folder).
|
||||
- `strategy_template.py`
|
||||
|
||||
### Standalone / Potentially Unused Core Files:
|
||||
The following files seem to have their logic already integrated into the main multi-process application. They might be remnants of a previous architecture and may not be needed as standalone scripts.
|
||||
- `address_monitor.py`
|
||||
- `position_monitor.py`
|
||||
- `trade_log.py`
|
||||
- `wallet_data.py`
|
||||
- `whale_tracker.py`
|
||||
|
||||
### Data / Log Files (Recommend archiving or deleting):
|
||||
- `hyperliquid_wallet_data_*.json` (These appear to be backups or logs).
|
||||
Reference in New Issue
Block a user