hyper/GEMINI.md

Project Overview

This project is a sophisticated, multi-process automated trading bot for the Hyperliquid decentralized exchange. It is written in Python and uses a modular architecture to separate concerns like data fetching, strategy execution, and trade management.

The bot uses a high-performance data pipeline with SQLite for storing market data. Trading strategies are defined and configured in a JSON file, allowing for easy adjustments without code changes. The system supports multiple, independent trading agents for risk segregation and PNL tracking. A live terminal dashboard provides real-time monitoring of market data, strategy signals, and the status of all background processes.

Building and Running

1. Setup

1. Create and activate a virtual environment:

   # For Windows
   python -m venv .venv
   .\.venv\Scripts\activate
   
   # For macOS/Linux
   python3 -m venv .venv
   source .venv/bin/activate
   

2. Install dependencies:

   pip install -r requirements.txt
   

3. Configure environment variables: Create a .env file in the root of the project (you can copy .env.example) and add your Hyperliquid wallet private key and any agent keys.

4. Configure strategies: Edit _data/strategies.json to enable and configure your desired trading strategies.

2. Running the Bot

To run the main application, which includes the dashboard and all background processes, execute the following command:

python main_app.py

Development Conventions

• Modularity: The project is divided into several scripts, each with a specific responsibility (e.g., data_fetcher.py, trade_executor.py).
• Configuration-driven: Strategies are defined in _data/strategies.json, not hardcoded. This allows for easy management of strategies.
• Multi-processing: The application uses the multiprocessing module to run different components in parallel for performance and stability.
• Strategies: Custom strategies should inherit from the BaseStrategy class (defined in strategies/base_strategy.py) and implement the calculate_signals method.
• Documentation: The WIKI/ directory contains detailed documentation for the project. Start with WIKI/SUMMARY.md.

Session Summary

Date: 2025-11-10

Objective(s): Fix urllib3 SSL compatibility warning and create sessionsummary agent following OpenCode.ai guidelines

Key Accomplishments:

• Resolved NotOpenSSLWarning by downgrading urllib3 from 2.5.0 to 1.26.20
• Updated requirements.txt to prevent future SSL compatibility issues
• Created sessionsummary agent in .opencode/agent/ following OpenCode.ai specifications
• Removed incorrect Python implementation and created proper markdown agent configuration

Decisions Made:

• Chose to downgrade urllib3 instead of upgrading SSL environment for stability
• Followed OpenCode.ai agent guidelines instead of creating custom Python implementation
• Configured sessionsummary as subagent with proper permissions and tools

Key Files Modified:

• requirements.txt
• GEMINI.md
• .opencode/agent/sessionsummary.md

Next Steps/Open Questions:

• Test trading bot functionality after SSL fix to ensure no regressions
• Integrate sessionsummary agent into regular development workflow
• Add .opencode/ to .gitignore if not already present

Session Summary

Date: 2025-11-11

Objective(s): Start new Gemini session and organize project files by creating .temp folder for examples and temporary files

Key Accomplishments:

• Created .temp folder for organizing examples and temporary files
• Updated .gitignore to include .temp/ directory
• Moved model_comparison_examples.md to .temp folder for better organization
• Established file management practices for future development

Decisions Made:

• Chose to use .temp folder instead of mixing examples with main project files
• Added .temp to .gitignore to prevent accidental commits of temporary files
• Followed user instruction to organize project structure for better maintainability

Key Files Modified:

• .gitignore
• .temp/ (created)
• model_comparison_examples.md (moved to .temp/)

Next Steps/Open Questions:

• Continue organizing any other example or temporary files into .temp folder
• Maintain consistent file organization practices in future development
• Consider creating additional organizational directories if needed

Session Summary

Date: 2025-11-11

Objective(s): Fix DashboardDataFetcher path resolution error causing file operation failures

Key Accomplishments:

• Identified root cause of file path error in dashboard_data_fetcher.py subprocess execution
• Fixed path resolution by using absolute paths instead of relative paths
• Added os.makedirs() call to ensure _logs directory exists before file operations
• Tested fix and confirmed DashboardDataFetcher now works correctly
• Committed and pushed fix to remote repository

Decisions Made:

• Used os.path.dirname(os.path.abspath(file)) to get correct project root
• Ensured backward compatibility while fixing the path resolution issue
• Maintained atomic file write pattern for data integrity

Key Files Modified:

• dashboard_data_fetcher.py
• GEMINI.md

Next Steps/Open Questions:

• Monitor DashboardDataFetcher to ensure no further path-related errors occur
• Consider reviewing other subprocess scripts for similar path resolution issues
• Test main_app.py to ensure dashboard displays data correctly

Session Summary

Date: 2025-11-11

Objective(s): Debug and fix DashboardDataFetcher path resolution error causing file operation failures

Key Accomplishments:

• Identified root cause of file path error in dashboard_data_fetcher.py subprocess execution
• Fixed path resolution by using absolute paths instead of relative paths
• Added os.makedirs() call to ensure _logs directory exists before file operations
• Tested fix and confirmed DashboardDataFetcher now works correctly
• Committed and pushed fix to remote repository
• Organized project files with .temp folder for better structure

Decisions Made:

• Used os.path.dirname(os.path.abspath(file)) to get correct project root
• Ensured backward compatibility while fixing path resolution issue
• Maintained atomic file write pattern for data integrity
• Added proper directory existence checks to prevent runtime errors

Key Files Modified:

• dashboard_data_fetcher.py
• GEMINI.md
• .gitignore
• .temp/ (created)

Next Steps/Open Questions:

• Monitor DashboardDataFetcher to ensure no further path-related errors occur
• Consider reviewing other subprocess scripts for similar path resolution issues
• Test main_app.py to ensure dashboard displays data correctly
• Continue improving project organization and file management practices

---

Project Review and Recommendations

This review provides an analysis of the current state of the automated trading bot project, proposes specific code improvements, and identifies files that appear to be unused or are one-off utilities that could be reorganized.

The project is a well-structured, multi-process Python application for crypto trading. It has a clear separation of concerns between data fetching, strategy execution, and trade management. The use of multiprocessing and a centralized main_app.py orchestrator is a solid architectural choice.

The following sections detail recommendations for improving configuration management, code structure, and robustness, along with a list of files recommended for cleanup.

---

Proposed Code Changes

1. Centralize Configuration

• Issue: Key configuration variables like WATCHED_COINS and required_timeframes are hardcoded in main_app.py. This makes them difficult to change without modifying the source code.
• Proposal:
  • Create a central configuration file, e.g., _data/config.json.
  • Move WATCHED_COINS and required_timeframes into this new file.
  • Load this configuration in main_app.py at startup.
• Benefit: Decouples configuration from code, making the application more flexible and easier to manage.

2. Refactor main_app.py for Clarity

• Issue: main_app.py is long and handles multiple responsibilities: process orchestration, dashboard rendering, and data reading.
• Proposal:
  • Abstract Process Management: The functions for running subprocesses (e.g., run_live_candle_fetcher, run_resampler_job) contain repetitive logic for logging, shutdown handling, and process looping. This could be abstracted into a generic ProcessRunner class.
  • Create a Dashboard Class: The complex dashboard rendering logic could be moved into a separate Dashboard class to improve separation of concerns and make the main application loop cleaner.
• Benefit: Improves code readability, reduces duplication, and makes the application easier to maintain and extend.

3. Improve Project Structure

• Issue: The root directory is cluttered with numerous Python scripts, making it difficult to distinguish between core application files, utility scripts, and old/example files.
• Proposal:
  • Create a scripts/ directory and move all one-off utility and maintenance scripts into it.
  • Consider creating a src/ or app/ directory to house the core application source code (main_app.py, trade_executor.py, etc.), separating it clearly from configuration, data, and documentation.
• Benefit: A cleaner, more organized project structure that is easier for new developers to understand.

4. Enhance Robustness and Error Handling

• Issue: The agent loading in trade_executor.py relies on discovering environment variables by a naming convention (_AGENT_PK). This is clever but can be brittle if environment variables are named incorrectly.
• Proposal:
  • Explicitly define the agent names and their corresponding environment variable keys in the proposed _data/config.json file. The trade_executor would then load only the agents specified in the configuration.
• Benefit: Makes agent configuration more explicit and less prone to errors from stray environment variables.

---

Identified Unused/Utility Files

The following files were identified as likely being unused by the core application, being obsolete, or serving as one-off utilities. It is recommended to move them to a scripts/ directory or delete them if they are obsolete.

Obsolete / Old Versions:

• data_fetcher_old.py
• market_old.py
• base_strategy.py (The one in the root directory; the one in strategies/ is used).

One-Off Utility Scripts (Recommend moving to scripts/):

• !migrate_to_sqlite.py
• import_csv.py
• del_market_cap_tables.py
• fix_timestamps.py
• list_coins.py
• create_agent.py

Examples / Unused Code:

• basic_ws.py (Appears to be an example file).
• backtester.py
• strategy_sma_cross.py (A strategy file in the root, not in the strategies folder).
• strategy_template.py

Standalone / Potentially Unused Core Files:

The following files seem to have their logic already integrated into the main multi-process application. They might be remnants of a previous architecture and may not be needed as standalone scripts.

• address_monitor.py
• position_monitor.py
• trade_log.py
• wallet_data.py
• whale_tracker.py

Data / Log Files (Recommend archiving or deleting):

• hyperliquid_wallet_data_*.json (These appear to be backups or logs).