Algorithmic trading just got an open-source reinforcement learning upgrade. TensorTrade is a newly released Python framework that lets developers build, train, and evaluate RL agents for trading — composing environments, action schemes, reward functions, and data feeds into fully customisable trading systems. If you've ever wondered whether a self-learning AI agent can beat a passive Buy-and-Hold strategy, TensorTrade was built to answer exactly that question.
What Is TensorTrade?
TensorTrade is an open-source framework (Apache 2.0 licence) designed specifically for reinforcement learning-based algorithmic trading. Unlike general-purpose ML libraries, it provides purpose-built, composable components for the full trading pipeline — from raw data ingestion through to order execution and portfolio management.
Under the hood it leans on the Python scientific stack you already know:
- NumPy & Pandas — data wrangling and feature engineering
- Gym — the standard RL environment interface
- Keras & TensorFlow — deep learning policy networks
- Ray / RLlib — distributed training at scale (recommended for serious runs)
- Optuna — automated hyperparameter optimisation
Requires Python 3.11 or 3.12.
Core Architecture: Four Components That Compose a Trading Agent
TensorTrade's design philosophy is modular composition. The central TradingEnv wires together four key components:
┌─────────────────────────────────────────────────────────────────┐
│ TradingEnv │
│ │
│ Observer ──────> Agent ──────> ActionScheme ──────> Portfolio │
│ (features) (policy) (BSH/Orders) (wallets) │
│ ^ │ │
│ └──────────── RewardScheme <───────────────────────┘ │
│ (PBR) │
│ │
│ DataFeed ──────> Exchange ──────> Broker ──────> Trades │
└─────────────────────────────────────────────────────────────────┘- ActionScheme — Translates agent output into market orders. The default is BSH (Buy / Sell / Hold), a clean discrete action space ideal for initial experiments.
- RewardScheme — Computes the learning signal after each step. The default PBR (Position-Based Returns) directly ties reward to portfolio performance, keeping incentives aligned with real trading goals.
- Observer — Generates the observation the agent receives at each timestep: windowed feature vectors derived from price history and any additional signals you engineer.
- Portfolio — Manages wallets and open positions. The default configuration holds USD and BTC, mirroring a typical crypto spot-trading setup.
The Exchange component simulates order execution — including configurable commission rates — while the DataFeed pipeline handles feature engineering upstream of the environment loop.
Getting Started: Installation in Four Commands
# Create a virtual environment (Python 3.12 recommended)
python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate # Windows: tensortrade-env\Scripts\activate
# Install the core library
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
# Optional: add Ray/RLlib for distributed training
pip install -r examples/requirements.txt
# Verify everything works
pytest tests/tensortrade/unit -vDocker users can spin up a Jupyter environment, the documentation server, or the full test suite with the single commands make run-notebook, make run-docs, and make run-tests respectively.
Training Scripts: From Quick Demo to Full Optimisation
TensorTrade ships with a progression of ready-to-run training scripts so you can go from zero to a tuned agent without writing boilerplate from scratch:
- train_simple.py — A basic demo with wallet tracking. The best first run.
- train_ray_long.py — Distributed training via Ray RLlib for longer, parallelised experiments.
- train_optuna.py — Bayesian hyperparameter search with Optuna to find the best configuration automatically.
- train_best.py — Runs the exact configuration that produced the best results in the team's published experiments.
To kick off your first training run immediately:
python examples/training/train_simple.pyResearch Results: Can a PPO Agent Beat Buy-and-Hold on BTC/USD?
The TensorTrade team conducted extensive experiments training PPO (Proximal Policy Optimisation) agents on BTC/USD price data. The results are illuminating — and refreshingly honest about the limitations:
| Configuration | Test P&L | vs Buy-and-Hold |
|---|---|---|
| Agent (0% commission) | +$239 | +$594 |
| Agent (0.1% commission) | −$650 | −$295 |
| Buy-and-Hold (baseline) | −$355 | — |
The headline finding: the agent demonstrates genuine directional prediction capability — it beats Buy-and-Hold at zero commission, proving the policy is learning something real. The problem is trading frequency. At a realistic 0.1% per-trade commission, the agent overtrades and commission costs swamp its prediction edge.
This is not a failure — it's a precise diagnosis. The framework identifies exactly where the gap needs to be closed, and the team has flagged three priority research directions: position sizing to reduce frequency, commission-aware reward schemes, and alternative action spaces.
Priority Research Areas and How to Contribute
TensorTrade is actively soliciting community contributions in the areas most likely to unlock real-world profitability:
- Trading frequency reduction — position sizing schemes and configurable minimum holding periods to cut unnecessary round-trips.
- Commission-aware reward schemes — reward functions that directly penalise commission cost, not just raw P&L.
- Alternative action spaces — continuous or hierarchical action schemes beyond the binary BSH default.
If you're an RL researcher or ML engineer looking for a high-impact open-source contribution with a clear research frontier, this is a strong candidate. See the project's CONTRIBUTING.md for guidelines, and join the community Discord for discussion.
Where TensorTrade Fits in Your AI Engineering Stack
TensorTrade isn't a black-box trading bot — it's a research and engineering platform. It belongs in your stack if you are:
- An ML engineer exploring applied RL beyond Atari and robotics environments
- A quant developer wanting to benchmark RL policies against classical strategies
- A researcher studying the intersection of deep learning and financial markets
- A developer building a custom algorithmic trading system who wants composable, testable components rather than a monolith
If you're newer to the broader AI engineering space, the 6-Month Agentic AI Engineer Roadmap provides a solid foundation in production ML systems before you dive into RL-specific tooling. And if you're building agent loops more broadly, the principles in Loop Engineering: The Four Layers That Separate Toy Agents from Production Agents apply directly to how TensorTrade's environment-agent feedback cycle is designed.
For those interested in the open-source ecosystem more broadly, TensorTrade sits alongside other high-quality projects covered in our 10 Open-Source GitHub Repos Every Founder & Vibe Coder Should Bookmark roundup. And if you're drawn to the theme of doing more with less compute — the same ethos behind projects like colibri, which runs a 744B-parameter model in a single C file — TensorTrade's composable, dependency-light design will feel familiar.
Quick Reference: Project Structure
tensortrade/
├── tensortrade/ # Core library
│ ├── env/ # Trading environments
│ ├── feed/ # Data pipeline
│ ├── oms/ # Order management
│ └── data/ # Data fetching
├── examples/
│ ├── training/ # Training scripts
│ └── notebooks/ # Jupyter tutorials
├── docs/
│ ├── tutorials/ # Learning curriculum
│ └── EXPERIMENTS.md # Research log
└── tests/Common Issues and Fixes
- "No stream satisfies selector" — Update to
v1.0.4-dev1+. - Ray installation fails — Run
pip install --upgrade pipfirst, then retry. - NumPy version conflict — Pin with
pip install "numpy>=1.26.4,<2.0". - TensorFlow CUDA issues — Install via
pip install tensorflow[and-cuda]>=2.15.1.
Conclusion
TensorTrade brings reinforcement learning to algorithmic trading in a principled, composable, open-source package. Its research results are honest: the agent can predict direction, but commission costs remain the critical unsolved problem. That transparency makes it more trustworthy as a research platform — you know exactly what has been tried, what worked, and where the frontier is.
Whether you're an ML practitioner looking to apply RL in a real-world domain, or a trading developer who wants to move beyond rules-based systems, TensorTrade is a well-structured starting point. The codebase is clean, the documentation is thorough, and the community is active.
Get started: TensorTrade on GitHub
