AI Engineering

TensorTrade: The Open-Source Python Framework for Building RL-Powered Trading Bots

TensorTrade is a new open-source Python framework that lets developers build, train, and evaluate reinforcement learning agents for algorithmic trading — using NumPy, Pandas, TensorFlow, and Keras. Can an RL agent beat Buy-and-Hold? The results are in.

By Aby Varghese July 13, 2026 6 min read

Algorithmic trading just got an open-source reinforcement learning upgrade. TensorTrade is a newly released Python framework that lets developers build, train, and evaluate RL agents for trading — composing environments, action schemes, reward functions, and data feeds into fully customisable trading systems. If you've ever wondered whether a self-learning AI agent can beat a passive Buy-and-Hold strategy, TensorTrade was built to answer exactly that question.

What Is TensorTrade?

TensorTrade is an open-source framework (Apache 2.0 licence) designed specifically for reinforcement learning-based algorithmic trading. Unlike general-purpose ML libraries, it provides purpose-built, composable components for the full trading pipeline — from raw data ingestion through to order execution and portfolio management.

Under the hood it leans on the Python scientific stack you already know:

NumPy & Pandas — data wrangling and feature engineering
Gym — the standard RL environment interface
Keras & TensorFlow — deep learning policy networks
Ray / RLlib — distributed training at scale (recommended for serious runs)
Optuna — automated hyperparameter optimisation

Requires Python 3.11 or 3.12.

Core Architecture: Four Components That Compose a Trading Agent

TensorTrade's design philosophy is modular composition. The central TradingEnv wires together four key components:

┌─────────────────────────────────────────────────────────────────┐
│                        TradingEnv                               │
│                                                                 │
│   Observer ──────> Agent ──────> ActionScheme ──────> Portfolio │
│   (features)      (policy)      (BSH/Orders)        (wallets)  │
│       ^                                                  │      │
│       └──────────── RewardScheme <───────────────────────┘      │
│                        (PBR)                                    │
│                                                                 │
│   DataFeed ──────> Exchange ──────> Broker ──────> Trades       │
└─────────────────────────────────────────────────────────────────┘

ActionScheme — Translates agent output into market orders. The default is BSH (Buy / Sell / Hold), a clean discrete action space ideal for initial experiments.
RewardScheme — Computes the learning signal after each step. The default PBR (Position-Based Returns) directly ties reward to portfolio performance, keeping incentives aligned with real trading goals.
Observer — Generates the observation the agent receives at each timestep: windowed feature vectors derived from price history and any additional signals you engineer.
Portfolio — Manages wallets and open positions. The default configuration holds USD and BTC, mirroring a typical crypto spot-trading setup.

The Exchange component simulates order execution — including configurable commission rates — while the DataFeed pipeline handles feature engineering upstream of the environment loop.

Getting Started: Installation in Four Commands

# Create a virtual environment (Python 3.12 recommended)
python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate  # Windows: tensortrade-env\Scripts\activate

# Install the core library
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

# Optional: add Ray/RLlib for distributed training
pip install -r examples/requirements.txt

# Verify everything works
pytest tests/tensortrade/unit -v

Docker users can spin up a Jupyter environment, the documentation server, or the full test suite with the single commands make run-notebook, make run-docs, and make run-tests respectively.

Training Scripts: From Quick Demo to Full Optimisation

TensorTrade ships with a progression of ready-to-run training scripts so you can go from zero to a tuned agent without writing boilerplate from scratch:

train_simple.py — A basic demo with wallet tracking. The best first run.
train_ray_long.py — Distributed training via Ray RLlib for longer, parallelised experiments.
train_optuna.py — Bayesian hyperparameter search with Optuna to find the best configuration automatically.
train_best.py — Runs the exact configuration that produced the best results in the team's published experiments.

To kick off your first training run immediately:

python examples/training/train_simple.py

Research Results: Can a PPO Agent Beat Buy-and-Hold on BTC/USD?

The TensorTrade team conducted extensive experiments training PPO (Proximal Policy Optimisation) agents on BTC/USD price data. The results are illuminating — and refreshingly honest about the limitations:

Configuration	Test P&L	vs Buy-and-Hold
Agent (0% commission)	+$239	+$594
Agent (0.1% commission)	−$650	−$295
Buy-and-Hold (baseline)	−$355	—

The headline finding: the agent demonstrates genuine directional prediction capability — it beats Buy-and-Hold at zero commission, proving the policy is learning something real. The problem is trading frequency. At a realistic 0.1% per-trade commission, the agent overtrades and commission costs swamp its prediction edge.

This is not a failure — it's a precise diagnosis. The framework identifies exactly where the gap needs to be closed, and the team has flagged three priority research directions: position sizing to reduce frequency, commission-aware reward schemes, and alternative action spaces.

Priority Research Areas and How to Contribute

TensorTrade is actively soliciting community contributions in the areas most likely to unlock real-world profitability:

Trading frequency reduction — position sizing schemes and configurable minimum holding periods to cut unnecessary round-trips.
Commission-aware reward schemes — reward functions that directly penalise commission cost, not just raw P&L.
Alternative action spaces — continuous or hierarchical action schemes beyond the binary BSH default.

If you're an RL researcher or ML engineer looking for a high-impact open-source contribution with a clear research frontier, this is a strong candidate. See the project's CONTRIBUTING.md for guidelines, and join the community Discord for discussion.

Where TensorTrade Fits in Your AI Engineering Stack

TensorTrade isn't a black-box trading bot — it's a research and engineering platform. It belongs in your stack if you are:

An ML engineer exploring applied RL beyond Atari and robotics environments
A quant developer wanting to benchmark RL policies against classical strategies
A researcher studying the intersection of deep learning and financial markets
A developer building a custom algorithmic trading system who wants composable, testable components rather than a monolith

If you're newer to the broader AI engineering space, the 6-Month Agentic AI Engineer Roadmap provides a solid foundation in production ML systems before you dive into RL-specific tooling. And if you're building agent loops more broadly, the principles in Loop Engineering: The Four Layers That Separate Toy Agents from Production Agents apply directly to how TensorTrade's environment-agent feedback cycle is designed.

For those interested in the open-source ecosystem more broadly, TensorTrade sits alongside other high-quality projects covered in our 10 Open-Source GitHub Repos Every Founder & Vibe Coder Should Bookmark roundup. And if you're drawn to the theme of doing more with less compute — the same ethos behind projects like colibri, which runs a 744B-parameter model in a single C file — TensorTrade's composable, dependency-light design will feel familiar.

Quick Reference: Project Structure

tensortrade/
├── tensortrade/           # Core library
│   ├── env/              # Trading environments
│   ├── feed/             # Data pipeline
│   ├── oms/              # Order management
│   └── data/             # Data fetching
├── examples/
│   ├── training/         # Training scripts
│   └── notebooks/        # Jupyter tutorials
├── docs/
│   ├── tutorials/        # Learning curriculum
│   └── EXPERIMENTS.md    # Research log
└── tests/

Common Issues and Fixes

"No stream satisfies selector" — Update to v1.0.4-dev1+.
Ray installation fails — Run pip install --upgrade pip first, then retry.
NumPy version conflict — Pin with pip install "numpy>=1.26.4,<2.0".
TensorFlow CUDA issues — Install via pip install tensorflow[and-cuda]>=2.15.1.

Conclusion

TensorTrade brings reinforcement learning to algorithmic trading in a principled, composable, open-source package. Its research results are honest: the agent can predict direction, but commission costs remain the critical unsolved problem. That transparency makes it more trustworthy as a research platform — you know exactly what has been tried, what worked, and where the frontier is.

Whether you're an ML practitioner looking to apply RL in a real-world domain, or a trading developer who wants to move beyond rules-based systems, TensorTrade is a well-structured starting point. The codebase is clean, the documentation is thorough, and the community is active.

Get started: TensorTrade on GitHub

Aby Varghese

AI and software enthusiast passionate about web technologies, automation, and developer tools. Writes about AI, testing, and modern software engineering.