MoE for Order Execution
Reinforcement learning with regime-specialized experts for crypto execution. Under review.
Joint work with Ali Habibnia and Alexander Ardaiz at the Dataism Lab for Quantitative Finance. A paper is currently under peer review, so this note describes the approach and the questions rather than the results.
The setup
Take a fixed inventory of Bitcoin and liquidate it over a fixed horizon, executing against real historical limit order book data — walking the actual ladder of resting bids rather than assuming a functional form for Market Impact. Score the result with Implementation Shortfall.
Two agents compete. The baseline is a single Double Deep Q-Network (DDQL). The proposal is a Mixture of Experts: $K$ specialist DDQL networks plus an unsupervised K-Means router that assigns each trading episode to a market regime based on realized volatility, spread, and order-flow imbalance — always using the previous episode's features, so no future information leaks into the routing decision.
The question that actually matters
Not "is the mean cost lower?" — that's the question everyone asks and it's the wrong one. RL training is stochastic: the same architecture on the same data with a different seed produces a different policy. So we retrain the entire pipeline 100 independent times and study the resulting distribution of outcomes.
That reframes the whole investigation. A specialized architecture might not lower the average cost at all, and still be valuable if it makes the outcome of training more reliable — if it narrows the spread of policies you might end up with. Those are different properties, and the field routinely conflates them under the word "risk."
The control that keeps it honest
$K$ experts have roughly $K$ times the parameters of one network. So any advantage might just be capacity, not routing. To separate them we also train a single network with parameter count matched exactly to each expert ensemble. If the effect survives that control, routing is doing real work. If it doesn't, we'd rather know.
We also swept a sensitivity parameter in the fill model expecting a robustness result, and got a clean null instead — the parameter never binds at the order sizes we trade. Reported as a null, because that's what it is.
What I took from it
Most of the intellectual work wasn't the architecture. It was building an evaluation that could tell a real effect from a lucky seed, and being willing to let the controls overturn the story we started with.