MoE for Order Execution

Joint work with Ali Habibnia and Alexander Ardaiz at the Dataism Lab for Quantitative Finance. The manuscript is under review, so this note describes the design, the methodology, and what the controls did to our own hypotheses, not results that are still in front of referees.

The setup

Take a fixed inventory of Bitcoin and liquidate it over a fixed horizon, executing against real historical limit order book data, walking the actual ladder of resting bids rather than assuming a functional form for Market Impact. Score the result with Implementation Shortfall.

Six configurations compete. The baseline is a single Double Deep Q-Network (DDQL). The proposal is a Mixture of Experts: $K \in {2, 4, 8}$ specialist DDQL networks plus an unsupervised K-Means router that assigns each trading episode to a cluster based on realized volatility, spread, and order-flow imbalance. It always reads the previous episode's features, so no future information leaks into the routing decision. Three dense networks, widened until their parameter count matches each expert budget, act as the capacity control: if a single fat network reproduces whatever MoE does, the answer was capacity, not routing.

The question that actually matters

Not "is the mean cost lower?". That's the question everyone asks, and it's the wrong one. RL training is stochastic: the same architecture on the same data with a different seed produces a different policy. So we retrain the entire pipeline 100 independent times per configuration, on shared seeds and shared evaluation episodes, and study the resulting distribution of outcomes.

That pairing is what turns a pile of summary statistics into inference. Same seed, same episodes, one architectural difference. That's what licenses paired bootstraps, permutation tests and McNemar, rather than eyeballing two averages.

What the controls did to us

No configuration improves mean execution cost. Every paired confidence interval straddles zero. It doesn't establish equivalence either. The intervals are wide enough to admit a real effect in either direction, and saying "statistically indistinguishable" would be claiming more than we measured.

Stability and tail risk turned out to be different things. Across-seed dispersion asks how reliably does training land somewhere decent. Per-run CVaR-95 asks how bad can one trade be, given the policy I ended up with. Those move independently, and in our results they moved in opposite directions. The field routinely collapses both into the word "risk". They are not the same axis and a strategy can improve one while quietly worsening the other.

And the mechanism isn't the one in the architecture's name. Episodes start at randomly sampled bars, which makes the previous episode's features statistically independent of the episode being routed. The router cannot be conditioning on market regime. It's partitioning data, nothing more. Whatever the architecture does, it does by ensembling. That killed the paper's original framing, and it should have.

Some seeds don't produce a worse policy, they produce a degenerate one. A meaningful fraction of vanilla runs converge on selling nothing at all and simply accepting the forced terminal liquidation. Averaged into a mean that's an unremarkable smear; counted as an event rate it's the most interesting thing in the distribution. You only ever see it by training the thing a hundred times.

The part I'd keep

Most of the intellectual work wasn't the architecture. It was building an evaluation that could tell a real effect from a lucky seed, then auditing our own pipeline hard enough that several of our published explanations didn't survive: the floor-sensitivity story, the regime-conditioning story, and our own reproduction-error figures, which we'd understated in the flattering direction.

The empirical results mostly held. The reasons we gave for them frequently didn't. Learning to report that gap plainly, rather than quietly patching the story and moving on, is the habit worth carrying, and it generalises well past this paper: a result you can't decompose into the choices that produced it isn't a finding yet.