Exactly Subtractable Learned Evolution-Strategy Updates

Why People Wanted It

A learned optimizer rarely starts from nothing. If a neural rule modifies a competent hand-designed optimizer, its score combines what the old algorithm already did well with whatever learning added. Comparing only against an external baseline cannot separate those contributions.

So the real question was never just whether a learned evolution strategy transfers. It was: how much of its advantage survives when you subtract the algorithm it was built on?

The Core Move

Put bounded multiplicative neural gates on parts of separable CMA-ES, initialize every gate to identity, and compare the full strategy with the same code forced to keep those gates at identity:

$g(v)=\exp(\ln 2\,\tanh(3v))\in[1/2,2],\qquad g(0)=1.$

The neural modules can modulate the classical updates, but their zero-output initialization leaves those updates unchanged—and the identity-gate run supplies an exact control for measuring the learned increment.

Quiz

What does the identity-gate control isolate in this learned evolution strategy?

The effect of changing the objective function while leaving the optimizer unchanged.

The effect of training the gates for a longer deployment budget.

The effect of learned modulation relative to the anchored core and shared fixed machinery.

The performance of bare CMA-ES with every other part of the implementation removed.

$ check

Why the Identity Gate Is an Exact Control

Here is the part that gets technical: the identity setting is not a separately reimplemented baseline, but a constraint on the learned strategy’s own code path. Each module’s final dense layer is initialized to zero, so its output is $v=0$ and its multiplicative gate is exactly one. Forcing every gate to one throughout a run removes the learned modulation and leaves separable CMA-ES plus the strategy’s fixed sampling machinery.

That distinction matters for attribution. The control shares the implementation and evaluation harness with the full method; the comparison therefore targets the learned modulation rather than a difference in code paths. It is not a comparison to bare CMA-ES: fixed machinery remains in both configurations.

Why Bounded Multipliers Make the Deviation Legible

Now the version for readers who want more of the machinery: the gate’s argument is passed through a hyperbolic tangent, then scaled and exponentiated. Since $\tanh(3v)$ lies between $-1$ and $1$, the multiplier stays between one-half and two. The exponential makes the deformation multiplicative, while the zero-output initialization pins the starting point exactly at one.

The three learned modules act on recombination weights, the step-size and mean-step updates, and per-coordinate axis scales. For example, the mean update takes the weighted step and scales it by its gate; when that gate is one, the usual separable CMA-ES mean update is recovered. Similarly, the step-size update’s log change is scaled by its gate, and the diagonal scale update’s exponent is modulated by a dimension gate. With all gates at one, these are the separable CMA-ES updates. The bounded range limits each learned multiplier, but it does not guarantee that a learned strategy will transfer well.

What the Subtraction Says—and Does Not Say

Here is the exact-control interpretation at the level of the reported comparison: against a same-seed CMA-ES reference normalized to 1.000, the identity-gate anchor scores 1.107 and the full method scores 1.191. The difference is a paired learned increment of 0.083, positive on all eight evaluation seeds; the anchor accounts for 55.8% of the full method’s above-reference margin.

That is a useful decomposition, not proof that the learned component is universally beneficial. The increment shrinks with the measured budget and reverses on final solution quality: the anchor’s mean final return is higher. So what exactly does the score reward, and would another objective tell a different story?

Minimal Causal Code

A plain scalar gate makes the anchoring logic visible. The research strategy applies several learned gates to different CMA-ES quantities; this loop isolates the identity-versus-modulation mechanism.

import torch


def gate(raw):
    return torch.exp(torch.log(torch.tensor(2.0)) * torch.tanh(3.0 * raw))


# Illustrative one-dimensional separable CMA-ES state.
mean = torch.tensor(0.0)
sigma = torch.tensor(1.0)
axis = torch.tensor(1.0)
raw_mean = torch.tensor(0.0)  # zero-output initialization

for generation in range(5):
    noise = torch.randn(16)
    candidates = mean + sigma * axis * noise
    fitness = candidates.square()
    elite = torch.argsort(fitness)[:4]
    weights = torch.full((4,), 0.25)
    weighted_noise = (weights * noise[elite]).sum()

    # Identity control: use 1.0 here for the whole run.
    mean_gate = gate(raw_mean)
    mean = mean + mean_gate * sigma * axis * weighted_noise

What You Gain, and What You Lose

The design gives you a clean way to ask what learning contributed: initialize at the classical update, constrain the gates to identity for a matched control, and measure the gap under shared code and evaluation conditions. Bounded multiplicative residuals also keep the learned update anchored to a known rule rather than allowing an unconstrained replacement.

But an exact anchor is not the same as an exact causal explanation of every behavior. The full strategy includes fixed sampling machinery that remains in the identity control, and the learned increment depends on metric and budget. In the reported comparison, the full method improves the anytime score over the anchor, while the anchor finishes with higher mean return. Read the learned gain as conditional, not as a universal improvement over the classical core.