Searched the wide web, source by source. Every pattern below is named, real, and linked to a live page — arXiv, GitHub, Wikipedia, or the original paper. Three families: the workflows that create algorithms, the science that proves them, and the patterns that make them land.
These are the systems that literally produce new algorithms. Every one runs the same loop: propose variants of a procedure → evaluate against a signal the procedure can't game → keep only what measurably wins → repeat on unseen instances. That loop is GOLD at machine speed.
Evolutionary LLM agent proposes and evaluates code variants; discovered improved matrix-mult and attention kernels.
arxiv.org/abs/2506.13131 · 304★LLM as mutation operator over programs + external evaluator selects candidates; found new math and bin-packing heuristics. Published in Nature.
github.com/google-deepmind/funsearch · 1,133★Learns superhuman game play from scratch via self-play RL combined with Monte-Carlo tree search.
arxiv.org/abs/1712.01815Neural-symbolic system: synthetic-data engine + LLM + symbolic geometry solver evolves olympiad proofs.
arxiv.org/abs/2502.03544RL-trained formal-math model combined with search produces Lean proofs at IMO silver-medal standard.
deepmind.google/blogAgent rewrites its own code in open-ended loops, benchmarks itself, and keeps only self-modifications that measurably improve.
arxiv.org/abs/2505.22954 · 2,416★LLM proposes ideas, runs automated experiments, and writes full papers in a closed loop.
arxiv.org/abs/2408.06292 · 14,717★Stores verbal self-reflections after failures as episodic memory, improving subsequent task attempts.
arxiv.org/abs/2303.11366Open-ended embodied loop: propose skill code, execute in Minecraft, commit successes to a growing program library.
arxiv.org/abs/2305.16291 · 7,260★A step-level process reward model scores each reasoning step and is used to search and filter correct solutions.
arxiv.org/abs/2305.20050Scale inference-time search and verification per problem — beyond parameter scaling.
arxiv.org/abs/2408.03314The statistics and testing science behind the gates. Campbell's law, the winner's curse, and forking paths tell you why single runs lie; always-valid inference, holdouts, k-fold, and interleaving are the mathematics of proof; mutation, metamorphic, fuzzing, chaos, and canary are the proof for code and systems.
The more an outcome measure is used for decision-making, the more it is corrupted by gaming and pressure.
wikipedia.org/wiki/Campbell's_lawThe selected "best" tends to be the luckiest outlier, overstating the true effect.
wikipedia.org/wiki/Winner's_curseHidden researcher degrees of freedom exploited until p<0.05 appears.
wikipedia.org/wiki/Garden_of_forking_pathsConfidence sequences valid at every stopping time, controlling false positives under optional stopping.
arxiv.org/abs/1810.08240Any-time-valid p-values via mixture likelihood ratios — peek and stop early without inflation.
arxiv.org/abs/1512.04922A running-product statistic controlling type-I error under optional stopping.
arxiv.org/abs/1906.07801Metrics that are not the objective but must not degrade — the cost gate's cousin.
arxiv.org/abs/1808.00114Run two identical variants to calibrate variance and false-positive rate before real experiments.
wikipedia.org/wiki/A/B_testingLock hypothesis, model, and analysis plan before seeing outcomes.
osf.ioPeer review of the protocol before results, so findings can't change publishability.
cos.ioEvaluate once on a frozen holdout; never iterate on the test set.
wikipedia.org/wiki/Holdout_methodOptimizing the public leaderboard overfits its slice; only the private LB measures truth.
kaggle.com/docs/competitionsUsers react to "new" regardless of quality, inflating short-term metrics that decay.
wikipedia.org/wiki/Novelty_effectEarly items strongly shape evaluation, distorting order-sensitive metrics.
wikipedia.org/wiki/Primacy_effectCompare algorithms within a single ranked list for high-sensitivity online evaluation.
arxiv.org/abs/2303.10094Rotate train/test splits to reduce single-split variance in generalization estimates.
wikipedia.org/wiki/Cross-validationTest universal invariants over automatically generated inputs rather than hand examples.
wikipedia.org/wiki/QuickCheckInject small mutants; a good suite kills them; survival rate measures test effectiveness.
wikipedia.org/wiki/Mutation_testingCheck relations over transformed inputs when no oracle exists; failures reveal bugs.
wikipedia.org/wiki/Metamorphic_testingAutomated generation of random structured inputs to surface crashes and exploits.
wikipedia.org/wiki/FuzzingDeliberately inject failures in production to verify resilience before real outages.
principlesofchaos.orgRoll out to a small subset first; monitor metrics; ramp or roll back on evidence.
wikipedia.org/wiki/Canary_releaseAdversarial probing of systems to surface failures before launch.
wikipedia.org/wiki/Red_teamingOpposing camps jointly design the experiment to test competing theories, neutral arbiter.
wikipedia.org/wiki/Adversarial_collaborationMany published findings fail to replicate — the reason every gate above exists.
wikipedia.org/wiki/ReproducibilityStandardized effect size making results comparable across scales and studies.
wikipedia.org/wiki/Cohen's_dThe persuasion patterns that make a standard stick — hook the gap, keep the loop open, anchor on the real number. The twist: the honesty itself is the hook. A standard that shows its own −0.144 correlation and 11 Goodhart flags is the only page that can't be a scam — and that's the most persuasive pattern on the site.
Trigger → action → variable reward → investment: the loop that builds habit-forming products.
wikipedia.org/wiki/Hooked_(book) · Nir EyalReciprocity, commitment/consistency, social proof, authority, liking, scarcity.
influenceatwork.comUnpredictable rewards after an action produce the strongest, most persistent engagement.
wikipedia.org/wiki/Reinforcement · SkinnerA perceived gap between what you know and what you want to know creates the itch to click.
Loewenstein 1994 · cmu.eduUnfinished tasks dwell in the mind — open loops drive return visits.
wikipedia.org/wiki/Zeigarnik_effectLosses feel roughly twice as painful as equivalent gains feel good.
wikipedia.org/wiki/Prospect_theoryThe first number seen disproportionately biases all later value judgments.
wikipedia.org/wiki/Anchoring_effectPeople keep investing in something they've already put effort into.
wikipedia.org/wiki/Sunk_costPeople overvalue things they partially created themselves.
wikipedia.org/wiki/IKEA_effectPeople ascribe higher value to things merely because they own them.
wikipedia.org/wiki/Endowment_effectExperiences are judged by their most intense moment and their ending.
wikipedia.org/wiki/Peak–end_ruleThe line we don't cross: no roach motels, no confirmshaming, no forced continuity.
wikipedia.org/wiki/Dark_pattern · BrignullFamily 1 is the engine. Propose → evaluate against an ungameable signal → keep winners → repeat. AlphaEvolve, FunSearch, and the Darwin Gödel Machine are GOLD at machine speed — but none of them ship the verification discipline as a standalone standard. That gap is real, and it's the positioning.
Family 2 is the receipt. freeze before you look → same fight for both sides → score against a sealed truth → demand the paired win survives bootstrap and correction → prove it isn't the prior, the proxy, or the compute → repeat on seeds never seen.
Family 3 is the dealer. Hook with the gap, anchor on the real number, keep the loop open — and let the honesty be the hook. Family 1 is mostly DeepMind/Sakana work; GOLD is the evaluation-and-verification discipline every one of those loops needs but none of them ship.