Self-Play for LLMs, AI for Biology, Formal Verification, and More | YC Paper Club

Scaling laws also hold in protein biology, and the ESM Cambrian model, by expanding data from 50 million to 2.8 billion samples, continued to improve its structure prediction performance, avoiding a plateau.

While AlphaFold3 requires handcrafted MSA features, ESM-Fold2, with single sequences only, outperformed AlphaFold3 50 to 47 in antibody design tasks, demonstrating the effectiveness of general pre-trained representations.

In LLM self-play learning, traditional methods, which only reward difficulty adjustment, have the challenge of generating artificially complex and meaningless problems.

Self-Guided Self-Play (SGS), where a conjurer generates problems related to the original problem and a guide evaluates relevance and complexity with a dual reward, improved a 7B parameter model to performance equivalent to a 670B model with 8 times the computational cost.

In voice AI agents, as traditional RAG causes high latency, it is effective to perform RAG streaming during user utterances and initiate searches at appropriate times based on partial questions.

Lean is both a theorem prover and a functional programming language, and the Mathlib library is the largest of its kind, containing over 1 million lines of formalized mathematics.

TorchLean is the first integrated framework for describing neural networks within Lean, allowing for the proof of equivalence between Flash Attention and standard Attention, as well as the permutation invariance of Attention without positional encoding.

Agent programming is analogous to an RTS game, where productivity is maximized by running multiple agents in parallel and balancing macro and micro management while maintaining high visibility.

Coding agents like Claude have extremely inaccurate task completion time predictions; tasks predicted to take 2 weeks based on human-centric training data can sometimes be completed in 20 minutes.

In agent-generated code, actively documenting structured wiki-style knowledge base files rather than the code itself is more contextually efficient for future agents.