Eva Silverstein:Learning Dynamics, Representations and Symmetry Breaking (September 24, 2026)

Value of the Information & Strength of the Argument

The talk provides substantial value by synthesizing multiple rigorous approaches to a central problem in AI research: the emergence of representations during training. The argumentation is solid, built on explicit mathematical derivations and supported by empirical plots comparing the proposed effective theories with actual network behavior. The step-by-step simplifications from realistic architectures to tractable models are carefully justified, and the presentation highlights both successes and limitations of each method. The discussion of symmetry breaking, including the introduction of novel mechanisms specific to attention, offers new conceptual insights that go beyond typical empirical analyses. The overall reasoning is coherent and logically structured, though the talk assumes a high level of background knowledge, limiting its accessibility.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is high: the speaker is a leading physicist, and the presented results are grounded in established theoretical methods (DMFT, symmetry analysis, field theory) and validated against numerical experiments. However, the talk serves as a review of ongoing collaboration work, and specific citations to original papers are only alluded to by author names, not provided in full. The only explicit source is the Simons Foundation event page. The title accurately describes the content, focusing on learning dynamics and symmetry breaking, and matches the delivered material. No factual inaccuracies were apparent, but the lack of detailed references limits independent verification. The content is clearly aimed at experts, but this does not detract from its scientific value.

The title accurately reflects the content: learning dynamics, representation learning, and symmetry breaking are the main themes.

8/10

The presentation is based on established physics methods, includes theoretical derivations and empirical comparisons, and is given by an expert. However, it is a summary of multiple works without full peer-review verification, and only one source link is provided.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and overview of the problem: understanding how neural networks develop representations through learning, with emphasis on physics-based approaches.
• 07:00Presentation of neural low-rank filtering as an effective theory for layer-wise feature learning, with empirical validation.
• 15:00Spectral analysis of the neural network's kernel using dynamical mean field theory; discussion of eigenvalue bulk and outliers, and implications for training stability.
• 22:00Introduction of a simplified transformer toy model with attention, demonstrating the advantages of sequential data processing and rigorous bounds on learning steps.
• 31:00Analysis of stochastic gradient descent dynamics, exploring phases of exploration vs. convergence and the role of gradient noise in navigating the loss landscape.
• 38:00Discussion of discrete and continuous symmetries in neural networks, including symmetry breaking in attention and the analog of anomalies, offering novel interpretability tools.

• Simons Collaboration on the Physics of Learning and Neural Computation Annual Meeting 2026 — Event page where this talk and other collaboration research are presented.