Identifying Interactions at Scale for LLMs

Follow a connection

Related by topics and source metadata.

1. 1

   What exactly does word2vec learn?

   article

   Berkeley AI ResearchSeptember 1, 2025

   What exactly does word2vec learn, and how? Answering this question amounts to understanding representation learning in a minimal yet interesting language modeling task. Despite the fact that word2vec is a well-known precursor to modern language models, for many years, researchers lacked a quantitative and predictive theory describing its learning process. In our new paper , we finally provide such a theory. We prove that there are realistic, practical regimes in which the learning problem reduces to unweighted least-squares matrix factorization . We solve the gradient flow dynamics in closed f

   Original source ↗Frontier AI
2. 2

   RL without TD learning

   article

   Berkeley AI ResearchNovember 1, 2025

   In this post, I’ll introduce a reinforcement learning (RL) algorithm based on an “alternative” paradigm: divide and conquer . Unlike traditional methods, this algorithm is not based on temporal difference (TD) learning (which has scalability challenges ), and scales well to long-horizon tasks. We can do Reinforcement Learning (RL) based on divide and conquer, instead of temporal difference (TD) learning. Problem setting: off-policy RL Our problem setting is off-policy RL . Let’s briefly review what this means. There are two classes of algorithms in RL: on-policy RL and off-policy RL. On-policy

   Original source ↗Frontier AI
3. 3

   Information-Driven Design of Imaging Systems

   article

   Berkeley AI ResearchJanuary 10, 2026

   An encoder (optical system) maps objects to noiseless images, which noise corrupts into measurements. Our information estimator uses only these noisy measurements and a noise model to quantify how well measurements distinguish objects. Many imaging systems produce measurements that humans never see or cannot interpret directly. Your smartphone processes raw sensor data through algorithms before producing the final photo. MRI scanners collect frequency-space measurements that require reconstruction before doctors can view them. Self-driving cars process camera and LiDAR data directly with neura

   Original source ↗Frontier AI
4. 4

   Gradient-based Planning for World Models at Longer Horizons

   article

   Berkeley AI ResearchApril 20, 2026

   GRASP is a new gradient-based planner for learned dynamics (a “world model”) that makes long-horizon planning practical by (1) lifting the trajectory into virtual states so optimization is parallel across time, (2) adding stochasticity directly to the state iterates for exploration, and (3) reshaping gradients so actions get clean signals while we avoid brittle “state-input” gradients through high-dimensional vision models. Large, learned world models are becoming increasingly capable. They can predict long sequences of future observations in high-dimensional visual spaces and generalize acros

   Original source ↗Frontier AI
5. 5

   Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling

   article

   Berkeley AI ResearchMay 8, 2026

   Overview of adaptive parallel reasoning. What if a reasoning model could decide for itself when to decompose and parallelize independent subtasks, how many concurrent threads to spawn, and how to coordinate them based on the problem at hand? We provide a detailed analysis of recent progress in the field of parallel reasoning, especially Adaptive Parallel Reasoning. Disclosure: this post is part landscape survey, part perspective on adaptive parallel reasoning. One of the authors (Tony Lian) co-led ThreadWeaver ( Lian et al., 2025 ), one of the methods discussed below. The authors aim to presen

   Original source ↗Frontier AI
6. 6

   2026 BAIR Graduate Showcase

   article

   Berkeley AI ResearchJuly 1, 2026

   Congratulations to the Berkeley Artificial Intelligence Research (BAIR) Lab class of 2026! This year, BAIR celebrates another remarkable group of Ph.D. graduates whose curiosity, creativity, and perseverance have pushed the frontiers of artificial intelligence and machine learning. Their work spans the breadth of modern AI — robotics and embodied intelligence, large language models and reasoning, computer vision, generative modeling, AI safety, human-AI interaction, AI for science and healthcare, and much more. Along the way, they have published influential research, built systems with real-wo

   Original source ↗Frontier AIScience & psychologyUnexpected finds
7. 7

   Intelligence is Free, Now What? Data Systems for, of, and by Agents

   article

   Berkeley AI ResearchJuly 7, 2026

   ... government of the people, by the people, for the people ... — Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1 , and some providers are pushing costs below $0.10 . Across benchmarks, inference prices have fallen between 9x and 900x per year , with a median decline near 50x. Even frontier models are getting dramatically cheaper each generation, with open-source models following closely behind. And crucially, even if “Nobel-Prize-winning genius-level” intelli

   Original source ↗Frontier AI
8. 8

   Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

   article

   Berkeley AI ResearchJuly 26, 2026

   Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by supervising the contents of each belief state.. As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts, but at a significant performance cost, especially for human assistance domains where high quality data is scarce, e.g., collaborative code generation. We address this with ABBEL : a framework that isolates and supervises the information content of

   Original source ↗Frontier AI
9. 9

   From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon

   article

   Berkeley AI ResearchJuly 29, 2026

   Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago. With so much of computing now centered on AI, GPU kernels are a crucial com

   Original source ↗Frontier AISoftware engineering
10. 10

    Does Your Model Run on AWS Trainium? Look It Up

    article

    Hugging FaceOctober 23, 2026

    Original source ↗Frontier AI
11. 11

    LitPivot: Developing research ideas through dynamic contextualization within the literature landscape

    article

    Hugging FaceOctober 23, 2026

    Original source ↗Frontier AI
12. 12

    Bringing foundational models to the front lines of TB screening

    article

    Google ResearchOctober 23, 2026

    Generative AI

    Original source ↗Frontier AI