Beacon

Latest updates from supported sources.

AllCursor ChangelogCursor BlogAnthropic BlogAWS Bedrock ModelsDMIT VPS StockTencent Hy NewsXiaomi MiMo NewsBigModel NewsMiniMax NewsKimi BlogQwen BlogLongCat ResearchGemini BlogDeepSeek NewsDeepSeek UpdatesByteDance SeedxAI NewsMeta AI BlogArtificial Analysis ChangelogAWS Bedrock User GuideKiro ChangelogLil'LogOpenAI BlogOpenRouter Blog
Tencent Hy News · 2027-08-05

From LR to ELR: A Better Heuristic for Pretraining Dynamics

Understanding pretraining dynamics is crucial for designing effective training hyperparameters for large language models (LLMs), particularly the learning rate (LR). However, LR does not always reflect how much the function represented by the model actually changes, and the optimal LR often shifts with model and data scale. In this post, we identify that the effective learning rate (ELR), which controls the directional changes of model weights, is a more intrinsic heuristic than LR as a tunable hyperparameter. Specifically, ELR delivers more accurate loss prediction under the multi-power law (MPL) model and transfers more reliably across model scales. More broadly, the ELR perspective guides the design of better schedules, which outperform conventional LR-schedule baselines.

Open original

Tencent Hy News · 2026-04-30

Real life is where context gets hard

Previously, we built CL-Bench to test whether AI could learn complex new knowledge, such as rules, and use it effectively. But real life doesn't come with a rulebook. To be a helpful everyday assistant, an AI must do more than just understand complex rules; it should also understand complex human relationships and piece together clues from messy, scattered context.

Open original