K/20X LABS · AI_SETUP_FOUNDATIONS · DAILY RESEARCH BRIEF

AI Runtimes Optimize Performance, Phone AI Expands Multimodal and Agentic Capabilities

Published , 04:44 Bogota (UTC-5) · 27 sourced items, 19 new since the previous edition · Read the foundations review · RSS

Today in 5 points

Desk-side boxes (Mac Studio, DGX Spark, OEM)

Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language ModelsNEW

arXiv · · Desk-side boxes (Mac Studio, DGX Spark, OEM)

Sable is an argument system and single-GPU prover for bit-exact inference of integer-quantized transformers. It proves that a committed transformer produced a given token sequence bit for bit, fixing an integer function with no floating-point step. The prover

Why it matters: Provides a method for verifiable, bit-exact inference of large language models on a single GPU, enhancing trust and reproducibility for local AI.

Phone and edge AI

GigaEdge Spench: An SDK for Practical Streaming Speech EnhancementNEW

arXiv · · Phone and edge AI

GigaEdge Spench is an open-source SDK for streaming speech enhancement on CPU servers, mobile devices, and browsers. It supports building enhancement channels from neural denoising, normalization, and dynamics blocks, and includes 8 kHz INT8 and 48 kHz FP32 mo

Why it matters: Provides an SDK for practical speech enhancement on mobile devices, enhancing local AI audio processing capabilities.

Deep Personalization: Continuous Context Observation for Local Proactive AgentsNEW

arXiv · · Phone and edge AI

Deep personalization is introduced as a closed-loop process connecting continuous context observation with continual agent adaptation. This is initiated in Coco, a proactive desktop agent, where an Observer Agent monitors desktop activity and retains long-term

Why it matters: Describes a method for proactive local agents to understand user activity and personalize assistance, enhancing local agent capabilities.

PSL-RGHA: Parallel Split Learning with Reference Gradient Half-Space Alignment for Mitigating Gradient DivergenceNEW

arXiv · · Phone and edge AI

PSL-RGHA is a gradient-aligned Parallel Split Learning (PSL) framework that uses reference gradient half-space alignment to mitigate training divergence. PSL offloads computational workload to the server through layer-wise model partitioning, suitable for reso

Why it matters: Addresses training divergence in parallel split learning for resource-constrained mobile devices, improving federated learning capabilities on phones.

ASPIRE: An Agentic System-Identification Platform for Iterative Reasoning and ExplorationNEW

arXiv · · Phone and edge AI

ASPIRE is a framework for nonlinear system identification where a large language model agent selects and executes the identification method. It constructs closed-form, nonlinear static or dynamic models from measured data, optionally supplemented by system des

Why it matters: Describes an agentic system for model identification, showing how LLM agents can operate autonomously or interactively on local data.

Yonkers Rises

Simon Willison · · Phone and edge AI

This item describes using Claude Fable 5.2 Medium for 3D modeling and animation, specifically to generate a three.js animation of Yonkers as a mech stomping around Manhattan, Queens, and Brooklyn, using OpenStreetMap data.

Why it matters: Demonstrates the use of a generative AI model (Claude Fable 5.2) for creative tasks, potentially on local or accessible platforms.

v0.19.1

LiteRT-LM releases · · Phone and edge AI

LiteRT-LM `v0.19.1` adds support for multimodal (vision + audio) Gemma 4 models on the Artisan GPU backend. It enables bidirectional attention for vision processing on GPU Artisan models and supports LiteRT audio encoders with Artisan GPU text decoders.

Why it matters: Expands multimodal AI capabilities on mobile devices by supporting Gemma 4 models with vision and audio on Artisan GPUs.

From Chips to Skills, NVIDIA and Korea Expand Sovereign AI Collaboration

NVIDIA Blog · · Phone and edge AI

NVIDIA and Korea’s Ministry of Science and ICT (MSIT) are expanding collaboration across Korea’s sovereign AI ecosystem, including AI semiconductors, data center infrastructure, talent, startups, models, agents, and security. This allows Korean NPU and XPU com

Why it matters: Indicates a broader trend of NPU and XPU development and collaboration, impacting future phone AI hardware and capabilities.

Runtimes and quantization

b12283NEW

llama.cpp releases · · Runtimes and quantization

`llama.cpp` release `b12283` fixes SYCL CLAMP to support strided views by modifying `ggml_sycl_op_clamp` to use a unary helper function. This fix uses a pattern matching other stride-safe unary operations, and removes dead code and an unused block size definit

Why it matters: Improves `llama.cpp` runtime stability and compatibility for SYCL users, potentially affecting performance on compatible hardware.

Looping Deeper Without Growing the KV Cache: Recurrent Depth for Causal Encoder-DecodersNEW

arXiv · · Runtimes and quantization

This paper designs a looped causal encoder-decoder (CED) that achieves recurrent depth with zero round-wise KV growth. Recurrence is placed in the encoder to iteratively refine global KV representations, driving language-modeling gains and preserving long-cont

Why it matters: Offers a method to deepen computation in language models without increasing KV cache size, potentially improving efficiency for local inference.

Closing the BF16-FP16 Convergence Speed Gap in RL Post-Training with Stochastic RoundingNEW

arXiv · · Runtimes and quantization

This paper finds that standard mixed-precision BF16 training converges more slowly than FP16 training in reinforcement learning with low learning rates, attributed to weight sticking. Raising the BF16 learning rate reduces weight sticking and matches FP16's co

Why it matters: Addresses convergence issues in BF16 training for reinforcement learning, relevant for optimizing local model training and fine-tuning.

Deterministic, Reproducible Inference in Exact Integer Arithmetic, and Verifiable Distributed Inference as Its ConsequenceNEW

arXiv · · Runtimes and quantization

Tritify is an open-source runtime that executes the forward pass of real models in exact integer arithmetic, confining floating point to offline quantization and verification. It claims cross-hardware reproducibility and decision fidelity.

Why it matters: Provides a runtime for deterministic and reproducible inference using exact integer arithmetic, crucial for verifiable local AI deployments.

REAL Loss: Optimizing Real Expected Acceptance Length for Speculative DecodingNEW

arXiv · · Runtimes and quantization

This paper analyzes the covariance gap in block-level training objectives for speculative decoding and derives REAL, a real Expected Acceptance Length (EAL) training loss. Speculative decoding accelerates LLM inference by having a draft model propose multiple

Why it matters: Introduces a new training loss (REAL) to optimize speculative decoding, potentially leading to faster LLM inference on local hardware.

b12262NEW

llama.cpp releases · · Runtimes and quantization

`llama.cpp` release `b12262` includes a change for CUDA to skip graph compatibility checks when a graph is being reused. It lists support for various platforms including macOS Apple Silicon, Linux, Android, and Windows.

Why it matters: Improves CUDA performance for `llama.cpp` by skipping redundant checks, beneficial for local inference on NVIDIA GPUs.

v0.40.17NEW

Ollama releases · · Runtimes and quantization

Ollama `v0.40.17` improves ChatGPT CLI restore and graceful restart. It uses `mmap` for Vulkan integrated GPUs and supports concurrent generation for MLX. It also includes `llama.cpp` and MLX version bumps.

Why it matters: Enhances Ollama's runtime for local AI, improving stability, Vulkan GPU usage, and concurrent generation on Apple Silicon (MLX).

v0.40.16

Ollama releases · · Runtimes and quantization

Ollama `v0.40.16` introduces Ripple Preview, a 65B dense model for agentic coding. It notes Qwen3.8 Flash Next uses about 27 GiB less memory on Apple silicon. It also adds CUDA 13.5 support for Windows on ARM and updates `llama.cpp`.

Why it matters: Offers a new agentic coding model and significant memory reduction for Qwen3.8 on Apple Silicon, improving local model deployment efficiency.

Agent sandboxes (E2B and peers)

Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving

arXiv · · Agent sandboxes (E2B and peers)

CoPhy is a CognitivePhysical reinforcement learning framework for autonomous driving. It distills VLM knowledge into a BEV encoder, retaining cognitive ability at zero inference cost, and builds an auto-regressive BEV for a foresighted physical environment.

Why it matters: Presents a framework for autonomous driving that distills VLM knowledge for cognitive ability at zero inference cost, relevant for agent sandboxes.

Einstein World ModelsNEW

arXiv · · Agent sandboxes (E2B and peers)

Einstein World Models (EWMs) are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace. In an EWM, the LLM calls a world-module to produce short rollouts of scenes under consideration.

Why it matters: Proposes a blueprint for LLM-based reasoning systems that integrate visual-temporal rollouts, enhancing agent capabilities in sandboxes.

How Artificial Analysis Puts Frontier Models to the Test with E2BNEW

E2B Blog · · Agent sandboxes (E2B and peers)

Artificial Analysis uses E2B to run agentic benchmarks on public and pre-release models, scaling from zero to thousands of tasks quickly.

Why it matters: Highlights E2B's role in benchmarking agentic models, crucial for evaluating and developing local AI agents.

NVIDIA Vera brings faster compute to E2B sandboxes

E2B Blog · · Agent sandboxes (E2B and peers)

NVIDIA Vera brings faster compute and memory population to E2B sandboxes, with capacity for 1,550 loaded sandboxes on one server.

Why it matters: Enhances E2B agent cloud performance and capacity, directly benefiting agent sandboxes with faster compute.

e2b@2.63.0

E2B SDK releases · · Agent sandboxes (E2B and peers)

E2B SDK `e2b@2.63.0` adds a `strict` parameter to `pause()`. With `strict: true`, a pause is refused if the sandbox's guest agent does not respond. Without it, a memory snapshot is taken and marked. Resuming an unresponsive sandbox without `onResume: 'reboot'`

Why it matters: Improves control and error handling for pausing and resuming agent sandboxes, enhancing reliability for local agent development.

Open models for local use

Tabular Foundation Models in Financial PredictionsNEW

arXiv · · Open models for local use

This technical note introduces tabular foundation models (TFMs) to accounting research and evaluates their performance in predicting firms' next-year earnings changes. TFMs, such as Google Research's TabFM, are pretrained on synthetic tabular data and produce

Why it matters: Introduces efficient and accurate tabular foundation models that can run locally, offering an alternative to larger LLMs for specific tasks.

HaiRuo: Forecasting Ocean Eddies as Fields and Objects with a Satellite-Altimetry Foundation ModelNEW

arXiv · · Open models for local use

HaiRuo forecasts ocean eddies as fields and objects from satellite sea-level maps. HaiRuo-1B, a 1.05-billion-parameter foundation model, is pretrained without labels on 24 years of global satellite-altimetry observations, supplying 15-day physical fields and m

Why it matters: Introduces a specialized foundation model for ocean eddy forecasting, demonstrating the application of smaller, focused models for specific tasks.

Pyrite: Building and Benchmarking Protein-Ligand Pose Ranking AgentsNEW

arXiv · · Open models for local use

Pyrite is a pairwise pose-ranking benchmark for protein-ligand complexes. PyriteRep is a structured representation of molecular geometry, protein-ligand interactions, and physicochemical scores. Two LLM-based agents, PyriteReasoner and PyriteFeaturizer, are de

Why it matters: Introduces a benchmark and LLM-based agents for protein-ligand pose ranking, relevant for local computational chemistry applications.

OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn LearningNEW

arXiv · · Open models for local use

OptiMind is a system that improves small language models (SLMs) for optimization modeling through data cleaning, class-specific hints, and training with solver feedback. It addresses limitations of existing SLMs for translating natural-language problems into s

Why it matters: Improves small language models for optimization tasks, making them more effective for local deployment where privacy, latency, and cost are concerns.

The Wizard of Open Weights: Chris Alexiuk Makes AI Easy With Open Models

NVIDIA Blog · · Open models for local use

Chris Alexiuk, a senior product research engineer at NVIDIA, specializes in making open large language models practical for working developers. He advocates for open-source AI.

Why it matters: Highlights NVIDIA's focus on making open LLMs practical, which supports the development and deployment of local AI models.

Method

Sources: arXiv API, Apple Machine Learning Research, NVIDIA, Google Research, Google Developers, Microsoft Research, Hugging Face, MLCommons, MIT News, Nature Machine Intelligence, Communications of the ACM, and official GitHub release feeds (MLX, llama.cpp, Ollama, vLLM, MLC LLM, LiteRT-LM, E2B). Items are filtered by topic rules; summaries are AI-assisted (gemini-2.5-flash) and grounded only in each source's own abstract or post text. Always read the linked source before acting.

Archive