Deep-dives, tutorials and essays on AI, automation & the future of the web — written by builders, for builders.
Research ExplainedLLMsAgents17 articles
Standard position embeddings are additive and have poor long-range generalization. RoPE embeds positions via rotation: multiply Q, K by rotation matrices. Enables 100K+ token context.
Modern LLMs are Transformers. Understand the evolution: self-attention, positional encoding, scaling laws, and how each architectural change improved performance.
Agents make autonomous decisions by reasoning, planning, and calling tools. Understand the perception-decision-action loop and when to use agents vs. deterministic workflows.
Transformers require O(n²) attention. Mamba uses state space models for O(n) complexity with better scaling. Understand selective SSMs and why Mamba matches transformer quality at 1/5 the memory.
Flash Attention achieves 2-4× speedup on attention by changing memory access patterns. Understand I/O complexity, tiling, and how to optimize matrix operations on GPUs.
Different optimizers suit different problems. SGD is stable, Momentum accelerates, Adam is adaptive. Understand why each optimizer works and pick the right one.
Vision Transformers apply Transformers to image classification. Patch embeddings convert images to sequences, enabling the same architecture as NLP models.
model.fit() hides the training loop. GradientTape exposes it. Use it when you need per-batch gradient manipulation, custom loss combinations, or training dynamics that Keras callbacks can't express.
RoPE encodes position by rotating query and key vectors in complex space. It extrapolates beyond training length, transfers across fine-tuning, and adds zero parameters — here's the math.
DataLoader is more than a loop — it's a parallel data pipeline. Build a correct Dataset, write a proper collate_fn, and understand num_workers to eliminate training bottlenecks.
LangChain wraps every LLM provider behind the same Runnable interface. Swap OpenAI for Claude or a local Llama model without changing a line of your chain logic.
Keras gives you three model-building APIs. Sequential is a dead end for anything non-trivial. Functional handles 90% of production architectures. Subclassing gives you full control when you need it.
Weekly dispatches on AI automation, technical deep-dives, and perspectives from the frontier—delivered straight to your inbox.
No spam, ever. Unsubscribe in one click.