Exploring how Google likely uses older, cheaper TPU generations for profitable LLM inference through fleet strategy, XLA/JAX, quantisation, model routing, and prefill/decode disaggregation.
Read more →LLM Inference Routing & Optimisation: Part 1 - Prefill & Decoding Disaggregation
Read more →How the evolution from batch to streaming systems reshaped ML feature platforms, and why streaming-first is the future. Part 1 of a series on building Thyme.
Read more →Building fun web applications with FastHTML
Read more →LLM Inference Optimisation - Continuous Batching and vLLM
Read more →A dive into the FlytePropeller architecture and Go Worker Pools
Read more →Some PoC code for using Meta's LLM compiler optimisation models
Read more →Adlib's blog post on technical specialties.
Read more →How we built our new MLOps platform at Ultraleap
Read more →