TIL

This really made me feel old and out of touch. I tweeted about it too. I don’t always read Hacker News comments, but when I do, they’re 🔥 Someone posted an article titled “I won’t download your app. The web version is a-ok” on Hacker News, and a comment in the thread reframed my perspective: The “Smartphone is the internet”, while for most of us, the “Smartphone is an extension of the internet from our desktops.” … [The] very first touchpoint with “the internet” for younger folks is a smartphone display. They even do homework on these small screens! ...

This is a TIL of a TIL. Simon Willison wrong of using an NVIDIA DGX Spark with an Apple Mac Studio for faster inference, with all the details here. He talks about the two phases of inference: Prefill: Influence Time-To-First-Token (TTFT) Decode: Influence Tokens Per Second (TPS) Prefill Read the prompt Build a Key-Value Cache for each transformer layer in the model Bound by compute as it initializes the model’s internal state and does a lot of matrix multiplication. ...