Scaling Laws Be Damned

Scaling Laws Be Damned

Dec 23, 2024

The history of transformative software products teaches us that the big business opportunities are in the unglamorous problems that silently grind down productivity, plague daily workflows, and eat into profitability.

We've watched LLMs swing like massive hammers at all kinds of problems. They've changed how we discover information and generate content. But we've hardly scratched the surface in building actual products – not demos, not magical prototypes that fail to stick.

I'm talking about the hard work of constant iteration. Of finding not just strengths but the critical weaknesses of these technologies. Anyone who's spent time prompt engineering knows how quickly you hit diminishing returns. And no, making RAG your universal solution isn't the answer when there's a whole universe of approaches worth exploring. Nor is dumping lots of data into an LLM and expecting consistent magic.

Are LLMs hitting limits? Are we discovering scaling laws? Who cares. We've hardly begun to exhaust what the current generation can do.

Stop chasing technology. Start building products that people love. Everything else be damned.

The history of transformative software products teaches us that the big business opportunities are in the unglamorous problems that silently grind down productivity, plague daily workflows, and eat into profitability.

We've watched LLMs swing like massive hammers at all kinds of problems. They've changed how we discover information and generate content. But we've hardly scratched the surface in building actual products – not demos, not magical prototypes that fail to stick.

I'm talking about the hard work of constant iteration. Of finding not just strengths but the critical weaknesses of these technologies. Anyone who's spent time prompt engineering knows how quickly you hit diminishing returns. And no, making RAG your universal solution isn't the answer when there's a whole universe of approaches worth exploring. Nor is dumping lots of data into an LLM and expecting consistent magic.

Are LLMs hitting limits? Are we discovering scaling laws? Who cares. We've hardly begun to exhaust what the current generation can do.

Stop chasing technology. Start building products that people love. Everything else be damned.