It’s fascinating that the consequences of a Fair Use court ruling means that AI companies are required to destroy the books they digitize. It means that being upset that they’re destroying the books is being upset with Fair Use and book ownership.

I find it odd how people cling to boring assessments of technologies so they can make themselves comfortable. Saying LLMs hallucinate doesn’t feel right, because even going on the newly synthesized definition of it saying something that seems plausible but is inaccurate presupposes what accuracy is.

Looking at my Claude usage for a single session, it now doesn’t seem that wild to me that someone could spend tens of billions of tokens or more in a month. This huge context windows mean a single session with 500k+ tokens incurs millions of tokens of cost in a couple turns.

As an industry we’ve mostly focused on the big shiny components: CPUs, RAM, graphics cards, etc… but we almost never pay attention to the most important part: the motherboard. The more that I use agent harnesses, the more they feel like motherboards.

Writing is incredibly nuanced. There’s so many subtle things that give the tell of lazy framing, or what some might call the “AI slop” feel. It reminds me that writing is difficult precisely because it’s an art that requires a careful eye.

Running /usage in Claude and seeing hundreds of dollars spent is weird, and I thought it was inflated. It’s not, but that feels like it’s because the context is allowed to grow to such large token quantities that every message turn starts incurring nearly a million tokens in costs.

Thought about this some more and initially thought I was wrong here, because models technically use *tokens* not prose words. But no, it's right, because even x86 machine code is translated into microcode, which is what the processor can actually run.

The Nuanced Writer@skriptble.me · 2w ago

And I mean *machine code* not assembler code, because the processor doesn't run assembler code, it runs machine code. A lot of what harnesses are doing is taking your rough prompts and wrapping them with enough context and tooling so that the model can actually perform well.

There's a deep irony that so many of us laughed at the idea of "prompt engineers" and now we've all become prompt engineers.

The Nuanced Writer@skriptble.me · 2w ago

By the time the next one comes, we'll likely have the same reaction, just like we had the same reaction to init systems, operating systems, kernels, interpreters, compilers, and assemblers.

It's so weird that it's obvious to me now that we're rebuilding computers on top of LLMs and AI. Or, more accurately, they *are* computers. The models have their own architectures, which require specific inference engine support, and then you build an operating system on top of inference engines.

Social media algorithms are so weird. Sometimes I’m like “this video is gonna do terribly” and it performs well. Other times I’m like “oh this is gonna do great” and the algorithm pushes it to no one 😒

1 million token context windows cause such problems with usage caps. I’ve been going back and forth on Codex and Claude Code. At times I like both, at times I dislike both.

The Nuanced Writer@skriptble.me · 2w ago

Me: you’re at 98% of your context maximum, we should compact before we do anything else. Claude: sure! But what if I actually just finished up this work over here. And whoop, there goes 10% of my 5 hour usage window.

Me: you’re at 98% of your context maximum, we should compact before we do anything else. Claude: sure! But what if I actually just finished up this work over here. And whoop, there goes 10% of my 5 hour usage window.

But if someone comments or responds with openness and clear curiosity, I’m usually more willing to engage. I’ve spent enough time trying to convince the hardest to convince people of things. I’ve got better things to do.

The Nuanced Writer@skriptble.me · 2w ago

One of the things that it took me a long time to learn was how to actually protect my own peace. I used to respond to comments because people were so hilariously wrong they needed correcting. Now I look at those comments and ask “is this worth the energy?” and often decide no.

One of the things that it took me a long time to learn was how to actually protect my own peace. I used to respond to comments because people were so hilariously wrong they needed correcting. Now I look at those comments and ask “is this worth the energy?” and often decide no.

I said fuck it and I’m just gonna use Pi. It’s actually working, unlike Codex. Side note, my graphics cards get somewhere between 100 and 130 tokens per second and holy hell does that feel FAST. The ~2700 tokens per second of prefill is certainly helping.

The Nuanced Writer@skriptble.me · 2w ago

There’s really wonderful parts of using LLMs and coding agents. And then there are the walls you hit when you want to get off the basic path. Coding harnesses have felt pretty great, but now that I want to run local models alongside frontier models everything goes to hell.

There’s really wonderful parts of using LLMs and coding agents. And then there are the walls you hit when you want to get off the basic path. Coding harnesses have felt pretty great, but now that I want to run local models alongside frontier models everything goes to hell.

Social media is so weird. Had a Short that only got 4 views in 5 hours. Then I changed the title and 10 minutes later the view count was climbing. Why the title matters so much when most views are through the Shorts feed is lost on me but ¯\_(ツ)_/¯

Over the last few months I’ve managed to speed up my workflows significantly. A lot of it is because I can now create software via coding agents that would have never been worth it to build. That software makes my workflows much lighter which means I can scale to do so many more things.

A year ago the idea of not reading the code coming out of LLMs horrified me. I found it distasteful and it felt like a hyped up fad. A year later and a bunch of growth, it’s obvious to me that reading the code isn’t just a waste of time, it’ll probably make your software worse. What a year.

fallthrough@fallthrough.fm · 3w ago

We've published episode 80: Everything Agentic All at Once. Happy Listening!

I think part of the apprehension to letting the models write code without really checking it is a mistake software folks make. In the med device space there are two really important things that you have to do before releasing a product: verification and validation.

The debate around subscriptions going away is interesting, because it feels sort of wrong at this point. It seems to me that if the big AI labs are going to survive they need both the $20 per month folks using chat and the Enterprises paying per token. Losing either demographic is like catastrophic.

So we all just getting unlimited usage for the next while? It seems like Anthropic and OpenAI are just gonna continue the escalation until one can’t sustain it anymore.

OpenAI temporarily removing the 5 hour usage window and giving out usage resets like they’re candy is wild. This battle between them and Anthropic means that it’s a particularly good time to go build a bunch of stuff that requires powerful models. Burn those tokens while they’re free!

A fun thing about language is that 15 years ago this sentence would have been nonsense. Then again, 15 years ago we were poking each other on Facebook, so this phenomenon makes happens in both directions.

The Nuanced Writer@skriptble.me · 4w ago

When we said we wanted punchy headlines and writing, we didn’t mean that we want the reader punched in the face. Unfortunately, people posting on LinkedIn missed the follow up memo and have gone all punchmaxxing with their writing.

When we said we wanted punchy headlines and writing, we didn’t mean that we want the reader punched in the face. Unfortunately, people posting on LinkedIn missed the follow up memo and have gone all punchmaxxing with their writing.

I thought it wasn’t possible for everyone to be wrong pretty much all the time. Then I spent too much thinking about rhetoric and language. Now it’s entirely understandable why everyone is wrong all the time.

I continue to be amazed at how fast I can move with the help of LLMs and coding agents. I just did what would have been weeks of on and off research in an afternoon because I never had to debug the stumbling blocks myself, Claude or Amp just fixed things itself while I was doing other work.

How can there be so many takes on the problems of our industry and software in general and yet nearly all of them lack some deep and subtle nuance? I will posit that a core problem is that we spent somewhere around 30 years trying to make software easier instead of better.

I did not like the idea Google going chat first for search, but pretty much any time a search doesn’t surface the AI summary I get a little annoyed. I think I was always searching for an answer, and I had to delve through sludge to find that before. I don’t miss having to do that.

This is also true of capitalism and socialism. What people lament isn’t actually capitalism, it’s something closer to widespread dysfunction. Humans love to make category errors, but improving our lives depends on us resolving those errors so we can fix actual problems.

The Nuanced Writer@skriptble.me · last mo.

Spending my time observing the world for what it is, there’s a theme of people wanting something to be one thing (if you’re really advanced, two things). Everything is an unbounded number of things. AI/LLMs aren’t good and they aren’t bad. Questioning on that premise is a category error.