Episode 2: Your PR Shouldn’t Make Your Stomach Hurt

Garrett Dimon 00:42

Not any deep thoughts, but the more time goes on and the more I find ways to adjust my workflow, the more I believe in it. And not from like a black and white human user interfaces are gone. And more of a humans are gonna want the interface they're gonna want, right? So in your case.

John Nunemaker 01:09

Yeah.

Garrett Dimon 01:10

For instance, at this point, it feels like things are mostly dictation. I'm rapidly moving that way as well to hope maybe save my hands. and I, you know, there's so many different things in case of accessibility reasons or just personal preference or context that people can't use a screen-based UI. And so whether that's audio or whether in the future it's augmented reality glasses. I feel like increasingly what the MCPs and that approach gives is people can choose the interface that's right for them in the context they're in, at the time they're at. And that makes the MCP more of that baseline that enables whichever UI the person wants. So now we're gonna hopefully move more towards personal UIs. And I mean

My wild idea is that I think at some point what and I don't know how it would take shape or whatever, but I think people will have their own UIs that they design the way they prefer to see things, whether that's larger fonts, smaller fonts, denser fonts, more graphs and charts

John Nunemaker 02:29

Yeah.

Garrett Dimon 02:29

and visualizations, or more image-based interfaces. And they'll design that with their agent and then say, okay. Here's all of the MCPs in my life. I mean, hopefully we be find a more human-friendly thing than MCPs, because

John Nunemaker 02:45

Yeah, for sure.

Garrett Dimon 02:46

my all my non-tech friends' eyes glaze over when I start trying to like explain what I see as the future for and whatever that is, basically you've got your own person, totally personalized interface that works for you or multiple variations. And then you say, here's all of the information in my life. I'll ask you questions and you deliver this information to me in the ways that I need it, when I need it, how I need it. And that to me has been one of the like biggest epiphanies and all of that. Just, you know, when you're my if I had to choose one angle of my career to identify what I enjoy, where my passion is, it's designing interfaces for people, right? Playing that

John Nunemaker 03:32

Yeah.

Garrett Dimon 03:33

middle layer between people and computers and like. I'm gonna be useless. Or at least that parti that

John Nunemaker 03:38

Ha ha ha.

Garrett Dimon 03:39

portion of me is going to be and I honestly, like, you know, it's fun, but I'm kind of thrilled because the reality is none of us could ever design enough variety of interfaces to accommodate all of the different needs and preferences of every single human in the world. And so now that to me is just amazing because everybody gets exactly what they need, what they want. And we can all focus on how do we help them assimilate the information or um store or capture the information or whatever it is. And so to me that's exciting. And that's that's kind of what I see is that I think that's where it will end up.

John Nunemaker 04:22

Yeah, that's I mean that's what I see trending too. I just again like I noticed from my own usage, I stopped using interfaces when as soon as I get the MCP. Like I I haven't signed into Help Scout, Let's Jelly, you know, App Signal, Flare, Honey Badger, like any of those things since I started using MCPs for everything.

Garrett Dimon 04:42

I will say I think the key threshold is going to be read versus write. So like right now,

John Nunemaker 04:50

Yeah.

Garrett Dimon 04:51

I'm comfortable giving these things read access so they can assimilate and present information, but then they still have to leave it to me to actually perform the things. I think it's inevitable that it gets to the point where the AI is trying. I mean, plenty of people would probably argue it's already there, but

John Nunemaker 05:12

This guy. oh

Garrett Dimon 05:12

I don't trust it enough. So, like I got duet and I gave it read access to my fast mail. And I set

John Nunemaker 05:19

Okay.

Garrett Dimon 05:20

it up yesterday. And so this morning I woke up and um I had an alert in duet and I looked in there and it's like, um, tomorrow is my daughter's last volleyball game of the season. And so it's like, um, this mom is collecting money for the coaches at the game tomorrow. Da-da-da, don't forget. And it's like, my gosh. Like I would have totally and I coach, like I know that's what they do. The parents collect money at the last game. And then um, and it's like, wow. Like that would have like just normally, and Lauren's out of town. So normally Lauren would handle that. But and and

John Nunemaker 05:52

Yeah, yeah, yeah, yeah.

Garrett Dimon 05:53

so like when she's out of town, it's very hard for me to like slide into that role of information sponge and then like managing it all and Not not because it's difficult, but just because it's not natural because of my habits and daily routine and all that. It

John Nunemaker 06:10

Is Duet a separate app like?

Garrett Dimon 06:12

is, it's interesting, like, yeah, I mean, it's it's the standard. They're all creating a bunch of different apps and things that are based. Like, what's the difference between chat and cowork? Like, okay, there's files, but really is that a difference? Um, you know, or like what's the difference between, I mean, I guess it's codex versus chat GPT, is the code versus chat. with the Chat GPT app, but

John Nunemaker 06:35

Mm-hmm.

Garrett Dimon 06:35

then they merged codex into the Chat GPT app. And it's like, I think they're figuring out where all the lines and divisions and seams are and all this stuff. And it gives them a little more flexibility and playgrounds. Cause I guess if they were releasing just a single unified chat GPT, um that that would probably be confusing or a mess.

John Nunemaker 06:56

There's a lot more things to consider from an interface standpoint. If you the more things

Garrett Dimon 06:58

Well,

John Nunemaker 06:59

you have and want

Garrett Dimon 07:00

yeah. Well, it just the d I mean, I I can see it from a software delivery standpoint. It feels to me like it's all split up right now because they're all shipping so fast. And if you had one monolithic thing, there's less room to experiment in low risk ways. And so they're kind of just like going a whole bunch of different directions. And I mean that's what it feels like to me. But when you kind

John Nunemaker 07:22

Letting a thousand flowers bloom. Have you heard that?

Garrett Dimon 07:24

of, yeah.

John Nunemaker 07:25

Yeah. Yep.

Garrett Dimon 07:27

But yeah, but ultimately it it's almost this really raw, rough version of what I was saying where behind the scenes it's all the same thing. Right? Like it's all the agents and the MCPs and what have you. And it's just all these different interfaces to try out and figure out like what people actually like. And obviously the chat has just dominated so far. But to me, that feels like such a a default boring assumption. Like

Let's make everybody talk to these in chat. And it's like, okay,

John Nunemaker 07:58

Yeah.

Garrett Dimon 07:59

well, yeah, I mean, I guess, you know, that's not the place to innovate right now. But um that to me is is the most disappointing is that we're all just chatting with these things. Duet was interesting though, because it very proactively presented and there was something else that gave me um I don't remember what it was now. It's it's in there. I was like, Yeah, this all sounds good. Make me a list and let me know. And

John Nunemaker 08:21

Yeah.

Garrett Dimon 08:22

so like things and it's like just from reading my email. I'm like, my gosh, if I'd ever have to check my email again, that would be fantastic. Um, so we'll see.

John Nunemaker 08:30

I know it there's this thing in my stomach though where I'm like, if it has my email as my whole life. Like you know what I mean? It's not just that it has access to like read your email.

Garrett Dimon 08:38

It does. There's a lot of

John Nunemaker 08:39

It actually it can read all your recovery codes, your OTP codes,

Garrett Dimon 08:42

Mm-hmm.

John Nunemaker 08:43

like it like I so I gave my email access to to my dot because I was like, Okay, dots will be cool for codex. And then I was like, dots are stupid, I don't like them. And I was and so I

Garrett Dimon 08:54

I was underwhelmed.

John Nunemaker 08:55

and I had just like kind of ignored it. And then all of a sudden I was like, my gosh. OpenAI has all of my email. I guarantee they're not just pulling in the new ones and my inbox. They're pulling in all of my email historically to learn everything they can about me. And I was like, revoke, delete. I was like, no. I'm not maybe Claude, I would be okay with, maybe, even though they're probably evil as well. But I was like, wow, this is interesting. So I've also been using instinct. Have you used instinct at all?

Garrett Dimon 09:25

So I tried that, but it doesn't um I'm for whatever reason fully in bed with iCloud on calendars. And and so that kind of just

John Nunemaker 09:33

Yeah, yeah, that's the the killer for it. I use iCloud as well. Yeah.

Garrett Dimon 09:38

that put out the fire before it could get started. Um but so

John Nunemaker 09:41

Yeah, I've just been using it as like text v based like inference. So I can just be like, Hey, I need like this random thing for my truck. Like, can you give me a link to buy it somewhere? Preferably Amazon. But if not, that somewhere else is fine, you know? That's been great.

Garrett Dimon 09:55

So speaking of that, like obviously, you know, I told you in the past that Chat GPT helped me. So like we live so remote, finding appliance repair people and getting them

John Nunemaker 10:04

Yeah.

Garrett Dimon 10:04

to show up here. And it's just exhausting. Like it's such a and so finally I just you know, so I fixed our dryer with Chat GPT. It was like just the heating element need be replaced. So it was just undo a bunch of screws, buy a $30 heating element off Amazon, drop it in, dryer's

John Nunemaker 10:19

Nice. Yeah.

Garrett Dimon 10:21

fixed. Um, our freezer has been broken, but I'm like, that's much more complicated. I'm not gonna mess with the freezer. But I finally, because it's just been such a mess trying to get a appliance repair person to show up, and they're like diagnosing it over the phone, clearly in a way that's like, I don't want to drive all the way out there and deal with this. And so I'm just gonna

John Nunemaker 10:41

Yeah. Yeah.

Garrett Dimon 10:42

blow them off. And so finally I was like, All right, well, maybe, just maybe. And so I start troubleshooting the freezer with Claude. And um Yesterday I ordered like and it's this is one of those for whatever reason like ridiculously overpriced freezers like built in. And so that was the other part. It was like crap, like something like that. You can't just go buy a new one at Best Buy. Um, and so we kept putting it off, putting it off, thinking we really had to have a s like a specialist for this until I finally was just like, I'm done. And so yesterday I ordered $400 worth of parts and

John Nunemaker 11:19

Yeah.

Garrett Dimon 11:20

You know, they hopefully get here in a couple of days. I install and then the freezer that hasn't worked in a year or two is back and at it just from having enough and using cloud. And it's like, this is kind of absurd. Cause before, like, I don't know

John Nunemaker 11:32

It is, yeah.

Garrett Dimon 11:33

how many manuals you would have had to go through the Googling and like finding the the parts and the all of the stuff and then all these ad covered sites trying to, you know, catalog every repair manual in the world. And it's a horrible, unpleasant. you know, distracting experience. But with Claude, it's like, yeah, that's this element. Here's the part number. Here it is on Amazon. Here's a better price. And it's like, my gosh. Uh yeah.

John Nunemaker 11:57

Yeah, it's incredible.

Garrett Dimon 11:58

And it's kind of fun. Like as a soft as like an engineer, a software person, like it ironically, once you're not dealing with all the physical parts of it, it's fun.

John Nunemaker 12:08

Yeah. Well, the part that's painful and hard, which is like, you know, your your time to dopamine hit of like get getting the part and putting it in and fixing it is so much shorter that it's like

Garrett Dimon 12:14

Yes. That feedback loop.

John Nunemaker 12:17

I might as well do this. Yeah, the loop is there. So so

Garrett Dimon 12:20

Yeah.

John Nunemaker 12:20

speaking of loops, really, this is not related to loops, but uh

Garrett Dimon 12:23

It's a good sag.

John Nunemaker 12:25

I wrote I wrote my first skills today. And just I knew you yeah, I knew you would be proud of me. Cause um Garrett is like got 800 skills and they're all you know, they do everything for him. And he doesn't he doesn't type anything into AI. He yeah, he just

Garrett Dimon 12:39

Skill files. I don't actually have skills.

John Nunemaker 12:42

he just types skill files in all day long. That's what we always joke about. And I've always been very vanilla and not had any of those. And then um, you know, I we we talked a bunch last week, not on the podcast, but we'll talk about it now about like Nunabot and Workbench, which are our two tools that we use to like get stuff done. And so I I've been kind of iterating on Nunabot. And last night I I wrote up an article.

Um and you know, released it and I'll link in the show notes and stuff like that. But after I wrote it, I realized like I love the Nunabot PRs, like the format that they're in. And I also love um the the skill, which is like the way so since October, the way I've been basically making code is Opus five point five. What's that?

Garrett Dimon 13:29

It's still October. It's still October.

John Nunemaker 13:33

yeah, no, it is it is October. That's right, October 30th.

Garrett Dimon 13:34

Ha ha ha.

John Nunemaker 13:36

But like all of October, I've basically been Opus 5.5 for building, codex for review. It's been great. And so I'll do my my rip my build loop is basically Opus five five. This is what I want to build, build the thing, it builds it. Then I do simplify, which is basically like, you know, shave all the AI junk off of it, find performance issues,

Garrett Dimon 13:57

Mm-hmm.

John Nunemaker 13:57

security issues, all that kind of stuff. And then I do QA, which is a G stack skill, goes through and it says, okay, cool. Like, Click around and make sure everything actually works in a browser, you know? And that's been super helpful. And then on top of that, like if it's a CLI or an MCP, it will actually like install it, do all the things, which is great. And then the last round of stuff is like at this point, you've got working kind of shaved down software, but like let's look for holes and things we didn't think of when we built it. And that's where, you know, the codex review comes in. But the problem that I would I would always have is the codex review, I would do a fresh one over and over and over.

And I feel like that's what was causing a lot of my pain because then I would end up in these really big, huge pull requests that made, you know, my tum tum hurt. And I was like, This is yes, it will cause it'll just

Garrett Dimon 14:40

To fractal. It just the deeper you go, the more it expands.

John Nunemaker 14:45

Forever. And it just turns in this massive spider web and you feel caught in it. and a couple of times I actually like gave up on a PR and started over because I was like, I don't even know how to like sh you know, shave this back to something reasonable that does not terrify me to deploy. And so that's kind of been my process. So then what I what I started doing is basically I would do codex review, fix the findings, and then I would only from then on, I only review the fixes. I never review the entire thing again.

And that has been my trick to like not going down the complexity uh rabbit hole of like making a PR so big that I get uncomfortable with it. And once I figured that out in October, I was like, okay, now I have this way of making a small, concise PR with proof that it works and stuff like that. I mean, the next thing is just to like automatically do it, which is kind of where Nunabot came in. But then what I noticed is the Noonabot PRs, I love them and I can't wait to merge them, but all my manual PRs are like just clawed junk.

You know, they like Claude's saying things in the pull requests. And I'm like, this is this, I don't want to see this. And so this morning, finally I was like, Okay, I need to make a skill so that I can make a PR that is exactly like my Nunabot PRs when Nunabot's not involved. And so I was like, do this for me. And it did. I know you're supposed to write your own skills and like by hand and whatever, but I'm not I don't have time for that. So

Garrett Dimon 16:07

Oh no.

John Nunemaker 16:08

so I I kind of, you know, had it just kind of whip the skill together and then Tweaked a little bit. So now I have a Nunabot PR skill and a Nunabot review skill. And so the Nunabot review skill just does that simplify QA, codex, and then a loop up to three times of fixes and codex reviewing the fixes. And so you've got like a bounded way of, you know, kind of making a good PR or knowing that it's off the rails. And then, and so that's that part. And I can use that now anywhere. And then the other part is the the Noonabot PR, which is like

Give me a PR in the format that I absolutely love, which is basically why are we making this change? What changed? Give me proof that it works. Um,

Garrett Dimon 16:50

Mm-hmm.

John Nunemaker 16:51

you know, there's like what was what there's there's some other um I can't remember what the other things are, but like screenshots, rollback. Very, yes. It's very. Yep.

Garrett Dimon 16:56

I mean it's it's it's all pretty standard scientific method type of stuff. Right, like a hypothesis, you know, and the key, I think, is the depth, right? Because it is a fractal. It will go on and on and on. It's just like art. There is no technical finish point. You have to know when to stop. And to me, that was kind of the key realization. And it's funny because you and I took different routes to all of this. Whereas

John Nunemaker 17:28

Yes.

Garrett Dimon 17:28

I feel like you took a macro level um iteration based approach at the larger cycles. Me being more like UI centric and in a space where there's much more subjective detail went From a micro level, how do I make sure it doesn't do all the really dumb things? Like, you know, because as the simplest way that, you know, I've tried to explain it to non-technical people is, you know, any subject matter out there, um, or any content, any anything, half of it is below average. Right. And so maybe, maybe five percent of it is good.

And depending on my guess is that they just all slurped up everything. And so the vast majority, like they might have weighted it somehow, surely they had to, is it's trained on really bad ideas. Um and so I've spent a lot more time on the micro level, like make sure it's accessible, build it with progressive enhancement, don't just like slap together some JavaScript nonsense. when there's HTML elements that do it. And so like really giving it the process of like always build with HTML first. If you need it, add CSS. And then if all else fails, you can carefully discuss JavaScript with me. Um and so

John Nunemaker 18:55

Carefully discuss JavaScript.

Garrett Dimon 18:57

like I'm taking that micro level, but ultimately I kind of came to the same conclusion with my process. So I came up my favorite skill that I'm using now, and it's really just a combination of skills, but in a more like linear manner and it's my savage review. And so it runs um obviously all the automated tests and it runs reviewer. So reviewer handles is like, you know, it's my orchestrator for linting for like break man and bundle audit and you know any of those kind of deterministic tool command line tools. It runs through all of those and it

Reviewers designed so that it obviously is choosing only the files that are relevant ah so that it goes faster and isn't wasting time. But it gives a very, very set set of things that it's not gonna skip. Um and so it goes through all that, the automated tests, and then it does tidy. So I created tidy because codex doesn't have simplify, or at least it didn't at the

John Nunemaker 19:59

Mm-hmm.

Garrett Dimon 19:59

time. I don't know if it does now. And so tidy.

John Nunemaker 20:01

It doesn't. I just ported it. I I just said, you know, port a simplify. Yep.

Garrett Dimon 20:04

So that's kind of what tidy is, but I chose a different name so there wouldn't be any overlap. Um, and so that way I can switch between codex or chat GPT interchangeably, and whichever one I'm using, the Savage review will trigger an adversarial view from the other and from Copilot now. So, like if I'm using Claude, it will go through tidy, simplify, do all the you know more deterministic checks. And then it says, you know, in this case, if I've squad, so it sends off Copilot and um Codex. And then they do the reviews, and then it synthesizes everything. And then there's the two key elements here for my boundaries where I did the same thing as you, where it's thr limited to three cycles. But in addition to the three cycles in Workbench, I set up

A SQLite database. I originally tried with static files, but there were too it's too much contention and overriding. And the more you let the agents kind of spawn other agents, they would be competing for things. So I basically just came up with a very well reasonably simple like findings database storage method. And

John Nunemaker 21:17

Mm-hmm.

Garrett Dimon 21:18

so it stamps any any agent comes back with a finding. It stamps it with the branch, the commit, like the state the code, the whole entire code base was in at the moment that finding was found. And then it requires a disposition, which is it got addressed. well, actually, sorry, it prioritizes it, which is yesterday, now, next, someday. Um,

John Nunemaker 21:43

Okay.

Garrett Dimon 21:44

and so yesterday is okay, this is critical. Drop everything, create a new PR. This needs to go out

John Nunemaker 21:48

Yeah. Mm-hmm.

Garrett Dimon 21:50

by itself right now. Don't hold up. Um and it discusses all this stuff with me, but um and so anyways, it prioritizes them all. And then once they're prioritized, it's says, okay, you know, here's what we've got, and then it'll verify them and make sure they're real and not just some made up hypothetical

John Nunemaker 22:05

Yeah.

Garrett Dimon 22:06

concern. Cause obviously like the nice thing about

John Nunemaker 22:08

Happens, yeah.

Garrett Dimon 22:09

all of this is that at a certain level it's writing better code. Um, it's writing better code for us, but if you let it go too far, it will write scientifically perfect code. It's completely unnecessary. Um, and so the verification process forces after all three agents come back together with their three reviews. Um, it goes through every single one and it and it will also collate all the findings. So if there's duplicates or very similar things, and so it's categorizing all of these, and so they've all got a discrete entry.

And now they have to have a disposition or it won't move forward. And so that disposition can be this is nonsense, not important. Or if it's unsure, it'll it'll surface it and say, Hey, I'm not sure about this one. But then

John Nunemaker 23:00

Yeah. For me it goes into an inbox if it's not sure. So I have an inbox file. And so if it's not sure, 'cause I haven't switched to SQLite yet. And so it'll be like, I'm not sure.

Garrett Dimon 23:07

we I'm thinking about idiocracy now.

John Nunemaker 23:09

Yeah, it's like in I'm not sure. So I'm just gonna put it in the inbox and then I can just go through the inbox once in

Garrett Dimon 23:14

Have

John Nunemaker 23:14

a while and and answer.

Garrett Dimon 23:16

have you seen Idiocracy? And so not sure. His name? Oh anyways,

John Nunemaker 23:22

Forgot about that, yeah.

Garrett Dimon 23:24

Yeah, so then all of the findings and it'll it'll do that and it has to have a disposition for the findings. But then the problem would be once it starts fixing some of the findings, then the next round, his like previously before I like dove into how to design this to make it work, it would get confused about which findings were still relevant, which findings, and so that's where having each of the findings stamped with the code state and that would

John Nunemaker 23:47

Yeah.

Garrett Dimon 23:48

help. And so then the next pass would go through. And so by the end of

John Nunemaker 23:51

Mm-hmm.

Garrett Dimon 23:52

it, you know, there might be 80 findings. And probably at least 40 of them are just outright dismissed because they're overly pedantic things that are just not realistic. Um, and then, you know, most of them will just be normal, basic, minor edge case bug fixes that make sense. And then there might be one or two that are significant enough to discuss. But then that way I can build and then and I can run the savage review on plans too, if it's something I think is gonna get hairy.

Then it's like, let's see out what can we do to, you know, or like what what your question, um, what would have to be true for this to be half as big, you know, or half as complex? And

John Nunemaker 24:32

Yeah. Mm-hmm.

Garrett Dimon 24:34

um, but same thing. So that review then does everything, but then it's the PRs where things get great. And now my PRs, and again, so much UI-based stuff. Um, and so so much of what I've been doing is how do I verify UI when UI can have 20 different states?

John Nunemaker 24:54

Yeah, that's where it's great,

Garrett Dimon 24:54

Right. And so

John Nunemaker 24:56

right?

Garrett Dimon 24:56

and so I'm like iterate through and include a screenshot of every single state and the copy on every single page in every single state under every condition. And now I'm not reviewing the code of the PR, I'm reviewing the screenshots and reading the content that the people are gonna see. And then the really cool thing that I've been super excited about is historically, like if you're designing UI, there's infinite edge cases.

Right. So like I'm doing this, I've been doing this with fireside with the custom domain configuration. There's a million different ways somebody can misconfigure DNS for a custom

John Nunemaker 25:31

Mm-hmm.

Garrett Dimon 25:31

domain. And The return is not worth identifying the 20 different types of edge cases and manually creating a explanation interface that accommodates every single edge case where one of those might happen once a year. But with AI, it's like, all right, go through all of these edge cases and it's

John Nunemaker 25:55

Do it, yeah.

Garrett Dimon 25:56

like find the commonality. And so now we can handle 20 edge cases and give people a very precise explanation of what's going on with their domain and how to fix it. And in most cases, so just from the product experience standpoint, that's great. Like it's more pleasant for people to use. But then the bonus of all of that is support has gone down a ton because now the interfaces

John Nunemaker 26:20

Yeah.

Garrett Dimon 26:20

proactively provide the answers for these edge cases that before it's like, yeah, that's you know, somebody'd have to email in and ask. And so that's inconvenient for them to email us. But then obviously like answering support emails is less time. Building the actual product. Um, and so that's been great in that it's been able to help customers. It's made support easier on our end, which frees up our time to further improve the product. And so now it's all of a sudden, for the first time, been pragmatic and reasonable to create a user interface that answers every single question for customers. And

John Nunemaker 26:56

Yeah.

Garrett Dimon 26:57

like that was never possible before. Or I mean it was possible, but there's no way you would do it. It just wasn't just it. wasn't scalable. Now it's totally scalable. And so that to me is one of the coolest things that once you've got a structure, you're can say, okay, let's find every edge case and let's prepare a message or instructions or guidance or whatever for every single one of these. And that's been awesome.

John Nunemaker 27:25

Like it's the same thing for me on the back end side because like you're doing that for the front end. So you're like explain everything to the user. And like on the back end side, that's kind of what Nunabot does for me. Is like I'm like, look, it's not gonna solve

Garrett Dimon 27:35

Like failure modes.

John Nunemaker 27:37

its what.

Garrett Dimon 27:38

Like failure modes. All it finds and knows all the failure possibilities.

John Nunemaker 27:40

Yeah. Well, yes, but even more so, like I feel like one of the like what I would say even better is like automatic error fixing. You know, like every time a user does something that's

Garrett Dimon 27:50

Mm. Mm. Mm-hmm.

John Nunemaker 27:52

that throws an error, it pops, it goes in honey badger. Now I can be like, pull that in, see if it's legit. Do we need to fix it? How many people hit it? Like, and I can do all those kinds of things so that like they never hit that error. So then They don't actually ever no the next person never ends up in the same state where they have to contact support. So it's e it's the same thing. And you can even see like, okay, four people hit this, but only one of them reached out. You know what I mean? If you have your support in the context. Yeah. Yeah.

Garrett Dimon 28:17

It it's almost nerve-wracking how quiet support has gotten. Right? But in hindsight, almost all of our support requests were just those one off-edge cases. And so now it's like, is something wrong? That like it's

John Nunemaker 28:29

Yeah. They were they were usually like people don't understand DNS. Yeah. Or things like that. Yeah.

Garrett Dimon 28:35

it's deafening silence, I guess, or whatever. Um, but yeah, it's so

John Nunemaker 28:38

Well, that brings up another thing too that I've been thinking is like now you you can't just have the signals of like here's the bad flares. You know what I mean? You have to have the signals that are the good flares also. So like I've because I I mean we have entirely again, no support requests, but we've entirely swapped out our back end, you know, object storage for fireside from B2 to R2. And so like

that has

Garrett Dimon 29:02

Well not just B two, it was S3 and B two, and now it's all R2. Well

John Nunemaker 29:05

Yes. Now I guess it it's still technically a little bit of S3 and R2 right now, currently still. But soon it will be just R2. It's just in in phases. But like it's all going to R2, reads and writes and all that stuff. And like no one has said anything. And no one's and I like so what what I had to do when I was doing it is I was like, okay, well, support is very quiet. I don't want to wait until something's wrong. So I basically just set up claw monitors to say like look every

You know, whatever interval you think is is critical, feeling free to scale it up faster or slow it down whenever you feel comfortable. I want you to look at the number of uploads we've had from MP3s, general media images, transcripts, stuff like that. Look at the errors through Cloudflare, look at the errors in our app at Honey Badger, look at support requests. And I want you to say, like, this is going well every on some interval.

Garrett Dimon 30:03

Use all the signals that might hint that

John Nunemaker 30:04

Or or something's wrong. Yes. Use

Garrett Dimon 30:06

something's off.

John Nunemaker 30:07

Yep. And so it did that yesterday for like every it was like, I think 10 minutes should be good to start. And if we gain confidence, we'll go up to twenty five. And so like it was like ten minutes looking good. Ten minutes look we've had X uploads. You know, twenty minutes we've had X uploads. And I was like, I'm gonna bump it up to a half an hour now. And then it was like went did that for a little while. Yeah. Yeah. Yep. So I can roll it back if I need to.

Garrett Dimon 30:25

But all of that is a feature flag thing. And so like that's feature flags have been The thing, like I was always excited about the idea from, you know, you hear about it like, Amazon does this, Google does it. Like, well, of course they do. They're ginormous. And so once and I still am like you're still like indoctrinating me little by little, but that has been one of those things that realizing it's not as hard as I've always imagined it. And

John Nunemaker 30:55

Yeah. Mm-hmm.

Garrett Dimon 30:56

as long as you plan for it in the beginning, which now is part of All of my workbench. It's like if this

John Nunemaker 31:03

Yeah, yep.

Garrett Dimon 31:03

is something that makes sense for a feature flag, plan for that. And make sure that like we're not sprinkling the feature flag 30 places all over the code base when we don't need to. Let's design this around. And so it helps design better too, because then

John Nunemaker 31:14

Yeah. Yeah. Try to mm hmm. Yeah.

Garrett Dimon 31:18

it modularizes things in a way where you can have fewer of those splits. Um, and that's been huge. And then to be able to just tell the agent, plan this way. makes it so much easier. I don't ever forget about it or like realize it after the fact. I know ahead of time like, this is a perfect feature flag thing. And, you know, until I'm completely brainwashed, I need that like extra nudge to remind me like, hey, this can be a lot easier than you're making it. And that's been huge. Like we've got a couple of like, I don't know, three or four new features in Fireside right now that are flagged and I've been rolling out gradually. And they're just minor UI things, but

I don't have to worry about them blowing up in the wrong way. and then like all of this file storage stuff. When we updated the studio admin, throughout all of that, I was also kind of cleaning up a lot of the file storage stuff as we went so that it we didn't have it as sprinkled and fragmented. And so like images, mp3s, um, avatars, like we've got many different types of uploads, but then I they're at least a little more standardized across the board.

John Nunemaker 32:27

Mm-hmm.

Garrett Dimon 32:28

And then that kind of helped. I don't know, I guess I I didn't write the R2 code, so um, but

John Nunemaker 32:33

It helped. No, because there was several spots where like it would do like I I I I will admit I looked at some of the code. Um there was some spots where it was like studio media something in JavaScript dot like upload. It was like there was some obvious like yeah. Yeah, there I

Garrett Dimon 32:46

the JavaScript I definitely cleaned up. Yeah.

John Nunemaker 32:50

saw some of that. And I mean, you know, we and we I did both of it. I did with flags. So I have a R2 reads, which is what we started with because we were replicating

Garrett Dimon 32:57

Mm-hmm.

John Nunemaker 32:58

from B two to R two. I was like, the worst case scenario is like a wee bit, a smidge of replication lag, which I was like to move fast on this, I'm okay with that. Because by the time it's

Garrett Dimon 33:08

I mean tens of milliseconds, right?

John Nunemaker 33:10

yeah, I mean, you're you're talking under subsecond. So, like, in in it, you know, if you hit save, sometimes you would maybe get a 404, like in the preview for the transcript, but like nobody complained, and it was only for like a day until I turned on the rights, and so it wasn't I honestly, it wasn't a big deal. And then you the R2 rights are a flag too. And I remember I was working on it in Codex, and Codex was just like maybe it got it to a point where my my tummy was hurting. I was like, I'm I don't think I can merge this. I was like, I I kept I catch myself

Garrett Dimon 33:40

Yeah.

John Nunemaker 33:42

when I'm just asking questions from every single angle, like all of a sudden I I kind of recognized, like, I'm afraid that's what is happening here. So I was like, okay, I need to go back and I need to start over. So I went, I literally stopped in Codex and I went over to Claude and I was like, okay, we're gonna and it I it might have been that I went over to Claude and Claude solved it better. It might be that like Codex would have done it again in a new thread fine too. And I just took it to another place on that one. Um, but and I was just like, I want this to be as simple as possible. Here's what I'm thinking, here's this, and started over. And I've done this like I would say three or four times in the last month where I something got, you know, to three thousand lines and I was like, nope.

Garrett Dimon 34:21

Yep. uh.

John Nunemaker 34:23

And I was like, we're gonna shave this down. It's gotta be under 1k or under 500, you know. And Nunabot is like I it even has an explicit goal in it that is like 50 lines or fewer. Like it's like it's gotta be real and it will do a couple hundred, and that's okay. But I think if you tell it 50, then it breaks it, you know, by four X, no big deal. If you tell it 500 and it breaks by four X, you have a big deal.

Garrett Dimon 34:44

Yeah.

John Nunemaker 34:45

So, but yeah, it and so like I have the R2 reads, the R2 rights. So I switched reads over, I got confident there. I switched writes over. Um, yesterday. And and like these are things that have been worrying me for like a while. And then I turned them on and it's just like nothing. I mean, it's just radio silence. And even with um with codex, they had added like a column to the database table to say, like, okay, well, we need to know when rights switched from B2 to R2 for each podcast. And I was like, Do we really like we have that in the audit log for Flipper? Like, I don't think we really we can get it from there if we really need to. There's an

Garrett Dimon 35:16

Mm-hmm.

John Nunemaker 35:19

API for that, like. And it was like, well, no, we really need it for this reason. So then I went to Claude and I was like, you know, how simple can this be? Can we avoid, you know, whatever? And it and I was like, do not look at any of my prior art. I was like, don't look in at the docs.

Garrett Dimon 35:33

Right.

John Nunemaker 35:34

Don't look at anything that I have said before. Just give me the most simple thing that you think is right in the current state of the repo, you know? And

Garrett Dimon 35:41

Mm-hmm.

John Nunemaker 35:41

so I came up with this and it was like, I was and it was just one flag, our two rights. And I was like, cool, okay. So and I was like, well, if I enable one and I need to roll back. Is that a problem? Because now, you know, instead of uploading to B2, you've started uploading to R2. And they were like, No, as long as you leave R2 reads on, you're fine. And R2 reads have been on for like a week now. So, you know, because you you were writing to B2 and replicating to R2, now you're just writing to R2. So the only way you need to change anything in a rollback is if you roll the reads back, you need to replicate the the diff.

you know, over to B2. So that the that, you know, a read on B2

Garrett Dimon 36:24

Right.

John Nunemaker 36:24

is consistent with a read on R2. And I was like, oh, yeah, totally. I don't need a database column. I knew it. Like I I was positive that I didn't. But it was interesting how, like, you know, you get this big anytime I get this big thing, I almost look at it like I used to look at it as like, okay, man, I really got to be thorough with this. I got to figure out how to get this thing out. And I would say the last week, the the best thing that I have done is I've said, no, uh if it's

If it's big, I don't I don't say how can I get this out? I start a new thread, I throw away the old and I say, okay, this is what I want to do, but I want to do it in a more simple way, like this, you know, or give me options for more simple ways.

Garrett Dimon 37:03

The case where I've done that the most has been a lot of this box out stuff and billing. And I think I've thrown away seven PRs. And each time I bef right before I throw it away, I have the I'm like, go through, go back through this, look at what we've built and catalog all of the ways you've screwed up. And then it carries that forward. And then it's like, okay, this one didn't work either. Like you clearly, and so like a lot of this too was the whole learning how to use AI process. Um, because I was trying to design something very big at a high level and then break it down into very small parts, but then you've got dependencies and you know, this needs to happen before this and this, and um, and it would become a tangled web.

And so it was very difficult to do that. And like on a different thing where it's a very small focus change, then it's easier to do. But then, like for a big thing, it's like, well, let's use the most powerful one with a million token context. But the more I use the million token contexts, the more it felt like those would drift super fast because they were holding so much. And they'd get confused and misprioritize. And so by forcing the smaller context, not forcing, but like just by sticking with smaller contexts, it forced more modular compartmentalized thinking. And then it just left the larger holistic thinking to me. So instead of like trying to trust the

John Nunemaker 38:41

Yeah, exactly.

Garrett Dimon 38:42

agent to do larger holistic thinking. I gave up on that and said, okay, I will do that part. I the agents are so clearly and a lot of that was just experimentation. Like, what are they capable of? Um, because in some cases, I underestimate, I'm like, there's no way the agent's gonna do a good job on this, I'll just do it. Um and so a lot of it's been that figuring out the right size chunks um and how to connect the dots when the agent is not capable of seeing the bigger picture.

John Nunemaker 39:11

That's entirely that's that's the title of my post. Like my post was like, Don't make a human do what AI is amazing at. And I feel like

Garrett Dimon 39:18

Yeah.

John Nunemaker 39:19

it does take a lot of experimentation to figure out what it's amazing at and what it's not.

Garrett Dimon 39:22

Yes.

John Nunemaker 39:23

But I'm like, once you have it, where you have like workbench or you have, you know, Nunabot review or you have things like that where you can run run and make sure that the code gets that gets created initially gets shaved down into like something small, concise that you cannot, you know, say no to. Like it's so good

Garrett Dimon 39:40

Mm-hmm.

John Nunemaker 39:41

that you're like, Yes, this is a no-brainer. Why would I not do this? Then it's like, okay, next layer is like now do this for everything. Do this for every

Garrett Dimon 39:49

Mm-hmm.

John Nunemaker 39:50

security finding. Do the you know, you got to collect all that stuff somewhere. But like every support request, every security finding, every error that comes in, like all those things need to be

Garrett Dimon 40:01

Mm-hmm.

John Nunemaker 40:02

in some manner fed into some system, you know, analyzed to make sure it's safe, you know, first. Like, don't just prompt injection,

Garrett Dimon 40:07

Mm-hmm. yeah.

John Nunemaker 40:09

stuff like that's a real thing. But like Bring all those things in and then have this layer that just takes away an entire bit of your job. And so then, like you said, I think so well that I I hadn't really even thought of it that way. But like now your job is just the higher level like things that AI is not great at right now. And so at some point it will be better at that stuff too. And so we keep adding more layers on top of sediment or whatever you want to say. My in my post, what I the example I gave was like, you know, treat each repo like a painting you come back to, make one deliberate.

Improvement per pass and leave it better than you found it. So it's kind of like the Boy Scout model or whatever. Like you want each time you you touch that repo in some way prioritize, you know, does a customer support request go first? Does a security finding go first? You know, does a performance regression go first? Like all those kinds of things, or do one of each every run, whatever it is, it

Garrett Dimon 41:04

Mm-hmm.

John Nunemaker 41:04

doesn't really matter. And then you will kind of end up with this automatic system. And I think the only thing What I was thinking about yesterday or the day before, the only thing that is a little bit that that I think the next step of mine is like right now, it's all one big run. But I think what I want is I want schedules for uh finding findings. So I want

Garrett Dimon 41:28

Mm-hmm.

John Nunemaker 41:28

I I want pi a pipeline of like a few different things. So I want one set of workers that all they do is they just try to find things to fix in the repos. From all the things we just talked about. And then I want a second stage that's like, it just all it does is go ahead. Yes. Yeah. Objectively, these things have to happen. Yep. They're obvious. Like they

Garrett Dimon 41:43

But quantifiable deterministic things, right? Like obvious security N plus one performance issues, like yeah.

John Nunemaker 41:52

have to happen. There's a security issue. There's a nil, no method error. There's, you know, a support request. There's like, there's verifiable, here's why we got to do it. So that that will just happen all the time and it'll just be storing those somewhere. The second thing is there'll be something that's constantly reprioritizing them. So based

Garrett Dimon 42:08

Mm-hmm.

John Nunemaker 42:08

like if a new P1 security comes in, bam, it goes

Garrett Dimon 42:10

Big picture.

John Nunemaker 42:11

up to the top. Yeah. So it's like a higher level, it's saying what is the most important out of these. And it's not just from a technical basis. I think it also needs to be like all of my Nunabot stuff factors in the revenue of each of the apps. And whether I'm the primary

Garrett Dimon 42:23

Right, right. The significance, the risk.

John Nunemaker 42:25

exactly, or you're the primary or things like that. And then once it has that categorization done, I think it the third stage is like me saying, like, Yes, yes, yes, yes, yes, yes, yes, or no. Like, like it's not actually me

Garrett Dimon 42:39

Mm-hmm.

John Nunemaker 42:40

looking at the the PRs to merge. Right now, that's what it is. But I think if you have that step of me being like looking at the the work that it's gonna do and why it's gonna do it and saying, yes, I want you to do this, honestly, most of the time I don't need to see the PR at this point. It Yep. It's a hundred percent. It

Garrett Dimon 42:54

Well, especially with that fifty line limit, it's it's not gonna lose the thread and go off and get crazy.

John Nunemaker 43:02

it basically cranks out a PR. And I caught myself, I was. I was merging PRs, you know, and I actually caught myself about the I was probably like 20 or 30 PRs in that I was merging. And I had been looking at every the code and the PR body on every single one. Every single one of them was was fantastic. And all of a sudden, 20, 30 PRs in, I was just reading the PR body of like, you know, why we did it, what we did, the proof that it's working.

Garrett Dimon 43:30

Proof.

John Nunemaker 43:31

And, you know. What is the consequence if we have to roll this back or if something goes bad, you know? All that,

Garrett Dimon 43:36

Right. That's a that's always a

John Nunemaker 43:38

yeah, that's the last step on every PR now. And so, like, I was I was just reading that and I was like, Yeah, I mean, the worst case scenario is like somebody hits an error and the next run fixes it. Uh, so I was like, merge. And I just started merging and I was four or five merges in without looking at the code. And I was like, you know, I had to stop and be like, wait a second. Uh I'm not looking at the code anymore.

And I'm also not worried and no new errors have come in. Like this is I'm realizing that it actually is better than me at writing this stuff. You know, it's more

Garrett Dimon 44:10

Well, and

John Nunemaker 44:11

thorough.

Garrett Dimon 44:12

the same thing with uh the MCPs. So like now my whole workflow and workbench basically after I merge it um wakes up the thread in T3 and it's doing smoke tests and monitoring against what was deployed, right? So like that's part of the

John Nunemaker 44:34

Yep. Same.

Garrett Dimon 44:35

PR description. It's post deploy, these are the things to check. And so

John Nunemaker 44:38

Do this. Yep.

Garrett Dimon 44:40

Like

John Nunemaker 44:40

Great point.

Garrett Dimon 44:41

as a human, when I deploy, be like, I'm hungry. And I walk away and everything starts blowing up and my

John Nunemaker 44:46

Yeah, and you forget.

Garrett Dimon 44:47

phone and computer at the desk. And so then I come back and it's like, crap. And so now

John Nunemaker 44:49

Yes. Yeah.

Garrett Dimon 44:52

I deploy and it's check it's watching Honey Badger and it's looking for things specifically related to the code I just deployed. So like even if a bug does slip through, it is monitoring it on a level and at a speed, there's no way I would do. And it's doing it consistently every time without being asked, so that if the slightest chance something, you know, and as long as you're making sure the blast radius of like each one is small, then it can monitor and be like, hey, look. And so, you know, obviously we haven't done it yet, but there's gonna be a point where it's like scientist, the gym, just needs to be integrated into all of this automatically. Or, you know, or flipper starts having scientist-like functionality.

And it's all just talking to each other. And then as soon as Flipper notices something, it's capable of saying, like, you know, hey, go check this. Are you seeing it? And it's like, here in Flare, I found this. And then in Honey Badger, here's some exceptions. And

John Nunemaker 45:47

Yep. Yep.

Garrett Dimon 45:49

here's the intersection of all of this. And then it creates a PR and dings us and says, Hey, this should go out soon. Take a look. And then we just, yeah. And so this self-improvement cycle is awesome. And like that. I think the biggest simple thing, all of the tedious archaeology of writing software and fixing software is gone. All of the tedious data gathering. Like when I think of a feature,

John Nunemaker 46:18

Yeah.

Garrett Dimon 46:19

I'm like, all right, I think I want to go implement this. Can you go pull all the relevant data on a is this a problem? How many customers use this feature? How many customers don't use this feature? you know, blah, blah, blah, blah, blah. Like, should I build this feature or not? And it's like, here's the details. It'll help this situation, this situation. It's probably not going to help, you know. And I now have objective data before I start building anything. And then it can be like, okay, now let's understand the real scope of this feature. What am I overlooking? And

John Nunemaker 46:49

Yeah.

Garrett Dimon 46:50

what aspects and ripple effects are there gonna be that I could forget that are indirectly related features? And then it comes back and it's like,

John Nunemaker 46:56

Yeah.

Garrett Dimon 46:57

okay, wow, this scope's bigger than I thought it was gonna be. Capture all of this for the future. We're not gonna do it right now because It's too big of an effort to be worth it. And but ultimately,

John Nunemaker 47:06

Yeah, it's gonna be thousands of lines.

Garrett Dimon 47:08

what I'm figuring out is Well, on one hand, I'm more mentally exhausted because I'm just going 100 miles an hour and having fun. But I've never spent such a high percentage of my day doing things that I truly deeply enjoy. And on top of that, the manifestation of those ideas and that enjoyment is ending up at a higher quality. Because it can go a little deeper and have more knowledge and be like, here's an edge case you probably didn't think of. Or, you know, and even after 30 years, I've run into a lot of edge cases, but I surely haven't run into all of them. And so it helps then ship better code faster. And I'm having more fun because I'm able to do the things that I like and I'm good at. And the agent likely will not be super great at anytime soon. And that joy then compounds. Um, and so like um

This is gonna be off on a whole tangent, but obviously I, you know, I sent you some of the screenshots of the iOS app I started making. And I've never done iOS in my life. I've always been curious to, but I was like, I can't

John Nunemaker 48:16

Same. Same.

Garrett Dimon 48:18

go, I don't have to I can't go learn Swift and then all of this stuff. And like I know web stuff, I just gotta keep doing web stuff. And so I, you know, had an idea and started working on it. And I absolutely and I mean, and obviously there's a much lower blast radius to like. a a little iOS app for a specific thing. And so

John Nunemaker 48:38

Yeah, yeah.

Garrett Dimon 48:39

I'm not as worried, but I haven't looked at any of the code. But I mean, I am regularly like, that's too slow. Surely there has to be a way to optimize that safely.

John Nunemaker 48:47

Yeah. Yep.

Garrett Dimon 48:49

And it's like, yeah, there is. And like, I'll go do that. And so, you know, it fixes that. And that app, um, and like it's just been kind of more or less churning in the background. Like I give it the days like Go do all of this and then I'll go to the gym and do some test runs and record them. Um and I guess it might help for context. So the context for everybody else is this is an app to put on your phone and film you while you shoot free throws and then translate your body mechanics to the specific most efficient ways to shoot free throws and make a high percentage. Um base yeah. Um

John Nunemaker 49:30

Gary and I are both basketball lovers. So yeah, that's that's where the background.

Garrett Dimon 49:35

And so and I coach. And so I'm constantly working with, you know, the high school boys and girls and trying to, you know, help them refine their shots. And, you know, there's only one of me. And all of this was like, this app would be great. Cause then they could get the same knowledge and understanding and visually see the angles of their wrists and their elbows. And um, so I built this app thinking like maybe iOS can do this. And

It's been, I basically go to the gym and shoot and get some recordings. And like now it's good enough. I've started getting some recordings of other people so it's not just measured against me. And then I just feed it all in in the morning and have it work on it all day. And like I'm not too worried about if it does something weird or crazy because it's not going to take down a web app. Um, and

John Nunemaker 50:29

Yeah. Yeah.

Garrett Dimon 50:30

so like it's kind of been nice because I don't have to worry about. the blast radius. And it's just been pure like I'm still constantly like I s use the same workbench reviewer, all the tests, automated tests, all of that stuff. Um, but it's not stressful and it's been like just pure joy. And it's just this fun iterate in the background. Like there's no way there would be time for me to build this anywhere like what I've built. And it's almost like a

It feels in hindsight like this natural, inevitable outcome of building tools to automate software processes and it's like still ensure high levels of quality. But like all the plumbing, it's like building a house, and you don't have to worry a damn bit about electrical wiring or plumbing or anything in the walls. You just design the house you want to live in and and it works. And With software, it's so easy to be like, hey, I'm willing to bet you made this mistake and put this pipe in the wrong place. Go make sure you didn't do that. You know, and a lot of that's just intuition over the years, but that's been the only thing. And I still have yet to look at a line of this code. And it's and it's

John Nunemaker 51:45

Yeah, I know. Yeah.

Garrett Dimon 51:46

exhilarating. And like this is the it it just feels like, and like without being too corny, like the app I was born to build. It's like this intersection of the things that. are fun to me and like information design and basketball and math and angles and and it's been just fun and it would have never been possible.

John Nunemaker 52:08

Yeah, I completely agree. I just feel like and actually to the point that I've told a few people, I think I should stop writing Rails. Cause like I look at the code because it's Rails. I know. Like I I

Garrett Dimon 52:17

Okay, DHH. uh

John Nunemaker 52:20

started thinking that 'cause I'm like, well, if I if it's Rails and Ruby, I will look at it. If it's anything else, I'll be like, yuck. I don't want to look at this. But

Garrett Dimon 52:28

Yeah, but I don't know. So I mean, after that keynote, I was like, all right, is there any validity in that? And I mean, I think there can be. Um, but then at the same time, there's something to me about I think Ruby and Rails and the conventions. A, the conventions help reduce the ways that agents can go nuts.

John Nunemaker 52:57

Yeah.

Garrett Dimon 52:57

Um

John Nunemaker 52:58

Agreed.

Garrett Dimon 52:59

But also I feel so two things. One, I feel like Ruby has shaped my thinking a little bit in terms of like I want the things to read. well and be logical. And I feel like it gets me thinking in a way that's logical and designing and then communicating. And this is almost implicit and subconscious at this point, but I'm communicating this design, vague design that's in my head to the robots. And I couldn't do that in another language. So even though the exact execution might not be the same, the gap between what's in my head and what the agent creates

Is going to be smaller and more accurate.

John Nunemaker 53:47

Yeah.

Garrett Dimon 53:49

And then the other thing that I realized about Ruby is in theory, so like Ruby doesn't believe in one single way, right? There's so many different ways to do everything. And yet, we as a community have kind of just converged on in this context, you do it this way because this is the most efficient way. And in this, so even though there's all this fragmentation, the agents still kind of apply the right solution at the right time. And like occasionally I'll be like, all right, why did you use a struct there instead of data? Or, you know, um, but for the most part, even though that's all this fragmentation, it's there's still a level of convergence and conventions. And I and and this may be kind of just a goofy little quirk, but like that gives me a little bit of peace of mind. And

I like that. Or if I do need to look at some code, I can. I don't know.

John Nunemaker 54:50

Yeah, same. I'm not getting rid of Ruby or Rails, but it has made me think that because well, the times when I don't look at it, like I will say, just a fun fact. Um, we we gotta head out for this podcast, but um one thing we might want to talk about next time is I am potentially not potentially, I have already rewritten a big chunk of Flare in so it can run on Cloudflare entirely. And it's super fascinating. And it's basically the same.

Garrett Dimon 55:15

Mm mm. I love how you drop these things on me, like yeah.

John Nunemaker 55:19

Right at the end. And then you just like boom, mind explosion. And I'm like recording off. uh So that's we're doing that again. So ingest is completely rewritten. And it's almost like I literally just have to like sit down and hit the switch. And it will all go through like basically like through uh node and cloudflare and all that kind of stuff using their queues their durable objects, and there's like this really cool design to be insanely cheap. Like it's even cheaper than the one that I did in Ruby. and the one I did in Ruby's.

I mean, it's really cheap. So and I am like, then it's just Infiniscale because as long as Cloudflare's up, we're up. I don't have to manage servers on it. Um it's it's pretty sick. So so that's maybe next time we talk through like that. And and I mean, have I looked at the code? No, I have not. but I have ran verification of all the error states and everything and compared through

Garrett Dimon 56:12

Right.

John Nunemaker 56:13

A, through B, and they're identical. So

Garrett Dimon 56:16

Right. And I mean, like it's like unit tests, integration tests, system tests, like there's all these levels. But you know, we don't write Ruby and then write unit tests and then be like, all right, now I need to go check the machine code. What did it translate this to?

John Nunemaker 56:30

We don't.

Garrett Dimon 56:30

And so as long as you know to and are actively writing the things that verify the results in deterministic ways, then Does the specific algorithm choice of the LLM matter? Well, yes, if there's a performance issue, but then you can say, like, hey, whatever you did is too slow. It needs to be faster. But the test are still there. So it's not just going to

John Nunemaker 56:57

Yeah. Yep.

Garrett Dimon 56:59

make it faster and wrong. It's going to be faster and still accurate because it still has to, you know, so those kind of things are like parity when you're migrating a big thing and you can easily whip together a test that's like make sure the before and after still has the same output. Um And then you can delete the old stuff easy. And you know, and so as long as you know to do that and treat it all like a scientific experiment, it feels almost irrelevant what the underlying language is.

John Nunemaker 57:26

It does. Yeah. I'm I'm still going to use Ruby and Rails, just like you were talking earlier, but

Garrett Dimon 57:29

Yeah.

John Nunemaker 57:30

it does feel a lot less relevant. And I think it's more relevant to go hard on Cloudflare and just the amazing primitives that they have been shipping. That it's on real

Garrett Dimon 57:39

Yeah.

John Nunemaker 57:39

versus like, you know, I know I can redo it in Ruby or whatever if I have to. but I love the idea of like, especially with security and things like that going the way they are right now. I mean, that's a whole nother topic, but yeah.

Garrett Dimon 57:51

Yeah.

John Nunemaker 57:52

So awesome. Well, Speaking of basketball, I gotta head to Noon Hoops and, you know, go slay some old middle aged dudes.

Garrett Dimon 58:03

Right, right.

John Nunemaker 58:04

I think that's all for today. That was a good talk though. All right. Later.

Garrett Dimon 58:06

All right. Yeah, I'm stoked. All right, Man. See ya.