Протиставний розробник: Як ваш кодовий агент стає ефективнішим через аргументацію

The biggest problem in AI right now can be described with a single word, sick fancy. You've probably heard of this before, but it means that AI models tend to agree with us and reinforce our opinions and their own way too much. They have too much bias. And the problem here is that fancy, it actually seems to be getting worse as large language models are getting more powerful, not better. Now, when people talk about psychopancy and AI, they're usually referring to the more broader sense of just any conversation with a large language model. But honestly, this hits the most with AI coding because the worst thing you can do is have a coding agent evaluate its own work. You know, the conversation where you just injected a ton of your opinions and there's a lot of bias built up as the agent wrote the code. It's going to give itself a gold star every time because it's like a student grading their own homework. They might point out a couple of flaws to make the review seem legit, but in the end, the bigger problems are stuffed under the rug and they're going to cause big problems when you're using your codebase for real. Okay. Well, that kind of sucks. So, cool. What's the solution here? Well, if you haven't tried giving your coding agent a sparring partner before, you're definitely going to want to after watching this video. I'm talking a totally separate coding agent with its own contact session where its sole job is to rip apart the main agents implementation. I call it adversarial dev because you're creating the second agent to be nitpicky to play devil's advocate and question even the deeper decisions made in the codebase. And yes, going from a single agent to a multi-agent setup has some overhead and we'll talk about that as well. But man, the results that we get from this speak for themselves. So, if you are wondering, yes, I'm going to get very practical for you in this video, showing you exactly how to create this adversarial dev architecture. We're going to take a lot of inspiration from Enthropic, and so I'll cover this new blog post from them. One unfortunate thing about this though is they share a lot of good stuff here, but they haven't actually opened anything like they have in previous blog articles like this one. And so, I've taken it upon myself to create everything from the ground up. I've created a Claude and Codeex version of these harnesses. And you could even mix these to have Claude be the implementator and Codeex be the evaluator. And so I'll show you how I built this. And then of course I've got a diagram here to explain everything. So I'm going to break it down nice and simple. And really this is the future of AI coding, having separate coding agents work together to accomplish larger tasks more reliably. So let's get into this. So, if you've been following Anthropics research, you know that they've been working pretty hard on making Claude better at building frontends and improving their longunning coding agent harness, answering the question, how far can we push the limits for coding agents? And so, they've been building a lot on top of the harness that they covered in this blog post. I actually covered this in a video that I'll link to right here, but with their evolution now, they've been taking a lot of inspiration from generative adversarial networks. And this is where it gets really interesting. If you're not familiar with GANs, it's a machine learning approach where we have one model generate images and then another model is the discriminator. The discriminator's sole job is to figure out what images are real from some kind of real data set we feed in versus which ones were generated by the other model. And so the sole job of the generator is to trick the discriminator. And in getting better and better at tricking the discriminator, it actually makes more and more realistic images over time. So it's a very interesting approach to train it to make accurate images. And so we're taking inspiration from that now applying it to large language models and coding agent harnesses. And so we're extrapolating that idea into having the generator and evaluator agent just like I was saying earlier. And so this article is a very interesting read. I'll link to this in the description. Of course, they talk about why naive implementations fall short as in just having a single coding agent session. And then they get into their approach. They actually have three agents here. So they have the planner to help create the initial spec. And then we go through the loop of the generator and the evaluator. And I could just read through this entire article, but I wanted to make it really easy for you to grasp everything here. So I've created a diagram to explain it all. So we'll get to that in just a sec. But one more thing I want to show you here is the kinds of things that we can build with this harness. I'll show you both Anthropic and myself. So for their testing, they tried using both the single agent and this harness to build a retro game maker. So you can define your own sprites and even play the game after. And so what we're looking at here is what the single agent made. It looks pretty cool overall, but when they got to the actual gameplay, nothing was working. So, as far as the first pass, you know, the first iteration went, it was pretty underwhelming. And so, of course, you can iterate, but the point of a harness is you don't have to. You can just send off a task and it'll go through the process of iterating itself. And so, that's exactly what they did. This is the second version built with the harness where we had the evaluator being very nitpicky, basically fighting with the implementator. And so we have our opening screen, the sprite editor that looks even a lot more elaborate. The game design, so building your different levels, and then the game play this time it actually worked. And so this isn't like a video that we can watch or anything, but they were able to play the game. And pretty cool. I don't want to just take the article at face value, though. And so with this whole harness that I built for you, I also ran it entirely to build a pretty elaborate application. I created this app where it ingests my YouTube content and it has a full rag pipeline working in the back end and then we have a front end here where we can ask any questions that we want. So I can send this in. This actually works with a large language model under the hood. It has a very very polished UI. Like I'm super impressed overall. We got token streaming and everything like really really cool. So this is the kind of thing that even like shows the sources at the bottom here. Like I know that a single coding agent session would not have been able to create something this elaborate, but I did oneshot it, right? Like of course there's a lot more agents doing work under the hood. But the point is for me, I just sent in a single request for this application and I had it rip through it over the course of about 4 hours actually iterating on this. And man, it's super impressive. Like these are the kinds of things we can do when we let agents work together and just go ham. And yeah, 4 hours does mean a lot of tokens. But the thing is, these harnesses, they enable us to start using faster and cheaper models and still get really good results. Plus, the reliability is just really worth the overhead for me. So, we'll talk about that more at the end of the video, but right now, I want to get into the harness diagram here and show you exactly how this works, how you can run my example for yourself as well. The sponsor of today's video is Scribba. I've covered their full stack developer course before, but not their AI engineering path. And it is very valuepacked. It covers building AI agents, rag, model, context, protocol, prompt engineering, everything you need. And they built it in partnership with Hungface and Langchain, and you get to build things the entire time you're going through the course. So, this is not just your average data science or machine learning course. It's made for web developers who want to build aic applications. And what makes Scribba stand out over all other code learning platforms is at any point we can pause the lesson like right here. And then we can edit the instructor's code. So it's super easy to learn by doing. So for example, I can change the GPT model here. And then I can also change the system prompt for the agent in this example. And then I can run the application. I can test things live. Take a look at this. So I'll go ahead and say hi. And now we'll get a response in Pirate Talk. Very cool. There are also challenges built throughout the lessons with AI for instant feedback, so you're never just passively watching, you're always building. Scribble also has some free courses for you to try things out, but I would highly recommend trying the AI engineer path. I'll have a link in the description to save 20% on the pro plan, which unlocks everything in Scribba. So, the entire system starts off with your user prompt. Very simply, you just have to specify what you want to build. And the beauty of this is the planner is going to expand your prompt into a full spec that's optimized for the harness. So you don't have to worry about being overly specific. And so within the repository that I'll link to in the description where I have both the Claude and Codeex harnesses. I also have the example prompt. This is exactly what I sent in to both harnesses to build versions of the rag application that I shared earlier. And so yeah, build a full stack rag app that looks and feels like a premium AI chat platform like chatgbt or claw.ai blah blah blah. I describe my text stack. Basically injecting all of the opinions I want to, but you can be as specific or unspecific as you would like. So you take this as your initial prompt and then you send that into the planner. And the planner's sole job, I'll actually show you what it looks like here. In both the Claude and Codex harnesses, we have the planner here. its sole job is to take in the user prompt and then produce the spec and that is going to guide the rest of our harness. And so you can see with both the claude and codeex versions, we have the evaluator, right? We got the generator, we have the harness that ties it all together, the planner. All these I've set up very purposely in the exact same way to show you that we don't have to stick to anthropic just because we're taking inspiration from an anthropic article. And then within the uh shared folder here, I have all the prompts for the different stages. So like this is our planner here. You are a product architect. Your job is to take a brief user description. That's our prompt markdown. And then produce a comprehensive product specification. And then everything else in the prompt here is just describing the structure that we want for our spec so we can make it reliable and repeatable between all of our different harness executions. Very, very neat. And then once we have our spec, this is where we go into our sprint cycle. And this is the most fascinating part of the entire system because before we go into writing a single line of code, we're actually going to have the generator and evaluator negotiate a contract. So they're going to decide upfront, here are the different phases that we're going to go through to break up the spec, and here's how we're going to evaluate each phase. So yes, it is adversarial, right? the evaluator is going to rip apart the generator's implementation, but they're agreeing on what that battle looks like up front. So, it's very ethical. They have some etiquette that they're establishing here going into what this build is going to look like. So, let me go back into the codebase and show you my example of this. So, what you're looking at right here is the final spec that the planner agent produced when it was building my rag YouTube chat application. And so, you can look at the harness.ts ts if you want to like see the full workflow how this is coded but once we have the spec from the planner then again we send that into negotiation both the implementator and evaluator agents are going to agree on how they're going to split the larger spec into manageable chunks through these different sprints we definitely don't want to send this entire thing into the agent to knock out at once and so we're splitting up the work so here's our first sprint here are the different features that we need our implementator to build And then here's the criteria that our evaluator agent is going to judge against. And you can see that we have a description of like here's what the evaluator is going to judge. And then we have the threshold because it's going to give a score of 1 to 10 for each one of these criteria. And so as long as it meets the threshold or exceeds, then the implementator did a good job, right? Like we can move on from that single criteria. If everything passes here, then we can move on to the next sprint. And so going back to the idea of GANs, right? Because this is a GAN inspired harness. Essentially, the generator is trying to, you could say, trick the evaluator into thinking that the application is working. And if the evaluator is judging correctly, then it's not really being tricked. It's like the application is actually good. But you you get the idea there of how we're relating it back to having these adversarial agents that are battling it out just like with the old machine learning approach because basically the generator once it gets feedback from the evaluator and it retries a maximum of three times its sole job is just to appease the evaluator. So we rely on the criteria of actually pointing towards the application working correctly. it. As long as that's the case, then if it's doing everything in its power to appease or trick the evaluator, then the application is working. We go through iterating on all the sprints and then we end up with a working application. And so, yeah, this is not going to be perfect. Coding agents are never going to be perfect. But the idea here is that instead of us having to guide these different sprints and us having to work through all the AI slop and provide all the feedback oursel, we're setting up a system where a lot of that is taken care of for us. us and that is a beautiful thing. So I would encourage you to just download this repository and try out either the Claude or Codeex harness. It's going to take you 5 minutes to get up and running and you can build incredible things very quickly. Not that you would use this harness to create something that's immediately production ready, but helping you create proofs of concepts very quickly getting started on your next project. That's what I use these kinds of harnesses for all of the time. And then yeah, at some point this is going to be the way to build anything with AI coding assistance. And so this is worth getting in on right now. And so within the readme, I have a quick start here. Super easy. We can use either cloud code or codec. And to my knowledge, you are allowed to use your enthropic or codec subscription with the harness. So you don't have to pay for API credits. I know there's been a lot of controversy recently around if we can use our subscription with the agents SDK. Even some things in the anthropic docs that said that you couldn't, but uh the is one of the members of the anthropic team, he clarified that we are allowed to use our subscription with the Asian SDK or headless cloud mode as long as it is for local development and experimentation. So if you're building a business on top of the Asian SDK, as in other people would be using your script subscription, then it's against the terms of service and you should use your API key. So for what we're doing here, when you run the harness locally, you can use your subscription for either. So you don't have to pay for API credits. That would be obviously quite expensive with a longunning harness. And so you just have to have your authentication set up with either one of these, whatever one you want to run. And then you clone the repository, install the dependencies, and then it's just a single command to kick off either of the harnesses. And the best part is you can use your coding agent to help with this as well. So you can say, you know, I want to build XYZ. This is where you can describe your application, your text stack, and things like that. And then it can help you create the prompt MD that we have as our starting point. And you can even have it read the one that I have checked into the repo here if you just want a starting point, right? like the general structure that I use for my prompt and then your coding agent can read the readme so it knows how to kick off the harness for you. I've never actually run this command myself as I was prepping this video and building this and doing all my testing. I would always just have my claw code or codeex run this bun command to kick off the whole process. And so very easy to get up and running doing this yourself. All you have to work on is creating this initial prompt, but then all the other prompts for the planner and we have the generator system prompt for the software engineer and the evaluator. You're a skeptical QA engineer. Like all the other prompts are already built into the system here. So, you just get to let it go and it'll create the spec and then create the sprints and work through them automatically and then report back to you when you have the final application. It's so fun to just watch it go and then come back to this beautifully working application. So, the last thing that I want to hit on really quickly here is the downside of these harnesses. Obviously, it's going to be token cost, right? When you have this full specking process with the loop of implementation and evaluation and iteration, it's going to go on for a long time. I mean, that's why these are called harnesses for longunning agents. But the thing is, this is so worth it to me. I mean, look at what I was able to build here in just a single shot. I mean, yes, there's a lot of tokens that we spent here, but it would take a while to get to this point if I was working with a single agent myself. And the kicker here is I use Cloud Sonnet 4.6 to build the entire thing. You can see in the configuration I have for the model I use for both Claude and Codeex. And let me tell you, Opus 4.6 is way better than Sonnet 4.6. I have used both extensively. And Opus 4.6 would not have been able to oneshot this without the harness. and Claude Sonnet did with the harness. And so you can try this out yourself and and try smaller models and you'll see that like we could actually save tokens with these harnesses because now we're leaning more into the system we build for AI coding versus always relying on the most reasoning power possible with these bigger large language models. So it's definitely worth a shot. And so that's everything that I have for you on this harness here. And I will say I'm going to be doing a lot more content soon on these harnesses and just showing you how far you can take them. And so if you appreciated this video and you're looking forward to more things on AI coding and these harnesses, I would really appreciate a like and a subscribe. And with that, I will see you in the next video.