Speaker A
GPT 5.6 is finally out. I've been waiting for this for way longer than I would like to admit. I was lucky enough to get early access to it. OpenAI gave me access. I think it was like a month
Ben Davis reviews GPT-5.6, comparing it to Fable, discussing its capabilities, pricing, and use in agent orchestration.
Key Takeaways
- GPT-5.6 represents a significant step forward in AI model capabilities and usability.
- While Fable is smarter in some respects, GPT-5.6 offers better practical performance and flexibility.
- Long-running orchestration and sub-agent management are key strengths of GPT-5.6.
- Government restrictions impact access to the most advanced models, limiting availability to large organizations.
- Firecrawl APIs complement GPT-5.6 by enabling efficient web data extraction and monitoring for agent workflows.
Summary
- GPT-5.6 is a next-generation model from OpenAI, released as three models: Luna, Terra, and Saul.
- Ben Davis received early access and has extensively tested GPT-5.6, finding it very capable and often better than Fable in many ways.
- The model excels at long-running tasks, orchestration of sub-agents, and handling complex workflows.
- Fable is considered smarter in understanding unsaid intent and general directions, but GPT-5.6 is more versatile and easier to use in practice.
- The video discusses the evolution of model generations from Sonnet 35 to Opus 45, Fable 5, and now GPT-5.6.
- Ben shares personal experiences of losing access to GPT-5.6 due to government restrictions and the impact of stepping back to GPT-5.5.
- Firecrawl is introduced as a sponsor, providing APIs for web scraping and monitoring, useful for agent workflows.
- The video covers pricing, benchmarking, and practical applications of GPT-5.6, including its integration with Hermes Agent.
- Challenges with previous models like 5.5 are contrasted with improvements in 5.6, especially in compaction and long-duration runs.
- The video ends with a discussion on industry realities, government restrictions, and the accessibility of advanced models.
Chapters
- 00:00Introduction and Early Access to GPT-5.6
- 01:34Overview of the Three GPT-5.6 Models: Luna, Terra, Saul
- 03:28Sponsor Break: Firecrawl API Overview
- 04:58Comparison Between GPT-5.6 and Fable
- 06:49Capabilities and Long-Running Orchestration
- 10:19Use Cases and Practical Applications
- 12:43Model Tiering and Use Case Recommendations
- 17:00Industry Challenges and Government Restrictions
- 19:38Conclusion and Final Thoughts
Full Transcript — Download SRT & Markdown
Speaker A
or so thereabouts. Might have been honestly even longer than that. Time is weird right now. But I've had this model for a bit. I've been using it a ton.
Speaker A
This is probably, of all the models that I've early tested and also just used in the past, it's the one I've used the most. This is a very, very, very good model. I would honestly argue better than Fable in many ways. Not quite
Speaker A
smarter, but we'll get to that. I have a lot that I want to say about this thing.
Speaker A
I'm going to try and break this into more reasonable sections because there's just, there's so much to go over here and probably put some other stuff in future videos. But the general things I want to cover here are the generations of models
Speaker A
because this is a new next generation model in many, many ways. How it compares to Fable, how it's benchmarking, how it's priced, how I've been actually using it, how it works in Hermes Agent, the new tiering system opening I just
Speaker A
introduced, and also the unfortunate requisite government crash out at the end of this video. So, if you want to see that, make sure you stay tuned. And let's just get into it. If you somehow have not seen this yet, 5.6 is the
Speaker A
newest model from OpenAI. It's actually three different models. Too in the past, the way they would do things is they would have a mainline GPT model, and then they would also have a mini and a nano model that would probably be
Speaker A
released sometime after that big model. I think the last time they did a synchronized drop like this was GPT-5.
Speaker A
This one is another one of those drops where they are dropping three models at once: Luna, Terra, and Saul, which is the small model Luna that I believe is kind of the replacement for mini, like a haiku tier model. Then they have Terra,
Speaker A
which is a sonnet tier model, and now Saul, which is the big boy. That is the opus tier, borderline punching into mythos tier model we have just gotten.
Speaker A
And I adore this thing. It is a true next generation model. This is a point that I admittedly did steal from Theo, but I like the way of phrasing these things in generations where the first generation of like models that were
Speaker A
actually useful was Sonnet 35, which was a model that could actually call tools. You could make something like Claude Code for the first time because it could run in a loop, call the tools, do the thing. It was useful. But then we got to
Speaker A
the next generation, which was Opus 45 that could do the same thing as Sonnet 35, but could run for much longer, handle much more difficult problems, and was just generally more capable in every way. It felt like a step function up.
Speaker A
And then this new third generation was really kicked off by Fable 5. The only models that we have in this class, I would argue right now, are Fable 5, Sonnet 5, and GPT 5.6. I didn't end up making a video about Sonnet 5, but I did
Speaker A
defend it a lot on Twitter. Most people hated that model. I understand why. It is honestly not a good model. But the thing it did have that none of the other models had at the time, especially since 5.6 and Fable 5 were in government
Speaker A
lockdown jail at the time, is the ability to orchestrate and run tons of different sub-agents like spawning itself over and over again to solve more complex problems and just run for honestly obscene amounts of time while easily
Speaker A
being able to be steered during that run. The way I used 5.6 is very different from the way I used basically any other model before it. I unfortunately did actually lose access to it at one point because of all the
Speaker A
government stuff. So, I had it for a while, then I lost it and had to go back down to 5.5, which is a very annoying but unique circumstance to get a little bit of perspective on how these step
Speaker A
functions feel. And I will tell you, God, 5.5 feels bad to go back to. Like, it felt terrible. Going from having 5.6 in this new system back down to 5.5, the flaws get really obvious. Like, stepping up from 5.5 to 5.6 is good, but going back
Speaker A
down feels atrocious. And we're going to talk a lot more about why right after the sponsor break. Today's sponsor is one of the easiest recommendations I have ever made. It is Firecrawl. They have incredible APIs for search, scrape,
Speaker A
which scrape in this case just means taking a URL and then turning that into agent-friendly and ready markdown files.
Speaker A
It's great. You can also use it to interact with a page, crawl to get all of the different pages on a site, and even now monitor the web with their new monitor API. This is very, very useful for something like a Hermes agent where
Speaker A
maybe you need to track some data source. Every time something changes in there, you need to kick off some workflow that you have saved in your Hermes agent. This will just handle the really obnoxious part of trying to
Speaker A
discern something simple like a copyright change from, oh wait, they added a new pricing plan to this. I've tried to set this up for myself before, but actually dipping a page in a way that makes sense to only get the real
Speaker A
changes and not the little changes is a remarkably difficult task that they do a phenomenal job with. It's a great API and honestly it's just a great product.
Speaker A
Their MCP server is one of the best I've used. CLI is great, REST API is great.
Speaker A
They have SDKs for Python, Node, Go, and Rust. And honestly, I think my recent usage basically just speaks for itself.
Speaker A
All my agents use Firecrawl for a reason, and yours should too at davis7.link/firecrawl. I think the easiest way to explain the new generation and how all this stuff actually feels is by comparing it to Fable. I got asked this question a lot
Speaker A
and ended up writing this out on Twitter where I do still think that overall Fable is a smarter, quote-unquote, model than 5.6 is, but there's a lot of nuance there where like, yeah, it is smarter, but it's not better in every way, if
Speaker A
that makes any sense. It's like Fable is magical because it is so gigantic. It is the biggest model that I think we have publicly available. At least the biggest competent model we have available. And the best way I found to describe the big
Speaker A
model smell that it has is it's really good at understanding the unsaid intent behind your prompt. Whenever you send the agent some instruction and tell it to do something, there's a lot of things that you're not explicitly saying. To
Speaker A
use kind of a bad example here, if you just tell it to make the homepage better, that's a very nebulous statement that different people will interpret different ways and different models will interpret different ways. Fable is much better at just taking these general
Speaker A
directions you give it that are more directional rather than specific and just kind of inferring your intent and just doing the thing you actually wanted it to do. Sometimes not even realizing ahead of time that that's the thing you
Speaker A
wanted. I noticed a lot when using Fable that whenever it would finish a turn and it would maybe suggest the next thing to do, it was almost always either A, the thing I was originally going to ask it
Speaker A
to do or B, something that I hadn't thought of that was better than what I was going to ask it before. It's remarkably intelligent and capable and can run for crazy amounts of time. It's really, really good at orchestrating
Speaker A
tons of sub-agents. I love this new way of building stuff where instead of just doing the one main single-threaded thing that I've been doing for quite a while, I'll typically have it spin off a sub-agent for researching, for implementing,
Speaker A
for reviewing. I'll have it spin up loops within itself effectively. Like the loops thing that everyone keeps talking about is just basically having the model give itself feedback over and over again until it hits some desired end state or goal. Like, for example,
Speaker A
whenever I make a PR now, I will just tell the model, hey, babysit this PR until it's ready. And what that means is it needs to
Speaker A
their review, handle any reviews that they left, make sure that they make sense, if they do, actually make those changes, push it back up, and keep going over and over and over again until that PR is ready to go. So that by the time I
Speaker A
look at it, it's not a first pass. It's done several passes, getting those adversarial reviews in there to ultimately produce far better outputs.
Speaker A
And 5.6 shares all of these behaviors. It can do all of the same stuff that Fable can, in some cases, better. The main difference is that 5.6 6 is much more of a blunt instrument. The differences between OpenAI and Enthropic
Speaker A
have never been more clear than with these two models. They are both next generation Frontier models, but one of them has the kind of uncanny personified feel of Claude models. I don't know, maybe that might be worth a full video
Speaker A
at some point in the future. But like, you've used Claude models, you know what I'm talking about. It is that on steroids. It's really good at understanding what you're asking for.
Speaker A
It's just a very intelligent, capable model. 5.6 is a blunt instrument. It just does what you say. The behavior of 55 where it will just kind of grind and get the task done by any means necessary is still very much present here. It has
Speaker A
better discernment on these things, but not nearly as good of discernment as Fable has. Fable will write more beautiful code. It will oftent times make more, for lack of a better word, tasteful decisions, if that makes any sense. It's better at cutting fluff than
Speaker A
5.6 is, but 5.6 is still a huge step up in both of those departments from 5.5.
Speaker A
One of the craziest accidental AB tests that happened here was I had my Hermes agent running using 5.6 and then when we lost 5.6, I moved it back down to 5.5 and I went from using it every single
Speaker A
day, relying on it pretty heavily, loving it to barely touching it at all because you instantly just notice 5.5 will do so many more tool calls. Its outputs will be three to four times longer than 5.6's will be. It's much
Speaker A
worse at pulling the correct data sources and using the right skills at the right time. 5.5 just kind of flails around in a way that 5.6 six doesn't and once you see it and feel it, it's pretty impossible to go back to the previous.
Speaker A
One of the biggest issues with 5.5 was the fact that it really did struggle with compaction and running for very long periods of time and orchestrating work. 5.6 does not have that problem. I have had runs go for 20 plus hours
Speaker A
perfectly coherent doing the right thing. The biggest issue though is that like I said, it's a blunt instrument where it will just solve the problem by any means necessary. It will write a thousand plus tests. It will add a
Speaker A
bajillion try catches. It will do whatever random patches it needs to to solve the problem and accomplish the goal by any means necessary, which is sometimes a good thing. But if you're dealing with something that's not super well specified and it finds a thread and
Speaker A
starts going down a bad path, you're going to end up in a really obnoxious place. Like to give you a weird example of this, this is one of the many things I built using 56 over the last couple
Speaker A
weeks. You can see on June 22nd, I had a very, very big day of tokens. Theo and I were pushing this thing very hard, trying to see how much we can do with this thing and just seeing like what is
Speaker A
actually possible if we just push the limits of its capability and letting it burn as many tokens as humanly possible.
Speaker A
Almost all the tokens it used that day were for this project TX9 which was initially architected as like a custom VM for running Hermes agent exeutor cloud codecs and a couple other things just in a really nice easy to use box
Speaker A
that you could zip up put on any machine open it up put in the password get it running like I have a bunch of Hermes agents I want an easy way to manage and control these things and my first couple
Speaker A
attempts at this were a little underspecified and there are a lot of weird edge cases with obviously dealing with VMs and stuff like that so when I sent 5.6 off on its journey to go try and build this thing. It sure as hell
Speaker A
tried to do it. And it made something that was like objectively probably pretty correct, but also massively over complicated, massively overengineered, really hard to work with and look at and deal with. It was hundreds of crazy shell scripts, thousands of tests, not
Speaker A
something I would actually want to use. I had to just basically take all that, throw it away, and rearchitect this in a way that made a lot more sense and give it a lot more direction on like, no,
Speaker A
this is the shape of the API. This is the way I want this to work. here's how you should do it. And after a few more days of work, I ended up with the current version that is live public. If
Speaker A
you want to use it, you're welcome to try it. It's still very much like alpha testing this out. I am currently porting my Hermes agents over to it right now. I have two on it right now. Probably going
Speaker A
to end up with like five by the end of the week. We'll see how that ends up going. But once I cleaned everything up, it did a great job here. The architecture is good. The code is good.
Speaker A
I'm impressed. Okay, so I'm reacting to this live here. Deep SWE, which is at this point basically my favorite benchmark. It's the only one I found that really maps to reality and also exposes the things I want like cost,
Speaker A
tokens, steps, etc. They have 5.6 soul on max reasoning at 73% 3% over Fable 5, which is fascinating. This graph is a little hard to parse. There's a lot of stuff going on here, but the important piece right here is 5.6 6 Saul is
Speaker A
outperforming Fable in basically every way except for low reasoning of Fable is about on par with the medium reasoning of 5.6 Saul. That sounds about right.
Speaker A
But then after that point, it is benching higher and is also benching far cheaper. The XI tasks are about $4.70 on 56 and they're about $13 on Fable 5.
Speaker A
Comparing the output tokens, not quite as big of a difference, but still pretty decent difference in between these and the agent steps. Once again, 5.6 is on top. Like, this is the really good trend that I'm now seeing from both anthropic
Speaker A
and openAI models, which is the gradual improvement in efficiency. I don't know if intelligence is the right word for it. I would argue it's at least a subset of capability here where the discernment quote unquote of a model is really,
Speaker A
really important. To use a kind of weird example here, I have made a lot of thumbnails in my day for my channel, Theo's channel, a bunch of other stuff.
Speaker A
I've gotten much better at making thumbnails. When I first started out, it would take me a couple dozen iterations to get anything competent. And even on the compositions that I did end up using, it required me to do a lot of
Speaker A
steps and make a lot of things. It took a while. As I got better, I got more efficient. I had to take fewer steps, use fewer assets, it took less time. And that's kind of analogous to how models
Speaker A
and their tools work where like if the model can just implicitly understand a what you're asking it to do, and b the general shape of the codebase and how everything fits together faster, it will do the job in fewer steps. It doesn't
Speaker A
need to do as many grips. It doesn't need to do as many reads. It doesn't need to check its work as much. It can just kind of do the thing, run the big checks and end up with something like
Speaker A
this where you can see 55x high was averaging 82 steps versus 56 soul is averaging 61 steps. It's a really nice improvement to see. And we're also seeing this in anthropic. Fable is more efficient than obus. Sonnet 5 is not
Speaker A
more efficient than anything on here, but th this was just a strange model in every single way. I don't it's not worth going down that rabbit hole right now.
Speaker A
So, like from my experience and from clearly benchmarks, 56 Sol is probably the best model out there right now. And I was genuinely more excited for this to come back than for Fable to come back just because I used it a lot more. I got
Speaker A
a lot more use out of it cuz it's just such a good workhorse. It is so good in Hermes. It is so good in codeex. It is really nice to have in chat GBT. It is definitely more of an autistic model is
Speaker A
like kind of the way I describe the GBT models. It will just do the thing you ask it to do, which I like a lot. But I think that leads to the next question here of what do we do with Terra and
Speaker A
Luna? Obviously, Solo's incredible. It's the big boy model. Go try it with sub agents. Go push it harder. You're going to have a very good time. What do we do with these two? Luna, the way I'm kind of thinking about it, and just small
Speaker A
models in general, is I feel like the more times you need to run the same prompt over and over again, the more sense a small model makes. And basically what I'm talking about is if the prompt is basically a function, like it is
Speaker A
parse the sentiment of this comment and then you pass in a comment. It is a function that takes in a variable and then just produces the output to get like a JSON shape out of that. The smaller models make a lot more sense
Speaker A
here. You don't need the big boy to do that. Like even on deep SWE, if you look at Max Reasoning 56 Luna, which is a very cheap and fast model, it is not that far away from Soul and Terra. It's
Speaker A
a really good model, but it just like I think makes a lot more sense on more defined tasks. So stuff like classification, title generation, all the nonsense I've talked about with like GStack where you're basically creating a program out of a markdown file that has
Speaker A
a list of steps that the agent needs to run in order to do something. Luna excels at that. Like a lot of the cron jobs I have going for like pulling down all the data and analytics from a bunch
Speaker A
of YouTube channels, a bunch of Twitter accounts, and all that stuff. Saving them to a database, doing a couple aggregations on them, leaving some notes, that sort of thing, that step-by-step procedure. I just use Luna for that and it works really well. I've
Speaker A
been very happy with it from the testing I have done. I've not used it nearly as much as the other ones. Terra, on the other hand, I'm honestly kind of at a loss. I think the medium model like
Speaker A
trench is very real and I don't know exactly where these things fit. It is better than 55 and it is much cheaper.
Speaker A
It's half the price of 56 Saul and that is very clear especially on Max Reasoning. Like that's $5 on Terra versus $8 on Saul. still a very capable model and might end up making a lot of sense for something like a Hermes agent
Speaker A
where you need a lot of capability, but you don't need quite as much and it doesn't need to run for quite as long.
Speaker A
So maybe just having that good balanced price toerformance ratio might make a lot of sense. I'm not entirely sure, but I do still think for general day-to-day coding tasks, most of the time I'm just going to be sticking to Saul because the
Speaker A
bigger the model, the better it is at these less clearly defined tasks. I mentioned earlier the idea of pulling the unsaid intent out of the prompt.
Speaker A
That is a big model thing. It needs to keep a lot of things inside of it at once to run for a very long time to accomplish these big complicated tasks that make it so that the outputs will be
Speaker A
far better from something like Saul. And they'll also probably in some ways end up being cheaper and more efficient because it'll have to do less work.
Speaker A
It'll have to think less. It'll be able to accomplish the task versus just kind of spinning its wheels and not really getting a ton done. I could entirely be wrong about this. Maybe Terra has a ton of really good uses and we're just
Speaker A
sleeping on it. But overall right now, the two that I think I'm going to spend the most time with are Saul and Luna.
Speaker A
Luna for all the workflows, and Saul for basically everything else. There's also an interesting branding thing here that I'm really glad OpenAI has finally woken up and started doing where instead of naming their models like GPT56, then GBT
Speaker A
56 mini, then GPT56 Nano, they're just calling them like some other name like Luna, Terra, Saul, or Haiku, Sonnet, Opus. These are much better names and make the smaller models sound much less unpleasant. It's the same thing with
Speaker A
reasoning. They have renamed low reasoning to light reasoning because low just sounded bad. When 55 came out, I pushed low reasoning really hard. Reason being, it sounds worse than it is. It is actually quite useful. And it's the same
Speaker A
thing with 56. 56 on light reasoning is extremely useful. I think it's actually the default in the codec to just have 56 on light reasoning. That's worked wonders for a lot of the ways I use these things day-to-day where like okay
Speaker A
I need to get this installed on this machine or I need to SSH into this thing and reboot the Hermes agent and change these configs instead of doing that manually but just tell 56 on light reasoning to go do it. It'll do it and I
Speaker A
end up with this and even though it is the big huge model it doesn't really feel like it on light reasoning. It's very very fast. I wanted to mention briefly too the things that I've been building with this. Obviously the
Speaker A
model's the main thing here but I have been using this to get a lot done. And a lot of my work these days is more on the internal tools side of things where like managing the YouTube channels is like
Speaker A
the biggest thing I need to do. And there's a lot of pretty complicated work that needs to be done on the data side to do this really effectively as well as just making other random things that I just kind of wanted to have. like TX9 is
Speaker A
the kind of utility that I probably wouldn't have bothered with before this came out, but now that it is generally speaking a lot easier to print out useful software, especially when these agents can run for long periods of time
Speaker A
and kind of the way I've started to do development in general has changed a lot where before I was just doing everything mostly within the Codex desktop app on my machine. Now I'm doing basically everything by remoting into a T3 code
Speaker A
instance that is running on one of my Linux boxes in my home server. I need to do my Linux video that should be coming soon hopefully. But the general gist of this is Mac OS is great is the operating
Speaker A
system I want to be using myself dayto-day. I just there are too many things I need inside of it that I can't get from Linux. So I have to stay on it.
Speaker A
The problem is if you look at activity monitor while a big codeex run is going, especially with a bunch of sub agents.
Speaker A
You will realize that Mac OS is sending off a bunch of system MD processes to babysit and watch it and will eat all of your performance. Even on my fully loaded M5 Max MacBook, it will start chugging if I have too many running just
Speaker A
because of all the crazy nonsense they're doing with computer use and all the MCP servers. The the real sin of MCP, the fact that you have to spin up a new MCP server for every thread is just deranged. I hate it. It's annoying. But
Speaker A
all that comes together to make the performance just not great locally. And I'm much happier remoting into another box and letting it do work there. But that's another one of these things that's kind of unlocked by this next
Speaker A
generation of models where I'm a lot more comfortable just letting them control the computer and kind of just do their thing over there. Like to give you an example, earlier today in the Columbia project, I added in the
Speaker A
homepage for the pages project, which is just like my HTML publishing thing. It's a self-hostable CLI has a skill. It has some pre-builtin stylesheets so that the pages end up looking more competent every single time. I have this installed
Speaker A
on all my machines and then hosting my own version of it up on Railway. This is one of those things that will be published and open source very soon hopefully. I just need to clean this repo up. There's too much in it right
Speaker A
now. I needed a homepage, so I just told it to go make one and then closed my laptop. I fire off the job. It runs on the Linux machine which is on 24/7. The performance in Linux is just absurd. It
Speaker A
is so so good, especially when you're remoting in. And at the end of this, I ended up with it fully implemented. Got the link that I just showed there.
Speaker A
Worked the way I wanted it to. It deployed it up to Cloudflare for me.
Speaker A
That's another thing I've started doing more of is letting the agents just deploy random stuff up for these especially these side projecty type things. I've really been enjoying Cloudflare lately. Cloudflare has some of the best primitives in the industry
Speaker A
by a mile. They're just kind of annoying to work with and a lot of the SDKs are gross and there's a lot of weird docs you have to read. But now that the models are this good, I can just let
Speaker A
them go do it. I know the concepts. I know what a durable object is. I know where I want to stick them. I know what the code should look like. I don't want to deal with the config. I tell it to do
Speaker A
the config. And then I don't want to run Wrangler deploy. It'll run that for me.
Speaker A
I just don't think about these things nearly as much anymore. I just talk to it and tell it to do the thing and it does the thing. I have a skill in the Codex instance on all my machines that
Speaker A
tells it the general setup of my network, the IP addresses of all the different machines, how to SSH into each one, what the tail scale network looks like, all that stuff. So, if I want to get a preview of a dev server, I just
Speaker A
tell it to start up the dev server and put it on tail scale so I can see it and then it'll just work that way. Like I just use this as the core dev box for everything. It's amazing. My home server
Speaker A
is a weird one to put on the list of things that I made using 56, but in a lot of ways it was. It was very useful in doing the research for what I needed, how it should all fit together. The app
Speaker A
with pictures was incredibly useful, especially setting up like the manage switch and all that stuff. That that sucked. And now it's set up, getting all the pieces together, getting SSH set up between all the machines, getting files transferred over from one to the other.
Speaker A
You can kind of just spin up codecs, tell it to, hey, here's the IP of this machine. Can you make it so an SSH into it? and it'll either just do it or if it needs a password, it'll just give you
Speaker A
the command to run to set that up for it. And then now that it's set up, it can just do that whenever. I've gotten to the point where I'm kind of just constantly sending random stuff back and forth between my machines and just
Speaker A
passing it around. And I don't have to memorize a bunch of weird commands to do it. I just tell 56, hey, go do this, and it just does it instantly. I think it's a lot of why I like 56 better than Fable
Speaker A
is Fable can do all of the same stuff, but it's a lot slower and clunkier to do it. 56 on light, medium, or high reasoning or whatever will just do the thing you want it to pretty quickly and
Speaker A
efficiently. It's great. Columbia is one that I want to talk about in a separate video another time. Better get status.
Speaker A
This was like a really dumb thing I made yesterday just cuz I was getting annoyed. I do love this new dev setup, but one weird thing I am running into is it is getting harder to keep mental PRs
Speaker A
that are open, the status of all the different projects, what was last done. Like if you have five agents running at once and then you go back to it, you have to get that context again, see what's been happening, verify
Speaker A
everything, run it, test it. This dumb little going CLI that just prints out the URL for the repo, which is a surprisingly annoying thing to get from most of these interfaces. Like not even in T3. There's no button to just open up
Speaker A
the repo to go look at the PR tabs or whatever. And then it also shows you the work tree, what it's tracking, what branch it's on, the last commit, any draft PRs, any open PRs. This is really useful for doing a quick sanity check on
Speaker A
where a project currently is. If I forgot about a PR one of the agents kicked off last night, I can go back to it from here. I like having this. And this is just kind of the thing you can
Speaker A
really start doing more of now that these models have crossed another threshold here. Like generation 3, you can just print so much software. I have been surprised at how much software that I actually want. Now that it's borderline trivial to make something
Speaker A
like this, I've been making a lot of things like it. And I think over time this is just going to become more and more true as I just naturally automate and build out this crazy custom system that's mine that I just created by
Speaker A
describing it to 56 and it doesn't. But now I think unfortunately I need to talk about and address at least a little bit the realities of what's happened with this model drop and the government and where the general industry is. The
Speaker A
mythos and fable ban caused a lot of panic and as a result 56 was delayed.
Speaker A
I'm pretty sure was supposed to come out a couple weeks ago, but it had to be pushed back in order for them to add in additional safeguards, do additional verification, and make sure that the government was okay with them releasing
Speaker A
it generally to the public. This is a very annoying thing that I'm very annoyed happened this early. Did this have to happen eventually? Absolutely.
Speaker A
Are we at that point yet? No. I have so many qualms with Anthropic. I've talked about them many times in videos and in the podcast, but the one thing that I don't doubt them on is their ability to
Speaker A
make safe models. Their safeguards, especially on Fable, are so robust and so aggressive and now in some ways even more so. I don't think they're as bad as a lot of people say. Like I don't run into them all that much, but like if you
Speaker A
even look or smell like you're going to be talking about biology or whatever, it will instantly kick you down to opus.
Speaker A
And the jailbreak, quote unquote, that the government found which triggered the Fable and Mythos export control was so stupid. It was such a just like the government doesn't understand these things. It was just like a PR that had a
Speaker A
security bug in it and when the agent was asked to fix this bug, it was able to fix this bug. That is not a unique capability to Mythos. 40 could probably do that if we're being honest here. The
Speaker A
new regulatory regime is going to be very annoying to deal with going forward because the thing that I am very scared of happening and do not want to have happen is the best of the best models being restricted to smaller and smaller
Speaker A
numbers of people. Like the fact that for quite a while the only people who were able to use Mythos or 56 were Fortune 100s and the labs themselves sucks. That sucks a lot. I don't want to live in a world where that is the case.
Speaker A
And it does seem like, especially from reading through the system card, which is quite good for this model. I'd recommend going through it at some point. And everything they say here, OpenAI is doing a lot to make sure that
Speaker A
these models are secure. And I am hopeful that through all of the nonsense that they're going through in Washington, all the politics they're playing, we get to a point where you're able to have a pretty quick turnaround from model preview to model being
Speaker A
released and approved for the public. They note that 56 models are more capable than earlier models in both biology and cyber security, but do not cross the critical threshold in either category. In cyber security, our testing suggests that 56 is better at finding
Speaker A
and fixing vulnerabilities than at reliably carrying out autonomous endto-end attacks against hardened targets, giving defenders an opportunity to strengthen systems before weaknesses are exploited. In bio 56 can support legitimate research, but does not provide endto-end capability needed to
Speaker A
create, engineer, or synthesize a highly dangerous novel threat. The obvious issue here is the fact that in both of these domains, defense and offense are generally speaking dual use, where by fixing vulnerability, you are also finding a vulnerability. And as a
Speaker A
result, you can exploit that vulnerability if that patch hasn't gone out to all of the users of that software, which can be very dangerous, especially on really big open- source projects. Like the Glass Wing thing, as cringe as some of the branding was, did
Speaker A
make a lot of sense to give all of these companies a head start to harden up these very important pieces of software so that when Mythos and Fable are released to the world, there's a much lower chance of some catastrophic damage
Speaker A
happening. I've historically really liked the way OpenAI does their safeguards and I think 56 seems to be in the same boat where 56's safeguards are layered for greater accuracy and redundancy and designed to adapt quickly as new attacks emerge. Protections
Speaker A
trained into the model work alongside real-time checks, continuous monitoring, and account level enforcement to help the system remain safe even when a particular layer does not work as intended. The way they're doing this seems to be different from the way
Speaker A
Anthropic is doing it where like there seems to just be a smaller dumber agent that is hunting for naughty words or something in your prompts which will instantly just kick you out back down to Opus 48. The way OpenAI does it is much
Speaker A
more gradual and much more context aware to where I have had zero issues with security classifiers going off or it being loized or it refusing to do something. They're quite good at this and they even call out here in many
Speaker A
systems classifier flags alone decide what to block relying on lower intelligence models that are harder to change in order to prevent harm which is a direct shot at enthropic. Our approach adds a reasoning monitor that reviews the conversation to determine if there
Speaker A
is a potential for harm. This design is intended to enable defensive work while blocking serious misuse with the most sensitive capabilities reserved for verified users through trusted access.
Speaker A
And they also have I believe 56 cyber which is not publicly available. that's the one that has all the cyber safeguards turned off that the big companies can use to harden the big systems. I I think that's a fine
Speaker A
compromise. Like I think not releasing that publicly, as much as I do want to be on the side of like everyone should have access to all of these things, it shouldn't be limited. The potential for downside here is just so great that I
Speaker A
get it. I I'm okay with that. And from everything I've heard from OpenAI, the people I've talked to there, they are taking this very seriously and doing a very, very good job of trying to mitigate the situation, ensure that as
Speaker A
many people as humanly possible have access to these things and can actually benefit from them. I'm very hopeful that going forward when we get into 57 or GBT6 or whatever ends up coming next, we don't have issues getting access to it
Speaker A
like we did with this one. Time will tell, but I am hopeful. And to the government, please, please just listen to these labs. They they understand this stuff a lot better than you do. I promise. That all said, I think my
Speaker A
closing thought here is that 56 is an incredible model. It is everything good about 55 improved or fixed. Kind of the ultimate model in that series. I don't know how much further they can push this pre-training cuz I believe it's the same
Speaker A
base model as 55. They just added a ton of reasoning on top of it. And this chart here is frankly very impressive.
Speaker A
Fable 5 is a much bigger model than 56 is. Like this is a new pre-train, a new generation. This is technically speaking an old generation model. Even if it feels next generation in capability.
Speaker A
Like OpenAI is just insane at RL. Their reinforcement learning pipelines have always been incredible. They're super super good at it. If they're able to pull this differential off of 73 versus 70 on 56. GPT6 is going to be a
Speaker A
fascinating beast. I don't know how that one's going to turn out to be super honest, but we might be getting into some pretty absurd territory here. If you asked me a year ago if the crazy scaling stuff could ever possibly happen
Speaker A
and we could get a magical god machine out of these LLMs, I probably would have said no. If you asked me that now, I would say maybe. These things are getting to pretty absurd levels of capability. It doesn't seem like they're
Speaker A
slowing down anytime soon. So, we might be on the exponential. Things might be about to get really insane really fast.
Speaker A
I don't know how else to end this one. This is the model that pushed me over the edge and in many ways gave me full AI psychosis. I'm guessing that's going to happen to a lot of other people and
Speaker A
it's going to be a very strange couple years from here. We'll we'll see how this all turns out.
Topics:GPT-5.6Fable AIBen DavisOpenAIAI model comparisonHermes AgentFirecrawl APIAI orchestrationAI benchmarkingAI pricing
