Astra Landed, Fable Led - The Week the Scorecard Broke (and the calculator in your pocket).

Ah, yet another big week in AI. This is The AI Exhale, hosted by Vibe CFO with Jez and Cam. Five minutes, one thing that matters, or maybe two. We ignore the rest so you can too.

Jez: OpenAI have come out with GPT-6 Astra. Just a couple of days before that, Anthropic released Claude Fable 5.1. Speaking on the OpenAI one, we had the company president, Greg Brockman, saying, welcome to the AGI era, and then declining to define exactly what AGI is. So let's give the audience a bit of an education on that. But the interesting part is that in both of those releases, each respective company has rated their model against the other. Where have they rated them? On their own scorecards.

So Anthropic said that theirs is ahead on intelligence power. And then equally, OpenAI has their own scorecard with Astra ahead. Now, what do we like? We like independent indexing, something we're going to do with our podcast, actually, now that we've done enough episodes. We're tracking nicely. And Artificial Analysis is actually putting Fable ahead.

Cam: What we're finding is that these frontier models are so commoditised. Nobody has any allegiance to any of them. If you tell the story for the past couple of years, let's say OpenAI was winning. And then around about December last year, Anthropic just started kicking butt and it went miles in front and everybody was coming past it. The annualised recurring monthly revenue just exploded, and it's going to have a bigger TAM and it's going to have a bigger IPO.

And then in June, OpenAI just launched some new models and went super, super cheap. And the developers, it was a monster switch. What does that tell you? It means that we just go looking forward. We don't have any allegiance really long-term, do we?

Jez: Yeah, no allegiance. That's for sure. We like to bounce between the two. But is there a differentiation?

Cam: I would personally go to OpenAI for speech. I would talk to it. I think it's exceptionally better. I would use OpenAI for creating images. I think it's exceptionally better. But I wouldn't use it for coding or finance work, like in Excel or anything, personally.

Jez: What about costings?

Cam: Well, costings, they're looking about the same. The one thing that's always forgotten is how efficient these tokens are. You just go, this is their best model and here's their input and output token costs. But which ones actually solve problems? I think you're starting to see that now on social media, where they go, here's a Ferrari model, and you watch them side by side. I don't know if you've seen that, and you go, which one looks better?

Jez: But also what they do, they do the price, right?

Cam: Because the actual price to make that Ferrari is the important point. It's not which one's smarter. It's which one efficiently uses tokens to get a better outcome as well, which I think we're coming into that sort of age now as well, of value, speed and quality.

Jez: Well, Ferrari is definitely quality. Cost effectiveness? They're known to just pour money in, but that's the beauty of the Italians, right?

Cam: Jez, what do you think AGI is? OpenAI define it as the ability to outperform humans at most economically valuable work. Because if you look back at a calculator in the 90s, that outperformed humans. We couldn't compete with it. That's a form of AGI in a subset of the machine, right?

Jez: Yeah. Oh, and I might say to that, I remember the teachers at school saying, you won't have a calculator in your pocket, so you've got to learn it. Well, we do. We have it on our phone. What I would answer, gut feel, is it's a big marketing spiel again. What we actually have here is a sales pitch of being the frontier model, which you're actually not.

So the product that came out later hasn't proven to be the one ahead. So again, it's a real transparent showcase of the fact that these guys are just competing for disruption and to get eyes on them. But yeah, in terms of that human cognitive capabilities piece, there is something in there. We hosted our webinar earlier today, one of our monthlies, where we talked about the human in the loop element, and we're going to remain true on that. So I think, harness what's there. If it does connect up more systems and makes my life easier, so that I can review and sign off, then that's of interest to me. But to me, it's a little bit of sales fluff.

Cam: It seems like it. And they've got every reason to. They should be in panic mode over two things. One, the open source movement, and two, the margins. I think lots of businesses now, whether you're a business or whether you're a SaaS selling to business, most people suggest that you build on the frontier, but you don't scale on the frontier. That's just too expensive, right? We can't give frontier intelligence at scale.

So yeah, I think that route of saying AGI is here and we're the leaders, and be careful with everything because you should be sticking with us, is obviously just marketing panic. They've got challenges on the frontier, but they've also got challenges at the open source level as well. They need to get better at routing. I think routing is still crap. I don't want to have to think what model. I reckon we're going to look back on this time of choosing our models.

It's absolutely stupid, because we just constantly choose the top model because we're not sure and we want the best answer. The routing should do that for us. And I think that they will eventually get there, where it's exceptional routing. That's our answer.

And that's your five-ish minutes back. Vibe CFO, Australian founded. An AI CFO for small and medium businesses, out of the box, governed and secure. vibecfo.ai. Now you know.