Testing GPT Astra vs Fable 5.1
Want to back the stream? The «داعم» membership on YouTube is EGP 60 a month: badges, emoji, and I answer your questions first on stream. Become a supporter
The gist
The stream centers on a practical, partly blind comparison of GPT Astra and Fable 5.1 using matched prompts for games, an SVG drawing, landing pages, and agentic tasks. Results were mixed: Astra often appeared stronger visually and in the controls of some games, while Fable sometimes delivered more challenging gameplay or more inventive design ideas. Speed varied by task and audience preferences split, so no clear overall winner emerged. Luqman discusses the limits of single tests, benchmarks, and model judges, arguing for evaluation grounded in domain expertise, bias reduction, critical-thinking and ideation tests, alongside efficiency, price, usage limits, and tool integration. The discussion also covers routing planning and execution to different models, sub-agent orchestration, documenting and compacting long coding sessions, preconfigured bots, video automation, backups, and changing developer roles. Claims about AGI, job displacement, and near-term end-to-end AI development remain attributed predictions or concerns rather than established conclusions.
104 moments from the stream
- 0:46:05
No Single Model Is Best for Every Task
Responding to an audience question, the speaker agrees that users should avoid automatically defaulting to one model. Some models are clearly stronger in particular areas, while closely matched models may suit different tasks, so he does not believe an absolute best model exists at this stage.
- 0:51:32
Speed Alone Is Not Enough for Model Comparison
The speaker discusses the difficulty of interpreting speed tests when tools do not report time consistently or use different execution settings. He notes that model usage can be measured numerically but says he did not calculate it for this comparison, leaving the speed observations partial rather than a complete benchmark.
- 0:54:03
Designing a Constrained Arabic Language Test
The speaker proposes an Arabic-language test asking for a Modern Standard Arabic short story of exactly fifty words, defining how words are counted and requiring successive sentences to begin with ordered letters. The idea tests compliance with layered Arabic constraints, but this page shows the test design rather than its outcome.
- 0:56:27
A Koshari Landing-Page Generation Test
The speaker prepares a shared prompt asking each model to build a koshari landing page with strong design, motion graphics, varied techniques, and a clear call to action. The aim is to push both models toward their best practical output and compare capability and time, but this page does not contain the final result.
- 0:59:33
Balancing Efficiency and Output Quality
The speaker says he currently leans toward Codex for efficiency rather than because he believes its model is inherently stronger. His qualification reflects the subjective quality judgments in the shown tests, where audience preferences were divided.
- 1:01:47
Testing a Website Redesign with Minimal Guidance
The group considered two minimal-guidance tests: generating a landing page from a brand/page or post reference, and separately giving an agent an existing website link with only a request to revamp it. Both remained proposed tests rather than completed results.
- 1:04:18
Branching, Steering, and Interrupting AI Conversations
The discussion distinguished several agent-conversation controls: forking into a new branch, opening a side conversation, and interrupting a running task with new guidance. The speaker noted that names and mechanisms differ across tools and expressed a personal preference for one tool’s steering experience.
- 1:10:02
Interim Observation on Agent Test Speed
During the stream, the speaker noted that two ongoing tasks were taking a long time and that Fable had been slightly faster than GPT in two earlier tests. This was an interim speed observation, not a final quality judgment or completed benchmark result.
- 1:10:56
Rebuilding a Well-Known Landing Page as a Stress Test
The discussion suggested asking the model to rebuild a well-known service landing page such as Resend, aiming for a demanding task that exposes the model’s limits rather than an easy test.
- 1:12:37
Consumer Subscriptions, Automation, and APIs
The speaker discussed an audience response saying that sending automated traffic through an Anthropic consumer subscription may violate its terms and that automated use should go through the API. He considered this consistent with Claude’s observed behavior while retaining uncertainty around some details.
- 1:14:09
Qualified Impressions of Astra, Fable, Codex, and Claude
An audience member reported a favorable impression of Astra over Fable in another test, but the speaker clarified that he had not tested that same task. He also expressed a personal preference toward Codex and a sense that Claude was slipping, presenting these as impressions rather than conclusive results.
- 1:15:10
Comparing Astra and Fable 5.1 Across Varied Tasks
The speaker showed a comparison in which the models’ identities were hidden from the audience. Tasks included a Flappy Bird-like game, a camel drawn using inline SVG only, and an ongoing website test. Astra’s game code appeared cleaner, while Fable 5.1 produced harder physics, more challenging gameplay, and delivered faster. He stressed that evaluation was still underway, also prioritizing intelligence, critical thinking, and ideation, and had not yet tested the models in his real work.
- 1:18:42
An Opinion on OpenAI’s Design and Graphics Advantage
The speaker suggested that OpenAI may be stronger in graphics and design, linking that view to its history and stronger image-generation capabilities relative to Anthropic. He framed this as a reasoned personal assessment rather than a definitive test.
- 1:19:30
Qualified Conclusion from the Flappy Bird Test
After playing the outputs, the speaker said parts of the comparison remained subjective but judged Fable 5.1 stronger in this specific test because of its more difficult gameplay and physics, as well as faster delivery. He did not generalize that result to the other tests.
- 1:23:22
How to Choose a Useful, Decisive AI Model Test
The stream discusses the tradeoff between testing models in a domain the evaluator knows well, which makes quality differences easier to judge, and using a broader visual task that more viewers can understand. 3D and motion-graphics tasks were suggested, but their potentially long runtime prompted a search for a shorter, more decisive test.
- 1:27:54
Why Model Efficiency May Matter More Than Small Quality Differences
The speaker’s provisional view was that the comparison remained subjective and that practical efficiency and usability might matter more than quality alone. During a long-running test that was not yet complete, he said Astra could pull ahead if it solved issues Fable 5.1 had not resolved, so this segment did not establish a final winner.
- 1:30:49
Why One Test or Benchmark Cannot Determine the Best Model
Luqman explains that solving one particular problem does not establish a model’s absolute superiority, especially when frontier models are closely matched. He argues that benchmarks can be context-dependent or biased and that interpreting more objective evaluations requires expertise and a learning curve; he also considers giving the same analysis to both models and comparing their interpretations.
- 1:36:18
Observing Astra and Fable on a Long Agentic Coding Task
Luqman follows a long coding prompt involving hex-field motion and agentic coding skills. In this live run, Astra finished before Fable while both used a high setting, and Fable later appeared to hang; this is an observation from one run, not a general verdict on either model.
- 1:39:14
An Opinion on the Workflow Leap from GPT 5.5 to GPT 5.6
Luqman describes his personal experience of moving from GPT 5.5 to GPT 5.6 as a noticeable improvement in workflow efficiency, with broader speed and model options. He says this reduced his reliance on Claude, while presenting it as a workflow-based personal assessment rather than a universal benchmark result.
- 1:43:00
Why Test Models Directly Instead of Relying on Benchmarks?
Luqman explains that he began questioning how well benchmark scores reflect his practical needs, so he gives different models the same prompt and compares their results directly to judge which is more suitable and why.
Got a question about something in this stream?
Ask in the group