Testing GPT Astra vs Fable 5.1
Want to back the stream? The «داعم» membership on YouTube is EGP 60 a month: badges, emoji, and I answer your questions first on stream. Become a supporter
The gist
The stream centers on a practical, partly blind comparison of GPT Astra and Fable 5.1 using matched prompts for games, an SVG drawing, landing pages, and agentic tasks. Results were mixed: Astra often appeared stronger visually and in the controls of some games, while Fable sometimes delivered more challenging gameplay or more inventive design ideas. Speed varied by task and audience preferences split, so no clear overall winner emerged. Luqman discusses the limits of single tests, benchmarks, and model judges, arguing for evaluation grounded in domain expertise, bias reduction, critical-thinking and ideation tests, alongside efficiency, price, usage limits, and tool integration. The discussion also covers routing planning and execution to different models, sub-agent orchestration, documenting and compacting long coding sessions, preconfigured bots, video automation, backups, and changing developer roles. Claims about AGI, job displacement, and near-term end-to-end AI development remain attributed predictions or concerns rather than established conclusions.
104 moments from the stream
- 0:01:37
Designing a Blind Comparison Between Two Models
Luqman explains why he wants to test the two models himself across tasks such as critical thinking, design, and debugging. He proposes showing outputs without revealing which model produced each one so the audience can vote with less bias.
- 0:05:48
Models May Have Different Strengths
As the test begins, the discussion includes an audience view that both models are strong but may excel in different areas, including design. At this stage, this is a reported opinion rather than a result established by the test.
- 0:07:04
Testing a Reported Design Advantage
Luqman says he has heard that Astra is better at design, but emphasizes that he will test the claim himself. He starts with a single-file game-clone task and plans to compare timing and outputs rather than relying only on benchmarks or other people's opinions.
- 0:08:11
Choosing a Coding Tool for Efficiency
Luqman gives his view that pairing a coding editor with an assistant model may deliver greater efficiency than relying on other tools, while still describing Codex as good. This is his recorded assessment, not a benchmark established in the segment.
- 0:09:10
Reducing Bias When Evaluating Outputs
Luqman explains that he sent the same prompt to both models, will hide the source of each output, and plans to open both games with the audience for evaluation and further test suggestions. The aim is a practical judgment less influenced by model names.
- 0:10:02
Criteria for Choosing an AI Coding Assistant
Luqman discusses his preliminary preference among Claude and GPT/Codex tools as a personal judgment based on breadth across design and programming, efficiency, usage limits, cost, and accumulated familiarity. No systematic comparison is presented to validate these impressions.
- 0:11:29
Learning Through Vibe Coding Without a Traditional Programming Background
Luqman describes himself as a hobbyist who has practiced vibe coding for some time without a traditional programming background, while benefiting from people with programming expertise.
- 0:11:48
Blind GPT Astra vs Fable 5.1 Comparison Using Flappy Bird
Luqman gives the same short prompt to the products labeled GPT Astra and Fable 5.1 to create an HTML Flappy Bird clone, planning to show both games and collect votes before revealing the labels to reduce bias. He initially feels Astra appears faster but gives no final result here. He also explicitly says he does not know whether GPT, Codex, or Claude was the underlying model that produced at least one displayed game, so the product label is not confirmed model-level attribution. He further discusses why creativity, critical thinking, or the quality of a marketing strategy is difficult to judge from a single output and may require longer-term validation.
- 0:14:00
Why an Assistant Can Feel Familiar and Candid Over Time
Luqman and a participant share the impression that Claude understands context and preferences better after extended use and may challenge the user rather than merely agree. Luqman attributes that feeling to memory that gathers conversational information and applies it across sessions; these are attributed experiences and observations, not a validated comparative test.
- 0:20:32
Expectations for GTA 6’s Impact and Broad Audience
A brief opinion that GTA 6’s long development and strong interest across a broad age range could give its launch major economic impact; this was presented as speculation, not an established fact.
- 0:22:31
Organizing Agents Like a Company and Validating Their Roles
An explanation of designing a multi-agent system as an organizational chart, with executives assigned distinct personalities, responsibilities, and departments. The next step is interviewing the agents to verify that each understands its goal, specialty, and connections to the wider system, whose interdependencies can become as complex as a real company.
- 0:24:09
Speed and Tool Philosophy as Comparison Criteria
While waiting for the test to finish, completion speed was treated as an important criterion. The discussion also noted that AI tools can embody different working “philosophies,” including how they approach workflows, files, and memory. Some tool names were unstable in the captions, so detailed results are not attributed where the evidence is unclear.
- 0:26:22
Blind GPT Astra vs Fable 5.1 Comparison Using a Flappy Bird-Style Game
A similar prompt was sent to GPT Astra and Fable 5.1 to generate a Flappy Bird-style game, and the two outputs were initially shown without identities to reduce audience bias. Both were tried for gameplay, sound effects, and ease of control. The speaker generally preferred the left version during the test, while one participant considered the right version cleaner; the speaker then believed the left was Fable and the right was Astra. This remained a limited subjective comparison, not a comprehensive verdict on either model.
- 0:30:15
Comparing Design Quality and Gameplay Challenge
In a same-prompt generated-game comparison, one result appeared visually cleaner and easier, while the other offered more challenge and physics that the speaker considered better. The conclusion was that the judgment is subjective, with no outright winner because ease, challenge, and visual design can favor different results.
- 0:34:24
How Can Ideation and Critical Thinking Be Tested?
The speaker discusses how difficult it is to distinguish models that produce similar design work and argues that a more meaningful test would examine unexpected ideation and thinking beyond familiar patterns. These abilities are presented as harder to measure than direct output comparisons or ideas reproduced from training.
- 0:36:24
Testing Models Through a Head-to-Head Debate
After the observed run did not show a clear speed difference, the discussion suggests asking each model to design a test for desired abilities and then running those tests in fresh conversations. Another proposed method is to assign opposing positions on a controversial question and have the models debate, as a possible probe of argumentation and critical thinking rather than a definitive verdict.
- 0:37:43
Why Benchmarks Alone May Not Be Enough
The speaker wants a practical comparison that helps the audience understand differences between Astra and Fable 5.1, while expressing reservations about relying on benchmarks alone and noting that benchmark creators also discuss their limitations.
- 0:39:47
Matching Model Tiers for a Fair Comparison
The speaker raises a methodological question about which Fable and Astra tiers should be compared: whether their highest tiers should face each other or another tier should be used. The excerpt identifies a fairness issue in the comparison setup but ends before resolving it.
- 0:42:08
Can Models Think Outside the Box?
The speaker discusses the view that current models need clear context and constraints, then presents—explicitly as an unsettled idea attributed to others—the possibility that future AGI systems could think more independently. He stresses that such a stage is hard to imagine and that circulating predictions about AI surpassing human intelligence concern him.
- 0:44:01
A Blind SVG Camel-Drawing Test
GPT Astra and Fable 5.1 were asked to draw a camel using inline SVG only, with output identities hidden as much as possible to reduce bias. Audience preferences split between the two drawings, and the speaker considered the result subjective and inconclusive. He also observed that Fable finished this task before Astra, without treating that as a comprehensive performance measure.
Got a question about something in this stream?
Ask in the group