#Multimodal

AIThis Next Token episode discusses Personal Agents, including Dots in Codex, memory and cloud computer permissions, and whether agents should act as assistants or digital twins. The hosts also cover Instinct's booking and business-travel model, hands-on projects built with Opus 5.5, and whether software, games, and hardware could become open source as AI makes rewriting easier.

AIBen Affleck described how visual effects work has long used machine learning, including convolutional neural networks that analyze image tensors to detect edges and features. He said these patterns help separate subjects from green screens and insert new backgrounds, and he called transformers the more advanced successors to those earlier methods.

AIBen Affleck says he writes Python, understands convolutional neural networks, and has worked extensively with GPUs. He also says he used his celebrity status to get private looks at Google and OpenAI's video models. The post asks whether he uses Veo, Omni, or Nano Banana, but the source does not confirm any of these.

AIThomas Wolf's post is a short, playful reply: "how do you like them convolutions," apparently referencing Ben Affleck's comments on convolutional neural networks. The quoted context reports Affleck describing his Python scripting, understanding of CNNs and tensors, GPU work, and private looks at Google and OpenAI's video models.

AIOn a livestream, Arthur Soroken interviews Amir and Ali Sadeghian, co-founders of Astrocade, about how generative AI and Gemini multimodal models are reshaping game development.

AIBoris Power says GPT-6 can work autonomously on 3D design for hours while its results keep improving, a gap other models failed to match because they could not recover from mistakes. Design Arena reports GPT-6 Astra took #1 on four leaderboards, including 3D Design at 1484 and Frontend at 1397, a month after release.

AIFelix Rieseberg, who posts from Anthropic's account, says this is his favorite Claude music video so far. He prefers Claude as a human-directed tool for making art rather than generating art unattended, though he enjoyed this one, which he calls a little meta.

AIEthan Mollick says he was wrong to think coordinating AI agents would require careful human-designed management structures. He points to personal agents like dots and Muse, and to a swarm of thousands of OpenAI agents that solved a Navier-Stokes problem in 88 hours with thin coordination. He argues many management problems stem from human limits, which agents lack, so people should mainly guide direction while agents handle organizing.

AILucas Beyer says he could build a multimodal model that scores how well any freeform text fits a given image, with calibrated scores and sigmoid-based yes/no judgments. He floats raising roughly xxxM in funding, open-weighting the model, and keeping it under 1B parameters, perhaps around 400M.

AIDeedy argues every open source repository should have a roughly seven-minute explainer video like the one made for SQLite, covering its purpose, a high-level code map, a query's path through the codebase, core abstractions, and a real execution trace including join-order query planning. He says the video was generated with Opus 5.5 and Gemini 3.8 TTS, and he expresses amazement at how coherent and capable the model is.

AIDeepMind has published an essay arguing that AGI will emerge through "cooperative interactions among models, tools, institutions, and human participants" rather than from a single winning AI. The commentary supports the collective framing but rejects treating AI agents as having their own theory of mind, arguing that creating new moral subjects should remain humanity's remit.

AIRunway co-founder and Co-CEO Alex Germanidis explains why he sees world models as the endgame for AI video, with applications in robotics simulation and real-time generation. He also discusses how Sora pushed Runway into an intense competitive sprint and his vision of a fully neural operating system where interfaces are generated as pixels rather than HTML and code.

AIHowie.serious says Opus 5.5 with only medium thinking enabled produced a video of impressive quality, suggesting a grim outlook for human video work. The quoted post says the model made a promo video for the Applore app in a few minutes, and the author calls the result remarkable.

AIThey show how much becomes possible with ultra-fast, multimodal models that are cheap enough to use freely. We are so focused on coding agents that it’s easy to forget how much else AI can do.

AIEthan Mollick says GPT-6 Astra and Fable 5.1 already enable transformative impact and can reliably handle weeks of human work when properly guided. He cites GPT-6 Astra turning the 1977 text adventure Zork into a 3D action-adventure game and Fable 5.1 reconstructing Umberto Eco's Milan library in 3D from videos, photos, and catalogues.

AIAlibaba Wan's post advises beginners making AI video to design a viewing experience and tell an emotion-triggering story rather than chasing the prettiest frame. It notes a single 30-second shot can be enough, citing the filmmaker behind Soulscape and Johnny Mai from Alibaba Cloud. The post promotes bringing Wan3.0 to teams via a sign-up form.

AISenseTime chief scientist Lin Dahua argues that native multimodal AI, which processes language, vision and other information in one shared model, is essential for AI to move beyond coding into industries and the physical world. SenseTime released the open-source SenseNova U1 in April and U1.5 Lite nearly four months later, and reported first-half 2026 revenue of RMB 2.91 billion, up 23.4% year-on-year. Lin's claim that a breakthrough could come within two years is the source's prediction, not a confirmed result.

AIIn a Paint recreation test, GPT-5.6 Astra built the image from layered geometric shapes, while Qwen3.8 Max worked pixel by pixel. Qwen's output looks closer to the original, but Raschka argues this single example does not show either model generalizes better or has stronger computer-use or visual understanding, and it illustrates how benchmarks comparing only final results can be misleading.

AIAt Guotai Junan Securities' 2026 Autumn Conference, SenseTime's Head of Capital Markets Philip Wong laid out three shifts reshaping AI: from single-modal to native multimodal, from token consumption to task delivery, and from single-point models to system-level full-stack capabilities. The post presents SenseTime's "One Model + One Token Factory + One Agent Harness" framework as built for these shifts.

AIWorld Labs co-founders discuss Atlas, a world model for spatial intelligence, as the key to unifying pixel-level generation and reconstruction. The source says Atlas can digitally capture a 3D representation of a space from three photos, where previously 100 to 300 were needed.

AIAndrej Karpathy gave Claude Opus 5 the first paragraph of Lord of the Rings with a 1M token budget and asked for a Three.js render. Opus spent about two hours writing 5500 lines of code that procedurally renders the story, which Karpathy calls janky but fun. He notes the model struggled to audit its work because it cannot efficiently perceive video or play the resulting game, relying on slow screenshots that led to several errors.

AIAt RAAIS, DeepMind VP of Research Raia Hadsell argued that the field focuses too much on language and should apply large-model training to worlds, robots, biology, and weather. The article cites DeepMind's DiffusionGemma, a 26-billion-parameter open text model that generates blocks by denoising rather than one token at a time, and the Genie-3 world model, which runs in real time for several minutes. It also describes world models as a source of synthetic training data for robots.

AI…refrigerator magnets." prompt injection test.

AIAt BAAI's "Reconstructing the World — Large Model Summit," President Wang Zhongyuan joined Zhu Jun, Luo Fuli, Liu Zhiyuan, An Bo, and other industry leaders in a roundtable. They discussed frontier model capability evolution, AI self-evolution, multimodality, and world models, with a focus on bridging the digital and physical worlds.

AIAs everyone talks about AI and automation, human creativity, story telling and productivity are even more important and essential to our society. @theworldlabs is founded on the premise of empowering human ingenuity and productivity. We are very grateful to be able to work with people like you! 🙏🌐

AIYou should be able “speak to” them not just with text chats, but by gesturing at a screen together, talking live, etc.

AIAt Google I/O, creators behind Flow, Project Genie, and Google Flow Music said human imagination, not the technology itself, shapes new storytelling. Designers Khyati Trehan and Kaloyan blend traditional design knowledge with vibe-coding to build Google Flow Tools, with Trehan saying that if the right tool doesn't exist, she can make it. Google Labs points users to Google Flow, Project Genie, and Google Flow Music at labs.google.

AI…in the new world We are at Step 1. Interaction Models are great real-time collaborative tools for humans. Here's a preview:

AIAndrej Karpathy says asking an LLM to structure its response as HTML and viewing it in a browser works well, and that slideshows have also worked for him. He argues vision is the preferred AI output channel, outlining a progression from raw text and markdown toward HTML and eventually interactive neural videos, while input methods like pointing and gesturing still need improvement.

AIAnthropic's Claude Sonnet 4.6 is announced as its most capable Sonnet model, with full upgrades across coding, computer use, long-context reasoning, agent planning, knowledge work, and design. It also features a 1M token context window in beta. The author notes that the model is versatile across classification, coding, computer use, and autonomous agents by adjusting effort and thinking modes.

AIChip Huyen praised projects at last weekend's Agentic Hackathon, which hosted by MongoDB and Cerebral Valley, where she served as a judge. Teams tackled long-running tasks such as memory management, recovery from mid-task failures, and consistency across steps and sub-agents, along with adaptive retrieval across databases, search indices, and websites. Finalist demos are scheduled in San Francisco tomorrow, with talks by Douglas Eck.