#Multimodal

AIJerry Liu argues that OCR, long dominated by brittle legacy systems, can be solved accurately and cheaply by applying agentic intelligence. He says a properly tuned agentic OCR dynamically allocates extra compute to complex elements, reviews and corrects failures, and builds semantic meaning across the page. He contends frontier models are overengineered for this task in cost and latency yet still struggle with complex edge cases.

AIDevelopers are using OpenAI's Decisions API to route requests to the right model, tool, or agent and to turn scaled inputs into labels, rankings, and scores. The post also lists uses including analyzing images and video frames, choosing buttons or form actions from screenshots, flagging risky tool calls, and categorizing large datasets.

AIOpenAI's GPT-6 Luna accepts text and image inputs and supports three output types: predicates that estimate the probability a statement is true, choices that select from predefined options with confidence scores, and scores that evaluate an input against a numeric range.

AIfal is now available inside ChatGPT and Codex, letting users generate images and videos without leaving the chat. Generated media can be browsed directly in the conversation, and users can access their fal Media Library from ChatGPT. The library can be pinned to the sidebar for quick access to assets in any chat.

AIGoogle's Gemini Live now includes Guided Vision, built with blind and low-vision community input, offering conversational real-time visual assistance. Users can share their camera to receive dynamic audio descriptions and verbal cues to help them explore their surroundings.

AIGoogle Research will demo Co-Director, a hierarchical multi-agent framework that optimizes video generation and consistency for long-form storytelling, at the #COLM2026 Google booth #107 today at 1:00 PM PT. The demo showcases interactive cinematic narratives, and the team's blog post details the approach.

AIvLLM announced day-0 support for EmbeddingGemma 2 from Google DeepMind, a bidirectional omni-modal embedding model that maps text, image, audio, video, and interleaved inputs into one vector space. Users can try it with the latest vLLM nightly build using the command vllm serve google/embeddinggemma-2 --runner pooling. The quoted Google post says the model is built on the Gemma 4 architecture and released under Apache 2.0.

AIPerplexity says its open-weights multimodal decision model pplx-decider-v1.1-27b is now available and scores highest on Hugging Face's new Decision Index 0.3 benchmark. The model costs $0.02 per million input tokens, half the price of v1.

AIGoogle DeepMind's Nano Banana 2.1, Google's latest image generation and editing model, is now available on Design Arena. The model brings improvements in visual design, mask-based editing, and subject consistency, aiming to produce more natural-looking images with greater control across editing workflows.

AIThe xai-cookbook example uses Grok to generate a Tailwind React component from a dropped-in screenshot. The page then screenshots its own render, overlays it on the original, and feeds the mismatches back so Grok can refine the layout, raising the match from 60% to 77% in one run.

AIxAI's storyboard-to-film example uses grok-4.7 to plan four shots from a one-line premise, then Grok Imagine draws and animates each one with text-to-speech narration. Every shot is an edit of the first keyframe, which keeps the main character consistent across scenes. The code is available in the xai-cookbook repository on GitHub.

AIGoogle's EmbeddingGemma 2 is an open embeddings model for on-device use that covers code, image, video, audio, and text. It comes in modular sizes from 270M text/code to 740M full multimodal, supports Matryoshka truncation down to 128 dimensions, and reports a 14% gain on MTEB Code over v1 under an Apache 2.0 license. The author's post highlights the release and a Hugging Face demo, while the benchmark table compares it with several models.

AIGoogle released EmbeddingGemma 2, a 740M-parameter open embedding model under Apache 2.0 that combines a 270M text model with vision (170M) and audio (300M) encoders. The 270M text model can run locally with 0.5GB of RAM, and the full multimodal model with 1GB, and Unsloth provides GGUF files and fine-tuning support.

AISGLang now supports EmbeddingGemma 2 from Google DeepMind on day zero. The multimodal embedding model maps text, code, images, video, and audio into one shared 768d space, with 8K context and 100+ languages. Its modular encoders range from a 270M text-only footprint to 740M for all modalities, with Matryoshka embeddings.

AIGoogle's EmbeddingGemma 2 is a 740M-parameter embedding model that outperforms some models more than twice its size while using about 191MB to 567MB of active RAM. It offers an 8K context window, 4x larger than the first generation, and can process up to 5.5 minutes of audio, 29 images, or 58 video frames in one pass.

AIGoogle Gemma announces EmbeddingGemma 2, a lightweight embedding model that maps text, code, images, video, and audio into a single unified embedding space. The model has a 740M parameter form factor with modular encoders, Matryoshka Representation Learning dimensions from 768 down to 128, and an 8K context window that is 4x larger than the text-only EmbeddingGemma. It is released under the commercially permissive Apache 2.0 license.

AIGoogle has released EmbeddingGemma 2, a compact 740M-parameter model built for on-device and edge applications. It natively embeds text, images, audio, and video into one shared space, enabling cross-format search without manual organization, translation, or labeling.

AIGoogle's new multimodal embedding setup processes image, audio, and video entirely offline with zero server calls. Its modular design lets developers drop unused vision and audio components to save memory, and flexible dimension sizes cut local database storage by up to 6x. It can also pair with Gemma 4 to build RAG pipelines with minimal memory and processing requirements.

AIGoogle DeepMind released EmbeddingGemma 2, a 740M-parameter embedding model, under an Apache 2.0 license. The post says it is competitive across benchmarks and outperforms some specialist models more than twice its size, and that developers can use it for multimodal search or pair it with Gemma 4 for on-device RAG. Weights are available on Hugging Face and Kaggle.

AIGoogle DeepMind introduced EmbeddingGemma 2, its first natively multimodal open model for on-device embeddings. The model expands beyond text to unify code, images, audio, and video in a shared embedding space.

AIGoogle introduces EmbeddingGemma 2, its first open, natively multimodal embedding model, covering text, code, image, video, and audio tasks. It has a 740M parameter form factor, is positioned for offline, privacy-first RAG when paired with Gemma 4, and the post claims it outperforms some specialist models more than twice its size. Weights are available now on Hugging Face.

AIMistral announced Mistral Large 4, which it describes as a natively multimodal model with 1T parameters and 49B active. Mistral says it is available via API now, with open weights to follow at the end of October, and a Hugging Face page is listed for the release.

Why it matters: The quoted Mistral announcement gives specific size, activation, and API details, and the open-weights timing matters for teams weighing open model options.

AIKling AI is showcasing videos creators have made with its Kling 4.0 and Kling 4.0 Flash models. The post is promotional and includes no specifications, benchmarks, pricing, or feature details.

AISimon Willison reports that Mistral can now generate his pelican SVG test, shared via a Markdown SVG renderer. The post links to a rendered result but gives no benchmark or scoring details. Background from Mistral's own announcement describes Mistral Large 4 as a 1T-parameter, natively multimodal model with 49B active parameters, available via API today and with open weights planned for end of October.

AIMistral announced Mistral Large 4, a natively multimodal model with 1T parameters and 49B active, now available via API. Open weights are scheduled for release at the end of October, with a countdown page on Hugging Face showing October 31, 2026.