#Multimodal

AIWith Guided Vision turned on, Gemini Live verbally prompts users to pan, tilt, or step back when their camera framing is too high, too close, or off to the side. These prompts help Gemini get the clear visual context it needs to answer questions.

AIGoogle's Gemini Live now includes Guided Vision, built with blind and low-vision community input, offering conversational real-time visual assistance. Users can share their camera to receive dynamic audio descriptions and verbal cues to help them explore their surroundings.

AIGoogle Research will demo Co-Director, a hierarchical multi-agent framework that optimizes video generation and consistency for long-form storytelling, at the #COLM2026 Google booth #107 today at 1:00 PM PT. The demo showcases interactive cinematic narratives, and the team's blog post details the approach.

AIGoogle releases EmbeddingGemma 2, its first native multimodal embedding model, built on Gemma 4 under Apache 2.0. It embeds over 100 languages, code, images, audio, and video into one vector, with an 8,192-token context and four sizes from 270M to 740M parameters. Matryoshka output dimensions of 768, 512, 256, or 128 are supported, and the model is available in Sentence Transformers and LiteRT-LM, with a reported 14% gain on MTEB Code.

Why it matters: The release extends an embedding model to text, code, images, audio, and video in one vector, a useful option for retrieval systems that mix media types.

AIvLLM announced day-0 support for EmbeddingGemma 2 from Google DeepMind, a bidirectional omni-modal embedding model that maps text, image, audio, video, and interleaved inputs into one vector space. Users can try it with the latest vLLM nightly build using the command vllm serve google/embeddinggemma-2 --runner pooling. The quoted Google post says the model is built on the Gemma 4 architecture and released under Apache 2.0.

AIGoogle researchers, working with Northwestern Medicine and Jacaranda Health, trained healthcare workers to perform "blind sweep" ultrasounds analyzed by machine learning models. The models estimated gestational age and fetal presentation as accurately as a trained sonographer in a study of 1,000 mothers each in Nairobi and Chicago. The AI processes results on the device, so it needs no electricity supply or Wi-Fi.

AIThe xai-cookbook example uses Grok to generate a Tailwind React component from a dropped-in screenshot. The page then screenshots its own render, overlays it on the original, and feeds the mismatches back so Grok can refine the layout, raising the match from 60% to 77% in one run.

AIxAI's storyboard-to-film example uses grok-4.7 to plan four shots from a one-line premise, then Grok Imagine draws and animates each one with text-to-speech narration. Every shot is an edit of the first keyframe, which keeps the main character consistent across scenes. The code is available in the xai-cookbook repository on GitHub.

AIBoris Cherny, who works at Anthropic, said he asked Opus 5.5 to build an interactive website companion for the latest Acquired episode on Home Depot. He noted that the watercolor illustrations on the site were also drawn by Claude.

AIGoogle's EmbeddingGemma 2 is an open embeddings model for on-device use that covers code, image, video, audio, and text. It comes in modular sizes from 270M text/code to 740M full multimodal, supports Matryoshka truncation down to 128 dimensions, and reports a 14% gain on MTEB Code over v1 under an Apache 2.0 license. The author's post highlights the release and a Hugging Face demo, while the benchmark table compares it with several models.

AIGoogle released EmbeddingGemma 2, a 740M-parameter open embedding model under Apache 2.0 that combines a 270M text model with vision (170M) and audio (300M) encoders. The 270M text model can run locally with 0.5GB of RAM, and the full multimodal model with 1GB, and Unsloth provides GGUF files and fine-tuning support.

AIGoogle's EmbeddingGemma 2 is a 740M-parameter embedding model that outperforms some models more than twice its size while using about 191MB to 567MB of active RAM. It offers an 8K context window, 4x larger than the first generation, and can process up to 5.5 minutes of audio, 29 images, or 58 video frames in one pass.

AIGoogle has released EmbeddingGemma 2, a compact 740M-parameter model built for on-device and edge applications. It natively embeds text, images, audio, and video into one shared space, enabling cross-format search without manual organization, translation, or labeling.

AIGoogle's new multimodal embedding setup processes image, audio, and video entirely offline with zero server calls. Its modular design lets developers drop unused vision and audio components to save memory, and flexible dimension sizes cut local database storage by up to 6x. It can also pair with Gemma 4 to build RAG pipelines with minimal memory and processing requirements.

AIGoogle DeepMind released EmbeddingGemma 2, a 740M-parameter embedding model, under an Apache 2.0 license. The post says it is competitive across benchmarks and outperforms some specialist models more than twice its size, and that developers can use it for multimodal search or pair it with Gemma 4 for on-device RAG. Weights are available on Hugging Face and Kaggle.

AIGoogle DeepMind introduced EmbeddingGemma 2, its first natively multimodal open model for on-device embeddings. The model expands beyond text to unify code, images, audio, and video in a shared embedding space.

AIGoogle introduces EmbeddingGemma 2, its first open, natively multimodal embedding model, covering text, code, image, video, and audio tasks. It has a 740M parameter form factor, is positioned for offline, privacy-first RAG when paired with Gemma 4, and the post claims it outperforms some specialist models more than twice its size. Weights are available now on Hugging Face.

AIGoogle DeepMind has released EmbeddingGemma 2, a 740-million-parameter embedding model that maps text, images, audio, and video into a shared space and runs on local hardware under an Apache 2.0 license. Matryoshka Representation Learning lets developers truncate output vectors from 768 dimensions to 512, 256, or 128, and the model supports an 8K-token context window. The model weights are available on Hugging Face and Kaggle, with Gemini Enterprise Agent Platform availability coming soon.

Why it matters: The release shows how a 740M-parameter multimodal embedder runs locally with a 768-to-128 dimension truncation option, useful for judging on-device retrieval designs.

AIMistral announced Mistral Large 4, which it describes as a natively multimodal model with 1T parameters and 49B active. Mistral says it is available via API now, with open weights to follow at the end of October, and a Hugging Face page is listed for the release.

Why it matters: The quoted Mistral announcement gives specific size, activation, and API details, and the open-weights timing matters for teams weighing open model options.

AIKling AI is showcasing videos creators have made with its Kling 4.0 and Kling 4.0 Flash models. The post is promotional and includes no specifications, benchmarks, pricing, or feature details.

AISimon Willison reports that Mistral can now generate his pelican SVG test, shared via a Markdown SVG renderer. The post links to a rendered result but gives no benchmark or scoring details. Background from Mistral's own announcement describes Mistral Large 4 as a 1T-parameter, natively multimodal model with 49B active parameters, available via API today and with open weights planned for end of October.

AIMistral announced Mistral Large 4, a natively multimodal model with 1T parameters and 49B active, now available via API. Open weights are scheduled for release at the end of October, with a countdown page on Hugging Face showing October 31, 2026.

AIMistral has announced Mistral Large 4, a natively multimodal model with 1T total parameters and 49B active parameters. The company says it is the best open-weights model from the US or Europe on aggregated benchmarks, and that it is available via API today, with open weights due at the end of October.

AIMistral AI's Mistral Large 4 is listed on OpenRouter as a frontier multimodal model accepting text and image input. The listing says it is built for reasoning, coding, and agentic workloads and offers a 1M-token context window. The feed excerpt is truncated, so further details such as pricing or availability are not confirmed here.

AIJulien Chaumond reposted Mistral's announcement of Mistral Large 4, a 1T-parameter natively multimodal model with 49B active parameters. Mistral says it is available via API today, with open weights scheduled for release at the end of October, and is working privately with cybersecurity partners.

Why it matters: The post lays out Mistral Large 4's scale, multimodal design, and availability timeline, which helps readers gauge the open-weights landscape outside China.

AIArthur Mensch says Mistral trained its model on its own compute, and reinforcement learning shows no sign of saturating. The post accompanies Mistral's announcement of Mistral Large 4, a 1T-parameter natively multimodal model with 49B active parameters, available via API today and with open weights planned for end of October.

AIMistral's ML4 model matches the best open-weight models on DeepSWE, AutomationBench, and AA-Briefcase, and reaches state-of-the-art results on finance and legal workflows and complex multimodal grounding benchmarks. The post says it can navigate terminal workflows, work across spreadsheets, slides, and PDFs, and reason over scientific and multimodal tasks.

AIMistral has launched a preview of Mistral Large 4 (ML4), a 1T-parameter multimodal model with 49B active parameters. The company says it is the strongest open-weight model from the US or Europe on aggregated benchmarks and is available via API now, with open weights planned for the end of October.

Why it matters: The post gives parameter counts, a preview timeline, and an open-weights release date, which help readers judge how Mistral's model compares with other open-weight options.

AIMistral AI introduced Mistral Large 4, a natively multimodal model with 1T parameters and 49B active parameters. The company says it is the best open-weights model from the US or Europe on aggregated benchmarks and is available via API today, with open weights due at the end of October.

AIMistral AI launched a public preview API for Mistral Large 4, a 1 trillion-parameter natively multimodal model with 52 billion active parameters, and says it will release the weights by the end of the month. The company reports 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, 28.3% on Terminal-Bench 4, and 59.9% on AutomationBench. The model was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's datacenters in Europe.

Why it matters: The post gives benchmark figures and a weights timeline for an open-weight model, letting readers compare it with other open models and judge its access terms.

AIHuawei's Mate 90 Pro Max starts at 9,499 yuan for the 12GB+512GB model, and the 16GB+512GB Collector's Edition starts at 10,999 yuan. Its most notable upgrades are camera-related, including a "原片" (original-style) portrait option that reduces beautification and a 200-megapixel telephoto lens with about 4x optical zoom.

AIvLLM-Omni now supports Kandinsky 6.0 Video from launch day, with inference ready at release. Kandinsky 6.0 Video generates 5-second clips with synchronized audio and lip-sync from text or image inputs. Kandinsky's code and checkpoints are released under the MIT license, with Lite (3B) and Pro (29B) variants.

AIThe guide offers a prompt structure for cyberpunk visuals built from subject, environment, lighting, camera, style, and quality modifiers, with magenta and cyan neon, rain-slicked reflections, and fog named as key mood elements. It presents 15 ready-to-use prompts and argues that free tools suit testing directions, while full access is needed for commercial campaigns.

AIGoogle released EmbeddingGemma 2, an open embedding model under the Apache 2.0 license that maps text, code, images, video, and audio into a shared 768-dimensional space. Developers can load a 270M-parameter text and code setup, or add vision and audio encoders up to a 740M-parameter full multimodal model. Matryoshka truncation to 256 or 128 dimensions reduces vector storage, with the guide noting quality losses on image, video, and speech retrieval at lower dimensions.

Why it matters: The guide gives concrete encoder sizes and dimension-storage tradeoffs, showing how to choose a configuration for text, code, image, video, and audio retrieval.

AIGoogle DeepMind launched EmbeddingGemma 2, an open-weight 740M parameter model that maps text, images, video frames, and audio into one vector space. The model can run on-device, with about 567MB active RAM for the full multimodal model on a Google Pixel 11 Pro, and is available through Google AI Edge Gallery, Google AI Edge Foresight on Mac, and MediaPipe Tasks, with ML Kit support coming in the weeks ahead.

Why it matters: The post names concrete on-device apps, memory footprints, and latency figures, showing how a multimodal embedding model can power local search without cloud calls.

AIThe llama.cpp v0.6.0 release adds Clef support for text and vision, along with high-quality support for Qwen3.8-Flash-Next. It also brings a major Metal performance improvement and a new llama_batch_ext API, and the project website at llama.app has been refreshed.

AILiquid AI's d1 picks the next move by reading game screenshots, without text versions of the board. In Tetris, adding the screen raised cleared lines from 70 to 81, and in Wordle it solved 12/12 games in an average of 3.8 guesses.

AILiquid AI released d1 with vision support, accepting images, text, or both as inputs. In tests on six real applications, d1 matched or beat GPT-6.1 Sol on four while costing 19x to 200x less than both GPT-6.1 Sol and Claude Opus 5.5. It returns probabilities for yes/no, choice, or score questions in one forward pass, with text decisions in 200 to 300 ms.

AILiquid AI's vision-enabled decision model d1 inspects parts directly from camera images and is described as the best such model currently on the market. It reaches 85% to 97% accuracy across four VisA inspection tasks covering circuit boards, candles, cashews, and chewing gum. It understands each task from a short description without task-specific training.

AILiquidAI has released d1-omni-600M on Hugging Face, a 587M-parameter model that answers named yes/no, choice and score questions over text, images or up to 30 seconds of speech in a single forward pass. It returns typed answers with zero output tokens by reading the model's distribution over options, and is built on LFM2.5-Encoder-350M with a 16,384-token context length. The model is not a chat model and does not generate text.