#Multimodal

AITencent has open-sourced Youtu-Parsing-Omni, a 5B-parameter omni-modal model that outputs a single structured JSON covering layout, text, tables, formulas, ASR, OCR, and video segments. It scores 96.96 Overall on OmniDocBench, the highest among the compared models, and ships with weights on Hugging Face, a vLLM plugin, and inference examples.

AINVIDIA published the nvidia/agile_one_s_place_ssd_n17_24050 repository on Hugging Face, containing a GR00T N1.7 checkpoint 24050 model for placing an SSD with three cameras. The repository includes five ONNX graphs with external tensor files and two existing TensorRT BF16 engines, republished without retraining, re-export, or engine rebuild. Engine compatibility depends on the target GPU and TensorRT environment, and the files are not a robot deployment or safety qualification.

AIXiaomi released the MiMo-V2.5-TTS Series, three speech synthesis models for stock voices, voice design, and voice cloning. The models accept natural-language style instructions and inline audio tags, and the source says the three models are free of charge for a limited time on the Xiaomi MiMo API platform. Xiaomi also open-sourced integration Skills for agent applications on GitHub.

Why it matters: The release shows how a TTS family adds style instructions, inline audio tags, and voice design or cloning to speech synthesis, which matters for agent and creative workflows.

AIDevelopers are pairing frontier AI models with NVIDIA Omniverse libraries to turn simulation ideas into working applications, from humanoid warehouse simulators to autonomous-driving test environments. In the examples, developers direct AI agents through natural-language instructions and review results, while Omniverse provides GPU-accelerated physics, rendering and sensor simulation. One experiment reported a simulated Unitree G1 humanoid clearing a hurdle in 64 of 100 trials.

AIHospitals, academic centers, medtech firms, and pharma companies all face the same obstacle: imaging data is locked in clinical systems and hard to share. The EXAM study across 20 institutions showed federated learning, which shares model weights rather than patient data, improved AUC by 16% on average. Collaboration remains difficult due to scanner and protocol heterogeneity, privacy governance, and the lack of a common data substrate.

AIMeta is donating 1,000 Ray-Ban Meta AI glasses to four Singapore organisations serving people with disabilities, alongside a US$30,000 grant for accessibility training. The glasses help users who are blind or have low vision read text, identify objects and describe their surroundings. The grant will fund a free curriculum from the Singapore Association of the Visually Handicapped on using the glasses safely in daily life.

AIJohns Hopkins astrophysicist Brice Ménard, working as an Anthropic researcher, used Claude Science to produce the first complete map of the sky in ultraviolet light. Claude orchestrated agents to merge GALEX, Swift, and FIMS/SPEAR data, then predicted roughly a third of the sky that no UV telescope had observed, using relationships to visible, infrared, and radio data. Hidden test regions were reconstructed to within about 10% of real measurements, and each pixel is labeled measured or predicted with uncertainty estimates.

Why it matters: The post shows how an astrophysicist used Claude Science agents to merge UV surveys and predict missing sky regions, with a validation step that makes the method reusable.

AILuma announced that Claude Motion animations can now open directly in Luma through an MCP connection, letting creators restyle them and reframe them to 9:16, 1:1, 4:3, or 21:9. Claude Motion, currently in beta on Claude Team and Enterprise plans, generates animated explainers from prompts, while Luma's Ray and Uni video models produce final video files.

AIApple researchers introduced Normalizing Trajectory Models (NTM), which model each reverse diffusion step as a conditional normalizing flow trained with exact likelihood. The model matches or outperforms strong image generation baselines on text-to-image benchmarks in just four sampling steps while retaining exact likelihood over the generative trajectory.

AIOpenAI says ChatGPT now offers Intelligent UI, which returns answers with fully interactive user interfaces. GPT-6 is rolling out in ChatGPT, powered by GPT-6 Sol for Plus, Pro, Business, and Enterprise tiers and GPT-6 Luna for Free and Go tiers. The rollout begins globally today in the Chat tab for paid tiers, with Free and Go tiers following tomorrow.

AILiquid AI released two open-weight decision models, d1-3B and d1-omni-600M (experimental), built on its Liquid Foundation Models and available on Hugging Face. d1-3B scores 48.57 on the Decision Index 0.2.1, the highest among decision models under 10B parameters, and answers a question in 16 ms on an NVIDIA Jetson AGX Thor and under 50 ms on a Jetson Orin Nano. The models support text and images (d1-3B) or text with image or audio (d1-omni-600M).

AIGoogle is making its SynthID Detector available globally in English, letting anyone check whether an image, video, or audio file was made with AI from Google or partners including OpenAI, NVIDIA, Kakao, and soon Apple. The tool joins built-in verification in Search, the Gemini app, and Chrome, which now handle over 1 million requests daily. Google says SynthID has watermarked over 180 billion images and videos and 240,000 years of audio.

Why it matters: The source specifies which vendors' AI media the detector checks, helping readers judge how far the verification covers content they encounter online.

AIThe Technology Innovation Institute introduces Falcon-ASR, a 1.6 billion parameter speech recognition model for Arabic with a focus on the Emirati dialect. On six Arabic test sets it reports an average word error rate of 20.92%, versus 23.17% for the best published leaderboard result it compared against. The model also transcribes English, French, Spanish and Portuguese with the same weights, and a demo Space is available while API access and native apps are planned.

AILiquid AI released two open-weight d1 decision models, d1-3B and d1-omni-600M, on Hugging Face. d1-3B scores 48.57 on the Decision Index v0.2.1 public split and answers a single question in 8 ms on an NVIDIA GeForce RTX 4090 and 50 ms on a Jetson Orin Nano. d1-omni-600M is an experimental checkpoint that handles text with images or audio and scores 15.95 on the same index.

Why it matters: The release pairs open-weight decision models with measured latency across Apple, NVIDIA, and Jetson hardware, showing how edge deployment changes what is practical.

AIGoogle's Gemini Nano Banana 2.1 image generation and editing model is now available through ComfyUI Partner Nodes, succeeding Nano Banana 2 with a balance of price and performance. It accepts up to 14 reference images, outputs images up to 4K, and offers Minimal, Medium, and High thinking levels plus search grounding and 9:21 aspect ratio support.

AIGoogle DeepMind has released EmbeddingGemma 2, an open 740 million parameter model that maps text, images, audio, and video into one embedding space. It is built on the Gemma 4 architecture under an Apache 2.0 license and supports an 8K token context window. The company reports a code benchmark gain from 68.76 to 78.68 on MTEB Code and says the model can run on-device with about 567MB of active RAM for the full multimodal version on a Google Pixel 11 Pro.

Why it matters: The release shows how a 740M-parameter embedding model can cover text, code, images, audio, and video on local hardware, with memory and storage figures to compare against other on-device options.

AIGoogle researchers, working with Northwestern Medicine and Jacaranda Health, trained healthcare workers to perform "blind sweep" ultrasounds analyzed by machine learning models. The models estimated gestational age and fetal presentation as accurately as a trained sonographer in a study of 1,000 mothers each in Nairobi and Chicago. The AI processes results on the device, so it needs no electricity supply or Wi-Fi.

AIGoogle DeepMind has released EmbeddingGemma 2, a 740-million-parameter embedding model that maps text, images, audio, and video into a shared space and runs on local hardware under an Apache 2.0 license. Matryoshka Representation Learning lets developers truncate output vectors from 768 dimensions to 512, 256, or 128, and the model supports an 8K-token context window. The model weights are available on Hugging Face and Kaggle, with Gemini Enterprise Agent Platform availability coming soon.

Why it matters: The release shows how a 740M-parameter multimodal embedder runs locally with a 768-to-128 dimension truncation option, useful for judging on-device retrieval designs.

AIMistral AI launched a public preview API for Mistral Large 4, a 1 trillion-parameter natively multimodal model with 52 billion active parameters, and says it will release the weights by the end of the month. The company reports 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, 28.3% on Terminal-Bench 4, and 59.9% on AutomationBench. The model was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's datacenters in Europe.

Why it matters: The post gives benchmark figures and a weights timeline for an open-weight model, letting readers compare it with other open models and judge its access terms.

AIThe guide offers a prompt structure for cyberpunk visuals built from subject, environment, lighting, camera, style, and quality modifiers, with magenta and cyan neon, rain-slicked reflections, and fog named as key mood elements. It presents 15 ready-to-use prompts and argues that free tools suit testing directions, while full access is needed for commercial campaigns.

AIGoogle released EmbeddingGemma 2, an open embedding model under the Apache 2.0 license that maps text, code, images, video, and audio into a shared 768-dimensional space. Developers can load a 270M-parameter text and code setup, or add vision and audio encoders up to a 740M-parameter full multimodal model. Matryoshka truncation to 256 or 128 dimensions reduces vector storage, with the guide noting quality losses on image, video, and speech retrieval at lower dimensions.

Why it matters: The guide gives concrete encoder sizes and dimension-storage tradeoffs, showing how to choose a configuration for text, code, image, video, and audio retrieval.

AIGoogle DeepMind launched EmbeddingGemma 2, an open-weight 740M parameter model that maps text, images, video frames, and audio into one vector space. The model can run on-device, with about 567MB active RAM for the full multimodal model on a Google Pixel 11 Pro, and is available through Google AI Edge Gallery, Google AI Edge Foresight on Mac, and MediaPipe Tasks, with ML Kit support coming in the weeks ahead.

Why it matters: The post names concrete on-device apps, memory footprints, and latency figures, showing how a multimodal embedding model can power local search without cloud calls.

AILiquidAI has released d1-omni-600M on Hugging Face, a 587M-parameter model that answers named yes/no, choice and score questions over text, images or up to 30 seconds of speech in a single forward pass. It returns typed answers with zero output tokens by reading the model's distribution over options, and is built on LFM2.5-Encoder-350M with a 16,384-token context length. The model is not a chat model and does not generate text.

AILiquid AI has released d1-3B, a 3B parameter multimodal model post-trained to return calibrated, typed answers to yes/no, choice, and score questions in one forward pass. The source reports a Decision Index 0.2.1 score of 48.57, the highest among models under 10B in its table, and 8 ms per decision on an NVIDIA RTX 4090.

Why it matters: The source gives benchmark scores against named peer models and edge latency figures across several hardware targets, helping readers judge fit for on-device decision pipelines.

AILiquid AI introduces d1, its first decision model, now accepting both text and images. The company says d1 matches or beats GPT-6.1 Sol on four of six tested applications, at 19x to 200x lower cost and with faster answers on every task. d1 is available on the Liquid AI API and through Vercel and OpenRouter, with text-only support on those two platforms for now.

Why it matters: The post gives benchmark comparisons against named models along with per-token pricing and latency figures, which makes the cost and speed tradeoff checkable.

AIGoogle's September 2026 roundup highlights Gemini 4 Argon, a frontier model with a 1-million-token output limit aimed at complex tasks such as cybersecurity defense. Argon is rolling out first to trusted cyber defenders through the Fairwind Program, with developer, enterprise, and consumer access to follow after guardrail feedback. The post also covers Gemini 3.8 Flash, Connected Apps in Gemini, and WeatherNext 3.

AIApple Machine Learning Research reports that discrete diffusion steps match the training distribution only when simultaneously written token positions are conditionally independent given already-fixed tokens. The authors show that per-position distributions cannot determine such dependence, and on the synthetic ScanAndAdd task, confidence-ranked groups of two or more positions were dependent and produced a generated distribution 29 times the sampling-noise floor in total variation.

AINVIDIA has released PixelUMM, an encoder-free unified multimodal model with 15,199,672,064 parameters that handles text, image, and video understanding and generation directly in pixel space. It represents images as 16-by-16 RGB pixel patches on a Qwen3-8B language backbone, with iterative denoising for generation. The checkpoint is licensed for non-commercial research or evaluation only, while the source code is under Apache License 2.0.

AIGoogle is launching Guided Vision in Gemini Live on compatible Android devices, letting users share their camera for spoken descriptions and follow-up questions. The model was trained with Aira on tens of thousands of hours of visual interpretation and tested by more than 1,000 members of Aira's Trusted Tester network. The feature is not a medical device, mobility aid, or navigation tool, and it requires Android 9 or later.

Why it matters: The launch shows how a real-time visual model was trained and tested with blind and low-vision users, a practical reference for accessibility-focused AI design.

AIMeta names Ranveer Singh the first Brand Ambassador for Ray-Ban and Ray-Ban Meta in India and launches Ray-Ban Meta (Gen 3) there, starting at INR 44,300. Gen 3 offers up to nine hours of battery life, a 12 MP camera, and a 6-mic array that cuts more than 90% of background noise. Ray-Ban Meta Audio, weighing 43 grams, is coming soon.

AICloudflare's AI Search is now generally available, adding native image embeddings for visual retrieval, OCR for scanned PDFs, and a 10 MiB file limit up from 4 MiB. Billing begins November 1, 2026, charged per ingested token, stored GB-month, and query, with a free monthly allotment on all Workers plans.

AIManus 2.0 introduces Video Editor, which lets users refine videos Manus generates, including changes to music, captions, and cut timing, without regenerating the entire video. The article describes Manus creating explainers, launch films, and animations from a single prompt, drawing on web search, video models such as Seedance 2.5, and code for motion graphics.

AISuno has launched Speech in beta, which it describes as the first audio model that generates voice and music together as one cohesive track. Users type an idea or existing text, then describe the voice and musical style they want, and the beta is now open to everyone after a month of testing with a small group.

AIGoogle DeepMind announced Gemini 4 Argon, rolling out first to trusted cyber defenders through its Fairwind Program. Argon will launch at an introductory price of $2 per million input tokens and $10 per million output tokens, with output limits raised to 1M tokens. The post cites a 77.9% score on DeepSWE v1.1 and 91.7% on LVBench, and says broad availability will follow safeguard testing.

Why it matters: The post pairs Argon's benchmark claims with the phased release, pricing, and safeguard details, helping readers weigh its frontier-level capabilities against its access limits.

AIKling 4.0 extends native single-pass video generation from 15 to 30 seconds and adds Multiple Keyframes supporting up to 10 keyframe images, versus Start & End Frames in Kling 3.0. It also expands reference inputs to up to 15 combined assets, including up to 5 videos totaling 30 seconds, and adds 10-bit HDR at 1080p and 4K. The all-new Kling 4.0 will officially launch in October, and Kling 4.0 Flash became available to a limited group of early-access users on September 28.

AIKling AI says Kling 4.0 will support native video generation from 3 to 30 seconds, with up to 10 keyframes for timing control. The full model is scheduled for October 2026, with early access to the new creation page rolling out from September 28.

AIGoogle Research introduced Diffusion Controller, a framework that treats image generation as a continuous control problem rather than separate inference-time guidance and fine-tuning fixes. Its lightweight add-on "steering damper" network keeps the base model frozen and works on black-box or gray-box models, and it outperformed the industry standard on human preference matching. In a Stable Diffusion v1.4 test, the fully unlocked version achieved a 90% win rate over the baseline.

AIThe article presents a four-part prompt structure (action verb, target element, desired result, protection instructions) for AI photo editing, saying it preserves approved work across platforms. It identifies three common failure causes: unmatched light direction, stacked edits in one prompt, and vague visual language. It states that simple skin retouching takes 2-3 minutes versus 15-30 minutes manually.

AIGoogle says Gemini 3.8 Flash, its most intelligent workhorse model, improves on 3.7 Flash in software engineering, agentic tasks, and multistep reasoning by running extra reasoning steps and calling tools iteratively. The post highlights four community builds, including a model rocket simulation, an animated ink-painting effect, a 3D dinosaur skeleton, and an interactive automatic transmission simulation. Developers can try the model through Google Antigravity and Google AI Studio.

AIShanghai AI Lab's InternLM released Intern-Decision-2B, a multimodal structured decision model fine-tuned from Qwen3.5-2B that returns calibrated answer distributions for multiple questions in one forward pass. It averages 84.68 across listed benchmarks with a 0.437 Brier score and 33.28 ms mean latency on a single RTX 4090. Model weights, a Python DecisionEngine API, and GitHub code are available, with support for up to 16 questions and eight images.