Amazon and Booking.com combine distillation and Medusa to speed up travel LLMs
AIAmazon Science reports that combining knowledge distillation with Medusa decoding let a 1.1-billion-parameter TinyLlama student match its 70-billion-parameter Llama-3.1 teacher while running 10 to 20 times as fast. The work was tested with Booking.com on the AI Trip Planner, Smart Filters, and accommodation description generation, and the authors say it maintained quality at less than one-third the cost.
AIWhy it matters.
The post explains how distillation and Medusa decoding were combined to cut LLM cost and latency for travel features, with concrete measured results for each use case.
Source details
Amazon Science · amazon.sciencePublished · added here
Original title: Speeding up language models for travel applications