are there studies on using chat interface for hotel discovery and booking or trip planning
Empirical studies across Human-Computer Interaction (HCI), AI benchmarking, and real-world Online Travel Agency (OTA) telemetry directly examine using chat interfaces for hotel discovery, trip planning, and booking.
Together, they validate every frustration you experienced:conversational interfaces work well for vague initial inspiration or simple post-booking customer service, but collapse during multi-variable discovery, constraint satisfaction, and checkout—especially for analytical users who already understand how data is structured.
1. HCI Studies: Chatbots vs. Visual Menus & Filters (GUIs)
Researchers have tested how users perform when forced to make multi-attribute decisions (like picking a hotel or filtering travel options) through a Conversational User Interface (CUI) versus a traditional Graphical User Interface (GUI).
-
**The "Repeated Use & Rational User" Penalty (Liu, Rietz, & Maedche, Karlsruhe Institute of Technology, Universal Access in the Information Society, 2024/2025):**This controlled lab experiment compared task completion times and cognitive friction between a chat interface and a graphical filter interface across repeated decision-making tasks.
-
**First-Time Novice vs. Repeat/Expert User:**On the very first attempt, complete novices found chat slightly easier because they didn't have to learn where the UI buttons were.However, by the second and third iteration (e.g., changing dates or tweaking filters), **the GUI became significantly faster than the chat interface.**Even when users knew the system well, chat forced them to spend time typing, reading, and waiting through serialized turns.
-
**High-Rationality Penalty:**The researchers measured each participant's "Rational Decision Style" (people who systematically evaluate structured attributes and compare tradeoffs—exactly how you described knowing the structure of the data).The study proved that highly analytical/rational decision-makers performed significantly worse and experienced higher frustration on chat interfaces, strongly preferring GUIs where they could inspect the underlying matrix directly.
-
**Autonomy Loss & Cognitive Load (Nguyen, Sidorova, & Torres, Computers in Human Behavior, 2022):**In an empirical comparison between chatbot interfaces and menu-based interfaces for service and booking tasks, researchers found that chatbots caused statistically higher cognitive load and lower perceived user autonomy than visual menus, leading directly to lower overall user satisfaction. Users felt trapped inside the bot's linear turn-taking rather than being in control of the search parameters.
-
**The "Specification Burden" in Travel Planning (Journal of Tourism & Recreation, June 2026):**A qualitative study ("ChatGPT, where should I go?") examining how travelers actually use LLMs for trip planning found that getting usable hotel or itinerary outputs required users to engage in tedious "prompt engineering as specification"—front-loading massive blocks of text detailing ages, budgets, safety constraints, and transit radii just to prevent generic outputs.While users liked LLMs for hedonic brainstorming ("Where should I go in May?"), they abandoned chat and switched back to traditional visual platforms for pragmatic verification (checking real prices, exact hotel locations, and booking).
2. AI Agent Benchmarks: Why Bots Fail Multi-Constraint Travel Searches
Computer scientists have built dedicated travel-planning benchmarks to test whether LLM agents using search tools can actually satisfy real-world hotel, flight, and budget constraints at the same time. The pass rates are shockingly low:
| Benchmark Study | Scope & Task Setup | Empirical Success / Pass Rate | Why the Agents Failed |
|---|---|---|---|
| TravelPlanner (Xie et al., Ohio State / ICML 2024) | 1,225 real-world travel queries requiring agents to use tools to search flights, hotels, and restaurants while respecting budget, dates, and room rules. | 0.6% to 4.4% final pass rate across frontier models (including GPT-4). | Agents constantly dropped constraints mid-search—booking a hotel in the wrong city, violating minimum-stay rules, exceeding total budget when flight prices changed, or hallucinating amenities. |
| TREK Benchmark (Qi et al., July 2026) | 800 complex travel tasks across a controlled sandbox of 212,530 records (107k flights, 39k hotels) graded by a strict, deterministic rule-based evaluator. | The strongest of 15 modern LLM agents succeeded on only 46.2% of solvable tasks. | Over 53% of the time, even 2026-era tool-calling agents failed to reconcile hard constraints (budget + party size + dates) with implicit needs (e.g., accessibility or family room sizing) or failed to recognize when a date/route combination was impossible. |
| TripScore (Oct 2025) | Real-world travel evaluation testing spatio-temporal feasibility and multi-step tool chaining. | Confirmed massive degradation whenever queries exceeded 3+ simultaneous constraints. | Changing a single variable (like your check-in date shift) forces the agent to invalidate and recompute the entire dependency tree, causing long tool-calling loops and state drift. |
When you asked the bot to check a hotel, then check baggage rules, and then shift the dates, you were forcing it to re-run a multi-step tool trajectory where modern agents fail more than half the time—and take minutes of chained API/browser calls per attempt.
3. Real-World OTA Earnings & Telemetry (Booking Holdings & Expedia, Q2/Q3 2026)
If you want the ultimate proof of whether consumers are actually using chat to book hotels, look at the hard numbers disclosed by the largest travel marketplaces on their recent 2026 earnings calls:
- Booking Holdings (August 2026 Earnings Disclosure): Despite 18 months of non-stop hype around ChatGPT, Perplexity, and Meta’s Muse, Booking Holdings’ CFO officially disclosed on August 4, 2026, that total paid and unpaid referral traffic coming from Large Language Models accounts for "significantly below 1%" of total room nights booked—with almost zero change over recent quarters.
- Expedia Group (Q2 2026 Earnings Call): Expedia built its own native conversational AI agents (including natural-language search and Property Expert). On their Q2 2026 call, Expedia CEO Ariane Gorin admitted openly that conversational AI features "are not driving conversion right now". Instead, Expedia noted that while chat gives them "60% more information about traveler intent" (because users type long sentences), users still rely on the visual interface to actually compare and convert.
4. Where Chat Actually Does Work in Travel (The Narrow Exception)
Interestingly, hospitality studies show there are two specific areas where chat interfaces do beat traditional menus:
- Unstructured "Fine-Print" Q&A on a Single Property: Once you have already visually picked a hotel on a map, using a property-specific chatbot to ask "Is the pool heated in November?" or _"Does this fare include a 23kg checked bag?"_works well because it saves you from digging through 10 pages of PDF terms and conditions (Booking Holdings reported a ~10% reduction in customer service cost per reservation from this exact use case).
- Curated Package Shortlisting for Overwhelmed Novices: Niche European leisure platforms like Zoover and Neckermann reported a 6% to 3x conversion lift among a small subset of holidaymakers who used an embedded AI assistant to narrow down thousands of all-inclusive beach packages into a 3-hotel shortlist—provided the bot immediately rendered visual cards inside the same website rather than leaving the user in a text-only chat window.
In short, the academic and industry data backs your gut instinct 100%: **chat is a high-friction bottleneck for structured multi-variable discovery.**For anyone who already understands how travel data is structured, a visual matrix with instant filters beats a slow conversational middleman every single time.