The Problem
During 2024's breakout moment for generative AI, Mercari decided to experiment with some public-facing projects utilizing LLMs.
At the same time that hype was building around the possibilities of generative AI in product applications, cracks in its capabilities were also being exposed. Chatbots could quickly jump off their perceived rails, hallucinating odd responses or being baited into problematic conversations by users. The unpredictability of this early generation of LLM chat applications was presenting as much brand risk as it was opportunity.
The Solution
I'd already been extensively tinkering with prompt engineering and paying close attention to developments in the technology. Part of that engagement had been investigating the flaws in the applications – getting a solid sense for how hallucinations were triggered and, generally, where the weaknesses in LLMs were.
My key point of advocacy on the project was to narrow the implied usage of the chat tool to the user: We would portray it primarily as a gift seeking guide.
This would primarily be achieved at a UI level through 1) a consistent greeting from the chatbot, 2) a series of sample prompts for the user, 3) our marketing and messaging around the tool.
This direction also tightened the approach from an engineering perspective. Behind the scenes, the ChatGPT implementation would always work toward generating a series of topical search prompts based on the conversation with the user. Those prompts would then be connected to our existing search experience, letting a more precise, situational ML model take over the critical final output step.
As we quickly built and tested the application, I brought in several content designers from my team to iterate attempts to break the tooling out of the rails we'd set up for it. Logs of problem conversations were then provided to the project's engineers to test adjusting parameters in the hidden LLM prompt as well as the structural logic.
In the end, we were able to release a well-tested and well-controlled LLM in 2 sprints of total work. Merchat AI became the biggest source of positive PR impressions for Mercari in years.