From chaos to clarity: How I restructured a chatbot’s content for better scale
The problem: Scaling a chatbot without a clear system
When I joined BMO’s Digital Messaging & Collaboration team, the product owner was focused on scaling chatbot content fast—but there was a fundamental issue:
👉 The chatbot inventory was a mess.
👉 Duplicate and overlapping intents created friction for QAs, developers, and designers.
👉 More importantly, it was hurting NLU training, reducing the accuracy of the chatbot’s responses.
I flagged that without a scalable structure, we were just piling content on top of a broken system. So I proposed a solution: run an open card sort to categorize and streamline the intent structure.
The PO pushed back. “Just keep training the chatbot,” they said. But I saw the bigger problem—we were scaling chaos.
The strategy: Escalating for change
Instead of letting it go, I escalated the issue to my UX manager and director, making the case that:
-
Unstructured intents were degrading chatbot performance.
-
Content designers were losing time searching for existing responses.
-
A strategic classification system would unlock scalability and improve sprint velocity.
With their support, I moved ahead—despite continued resistance from the PO—and led the effort to bring order to the system.
Participants from the UX Retail team categorized them into 15 groups for consistency.
The process: Working with constraints, driving structure
We didn’t have access to Optimal Sort, so I built a lean version of the card sort using FigJam and spreadsheets. I coordinated with the UX Retail team to run the sort and followed up with:
-
Auditing 200+ chatbot intents to find duplication and fragmentation
-
Creating naming conventions for better tracking and visibility
-
Defining slots to group related intents into unified conversational flows
I also uncovered inconsistencies—like the use of "HowdoI"—that were undermining the logic of our categorization system, and I flagged areas that required future cleanup.
By the end of the project, I had laid the foundation for a more structured, scalable chatbot experience:
​
-
📈 2x increase in sprint velocity by enabling intent tagging and story breakdown by topic
-
🧠Improved NLU training accuracy due to cleaner, more consistent intent categorization
-
🧹 Redundancy removed—grouped or eliminated similar intents, reducing confusion and improving authoring
​
This wasn’t just about organizing content. It was about reframing how the team approached chatbot scale—moving from reactive content creation to a system built to grow.
The impact: From messy content to scalable systems
2X
increase in sprint velocity (by breaking user stories into topic-based sub-tasks)
Areas for future improvement
-
Even with strong results, I identified ways to refine the process further:
-
Recruit more participants for broader insights
-
Ask users to name their categories to guide taxonomy
-
Standardize naming conventions across all chatbot properties
-
Revisit the “Other” category to avoid ambiguity and improve scalability
