Every mother has questions during and after a pregnancy. How often should I feel my baby kick? When should I go to the hospital? How do I know if my newborn is feeding enough? In many settings around the world, mothers get in touch with their providers, friends or search the internet for these answers. But what about a mother who only has a primary school education, living in a rural community in Kenya, whose connection to the outside world is simple feature phone?
Jacaranda Health is trying to find solutions to improve health outcomes for low-income mothers who seek care at public hospitals. We launched text messaging service called PROMPTS that provides pregnant women and new mothers with essential information. In a short period of time, over 11,000 mothers have signed up for the service. Our initial objective was to send a series of rigorously tested messages to the mothers to ‘prompt’ them to seek care. We soon realized that mothers have questions they want answered — lots of questions! Almost half of our users respond to the text prompts with three to five questions, and a small percentage have more than 20 questions. We set up a help-desk to answer these questions (via text message). Most of these questions are general in nature (“can I eat avocados during pregnancy?”), but at least 30 percent of our incoming questions could require an urgent response (“I’m bleeding, what should I do?”). As PROMPTS rapidly scales to hundreds of thousands of mothers across the country, we face an ethical challenge: how can we accurately answer thousands of questions and respond rapidly to urgent questions?
Use a chatbot? Chatbots were the rage a few years ago and promised a revolution in customer service. A personal assistant to order your groceries, help you child with homework and do your tax returns. Most chatbots are flow bots, or a menu based, ‘choose your own adventure’ in finding an answer. Customer service desks have long used menu-based options to direct their customers to sources of information (‘press 2 if you lost your bank card’). Flowbots ‘humanize’ this approach with apologies and emojis. More recently, AI and chatbots have been pitched as solutions to global health challenges (reviewed extensively here). There are several bot solutions out there for women’s reproductive health issues (Nivi, Pregnancy Bot, Sophie Bot, Lily.Health and Ada). We built and tested flowbot to see if it could support our challenges in supporting thousands of questions from pregnant mothers, but encountered a few fundamental challenges, illustrated by this chatbot interaction below:
User: “How can one increase milk supply without eating alot of food. I dont want to add alot of weight because am already overweight”
Bot: “I’m sorry, please say yes or no to receiving information about your first breast milk”
Challenge 1: Humans don’t have a common way of organizing information. Should the answer to the question above be classified under ‘weight gain/loss’ or ‘breastfeeding’? (Who hasn’t been lost in a series of menu options before pressing 0 repeatedly for an operator).
Challenge 2: Flow bots are not customer-centered: Remember, most of our users have a few burning questions they want answered. A flow bot wants you to follow a predetermined pathway to get to an answer — not a great customer experience.
Challenge 3: Language input is difficult to ‘understand’. Our help-desk receives questions in English and Kiswahili and a significant percentage of the questions are a combination of the two languages, including messages that contain slang.
Challenge 4: Context is important in customer service, but it is critical in healthcare. What if this particular user had asked a question about pregnancy weight loss first and had subsequently revealed the perceived challenges with milk-supply? Information organized for convenience rarely provides enough contextual detail for good health advice.
AI that helps the help-desk agent. We believe that the human presence is essential for our users, so we looked for an approach that could make the human workload more efficient. We designed a ‘Triage Bot’ that uses Natural Language Processing (NLP) to categorize the intent of thousands of user questions. NLP uses machine learning to ‘understand’ language in the way a human would. For example pain can be described as ‘paining’, ‘my side is hurting’, ‘strain in the side’. An NLP-based bot learns these distinctions through pattern recognition, and our Triage Bot also adds a priority level to the intents. A question about pain is determined to be a high level of urgency, whereas questions about nutrition are lower priority.
Triage Bot is now integrated within our help-desk processes. The bot reads each incoming question, assigns a priority level and suggests a response so that our help-desk agent can respond faster and more effectively. We continuously train Triage Bot to improve its intent classification, but it is already making a difference in our ability to help mothers. Since integrating the bot, our help-desk has responded to urgent or high priority questions in 50 percent of the time it took previously, with a 40 percent overall reduction in response time.
Lessons learned. AI can be incredibly powerful with the right use case. In our use case, we found that AI augmented the activity of a human being, as opposed to replacing the human presence. Maybe someday Triage Bot will graduate to answering questions about avocados, so that our help-desk agents can better support mothers who have an urgent need for human contact.
Article by Sathy Rajasekharan
Republished from Towards Data Science