PolyAI and Otter execs say voice AI needs major fixes

Despite the recent arrival of full-duplex models capable of simultaneous listening and speaking, artificial intelligence executives warn that conversational voice technology has not yet achieved its defining breakthrough moment. Speaking on stage at the HumanX conference, PolyAI Chief Technology Officer Shawn Wen and Otter Chief Marketing Officer Alex Gay detailed the underlying technical and operational bottlenecks currently slowing down autonomous voice tools across corporate settings.

A primary challenge involves processing latency and reasoning speed. While full-duplex architectures mark a significant engineering milestone, Wen noted that models must execute much faster reasoning to pull accurate answers without creating unnatural pauses. In enterprise customer service, establishing caller confidence requires agents that avoid sounding robotic while demonstrating they can autonomously resolve complex problems within the initial turns of a conversation.

Underlying accuracy in Automatic Speech Recognition, known as ASR, presents another crucial barrier to enterprise adoption. Both executives pointed out that missing key words or misinterpreting intent fundamentally undermines contextual understanding. Gay emphasized that transcription is merely the foundation for workplace productivity gains; if initial ASR accuracy fails, any subsequent automated actions become flawed, leading users to lose trust in the underlying platform.

Furthermore, as companies explore advanced applications like digital twins that represent individuals in meetings, systems must replicate emotive human expressions and support strategic debate. For practitioners, these constraints demonstrate that adopting voice AI demands prioritizing raw ASR precision over complex automation. Deploying organizations must also maintain strict operational transparency by notifying participants whenever conversations are being recorded or managed by synthetic intelligence.