OTRO: Oblivious Tokenization Path with Square-Root ORAM
OTRO addresses a security risk where CPU-side LLM tokenizers leak information in confidential computing environments. It introduces an oblivious path using Square-Root ORAM, employing replicated instances and epoch rotation to decouple slow rebuilds from fast serving. This ensures that observing the system only reveals prompt length, preserving confidentiality while maintaining minimal latency overhead.