On-Device Named-Entity Recognition: A Deployability Study of Accuracy, Cost, Reliability, and Confidence
This study evaluated nine named-entity recognition systems for local deployment, focusing on accuracy, cost, and reliability rather than just leaderboard scores. It found that bidirectional encoders are the most deployable choice because they offer high performance with low size and latency. The research also developed protocols to assess performance without human annotation.