InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write
InkSight converts offline handwriting from photos into digital ink using vision-language models that combine reading and writing knowledge. It overcomes the lack of paired data by training a multi-task model on recognition and derendering tasks. The system effectively transforms arbitrary handwritten text in diverse images into traceable digital strokes, showing high accuracy in human evaluation.