🧑‍🎓 I am a first-year PhD student in Natural Language Processing (NLP) group at the University of Sheffield, under the supervision of Dr. Marco Valentino and Prof. Nikolaos Aletras.

I obtained my master’s degree in Applied Mathematics from the School of Mathematical Sciences, Peking University, where I was supervised by Prof. Jinwen Ma.

🧐 My research interests include integrating neural and symbolic approaches within language models to enhance their reasoning capabilities, controllability, and interpretability.

😄 I am from Taichung, Taiwan and grew up in Beijing.

🥊🏋️ I enjoy sports. I do boxing and crossfit.

Selected Experience

Logic Before Language: Pre-pretraining on Formal Derivations Fosters Skill Acquisition and Compressibility

Paper

• Proposed logic pre-pretraining (Logic-PPT), which uses formal derivations to provide language models with richer structural and linguistic biases before natural language training.
• Demonstrated faster skill acquisition at the 100B-token scale, reaching 80% accuracy on linguistic tasks with 36B fewer training tokens than standard initialization.
• Showed that logic pre-pretraining induces a lower-rank, spectrally concentrated representation space and improves model compressibility, matching dense baseline performance at approximately 33% sparsity.

GeoUni: A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutions

Project Homepage | Paper ｜ Video

• Proposed the first unified multi-modal geometry expert model, GeoUni,capable of solving geometry problems, generating precise geometric diagrams using both formal and natural language, and creating geometry problems based on knowledge points.
• Proposed Geo-MAGVIT, a module specifically designed for the tokenization of geometric diagrams. By introducing topo-structural awareness loss and text region loss, it significantly improves the precision of geometry structure and text reconstruction.
• Combined GRPO and LoRA to train the Geo-Reasoning-Adapter, which effectively boosts geometric reasoning capability and seamlessly integrates into the unified model architecture.
• Established a novel diagram generation evaluation metrics, which includes the Geometry Semantic Matching Scores (GSMSs) and Geometry Pixel Matching Score (GPMS) to comprehensively evaluate the diagram generation task.

Diagram Formalization Enhanced Geometry Problem Solver

Project Homepage | Dataset | Paper Video

• Designed a multimodal framework integrating visual features and geometric formal languages for solving complex geometry problems.
• Proposed a synthetic data approach (SynthGeo228K dataset) for improving model training and diagram interpretation.
• Achieved an accuracy of 82.38% on the publicly available FormalGeo7k dataset, significantly outperforming existing multi-modal and language models, including GPT-4.