Vietnamese Natural Language Processing(Xử lý ngôn ngữ tiếng Việt)
Note: This page is moved to https://github.com/kanjirz50/vnlp-outline for interactive communication and editing.
Xin lỗi tôi không nói tiếng việt.
Outline
This page describes Vietnamese NLP tools, resources and some computation techniques related on Vietnamese.
If there is something missing or wrong information ,please send me a mail to "takahashi at jnlp.org".
Tools
Diacritics restoration
Vietnamese diacritics restoration tool
This tool is based on a point-wise diacritics restoration.
Word segmentation
DongDu
DongDu is a fastest and high accuracy word segmentation tool based on SVMs.
vnTokenizer
vnTokenizer provides Java interface.
I found some unofficial github repositories.
Part-of-Speech tagging
vnTagger
This tool provides Java interface.
Joint word segmentation and Part-of-Speech tagging
Vietnamese joint word segmentation and Part-of-Speech tagging based on SVM
This tool provides Python interface.
Vietnamese joint word segmentation and Part-of-Speech tagging based on CRF
This tool provides Python interface.
Other
Vitk -- a Vietnamese text processing toolkit
This toolkit is for large data processing, includes word segmentation, part-of-sppech tagging, dependency parsing.
Resources
Treebank
Vietnamese treebank
Corpus
VNESEcopus.txt
650 thousand sentences.
VNTQcorpus(small).txt
300 thousand sentences.
VNTQcorpus(big).txt
1.75 million sentences.
Bilingual Corpus
English-Vietnamese bilingual corpus
Technique
Unicode normalization
NFC
