Vietnamese Natural Language Processing(Xử lý ngôn ngữ tiếng Việt)

Note: This page is moved to https://github.com/kanjirz50/vnlp-outline for interactive communication and editing.

Xin lỗi tôi không nói tiếng việt.

Outline

This page describes Vietnamese NLP tools, resources and some computation techniques related on Vietnamese.

If there is something missing or wrong information ,please send me a mail to "takahashi at jnlp.org".

Tools

Diacritics restoration

Vietnamese diacritics restoration tool

Github, demonstration

This tool is based on a point-wise diacritics restoration.

Word segmentation

DongDu

Github, website

DongDu is a fastest and high accuracy word segmentation tool based on SVMs.

vnTokenizer

website

vnTokenizer provides Java interface.

I found some unofficial github repositories.

Part-of-Speech tagging

vnTagger

website

This tool provides Java interface.

Joint word segmentation and Part-of-Speech tagging

Vietnamese joint word segmentation and Part-of-Speech tagging based on SVM

Github, demonstration

This tool provides Python interface.

Vietnamese joint word segmentation and Part-of-Speech tagging based on CRF

Github, demonstration

This tool provides Python interface.

Other

Vitk -- a Vietnamese text processing toolkit

Github

This toolkit is for large data processing, includes word segmentation, part-of-sppech tagging, dependency parsing.

Resources

Treebank

Vietnamese treebank

website

Corpus

VNESEcopus.txt

website

650 thousand sentences.

VNTQcorpus(small).txt

website

300 thousand sentences.

VNTQcorpus(big).txt

website

1.75 million sentences.

Bilingual Corpus

English-Vietnamese bilingual corpus

website

Technique

Unicode normalization

NFC

Syllable normalization

Articles