What is text normalization?

Wkniuz48 report abuse

I am learning natural language processing (NLP). Recently I encountered the term "text normalization". Can somebody explain me, please, what is text normalization in NLP?

Answers

jeluw report abuse

There can be several definitions. What was the context where you learn about text normalization?

Wkniuz48 report abuse

I have read an article about how to improve the quality of text-to-sound conversion. They stated that text normalization is an important element of the improvement of the results.

jeluw report abuse

It is still not vivid enough. I know that text normalization is the process of unfolding text before feeding it to a machine learning model. But there can be other interpretations.

Wkniuz48 report abuse

Can you please tell more about text unfolding?

jeluw report abuse

Yes. You may notice that texts often include many special characters, abbreviations, numbers, etc. And, for example, numbers can be pronounced differently depends on the context. It is especially actual for languages like Russian, where words can be modified by tense, number, gender, etc. It is not very hard for a human to understand how to pronounce such abbreviations. But for machine learning models it is a hard task. That's why we need to unfold all these abbreviations before doing machine learning. This is called text normalization.

Wkniuz48 report abuse

Yes, probably this is what I have read about in that article. Thank you!

CabeBe report abuse

Wkniuz48 But what about other interpretations of this notion? I'm curious about them. Explain them too, please.

Wkniuz48 report abuse

Probably text normalization in NLP could be the processing of vectorized texts. As you may now, before feeding data to machine learning model you can preprocess it, perform feature engineering, etc. Maybe text normalization is the process where you remove stopwords from the text, perform stemming, lemmatization, framing, etc. All these text processing steps can be called text normalization also.

CabeBe report abuse

It really makes sense. Thanks for your explanation!

TomOstrovskyi report abuse __ edited

APA research paper example is beneficial for students unaware of APA style and its requirements. It enables you to submit your APA research paper without much trouble. This template is helpful in perfecting the formatting for textual citations and references. You just have to insert your written material wherever the template asks you to. A good APA research paper template should vividly illustrate the following main features of APA format

Add Answer

Need support?

Just drop us an email to ... Show more