Removing unwanted strings
To perform NLP, we need to ensure the data doesn't have any special characters, diacritics, HTML tags, or any other content that may make it difficult for the algorithms to work properly.
To see what Optimus can do for you, let's load a dataframe with text to help us demonstrate Optimus' features:
df = op.load.csv("text.csv", sep=";")
That will load a text we specially prepared from Wikipedia. Let's see its content:
text (object) "<a>http://google.com</a> <span>Transformers is an American and Japanese media franchise produced by the American toy company Hasbro and Japanese toy company Takara Tomy. It follows the battles of sentient, living autonomous robots, often the Autobots and the Decepticons, who can transform into other forms, such as vehicles and animals. The franchise encompasses toys, animation, comic books, ...
Now that we know the data, let's start transforming it...