For our purposes—learning how to clean data—it will be sufficient to load the smaller of these files into a single MySQL database table. Everything we need to do to learn, we can accomplish with the smaller file, the one called testdata.manual.2009.06.14.csv
.
As we are looking at the data, we may notice a few areas that will trip us up if we try to import this file directly into MySQL. One of the trouble spots is located at line 28 in the file:
"4","46","Thu May 14 02:58:07 UTC 2009","""booz allen""",
Do you see the triple quotation marks """
right before the booz
keyword and after the word allen
? The same issue comes up later on line 41 with double quotation marks around the song title P.Y.T
:
"4","131","Sun May 17 15:05:03 UTC 2009","Danny Gokey","VickyTigger","I'm listening to ""P.Y.T"" by Danny Gokey…"
The problem with these extra quotation marks is that the MySQL import routine will use the quotation marks to delimit the column text. This will produce...