Creates a tokenized, by TweetNLP, version of the message table.

Argument and Default Value



This will create a table called TABLE_tweettok (where TABLE is specified by -t) in the database specified by -d. The message column in this new table is a list of tokens.

Example on one message

Original message:

"@antijokeapple: What do you call a Bee who is having a bad hair day? A Frisbee." Hahah.

Tokenized message:

["\"", "@antijokeapple", ":", "What", "do", "you", "call", "a", "Bee", "who", "is", "having", "a", "bad", "hair", "day", "?", "A", "Frisbee", ".", "\"", "Hahah", "."]

Other Switches

Required Switches:

Optional Switches:

Example Commands

# General form
# Creates the tables: TABLE_tok
dlatkInterface.py -d DATABASE -t TABLE -c GROUP_BY_FIELD --add_tweettok