--add_tweetpos
Switch
--add_tweetpos
Description
Creates a POS tagged version of the message table using Carnegie Mellon University's TweetNLP part of speech tagger.
Argument and Default Value
None
Details
This will create a table called TABLE_tweetpos (where TABLE is specified by -t) in the database specified by -d. The message column in this new table is a list of tokens.
Example
Original message:
@someuser i think that curly hair is getting to you 😂
POS message:
{"tokens": ["@someuser", "i", "think", "that", "curly", "hair", "is", "getting", "to", "you", "&", "#128514", ";"],
"original": "@someuser i think that curly hair is getting to you 😂",
"probs": ["0.9994", "0.9898", "0.9999", "0.4810", "0.9903", "0.9992", "0.9955", "0.9959", "0.9967", "0.9992", "0.9806", "0.3757", "0.9448"],
"tags": ["@", "O", "V", "D", "A", "N", "V", "V", "P", "O", "&", "#", ","]}
Other Switches
Required Switches:
Example Commands
# creates the table msgs_tweetpos
./dlatkInterface.py -d dla_tutorial -t msgs -c message_id --add_tweetpos
mysql> select message from msgs_tweetpos limit 1;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| message |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"probs": ["0.9990", "0.9993", "0.9999", "0.9853", "0.9934", "0.9958", "0.9813", "0.9890", "0.9999", "0.9994", "0.9973", "0.7924", "0.9962", "0.9963", "0.9934", "0.9776", "0.9931", "0.9997", "0.9997", "0.9997", "0.9505", "0.9997", "0.8819", "0.9984", "0.9925", "0.9268", "0.9984", "0.9964", "0.9957", "0.9996", "0.6084", "0.5645", "0.9990", "0.9986", "0.9735", "0.9791", "0.9904", "0.9991", "0.5527", "0.9695", "0.9981", "0.9985"], "tags": ["V", "O", "V", "O", ",", "D", "N", "V", "V", "O", "V", "T", "P", "D", "N", ",", "O", "V", "R", "R", "A", "P", "O", ",", "R", "V", ",", "&", "O", "V", "A", "V", "P", "D", "N", "D", "N", ",", "P", "L", "A", ","], "tokens": ["can", "you", "believe", "it", "??", "my", "mom", "wouln't", "let", "me", "go", "out", "on", "my", "b'day", "...", "i", "was", "really", "really", "mad", "at", "her", ".", "still", "am", ".", "but", "i", "got", "more", "presents", "from", "my", "friends", "this", "year", ".", "so", "thats", "great", "."], "original": "can you believe it?? my mom wouln't let me go out on my b'day...i was really really mad at her. still am. but i got more presents from my friends this year. so thats great."} |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+