Twitter API

 

API Authentication

The package tweepy is great at handling all the Twitter API OAuth Authentication details for you. All you need to do is pass it your authentication credentials. In this interactive exercise, we have created some mock authentication credentials (if you wanted to replicate this at home, you would need to create a Twitter App as Hugo detailed in the video). Your task is to pass these credentials to tweepy's OAuth handler.

• Import the package tweepy.
• Pass the parameters consumer_key and consumer_secret to the function tweepy.OAuthHandler().
• Complete the passing of OAuth credentials to the OAuth handler auth by applying to it the method set_access_token(), along with arguments access_token and access_token_secret

import tweepy

# Store OAuth authentication credentials in relevant variables

access_token = "1092294848-aHN7DcRP9B4VMTQIhwqOYiB14YkW92fFO8k8EPy"

access_token_secret = "X4dHmhPfaksHcQ7SCbmZa2oYBBVSD2g8uIHXsp5CTaksx"

consumer_key = "nZ6EA0FxZ293SxGNg8g8aP0HM"

consumer_secret = "fJGEodwe3KiKUnsYJC3VRndj7jevVvXbK2D5EiJ2nehafRgA6i"

# Pass OAuth details to tweepy's OAuth handler

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)

auth.set_access_token(access_token,access_token_secret)

• Create your Stream object with authentication by passing tweepy.Stream() the authentication handler auth and the Stream listener l;
• To filter Twitter streams, pass to the track argument in stream.filter() a list containing the desired keywords 'clinton', 'trump', 'sanders', and 'cruz'.

# Initialize Stream listener

l = MyStreamListener()

# Create your Stream object with authentication

stream = tweepy.Stream(auth, l)

# Filter Twitter Streams to capture data by the keywords:

stream.filter(track=['clinton', 'trump', 'sanders', 'cruz'])

Now that you've got your Twitter data sitting locally in a text file, it's time to explore it! This is what you'll do in the next few interactive exercises. In this exercise, you'll read the Twitter data into a list: tweets_data.

Be aware that this is real data from Twitter and as such there is always a risk that it may contain profanity or other offensive content (in this exercise, and any following exercises that also use real Twitter data).

• Assign the filename 'tweets.txt' to the variable tweets_data_path.
• Initialize tweets_data as an empty list to store the tweets in.
• Within the for loop initiated by for line in tweets_file:, load each tweet into a variable, tweet, using json.loads(), then append tweet to tweets_data using the append() method.
• Hit submit and check out the keys of the first tweet dictionary printed to the shell.

# Import package

import json

# String of path to file: tweets_data_path

tweets_data_path = 'tweets.txt'

# Initialize empty list to store tweets: tweets_data

tweets_data = []

# Open connection to file

tweets_file = open(tweets_data_path, "r")

# Read in tweets and store in list: tweets_data

for line in tweets_file:

    tweet = json.loads(line)

    tweets_data.append(tweet)

# Close connection to file

tweets_file.close()

# Print the keys of the first tweet dict

print(tweets_data[0].keys())

# Import package

import pandas as pd

# Build DataFrame of tweet texts and languages

df = pd.DataFrame(tweets_data, columns=['text','lang'])

# Print head of DataFrame

print(df.head)

Now that you have your DataFrame of tweets set up, you're going to do a bit of text analysis to count how many tweets contain the words 'clinton', 'trump', 'sanders' and 'cruz'. In the pre-exercise code, we have defined the following function word_in_text(), which will tell you whether the first argument (a word) occurs within the 2nd argument (a tweet).

import re

def word_in_text(word, text):
    word = word.lower()
    text = text.lower()
    match = re.search(word, text)

    if match:
        return True
    return False

You're going to iterate over the rows of the DataFrame and calculate how many tweets contain each of our keywords! The list of objects for each candidate has been initialized to 0.

• Within the for loop for index, row in df.iterrows():, the code currently increases the value of clinton by 1 each time a tweet (text row) mentioning 'Clinton' is encountered; complete the code so that the same happens for trump, sanders and cruz.
• Initialize list to store tweet counts

  [clinton, trump, sanders, cruz] = [0, 0, 0, 0]

  
  

  # Iterate through df, counting the number of tweets in which

  # each candidate is mentioned

  for index, row in df.iterrows():

      clinton += word_in_text('clinton', row['text'])

      trump += word_in_text('trump',row['text'])

      sanders += word_in_text('sanders', row['text'])

      cruz += word_in_text('cruz', row['text'])