Python Data Science Coding Reference

  # Import package from   urllib . request   import   urlretrieve # Import pandas import   pandas   as   pd # Assign url of file: url url  =  'https://s3.amazonaws.com/assets.datacamp.com/production/course_1606/datasets/winequality-red.csv' # Save file locally urlretrieve ( url , 'winequality-red.csv' ) # Read file into a DataFrame and print its head df  =  pd . read_csv ( 'winequality-red.csv' ,   sep = ';' ) print ( df . head ()) Importing excel file online # Import package import   pandas   as   pd # Assign url of file: url url  =  'http://s3.amazonaws.com/assets.datacamp.com/course/importing_data_into_r/latitude.xls' # Read in all sheets of Excel file: xls xls  =  pd . read_excel ( url ,   sheet_name = None ...

  Exercise Exercise Pandas and The Hello World of SQL Queries! Here, you'll take advantage of the power of  pandas  to write the results of your SQL query to a DataFrame in one swift line of Python code! You'll first import  pandas  and create the SQLite  'Chinook.sqlite'  engine. Then you'll query the database to select all records from the  Album  table. Recall that to select all records from the  Orders  table in the Northwind database, Hugo executed the following command: df = pd.read_sql_query("SELECT * FROM Orders", engine) # Import packages from   sqlalchemy   import   create_engine import   pandas   as   pd # Create engine: engine engine  =  create_engine ( 'sqlite:///Chinook.sqlite' ) # Execute query and store records in DataFrame: df df  =  pd . read_sql_query ( "Select * FROM Album" ,   en...

  Here, you're going to fire up your very first SQL engine. You'll create an engine to connect to the SQLite database   'Chinook.sqlite' , which is in your working directory. Remember that to create an engine to connect to   'Northwind.sqlite' , Hugo executed the command engine = create_engine('sqlite:///Northwind.sqlite') Here,  'sqlite:///Northwind.sqlite'  is called the  connection string  to the SQLite database  Northwind.sqlite . A little bit of background on the  Chinook database : the Chinook database contains information about a semi-fictional digital media store in which media data is real and customer, employee and sales data has been manually created. The Hello World of SQL Queries! Now, it's time for liftoff! In this exercise, you'll perform the Hello World of SQL queries,  SELECT , in order to retrieve all columns of the table  Album  in the Chinook database. Recall that the query  SELECT *  selects all c...

 # Create a DataFrame of female drivers stopped for speeding female_and_speeding = ri[(ri.driver_gender == 'F') & (ri.violation == 'Speeding')] # Create a DataFrame of male drivers stopped for speeding male_and_speeding = ri[(ri.driver_gender == 'M') & (ri.violation == 'Speeding')] # Compute the stop outcomes for female drivers (as proportions) print(female_and_speeding.stop_outcome.value_counts(normalize=True)) # Compute the stop outcomes for male drivers (as proportions) print(male_and_speeding.stop_outcome.value_counts(normalize=True))

 # Define plot_pop() def plot_pop(filename, country_code):     # Initialize reader object: urb_pop_reader     urb_pop_reader = pd.read_csv(filename, chunksize=1000)     # Initialize empty DataFrame: data     data = pd.DataFrame()          # Iterate over each DataFrame chunk     for df_urb_pop in urb_pop_reader:         # Check out specific country: df_pop_ceb         df_pop_ceb = df_urb_pop[df_urb_pop['CountryCode'] == country_code]         # Zip DataFrame columns of interest: pops         pops = zip(df_pop_ceb['Total Population'],                     df_pop_ceb['Urban population (% of total)'])         # Turn zip object into list: pops_list         pops_list = list(pops)         # Use list comprehension to create new Dat...

  Writing a generator to load data in chunks (2) In the previous exercise, you processed a file line by line for a given number of lines. What if, however, you want to do this for the entire file? In this case, it would be useful to use  generators . Generators allow users to  lazily evaluate  data . This concept of  lazy evaluation  is useful when you have to deal with very large datasets because it lets you generate values in an efficient manner by  yielding  only chunks of data at a time instead of the whole thing at once. In this exercise, you will define a generator function  read_large_file()  that produces a generator object which yields a single line from a file each time  next()  is called on it. The csv file  'world_dev_ind.csv'  is in your current directory for your use. Note that when you open a connection to a file, the resulting file object is already a generator! So out in the wild, you won't have to ex...

 # Define count_entries() def count_entries(csv_file,c_size,colname):     """Return a dictionary with counts of     occurrences as value for each key."""          # Initialize an empty dictionary: counts_dict     counts_dict = {}     # Iterate over the file chunk by chunk     for chunk in pd.read_csv(csv_file,chunksize=c_size):         # Iterate over the column in DataFrame         for entry in chunk[colname]:             if entry in counts_dict.keys():                 counts_dict[entry] += 1             else:                 counts_dict[entry] = 1     # Return counts_dict     return counts_dict # Call count_entries(): result_counts result_counts =count_entries('tweets.csv',10,'lang') # Print result_counts print...