One Sample T Test in Python

  1 Sample T Test, which compares a sample mean to a hypothetical population mean.

ttest_1samp requires two inputs, a sample distribution (eg. the list of the 50 observed purchase prices) and a mean to test against (eg. 1000):

tstat, pval = ttest_1samp(example_distribution, expected_mean)
print pval

It also returns two outputs: the t-statistic (which we won’t cover in this course), and the p-value — telling us how confident we can be that the sample of values came from a distribution with the specified mean.

One Sample T-Test II

In the last exercise, we got a p-value that was much higher than 0.05, so we cannot reject the null hypothesis. If we conduct another experiment and take a new sample of orders, will we get the same result? Not necessarily!

Just because we don’t have enough data to detect a difference doesn’t mean that there isn’t one. Generally, the larger the sample(s) we have, the smaller a difference we’ll be able to detect. You can learn more about the exact relationship between sample size and detectable differences in the Sample Size Determination course.

It’s also possible that the true mean order price really is 1000 Rupees, but a single sample still leads us to incorrectly reject the null hypothesis. Remember that this is called Type 1 Error and the significance threshold we use for our test should be equal to the type 1 error rate under the null hypothesis. In other words, if we set a 0.05 significance threshold and the true mean purchase price is truly 1000 Rupees, we still expect to incorrectly reject the null (and say that the mean is not 1000 Rupees) in 5% of experiments.

To build intuition for the limitations of conclusions based on any individual sample, let’s explore some more data from BuyPie.com and see whether we consistently observe the same results

We have loaded a dataset daily_prices into the editor that represents the purchase prices of orders from BuyPie.com in the last 1000 days. Each entry daily_prices[i] is an array of entries representing the price per order on day i.

We predicted that the average price of an order would be 1000 Rupees, and we want to know if the actual data differs from that.

We have made a for loop that goes through the 1000 inner lists. Inside this loop, perform a 1 Sample T-Test with each day of data (daily_prices[i]). For now, just print out the p-value from each test.

.

If we get a pval < 0.05, we can conclude that it is unlikely that our sample has a true mean of 1000. Suppose that the true mean purchase price really is 1000 Rupees. Thus, if we get a p-value less than 0.05, we will incorrectly reject the null hypothesis test and say that the experiment indicates the mean purchase price is not 1000 Rupees.

Every time we get an incorrect result within the 1000 experiments, add 1 to incorrect_results. Since our significance threshold was 0.05, we expect about 5% or 50 of the 1000 experiments to return an incorrect result.

from scipy.stats import ttest_1samp
import numpy as np

incorrect_results = 0 # Start the counter at 0

daily_prices = np.genfromtxt("daily_prices.csv", delimiter=",")

for i in range(1000): # 1000 experiments
   #your ttest here:
   t_stat,pval = ttest_1samp(daily_prices[i],1000)
   if pval < 0.05:
     incorrect_results +=1

   
   #print the pvalue here:
   print pval
  
print "We incorrectly thought that the distribution was different in " + str(incorrect_results) + " out of 1000 experiments."

Comments

Post a Comment

Popular posts from this blog

Binomial Test in Python

  Binomial Test Let’s imagine that we are analyzing the percentage of customers who make a purchase after visiting a website. 1000 customers visited the site this month, and 58 of them made a purchase. The marketing department reports that historical data suggests about 72 of every 1000 visitors make a purchase. Thus, they estimate that the probability of any particular customer making a purchase is 7.2%. We would like to know if this month’s number, 58 purchases, is significantly different from normal or a reasonable fluctuation due to random chance. In previous exercises, we collected samples of numerical information (eg. order price) and then used the mean and standard deviation of those samples to make comparisons. In contrast, we now have a sample where each unit (a visitor) falls into one of two discrete categories: “made a purchase” “did not make a purchase” Instead of comparing sample means, we want to compare the percent in the “made a purchase” category to some expectatio...
Read more

Slicing and Indexing in Python Pandas

 # Make a list of cities to subset on cities = ["Moscow", "Saint Petersburg"] # Subset temperatures using square brackets print(temperatures[temperatures["city"].isin(cities)]) # Subset temperatures_ind using .loc[] print(temperatures_ind.loc[cities]) temperatures_ind = temperatures.set_index(['country','city']) # List of tuples: Brazil, Rio De Janeiro & Pakistan, Lahore rows_to_keep = [('Brazil','Rio De Janeiro'),('Pakistan','Lahore')] # Subset for rows to keep print(temperatures_ind.loc[rows_to_keep]) # Sort temperatures_ind by index values print(temperatures_ind.sort_index()) # Sort temperatures_ind by index values at the city level print(temperatures_ind.sort_index(level="city")) # Sort temperatures_ind by country then descending city print(temperatures_ind.sort_index(level=["country", "city"], ascending = [True, False])) Slice rows with code like  df.loc[("a", ...
Read more

Python Syntax and Functions Part2 (Summary Statistics)

The  .agg()  method allows you to apply your own custom functions to a DataFrame, as well as apply functions to more than one column of a DataFrame at once, making your aggregations super efficient. In the custom function for this exercise, "IQR" is short for inter-quartile range, which is the 75th percentile minus the 25th percentile. It's an alternative to standard deviation that is helpful if your data contains   import numpy as np def iqr(column):     return column.quantile(0.75) - column.quantile(0.25) # Update to print IQR and median of temperature_c, fuel_price_usd_per_l, & unemployment print(sales[["temperature_c", "fuel_price_usd_per_l", "unemployment"]].agg([iqr, np.median])) Calulating Cumulative Statistics Syntax = df.drop_duplicates sales_1_1 = sales_1_1.sort_values("date") # Get the cumulative sum of weekly_sales, add as cum_weekly_sales col sales_1_1["cum_weekly_sales"] = sales_1_1["weekly_sales...
Read more