Skip to content

Commit

Permalink
Merge pull request quantopian#264 from quantopian/mle-exercises
Browse files Browse the repository at this point in the history 
Adding in MLE exercises
  • Loading branch information
@rapetter94
rapetter94 authored Jul 2, 2018
2 parents a15fd9c + ca965ee commit abdb327
Show file tree
Hide file tree
Showing 8 changed files with 26,812 additions and 51 deletions.
57 changes: 35 additions & 22 deletions notebooks/lectures/Confidence_Intervals/answers/notebook.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercises: Confidence Intervals - Answer Key\n",
"# Exercises: Confidence Intervals - Answer Key\n",
"By Christopher Fenaroli and Delaney Mackenzie\n",
"\n",
"## Lecture Link:\n",
"https://www.quantopian.com/lectures/confidence-intervals\n",
"\n",
"###IMPORTANT NOTE: \n",
"### IMPORTANT NOTE: \n",
"This lecture corresponds to the Confidence Intervals lecture, which is part of the Quantopian lecture series. This homework expects you to rely heavily on the code presented in the corresponding lecture. Please copy and paste regularly from that lecture when starting to work on the problems, as trying to do them from scratch will likely be too difficult.\n",
"\n",
"When you feel comfortable with the topics presented here, see if you can create an algorithm that qualifies for the Quantopian Contest. Participants are evaluated on their ability to produce risk-constrained alpha and the top 10 contest participants are awarded cash prizes on a daily basis.\n",
Expand Down Expand Up @@ -103,9 +103,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise 1: Determining Confidence Intervals\n",
"# Exercise 1: Determining Confidence Intervals\n",
"\n",
"##a. Mean\n",
"## a. Mean\n",
"\n",
"Determine the mean of the following artificial data in `sample`."
]
Expand Down Expand Up @@ -137,7 +137,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##b. Standard Deviation\n",
"## b. Standard Deviation\n",
"\n",
"Determine standard deviation of the sample."
]
Expand Down Expand Up @@ -167,7 +167,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##c. Standard Error\n",
"## c. Standard Error\n",
"\n",
"Using the standard deviation and `sample_size`, determine the standard error for the sample."
]
Expand Down Expand Up @@ -197,7 +197,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##d. Confidence Intervals\n",
"## d. Confidence Intervals\n",
"\n",
"Using the standard error and mean, determine 95% `(Z = 1.96)`, 90% `(Z = 1.64)`, and 80% `(Z = 1.28)` confidence intervals for the sample. "
]
Expand Down Expand Up @@ -236,7 +236,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise 2: Interpreting Confidence Intervals\n",
"# Exercise 2: Interpreting Confidence Intervals\n",
"\n",
"Assuming our interval was correctly calculated and that the underlying data was independent, if we take many samples and make many 95% confidence intervals, the intervals will contain the true mean 95% of the time. Run 1000 samples and measure how many of their confidence intervals actually contain the true mean."
]
Expand Down Expand Up @@ -283,9 +283,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise 3: Central Limit Theorem\n",
"# Exercise 3: Central Limit Theorem\n",
"\n",
"##a. Plotting Sample Means - Normal\n",
"## a. Plotting Sample Means - Normal\n",
"\n",
"Assuming our samples are independent, the distribution of the sample means should be normally distributed, regardless of the underlying distribution. \n",
"\n",
Expand Down Expand Up @@ -337,7 +337,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##b. Plotting Sample Means - Exponential\n",
"## b. Plotting Sample Means - Exponential\n",
"\n",
"Draw 500 samples of size `sample_size` from a new exponential distribution, plot the means of each of the samples, and check to see if the distribution of the sample means is normal."
]
Expand Down Expand Up @@ -387,7 +387,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##c.i Plotting Sample Means - Autocorrelated\n",
"## c.i Plotting Sample Means - Autocorrelated\n",
"\n",
"Draw 500 samples of size `sample_size` from a new autocorrelated (dependent) distribution, plot the means of each of the samples, and check to see if the distribution of the sample means is normal."
]
Expand Down Expand Up @@ -438,7 +438,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##c.ii Plotting Sample Standard Deviations - Autocorrelated\n",
"## c.ii Plotting Sample Standard Deviations - Autocorrelated\n",
"\n",
"Draw 500 samples of size `sample_size` from the same autocorrelated distribution, plot the standard deviations of each of the samples, and check to see if the distribution of the sample standard deviations is normal."
]
Expand Down Expand Up @@ -497,9 +497,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise 4: Small Sample Sizes\n",
"# Exercise 4: Small Sample Sizes\n",
"\n",
"##a. Error Due to Small Sample Size\n",
"## a. Error Due to Small Sample Size\n",
"\n",
"Run 100 samples of size `small_size` and measure how many of their 95% confidence intervals actually contain the true mean."
]
Expand Down Expand Up @@ -542,7 +542,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##b. T-distribution Correction\n",
"## b. T-distribution Correction\n",
"\n",
"Run 100 samples of size `small_size`, this time accouting for the small sample size using a t-distribution, and measure how many of their 95% confidence intervals actually contain the true mean."
]
Expand Down Expand Up @@ -595,9 +595,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise 5: Dependence\n",
"# Exercise 5: Dependence\n",
"\n",
"##a. Error due to Dependence\n",
"## a. Error due to Dependence\n",
"\n",
"Run 100 samples of the following autocorrelated distribution and measure how many of their 95% confidence intervals actually contain the true mean. (Use the helper function `generate_autocorrelated_data(theta, noise_mu, noise_sigma, sample_size)` to generate the samples)"
]
Expand Down Expand Up @@ -643,7 +643,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##b. T-distribution Correction\n",
"## b. T-distribution Correction\n",
"\n",
"Run 100 samples from the autocorrelated distribution, this time attempting to account for the autocorrelation using a t-distribution, and measure how many of their 95% confidence intervals actually contain the true mean to see if the correction works."
]
Expand Down Expand Up @@ -687,7 +687,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##c. Newey-West Matrix\n",
"## c. Newey-West Matrix\n",
"\n",
"Use the `newey_west_matrix` helper function to compute an adjusted (robust) covariance matrix for a single sample of the autocorrelated data. "
]
Expand Down Expand Up @@ -720,7 +720,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"##d. Newey-West Correction\n",
"## d. Newey-West Correction\n",
"\n",
"Run 100 samples of the following autocorrelated distribution, this time accounting for the autocorrelation by using a Newey-West correction on the standard error, and measure how many of their 95% confidence intervals actually contain the true mean to see if the correction works. (Use the helper function `newey_west_SE` to find the corrected standard error)"
]
Expand Down Expand Up @@ -760,6 +760,19 @@
"print \"After accounting for autocorrelation by finding a Newey-West standard error, the actual number of confidence intervals containing the population mean is about what we expected.\"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"Congratulations on completing the Confidence Intervals exercises!\n",
"\n",
"As you learn more about writing trading models and the Quantopian platform, enter the daily [Quantopian Contest](https://www.quantopian.com/contest). Your strategy will be evaluated for a cash prize every day.\n",
"\n",
"Start by going through the [Writing a Contest Algorithm](https://www.quantopian.com/tutorials/contest) tutorial."
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -790,4 +803,4 @@
},
"nbformat": 4,
"nbformat_minor": 1
}
}
26 changes: 19 additions & 7 deletions notebooks/lectures/Confidence_Intervals/answers/preview.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13003,10 +13003,10 @@ <h1 id="Exercise-4:-Small-Sample-Sizes">Exercise 4: Small Sample Sizes<a class="
<span class="n">sample_mean</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">samples</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
<span class="n">sample_SE</span> <span class="o">=</span> <span class="n">stats</span><span class="o">.</span><span class="n">sem</span><span class="p">(</span><span class="n">samples</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
<span class="k">if</span> <span class="p">((</span><span class="n">POPULATION_MU</span> <span class="o">&gt;=</span> <span class="o">-</span><span class="mf">1.96</span> <span class="o">*</span> <span class="n">sample_SE</span> <span class="o">+</span> <span class="n">sample_mean</span><span class="p">)</span> <span class="ow">and</span> <span class="p">(</span><span class="n">POPULATION_MU</span> <span class="o">&lt;=</span> <span class="mf">1.96</span> <span class="o">*</span> <span class="n">sample_SE</span> <span class="o">+</span> <span class="n">sample_mean</span><span class="p">)):</span>
<span class="n">correct</span> <span class="o">+=</span> <span class="mi">1</span><span class="p">;</span>
<span class="n">correct</span> <span class="o">+=</span> <span class="mi">1</span>

<span class="k">print</span> <span class="s">&quot;Expected Result:&quot;</span><span class="p">,</span> <span class="o">.</span><span class="mi">95</span> <span class="o">*</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">&quot;Actual Result&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Actual Result:&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Due to the small sample size, the actual number of confidence intervals containing the population mean is much lower than what we would expect given a correctly calibrated interval.&quot;</span>
</pre></div>

Expand Down Expand Up @@ -13061,10 +13061,10 @@ <h2 id="b.-T-distribution-Correction">b. T-distribution Correction<a class="anch
<span class="n">sample_SE</span> <span class="o">=</span> <span class="n">stats</span><span class="o">.</span><span class="n">sem</span><span class="p">(</span><span class="n">samples</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
<span class="n">h</span> <span class="o">=</span> <span class="n">sample_SE</span> <span class="o">*</span> <span class="n">stats</span><span class="o">.</span><span class="n">t</span><span class="o">.</span><span class="n">ppf</span><span class="p">((</span><span class="mi">1</span><span class="o">+</span><span class="mf">0.95</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">samples</span><span class="p">[</span><span class="n">i</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="p">((</span><span class="n">POPULATION_MU</span> <span class="o">&gt;=</span> <span class="n">sample_mean</span> <span class="o">-</span> <span class="n">h</span><span class="p">)</span> <span class="ow">and</span> <span class="p">(</span><span class="n">POPULATION_MU</span> <span class="o">&lt;=</span> <span class="n">sample_mean</span> <span class="o">+</span> <span class="n">h</span><span class="p">)):</span>
<span class="n">correct</span> <span class="o">+=</span> <span class="mi">1</span><span class="p">;</span>
<span class="n">correct</span> <span class="o">+=</span> <span class="mi">1</span>

<span class="k">print</span> <span class="s">&quot;Expected Result:&quot;</span><span class="p">,</span> <span class="o">.</span><span class="mi">95</span> <span class="o">*</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">&quot;Actual Result&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Actual Result:&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;After using the t-distribution to correct for the smaller sample size, the actual number of confidence intervals containing the population mean is about what we expected.&quot;</span>
</pre></div>

Expand Down Expand Up @@ -13133,7 +13133,7 @@ <h1 id="Exercise-5:-Dependence">Exercise 5: Dependence<a class="anchor-link" hre


<span class="k">print</span> <span class="s">&quot;Expected Result:&quot;</span><span class="p">,</span> <span class="o">.</span><span class="mi">95</span> <span class="o">*</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">&quot;Actual Result&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Actual Result:&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Because the underlying data was autocorrelated, the actual number of confidence intervals containing the population mean is much lower than what we expected.&quot;</span>
</pre></div>

Expand Down Expand Up @@ -13191,7 +13191,7 @@ <h2 id="b.-T-distribution-Correction">b. T-distribution Correction<a class="anch


<span class="k">print</span> <span class="s">&quot;Expected Result:&quot;</span><span class="p">,</span> <span class="o">.</span><span class="mi">95</span> <span class="o">*</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">&quot;Actual Result&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Actual Result:&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;We did not see a significant improvement in the actual number of confidence intervals containing the population mean. This is because a t-distribution only corrects for small sample sizes, not autocorrelation.&quot;</span>
</pre></div>

Expand Down Expand Up @@ -13293,7 +13293,7 @@ <h2 id="d.-Newey-West-Correction">d. Newey-West Correction<a class="anchor-link"

<span class="k">print</span> <span class="s">&quot;New Standard Error:&quot;</span><span class="p">,</span> <span class="n">sample_SE</span>
<span class="k">print</span> <span class="s">&quot;Expected Result:&quot;</span><span class="p">,</span> <span class="o">.</span><span class="mi">95</span> <span class="o">*</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">&quot;Actual Result&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;Actual Result:&quot;</span><span class="p">,</span> <span class="n">correct</span>
<span class="k">print</span> <span class="s">&quot;After accounting for autocorrelation by finding a Newey-West standard error, the actual number of confidence intervals containing the population mean is about what we expected.&quot;</span>
</pre></div>

Expand Down Expand Up @@ -13322,6 +13322,18 @@ <h2 id="d.-Newey-West-Correction">d. Newey-West Correction<a class="anchor-link"
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<hr>
<p>Congratulations on completing the Confidence Intervals exercises!</p>
<p>As you learn more about writing trading models and the Quantopian platform, enter the daily <a href="https://www.quantopian.com/contest">Quantopian Contest</a>. Your strategy will be evaluated for a cash prize every day.</p>
<p>Start by going through the <a href="https://www.quantopian.com/tutorials/contest">Writing a Contest Algorithm</a> tutorial.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
Expand Down
Loading

0 comments on commit abdb327

Please sign in to comment.