Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace spring20 urls with master #451

Merged
merged 3 commits into from
Aug 8, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion setup_colab.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

apt-get -qq update
apt-get -qq install -y xvfb
wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/xvfb -O ../xvfb
wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/xvfb -O ../xvfb
2 changes: 1 addition & 1 deletion week01_intro/crossentropy_method.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" !touch .setup_complete\n",
"\n",
"# This code creates a virtual display to draw game images on.\n",
Expand Down
2 changes: 1 addition & 1 deletion week01_intro/deep_crossentropy_method.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion week01_intro/primer_python_for_ml/recap_ml.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@
"\n",
"import sys\n",
"if 'google.colab' in sys.modules:\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week01_intro/primer_python_for_ml/train.csv"
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week01_intro/primer_python_for_ml/train.csv"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion week01_intro/seminar_gym_interface.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion week02_value_based/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@

## Homework description:

The main assignment is `seminar_vi.ipynb`[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week02_value_based/seminar_vi.ipynb) notebook in this week's folder. It has no requirements besides the most basic data science libraries (e.g. numpy) so you should be able to run it locally.
The main assignment is `seminar_vi.ipynb`[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week02_value_based/seminar_vi.ipynb) notebook in this week's folder. It has no requirements besides the most basic data science libraries (e.g. numpy) so you should be able to run it locally.

__Note:__ if you have any difficulty using graphviz, just set `has_graphviz=False`.
4 changes: 2 additions & 2 deletions week02_value_based/seminar_vi.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week02_value_based/mdp.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week02_value_based/mdp.py\n",
" !touch .setup_complete\n",
"\n",
"# This code creates a virtual display to draw game images on.\n",
Expand Down
12 changes: 6 additions & 6 deletions week03_model_free/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
* Russian materials:
- Lecture - [video](https://yadi.sk/i/jcQ1Bg8n3SrhuQ)
- Seminars
- Q-learning seminar - [video](https://yadi.sk/i/dQmolwOy3EtGNK) (older track - [assignment](https://github.com/neer201/Practical_RL/tree/spring20/week03_model_free/crawler_and_pacman/seminar_py2))
- SARSA & stuff - [video](https://yadi.sk/i/XbqNQmjm3ExNsq)
- Q-learning seminar - [video](https://yadi.sk/i/dQmolwOy3EtGNK) (older track - [assignment](https://github.com/neer201/Practical_RL/tree/master/week03_model_free/crawler_and_pacman/seminar_py2))
- SARSA & stuff - [video](https://yadi.sk/i/XbqNQmjm3ExNsq)
* English materials:
- Lecture by David Silver (english) - [video part I](https://www.youtube.com/watch?v=PnHCvfgC_ZA), [video part II](https://www.youtube.com/watch?v=0g4j2k_Ggc4&t=43s)
- Alternative lecture by Pieter Abbeel (english) - [video](https://www.youtube.com/watch?v=ifma8G7LegE)
Expand All @@ -18,9 +18,9 @@

### Assignments

Just as usual, start with
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week03_model_free/seminar_qlearning.ipynb)
Just as usual, start with
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week03_model_free/seminar_qlearning.ipynb)
`seminar_qlearning.ipynb` _Implement q-learning agent and test it on Taxi and CartPole with binarizer_

and then proceed to
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week03_model_free/homework.ipynb) `homework.ipynb` _Implement EV-SARSA agent, experience replay + bonus tasks_
and then proceed to
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week03_model_free/homework.ipynb) `homework.ipynb` _Implement EV-SARSA agent, experience replay + bonus tasks_
12 changes: 6 additions & 6 deletions week03_model_free/crawler_and_pacman/seminar_py2/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Disclaimer
This assignment is not supported now. You can do it at your own risk.
This assignment is not supported now. You can do it at your own risk.

_this assignment borrows code from awesome [cs188](http://ai.berkeley.edu/project_overview.html)_
This homework assignment works on __python2 only__. If you stick to py3, consider seminar_alternative. Or just install it for this homework alone and remove afterwards.
Expand All @@ -17,8 +17,8 @@ python pacman.py -p PacmanQAgent -x 5000 -n 5010 -l smallGrid # example
```
* Make sure you can tune agent to beat ./run_crawler.sh
* on windows, just run `python crawler.py` from cmd in the project directory
* other ./run* files are mostly for your amusement.
* ./run_pacman.sh will need more epochs to converge, see [comments](https://github.com/yandexdataschool/Practical_RL/blob/spring20/week03_model_free/crawler_and_pacman/seminar_py2/run_pacman.sh)
* other ./run* files are mostly for your amusement.
* ./run_pacman.sh will need more epochs to converge, see [comments](https://github.com/yandexdataschool/Practical_RL/blob/master/week03_model_free/crawler_and_pacman/seminar_py2/run_pacman.sh)
* on windows, just copy the type `python pacman.py -p PacmanQAgent -x 2000 -n 2010 -l smallGrid` in cmd from assignemnt dir


Expand All @@ -35,14 +35,14 @@ The problem with those environments is that they have a large amount of unique s
* where is nearest food
* 'center of mass' of all food points (and variance, and whatever)
* is there a wall in each direction
* and anything else you see fit
* and anything else you see fit

Here's how to get this information from [state](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L49),
* Get pacman position: [state.getPacmanPosition()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L128)
* Is there a wall at (x,y)?: [state.hasWall(x,y)](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L189)
* Get ghost positions: [state.getGhostPositions()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L144)
* Get all food positions: [state.getCapsules()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L153)

You can call those methods anywhere you see state.
* e.g. in [agent.getValue(state)](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L52)
* Defining a function that extracts all features and calling it in [getQValue](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L38) and [setQValue](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L44) is probably enough.
Expand Down
14 changes: 7 additions & 7 deletions week03_model_free/crawler_and_pacman/seminar_py3/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
Contributors: [Vlad Lyalin](https://github.com/Guitaricet)

# Disclaimer
This assignment is not supported now. You can do it at your own risk.
This assignment is not supported now. You can do it at your own risk.

# Requirements
__Ubuntu__ - not tested

__MacOS__ Python 3.7.0 (3.6, 3.8 and other cause on MacOS [problems](https://stackoverflow.com/questions/57400301/how-to-fix-tkinter-every-code-with-gui-crashes-mac-os-with-respring) with _Tkinter_)

Anaconda users just can create new_env:
```bash
conda create -n pacman_env python=3.7.0
Expand All @@ -31,8 +31,8 @@ python pacman.py -p PacmanQAgent -x 5000 -n 5010 -l smallGrid # example
```
* Make sure you can tune agent to beat ./run_crawler.sh
* on windows, just run `python crawler.py` from cmd in the project directory
* other ./run* files are mostly for your amusement.
* ./run_pacman.sh will need more epochs to converge, see [comments](https://github.com/yandexdataschool/Practical_RL/blob/spring20/week03_model_free/crawler_and_pacman/seminar_py2/run_pacman.sh)
* other ./run* files are mostly for your amusement.
* ./run_pacman.sh will need more epochs to converge, see [comments](https://github.com/yandexdataschool/Practical_RL/blob/master/week03_model_free/crawler_and_pacman/seminar_py2/run_pacman.sh)
* on windows, just copy the type `python pacman.py -p PacmanQAgent -x 2000 -n 2010 -l smallGrid` in cmd from assignemnt dir


Expand All @@ -49,14 +49,14 @@ The problem with those environments is that they have a large amount of unique s
* where is nearest food
* 'center of mass' of all food points (and variance, and whatever)
* is there a wall in each direction
* and anything else you see fit
* and anything else you see fit

Here's how to get this information from [state](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L49),
* Get pacman position: [state.getPacmanPosition()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L128)
* Is there a wall at (x,y)?: [state.hasWall(x,y)](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L189)
* Get ghost positions: [state.getGhostPositions()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L144)
* Get all food positions: [state.getCapsules()](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/pacman.py#L153)

You can call those methods anywhere you see state.
* e.g. in [agent.getValue(state)](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L52)
* Defining a function that extracts all features and calling it in [getQValue](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L38) and [setQValue](https://github.com/yandexdataschool/Practical_RL/blob/7a559f8/week03_model_free/seminar_py2/qlearningAgents.py#L44) is probably enough.
Expand Down
4 changes: 2 additions & 2 deletions week03_model_free/homework.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" !touch .setup_complete\n",
"\n",
"# This code creates a virtual display to draw game images on.\n",
Expand Down Expand Up @@ -425,7 +425,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" !touch .setup_complete\n",
"\n",
"# This code creates a virtual display to draw game images on.\n",
Expand Down
2 changes: 1 addition & 1 deletion week03_model_free/seminar_qlearning.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion week04_[recap]_deep_learning/seminar_pytorch.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week04_%5Brecap%5D_deep_learning/notmnist.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_%5Brecap%5D_deep_learning/notmnist.py\n",
"\n",
" !touch .setup_complete"
]
Expand Down
4 changes: 2 additions & 2 deletions week04_[recap]_deep_learning/seminar_tensorflow.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@
" %tensorflow_version 1.x\n",
" \n",
" if not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week04_[recap]_deep_learning/mnist.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_[recap]_deep_learning/mnist.py\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
8 changes: 3 additions & 5 deletions week04_approx_rl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@

## Practice

* Seminar: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week04_approx_rl/seminar_pytorch.ipynb)
* Homework (main): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week04_approx_rl/homework_pytorch_main.ipynb#scrollTo=KVvvo7k_ap8w)
* Homework (debug): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/spring20/week04_approx_rl/homework_pytorch_debug.ipynb#scrollTo=KVvvo7k_ap8w)
* Seminar: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/seminar_pytorch.ipynb)
* Homework (main): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/homework_pytorch_main.ipynb#scrollTo=KVvvo7k_ap8w)
* Homework (debug): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yandexdataschool/Practical_RL/blob/master/week04_approx_rl/homework_pytorch_debug.ipynb#scrollTo=KVvvo7k_ap8w)



Expand All @@ -47,5 +47,3 @@ Begin with `seminar_<framework>.ipynb` and then proceed with `homework_<framewor


__Note: you're not required to submit assignments in all three frameworks. Pick one and go with it. Maybe switch it occasionally if you want more challenge. __


2 changes: 1 addition & 1 deletion week04_approx_rl/homework_pytorch_debug.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" \n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/atari_wrappers.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/utils.py\n",
Expand Down
2 changes: 1 addition & 1 deletion week04_approx_rl/homework_pytorch_main.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" \n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/atari_wrappers.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/utils.py\n",
Expand Down
6 changes: 3 additions & 3 deletions week04_approx_rl/homework_tf.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@
" %tensorflow_version 1.x\n",
" \n",
" if not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week04_approx_rl/framebuffer.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/week04_approx_rl/replay_buffer.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/framebuffer.py\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/week04_approx_rl/replay_buffer.py\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion week04_approx_rl/seminar_pytorch.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"source": [
"import sys, os\n",
"if 'google.colab' in sys.modules and not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
" !touch .setup_complete\n",
"\n",
"# This code creates a virtual display to draw game images on.\n",
Expand Down
2 changes: 1 addition & 1 deletion week04_approx_rl/seminar_tf.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
" %tensorflow_version 1.x\n",
" \n",
" if not os.path.exists('.setup_complete'):\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/spring20/setup_colab.sh -O- | bash\n",
" !wget -q https://raw.githubusercontent.com/yandexdataschool/Practical_RL/master/setup_colab.sh -O- | bash\n",
"\n",
" !touch .setup_complete\n",
"\n",
Expand Down
Loading