Update README

jakvah · Jul 28, 2020 · e83cff6 · e83cff6
1 parent ca9f4d5
commit e83cff6
Show file tree

Hide file tree

Showing 2 changed files with 32 additions and 10 deletions.
diff --git a/gym-drill/README.md b/gym-drill/README.md
@@ -1,6 +1,16 @@
 ## Custom Well Trajectory Environment
 
-Gym-Drill is a custom, OpenAI Gym compatible environment modelling a subsurface reservoar. Its main purpose is to be used in a reinforcement learning context to train an agent to find the best path from a given starting point, to a set of target balls. The environment inflicts a curvature constraint (dogleg severity limitation) on the agents well path. An option to include Hazards (hard constraints that the path cannot intersect) into the subsurface environment also exist. 
+Gym-Drill is a custom, OpenAI Gym compatible environment modelling a subsurface reservoar. Its main purpose is to be used in a reinforcement learning context to train an agent to find the best path from a given starting point, to a set of target balls. The environment inflicts a curvature constraint (dogleg severity limitation) on the agents well path. An option to include Hazards (hard constraints that the path cannot intersect) into the subsurface environment also exist.
+
+### Table of Contents
+
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [Advanced use](#advanced-use)
+    - [Overwriting the starting conditions](#overwriting-the-starting-conditions)
+    - [Toggle hazards, Monte Carlo simulated training and episode log](#toggle-hazards,-monte-carlo-simulated-training-and-episode-log)
+    - [Adjust environment parameters](#adjust-environment-parameters)
+    - [Key functions to utilize when training](#key-functions-and-attributes-to-utilize-when-training)
 
 ### Installation
 
@@ -45,7 +55,7 @@ To initialize an instance of the environment with your own specificed parameters
 ```python
 env = gym.make(env_name,startLocation = STARTLOCATION, bitInitialization = BIT_INITIALIZATION)
 ```
-where `STARTLOCATION` is of type [**Coordinate**](gym_drill/envs/Coordinate.py) and `BIT_INITIALIZATION` is a list/tuple on the form `[initialAngle, initialAngularVelocity, initialAngularAcceleration]`. An example of creating an environment with custom parameters would be:
+where `STARTLOCATION` is of type [**Coordinate**](gym_drill/envs/Coordinate.py) and `BIT_INITIALIZATION` is a list/tuple on the form ```[initial_azimuth_angle, initial_azimuth_angular_velocity, initial_azimuth_angular_acceleration,initial_inclination_angle, initial_inclination_angular_velocity, initial_inclination_angular_acceleration```. An example of creating an environment with custom parameters would be:
 
 ```python
 import gym
@@ -55,7 +65,7 @@ import numpy as np
 from gym_drill.envs.customAdditions import Coordinate
 
 STARTLOCATION = Coordinate(0.0,0.0,0.0)
-BIT_INITIALIZATION = [0.0,0.0,0.0]
+BIT_INITIALIZATION = [0.0,0.0,0.0,0.0,0.0,0.0]
 
 env_name = 'drill-v0'
 env = gym.make(env_name,startLocation = STARTLOCATION, bitInitialization = BIT_INITIALIZATION)
@@ -77,7 +87,18 @@ env = gym.make("drill-v0",activate_hazards = False,monte_carlo = False,activate_
 ```
 
 #### Adjust environment parameters
-physical attributes, movement limitations, rewards, what is contained in the observation space 
-#### Key functions to utilize when training
+
+The environment and an agent exeperience in the environment is described by a set of variables that control physical attributes, movement limitations, rewards, what is contained in the observation space and more. These are all stored in the [environment_config](gym_drill/envs/environment_config.py) file. If you feel like changing aspects of the environment for yourself by tweaking these variables all you have to do is update the values inside this file.
+#### Key functions and attributes to utilize when training
+As the environment is OpenAI gym compatible it has all the attributes and functions you would expect to be in an OpenAI gym environement pr [the documentation](https://gym.openai.com/docs/). These include, but are not limited to:
+
+- A ``reset()`` function which resets the environment to its initial state. Note that even if you are [overwriting the starting conditions](#overwriting-the-starting-conditions) the Azimuth and Inclination angle will be drawn randomly, to ensure that the training is not beeing overfitted for one particular starting angle.
+- A ``step()`` function that accepts an ``action`` and executes that action in the environment. In accordance with the documentation the ``step()`` function returns:
+    - An ``observation`` object, containing the new state of the environment after the action has been executed
+    - A ``reward`` (float), which is the reward the agent recieved for exectuing the particular action 
+    - A ``done`` signal (boolean) indicating wheter or not the episode is over
+    - An ``info`` (dictionary) message containing diagnostic information useful for debugging. 
+
+The only deviation from the functions describes in the documentation is that the ``render()`` function that most OpenAI gym environment use to display the environment has been replaced with two seperate functions. ``display_planes()`` for displaying the horizontal (xy) and vertical (zy) planes and ``display_3d_environment()`` which displays the path and environment in a 3D plot.
 
 *Last updated 28.07.2020*
diff --git a/gym-drill/gym_drill/envs/environment_config.py b/gym-drill/gym_drill/envs/environment_config.py
@@ -1,13 +1,12 @@
 """
-This config file completely describes the "physical" aspects of the environemnt aswell as its rewards system
+This config file completely describes the "physical" aspects of the environment aswell as its rewards system
 """
 
 import numpy as np
 
 # Limits on the angles. They should match the limits of angles describing a sphere in spherical coordinates
 MIN_INCL_ANGLE = 0
 MAX_INCL_ANGLE = np.pi
-
 MAX_AZIMUTH_ANGLE = 2*np.pi
 
 # Max values for angular velocity and acceleration for both angles
@@ -20,20 +19,21 @@
 # Step size. For each step the bit position gets updated with DRILL_SPEED multiplied with cos/sin of one of the angles
 DRILL_SPEED = 10
 
-# Screen size, environment should be square
+# Envrionment dimensions
 SCREEN_X = 2000
 SCREEN_Y = 2000
 SCREEN_Z = 2000
 
 # Step budget agent has available 
 NUM_MAX_STEPS = ((SCREEN_X+SCREEN_Y+SCREEN_Z)/DRILL_SPEED)*1.5
 
-# Target specs specifying where a target can exist
+# Target specs specifying where a target can exist and how big it (the radius) can be
 TARGET_BOUND_X = [0.25*SCREEN_X,0.75*SCREEN_X]
 TARGET_BOUND_Y = [0.25*SCREEN_Y,0.75*SCREEN_Y]
 TARGET_BOUND_Z = [0.40*SCREEN_Z,0.85*SCREEN_Z]
 TARGET_RADII_BOUND = [40,50]
 
+# Numbers of targets in the environment and number of targets stored in the observation space at all times
 NUM_TARGETS = 8
 TARGET_WINDOW_SIZE = 3
 
@@ -43,6 +43,7 @@
 HAZARD_BOUND_Z = [0,SCREEN_Z]
 HAZARD_RADII_BOUND = [100,150]
 
+# Numbers of hazards in the environment and number of hazards stored in the observation space at all times
 NUM_HAZARDS = 8
 HAZARD_WINDOW_SIZE = 1
 
@@ -68,7 +69,7 @@
 INCLINATION_REWARD_FACTOR = 0.5
 FINISHED_EARLY_FACTOR = 1 # Point per unused step
 
-# Generating environments with a Monte Carlo simulation
+# Monte Carlo simulation specs
 NUM_MONTE_CARLO_ENVS = int(1e4)
 MC_PATH_LENGTH_BOUND = [250,350]
 ENVIRONMENT_FILENAME = "environments.txt"