{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e8966967-97bc-406e-a2f4-4a62d8f9e895",
   "metadata": {},
   "source": [
    "[<img align=\"left\" src=\"https://colab.research.google.com/assets/colab-badge.svg\">](https://colab.research.google.com/github/pytorch/rl/blob/main/tutorials/envs.ipynb)\n",
    "\n",
    "# TorchRL envs (`torchrl.envs`)\n",
    "\n",
    "Environments play a crucial role in RL settings, often somewhat similar to datasets in supervised and unsupervised settings.\n",
    "The RL community has become quite familiar with OpenAI gym API which offers a flexible way of building environments, initializing them and interacting with them. \n",
    "However, many other libraries exist, and the way one interacts with them can be quite different from what is expected with gym.\n",
    "\n",
    "Let us start by describing how TorchRL interacts with gym, which will serve as an introduction to other frameworks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b331338",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install functorch torchvision\n",
    "!pip install \"gym[classic_control]\"\n",
    "!pip install dm_control matplotlib\n",
    "!pip install torchrl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f461815d-dfd2-4d48-8d9b-21cc25f55464",
   "metadata": {},
   "source": [
    "## Gym environments\n",
    "\n",
    "To run this part of the tutorial, you will need to have a recent version of the gym library installed, as well as the atari suite.\n",
    "You can get this installed by installing the following packages:\n",
    "\n",
    "```\n",
    "pip install gym atari-py ale-py gym[accept-rom-license] pygame\n",
    "```\n",
    "\n",
    "To unify all frameworks, torchrl environments are built inside the `__init__` method with a private method called `_build_env` that will pass the arguments and keyword arguments to the root library builder.\n",
    "\n",
    "With gym, it means that building an environment is as easy as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "09a90ffb-eba0-458e-912d-568ea006e15c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs.libs.gym import GymEnv\n",
    "from matplotlib import pyplot as plt\n",
    "from torchrl.data import TensorDict\n",
    "import torch\n",
    "env = GymEnv(\"Pendulum-v1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b508f501-20a0-4e44-b928-17410cf27eb6",
   "metadata": {},
   "source": [
    "The list of available environment can be accessed through this command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a2b3c152-be95-4140-ab2e-92c9df1c40bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ALE/Adventure-ram-v5',\n",
       " 'ALE/Adventure-v5',\n",
       " 'ALE/AirRaid-ram-v5',\n",
       " 'ALE/AirRaid-v5',\n",
       " 'ALE/Alien-ram-v5',\n",
       " 'ALE/Alien-v5',\n",
       " 'ALE/Amidar-ram-v5',\n",
       " 'ALE/Amidar-v5',\n",
       " 'ALE/Assault-ram-v5',\n",
       " 'ALE/Assault-v5']"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GymEnv.available_envs[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "330e470a-ec2e-436c-b1f7-ff2e1f4704c8",
   "metadata": {},
   "source": [
    "### Env specs\n",
    "\n",
    "Like other frameworks, TorchRL envs have attributes that indicate what space is for the observations, action and reward. \n",
    "Because it often happens that more than one observation is retrieved, we expect the observation spec to be of type `CompositeSpec`. Reward and action do not have this restriction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "36c1475f-c14a-4c76-ac0f-9fd9177ed5e1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Env observation_spec: \n",
      " CompositeSpec(\n",
      "    next_observation: NdBoundedTensorSpec(\n",
      "         shape=torch.Size([3]), space=ContinuousBox(minimum=tensor([-1., -1., -8.]), maximum=tensor([1., 1., 8.])), device=cpu, dtype=torch.float32, domain=continuous))\n",
      "Env action_spec: \n",
      " NdBoundedTensorSpec(\n",
      "     shape=torch.Size([1]), space=ContinuousBox(minimum=tensor([-2.]), maximum=tensor([2.])), device=cpu, dtype=torch.float32, domain=continuous)\n",
      "Env reward_spec: \n",
      " UnboundedContinuousTensorSpec(\n",
      "     shape=torch.Size([1]), space=ContinuousBox(minimum=-inf, maximum=inf), device=cpu, dtype=torch.float32, domain=composite)\n"
     ]
    }
   ],
   "source": [
    "print(\"Env observation_spec: \\n\", env.observation_spec)\n",
    "print(\"Env action_spec: \\n\", env.action_spec)\n",
    "print(\"Env reward_spec: \\n\", env.reward_spec)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab3e1a6b-06a8-47e6-b43d-d9cb4b82150a",
   "metadata": {},
   "source": [
    "Those spec come with a series of useful tools: one can assert whether a sample is in the defined space. We can also use some heuristic to project a sample in the space if it is out of space, and generate random (possibly uniformly distributed) numbers in that space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ad7d55fe-6dda-4757-ada8-5b8dddc41729",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "action is in bounds?\n",
      " False\n",
      "projected action: \n",
      " tensor([2.])\n"
     ]
    }
   ],
   "source": [
    "action = torch.ones(1) * 3\n",
    "print(\"action is in bounds?\\n\", bool(env.action_spec.is_in(action)))\n",
    "print(\"projected action: \\n\", env.action_spec.project(action))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fa103f09-c3a4-4c3e-b39c-6253599d0fec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "random action: \n",
      " tensor([-0.8754])\n"
     ]
    }
   ],
   "source": [
    "print(\"random action: \\n\", env.action_spec.rand())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41045768-1c5b-46a9-9941-a5797bb3185f",
   "metadata": {},
   "source": [
    "Envs are also packed with an `env.input_spec` attribute of type `CompositeSpec`. In brief, `input_spec` should contain all the specs of the inputs that are required for an env to exectute a step. For stateful envs (e.g. gym) this should include the action.\n",
    "With stateless environments (e.g. Brax) this should also include a representation of the previous state. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4d99bce-99ef-44f5-8406-de7da52cb23f",
   "metadata": {},
   "source": [
    "### Seeding, resetting and steps\n",
    "\n",
    "The basic operations on an environment are (1) `set_seed`, (2) `reset` and (3) `step`.\n",
    "\n",
    "Let's see how these methods work with TorchRL:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0e8bff90-5046-4888-8750-cfc7f050167a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        observation: Tensor(torch.Size([3]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(0)  # make sure that all torch code is also reproductible\n",
    "env.set_seed(0)\n",
    "tensordict = env.reset()\n",
    "print(tensordict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2936a240-8a03-4726-94fe-6151cc4f7f3e",
   "metadata": {},
   "source": [
    "We can now execute a step in the environment. \n",
    "Since we don't have a policy, we can just generate a random action:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "676c3ddc-3396-4e93-a474-2e0a403ec14d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def policy(tensordict):\n",
    "    tensordict.set(\"action\", env.action_spec.rand())\n",
    "    return tensordict\n",
    "policy(tensordict)\n",
    "tensordict_out = env.step(tensordict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a7364c-d8e9-44de-a9da-977e8d14c094",
   "metadata": {},
   "source": [
    "By default, the tensordict returned by `step` is the same as the input..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "903ad364-0843-4674-9625-11fdece3eb18",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert tensordict_out is tensordict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64aac817-a77d-4c8b-b6da-75a90f1ac1be",
   "metadata": {},
   "source": [
    "... but with new keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9cabe1d8-d904-4795-abc3-9b404ee9e9b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        action: Tensor(torch.Size([1]), dtype=torch.float32),\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        next_observation: Tensor(torch.Size([3]), dtype=torch.float32),\n",
       "        observation: Tensor(torch.Size([3]), dtype=torch.float32),\n",
       "        reward: Tensor(torch.Size([1]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensordict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "581b7ab6-f542-444f-970f-9755b18051cc",
   "metadata": {},
   "source": [
    "What we just did (a random step using `action_spec.rand()`) can also be done via the simple shortcut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2c08ee82-8d0e-4735-ba80-5944f393340e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        action: Tensor(torch.Size([1]), dtype=torch.float32),\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        next_observation: Tensor(torch.Size([3]), dtype=torch.float32),\n",
       "        reward: Tensor(torch.Size([1]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env.rand_step()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46b3e9c0-ad70-475b-90ae-11787b900ed3",
   "metadata": {},
   "source": [
    "The new key `\"next_observation\"` (as all keys starting with `\"next_\"`) have a special role in TorchRL: they indicate that they come after the key with the same name but without the prefix.\n",
    "\n",
    "We provide a function `step_mdp` that executes a step in the tensordict: it returns a new tensordict updated such that $t <- t'$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ad697e31-ec9d-4607-942a-93f96b5f0e85",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorDict(\n",
      "    fields={\n",
      "        observation: Tensor(torch.Size([3]), dtype=torch.float32),\n",
      "        some other key: Tensor(torch.Size([1]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n",
      "tensor(True)\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs.utils import step_mdp\n",
    "tensordict.set(\"some other key\", torch.randn(1))\n",
    "tensordict_tprime = step_mdp(tensordict)\n",
    "print(tensordict_tprime)\n",
    "print((tensordict_tprime.get(\"observation\") == tensordict.get(\"next_observation\")).all())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21925dd8-6492-401c-a09b-60ad6a7774d8",
   "metadata": {},
   "source": [
    "We can observe that `step_mdp` has removed all the time-dependent key-value pairs, but not `\"some other key\"`. Also, the new observation matches the previous one"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14d2951e-c263-4d06-903b-686191ecf97b",
   "metadata": {},
   "source": [
    "Finally, note that the `env.reset` method also accepts a tensordict to update:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "bc928092-ade0-46b3-836b-4f21caafc3a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        observation: Tensor(torch.Size([3]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensordict = TensorDict({}, [])\n",
    "assert env.reset(tensordict) is tensordict\n",
    "tensordict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14ae176d-a7af-4c3d-82fa-bf69d375bae8",
   "metadata": {},
   "source": [
    "### Rollouts\n",
    "\n",
    "The generic environment class provided by TorchRL allows you to run rollouts easily for a given number of steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "41d820f7-a063-4947-935d-6018f05c12ff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorDict(\n",
      "    fields={\n",
      "        action: Tensor(torch.Size([20, 1]), dtype=torch.float32),\n",
      "        done: Tensor(torch.Size([20, 1]), dtype=torch.bool),\n",
      "        next_observation: Tensor(torch.Size([20, 3]), dtype=torch.float32),\n",
      "        observation: Tensor(torch.Size([20, 3]), dtype=torch.float32),\n",
      "        reward: Tensor(torch.Size([20, 1]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([20]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    }
   ],
   "source": [
    "tensordict_rollout = env.rollout(max_steps=20, policy=policy)\n",
    "print(tensordict_rollout)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e0bb1e5-8661-4187-b97b-69bf64389a71",
   "metadata": {},
   "source": [
    "The resulting tensordict has a `batch_size` of `[20]`, which is the length of the trajectory. We can check that the observation match their next value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "eb449dec-640b-43d8-8f46-70a2de3cf469",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(tensordict_rollout.get(\"observation\")[1:] == tensordict_rollout.get(\"next_observation\")[:-1]).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "630dfdb7-d448-4c27-865f-4bb455d016b4",
   "metadata": {},
   "source": [
    "### frame_skip\n",
    "\n",
    "In some instances, it is useful to use a `frame_skip` argument to use the same action for several consecutive frames.\n",
    "\n",
    "The resulting tensordict will contain only the last frame observed in the sequence, but the rewards will be summed over the number of frames. \n",
    "\n",
    "If the environment reaches a done state during this process, it'll stop and return the result of the truncated chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "6b2cb9da-d976-410e-92d3-19ad75bd228e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        observation: Tensor(torch.Size([3]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env = GymEnv(\"Pendulum-v1\", frame_skip=4)\n",
    "env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be11c29c-1a68-4cd6-a8dd-4aebdf72785e",
   "metadata": {},
   "source": [
    "### Rendering\n",
    "\n",
    "Rendering plays an important role in many RL settings, and this is why the generic environment class from torchrl provides a `from_pixels` keyword argument that allows the user to quickly ask for image-based environments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "19cf0c74-ab92-4a8a-9c0a-984282a295ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "env = GymEnv(\"Pendulum-v1\", from_pixels=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "8b914a63-1734-4fa8-96b6-c2a859f273d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "tensordict = env.reset()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "39500114-9f7b-4150-bc11-ecdb7d38c3ff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13c1b5310>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAARbElEQVR4nO3dbYyV5Z3H8e9vnmdAeRwRGBBUbDW6WiWWpk3aaM1S21RjdKNptmRDwot1E7tt0mo27abJvmjf1LbpptFdm9JNW+1aE4kx6bJou9k0VaEoFREZ8AEQYRAYh8dhZv774lywIwzODXPuOWfm+n2Sk7mv6/7PnP8ww2/ux3MUEZhZvhpq3YCZ1ZZDwCxzDgGzzDkEzDLnEDDLnEPALHOlhICk5ZK2SuqW9GAZz2Fm1aFqXycgqRF4A7gN2AW8BNwXEa9V9YnMrCrK2BK4GeiOiB0R0Q88DtxRwvOYWRU0lfA15wM7h413AZ/8qE+YPXt2LFq0qIRWzOyUDRs27I+IzjPnywiBQiStAlYBLFy4kPXr19eqFbMsSHp7pPkydgd2AwuGjbvS3IdExKMRsTQilnZ2nhVOZjZOygiBl4AlkhZLagHuBdaU8DxmVgVV3x2IiAFJ/wD8DmgEfhYRm6v9PGZWHaUcE4iIZ4Fny/jaZlZdvmLQLHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDI3aghI+pmkfZJeHTY3U9JaSdvSxxlpXpJ+LKlb0iZJN5bZvJmNXZEtgZ8Dy8+YexBYFxFLgHVpDPAFYEl6rAJ+Wp02zawso4ZARPwPcOCM6TuA1Wl5NXDnsPlfRMWfgOmS5lapVzMrwYUeE5gTEXvS8nvAnLQ8H9g5rG5XmjuLpFWS1kta39PTc4FtmNlYjfnAYEQEEBfweY9GxNKIWNrZ2TnWNszsAl1oCOw9tZmfPu5L87uBBcPqutKcmdWpCw2BNcCKtLwCeHrY/FfTWYJlQO+w3QYzq0NNoxVI+jXwOWC2pF3APwPfA34jaSXwNvA3qfxZ4HagGzgK/F0JPZtZFY0aAhFx3zlW3TpCbQD3j7UpMxs/vmLQLHMOAbPMOQTMMucQMMucQ8Asc6OeHbDJLSIYOnaMDzZupG/zZoZOnKB94UKm3XwzrZdeiqRat2glcwhkLCI48e67vPPII/T95S8wOHh63b41a5i/YgUzPvMZ1OANxsnMP92MDRw6xNs/+Ql9L7/8oQAA6O/p4Z1HHqF3wwYql3/YZOUQyFREsH/tWg6/9to5awb7+tjzq18xdPToOHZm480hkKsIDv3xjzDKX/mj27ez58kniaGhcWrMxptDIGNFN/N7X3qJwSNHSu7GasUhYKPq37vXITCJOQQy1rF4caG6GBzk2M6doxfahOQQyJXElI99rFBpDAxwZOtWnyWYpBwCmZJES2cnam4uVN/f0wM+ODgpOQQy1nHFFTROmVKo9vCrrzJ04kTJHVktOAQy1tDWRtNFFxWqHervZ8AHByclh0DGGlpbmXrttYVqB/r6OLJlS8kdWS04BHIm0TJzZrHaoSEGent9cHAScghkTBJTr7228MHB3g0bRr3C0CYeh0DmWmbPRo2NhWpPHjhAnHGjkU18DoHMNV10Ee2XXVaotn//fk68+27JHdl4cwhkrqG9nZaCbwM3ePgwJw8eLLkjG28OgcxJomPJksL1R7ZtK7EbqwWHgNFx5ZWFa4+8/rrPEEwyDgGjqaOj8BmCk4cOMXTsWMkd2XhyCBhtXV20LVgweiFw7K23OHngQMkd2XhyCBhqaaGxo6NQbQwN+eDgJOMQMACmLV1arHBwkN7168ttxsaVQ8AAaLnkksK1A4cP+zUHJxGHgCGJ9ssuo7HgHYV9mzYx6FcgnjQcAgZAS2dn4eMCg0eO+LUFJhGHgAGgpibaFy4sVDt0/DhHXn+95I5svDgEDAA1NjLlqqsK1cbAAP09Pb5oaJIYNQQkLZD0vKTXJG2W9ECanylpraRt6eOMNC9JP5bULWmTpBvL/iZs7CRVDg4WvKPwxJ49vq14kiiyJTAAfCMirgGWAfdLugZ4EFgXEUuAdWkM8AVgSXqsAn5a9a6tFFOvvpqG1tZCtX2bN/u24kli1BCIiD0R8ee03AdsAeYDdwCrU9lq4M60fAfwi6j4EzBd0txqN27V19DeTmN7e6HawaNHGejrK7kjGw/ndUxA0iLgE8ALwJyI2JNWvQfMScvzgeHvVLErzVmda5oyhalXX12o9uTBgxx/+20fF5gECoeApKnAb4GvRcQHw9dF5TfhvH4bJK2StF7S+p6envP5VCtLYyNN06cXqx0cpH///lLbsfFRKAQkNVMJgF9GxFNpeu+pzfz0cV+a3w0MvxulK819SEQ8GhFLI2JpZ8EXtbBySeLiG24AqVD9B6+8Um5DNi6KnB0Q8BiwJSJ+MGzVGmBFWl4BPD1s/qvpLMEyoHfYboPVuZZLLoGGYhuI/fv2EQMDJXdkZSvy0/408LfALZJeTo/bge8Bt0naBnw+jQGeBXYA3cC/AX9f/batLC0zZ9I2b16h2hPvvcfJ998vuSMrW9NoBRHxv8C5tg9vHaE+gPvH2JfVSOPUqTTPnMnxAu9CPNDby8lDh2i99NJx6MzK4isG7UPU0EDHFVcUK47g6Pbt5TZkpXMI2FmKvmU5wJE33vBpwgnOIWBnaZk9m4aCFw319/T4jsIJziFgZ2mdN4/mGTMK1R7bsYPBw4dL7sjK5BCwszS2tRUOgaGBAfp9hmBCcwjY2RoauOi66wqVRn8/hzdv9nGBCcwhYCNqLnr5MJU3KvVtxROXQ8DOIon2yy+noa2tUH3fpk3EyZMld2VlcQjYiNrmzy/82gIDfX0M+gzBhOUQsBGpuZnWgpcPD/T1cWzHjpI7srI4BGxEDa2tTCn4RqXR30///v0+ODhBOQRsRJJo6ewsfFvxkS1bSu7IyuIQsHO66PrrUdOo95gBcPTNN8HvSjQhFfsJW5YaOzpoaG1lcNiR/w/6+3nqnXfoOX6cv543j+tmzEDS6TsKW2bNqmHHdiG8JWDn1DxzJlOWLDk97jt5ku9s3MhPtmzhiTff5B9ffJE/pZeG6+/poX/v3lq1amPgELBzamhu/tD7E+4+epQ/7tt3etx78iT/9e67tWjNqsghYB9p2k03nV5uaWig9Yw3J7m4uXm8W7IqcwjYR5p2003MXr4cgMVTp/Kt665jdmsrrY2N3DJ3LivT7kLLJZfQ4lcYmpB8YNA+UtPFF3Pp3Xdz5PXXOfbWW3yxq4sbZ83i2MAA86dMoa2xEbW0MOeuuwrfeWj1xVsCNqqWzk4W3n8/7YsXo4YG5nV0cMXFF9PW2EhDRweX3nMPsz//eVTwmgKrL94SsFFJYspVV3Hlt7/NgT/8gb5Nmxg8fpz2BQuY+dnPMuXqq2koeD2B1R//5KwQSbTMns2cu+5izl13VW4dTn/5vQUwsTkE7Lyc/g/v//iTho8JmGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmRs1BCS1SXpR0iuSNkv6bppfLOkFSd2SnpDUkuZb07g7rV9U8vdgZmNQZEvgBHBLRFwP3AAsl7QM+D7wcERcCRwEVqb6lcDBNP9wqjOzOjVqCETF4TRsTo8AbgGeTPOrgTvT8h1pTFp/q3zDuVndKnRMQFKjpJeBfcBaYDtwKCIGUskuYH5ang/sBEjre4Gz3pFC0ipJ6yWt70mvXW9m469QCETEYETcAHQBNwMfH+sTR8SjEbE0IpZ2dnaO9cuZ2QU6r7MDEXEIeB74FDBd0qlXJuoCdqfl3cACgLR+GvB+NZo1s+orcnagU9L0tNwO3AZsoRIGd6eyFcDTaXlNGpPWPxd+z2qzulXkNQbnAqslNVIJjd9ExDOSXgMel/QvwEbgsVT/GPAfkrqBA8C9JfRtZlUyaghExCbgEyPM76ByfODM+ePAPVXpzsxK5ysGzTLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy1zhEJDUKGmjpGfSeLGkFyR1S3pCUkuab03j7rR+UUm9m1kVnM+WwAPAlmHj7wMPR8SVwEFgZZpfCRxM8w+nOjOrU4VCQFIX8EXg39NYwC3Ak6lkNXBnWr4jjUnrb031ZlaHim4J/BD4JjCUxrOAQxExkMa7gPlpeT6wEyCt7031ZlaHRg0BSV8C9kXEhmo+saRVktZLWt/T01PNL21m56HIlsCngS9Legt4nMpuwI+A6ZKaUk0XsDst7wYWAKT104D3z/yiEfFoRCyNiKWdnZ1j+ibM7MKNGgIR8VBEdEXEIuBe4LmI+ArwPHB3KlsBPJ2W16Qxaf1zERFV7drMqmYs1wl8C/i6pG4q+/yPpfnHgFlp/uvAg2Nr0czK1DR6yf+LiN8Dv0/LO4CbR6g5DtxThd7MbBz4ikGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDKniKh1D0jqA7bWuo/zMBvYX+smCppIvcLE6nci9QpwWUR0njnZVItORrA1IpbWuomiJK2fKP1OpF5hYvU7kXr9KN4dMMucQ8Asc/USAo/WuoHzNJH6nUi9wsTqdyL1ek51cWDQzGqnXrYEzKxGah4CkpZL2iqpW9KDddDPzyTtk/TqsLmZktZK2pY+zkjzkvTj1PsmSTfWoN8Fkp6X9JqkzZIeqNeeJbVJelHSK6nX76b5xZJeSD09IaklzbemcXdav2i8eh3Wc6OkjZKeqfdeL1RNQ0BSI/CvwBeAa4D7JF1Ty56AnwPLz5h7EFgXEUuAdWkMlb6XpMcq4Kfj1ONwA8A3IuIaYBlwf/o3rMeeTwC3RMT1wA3AcknLgO8DD0fElcBBYGWqXwkcTPMPp7rx9gCwZdi4nnu9MBFRswfwKeB3w8YPAQ/VsqfUxyLg1WHjrcDctDyXynUNAI8A941UV8PenwZuq/eegQ7gz8AnqVxw03Tm7wTwO+BTabkp1Wkce+yiEqC3AM8Aqtdex/Ko9e7AfGDnsPGuNFdv5kTEnrT8HjAnLddV/2kT9BPAC9Rpz2nz+mVgH7AW2A4cioiBEfo53Wta3wvMGq9egR8C3wSG0ngW9dvrBat1CEw4UYn6ujulImkq8FvgaxHxwfB19dRzRAxGxA1U/sreDHy8th2NTNKXgH0RsaHWvZSt1iGwG1gwbNyV5urNXklzAdLHfWm+LvqX1EwlAH4ZEU+l6bruOSIOAc9T2aSeLunUJezD+znda1o/DXh/nFr8NPBlSW8Bj1PZJfhRnfY6JrUOgZeAJemIawtwL7Cmxj2NZA2wIi2voLLffWr+q+mI+zKgd9gm+LiQJOAxYEtE/GDYqrrrWVKnpOlpuZ3KsYstVMLg7nP0eup7uBt4Lm3VlC4iHoqIrohYROX38rmI+Eo99jpmtT4oAdwOvEFl3/Cf6qCfXwN7gJNU9vlWUtm3WwdsA/4bmJlqReXsxnbgL8DSGvT7GSqb+puAl9Pj9nrsGfgrYGPq9VXgO2n+cuBFoBv4T6A1zbelcXdaf3mNfic+BzwzEXq9kIevGDTLXK13B8ysxhwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWuf8DsXqv/xEXH5gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(tensordict.get(\"pixels\").numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f6d85ad-dcde-426f-b143-5e188c8c4afc",
   "metadata": {},
   "source": [
    "Let's have a look at what the tensordict contains:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "899fa1c2-e59a-40c6-bb50-450545984e8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        pixels: Tensor(torch.Size([500, 500, 3]), dtype=torch.uint8)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensordict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95654812-ccce-46be-bf96-17ff48abd65d",
   "metadata": {},
   "source": [
    "We still have a `\"state\"` that describes what `\"observation\"` used to describe in the previous case (the naming difference comes from the fact that gym now returns a dictionary and TorchRL gets the names from the dictionary if it exists, otherwise it names the step output `\"observation\"`: in a few words, this is due to inconsistencies in the object type returned by gym environment step method).\n",
    "\n",
    "One can also discard this supplementary output by asking for the pixels only:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "aee540c0-b51e-45df-be09-bf6a7549592a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "env = GymEnv(\"Pendulum-v1\", from_pixels=True, pixels_only=True)\n",
    "env.reset()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9df9805-6e58-4c10-912d-a4bd228e9a11",
   "metadata": {},
   "source": [
    "Some environments only come in image-based format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "30fd300a-7b91-490c-a5d6-2d34ca4635f8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "from pixels:  True\n",
      "tensordict:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        pixels: Tensor(torch.Size([210, 160, 3]), dtype=torch.uint8)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]\n"
     ]
    }
   ],
   "source": [
    "env = GymEnv(\"ALE/Pong-v5\")\n",
    "print('from pixels: ', env.from_pixels)\n",
    "print('tensordict: ', env.reset())\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f93140da-dc1c-4a09-94a9-626f5a1ff42d",
   "metadata": {},
   "source": [
    "___\n",
    "## DeepMind Control environments\n",
    "\n",
    "To run this part of the tutorial, make sure you have installed dm_control:\n",
    "\n",
    "```\n",
    "pip install dm_control\n",
    "```\n",
    "\n",
    "Make sure also to restart the notebook in between this demo and the previous, as gym and dm_control rendering can conflict.\n",
    "\n",
    "We also provide a wrapper for DM Control suite. Again, building an environment is easy: first let's look at what environments can be accessed. The `available_envs` now returns a dict of envs and possible tasks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1060ddb7-3880-473e-ab81-30e02add0e4d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'acrobot': ['swingup', 'swingup_sparse'],\n",
       " 'ball_in_cup': ['catch'],\n",
       " 'cartpole': ['balance',\n",
       "  'balance_sparse',\n",
       "  'swingup',\n",
       "  'swingup_sparse',\n",
       "  'three_poles',\n",
       "  'two_poles'],\n",
       " 'cheetah': ['run'],\n",
       " 'finger': ['spin', 'turn_easy', 'turn_hard'],\n",
       " 'fish': ['upright', 'swim'],\n",
       " 'hopper': ['stand', 'hop'],\n",
       " 'humanoid': ['stand', 'walk', 'run', 'run_pure_state'],\n",
       " 'manipulator': ['bring_ball', 'bring_peg', 'insert_ball', 'insert_peg'],\n",
       " 'pendulum': ['swingup'],\n",
       " 'point_mass': ['easy', 'hard'],\n",
       " 'reacher': ['easy', 'hard'],\n",
       " 'swimmer': ['swimmer6', 'swimmer15'],\n",
       " 'walker': ['stand', 'walk', 'run'],\n",
       " 'dog': ['fetch', 'run', 'stand', 'trot', 'walk'],\n",
       " 'humanoid_CMU': ['run', 'stand'],\n",
       " 'lqr': ['lqr_2_1', 'lqr_6_2'],\n",
       " 'quadruped': ['escape', 'fetch', 'run', 'walk'],\n",
       " 'stacker': ['stack_2', 'stack_4']}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torchrl.envs.libs.dm_control import DMControlEnv\n",
    "from matplotlib import pyplot as plt\n",
    "DMControlEnv.available_envs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "bb712ed0-aad8-4718-9dda-6eac875c78a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result of reset:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        orientations: Tensor(torch.Size([4]), dtype=torch.float64),\n",
      "        velocity: Tensor(torch.Size([2]), dtype=torch.float64)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    }
   ],
   "source": [
    "env = DMControlEnv('acrobot', 'swingup')\n",
    "tensordict = env.reset()\n",
    "print('result of reset: ', tensordict)\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4f5f4d5-b3c0-401d-8934-0dc9ebd3d72a",
   "metadata": {},
   "source": [
    "Of course we can also use pixel-based environments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "db64ab96-a5bc-4d77-990a-ab4b7357e291",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result of reset:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        pixels: Tensor(torch.Size([240, 320, 3]), dtype=torch.uint8)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAAD8CAYAAAARze3ZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAADAhklEQVR4nOz9Z7AlSZYeBn7uHupq8fTLl7KysmRXVWsxmEEPMAIDEBgMAWIJcokhlruzZgv+X6ztD/IPbLm2gma0taXZcAEjsITYIZcGMRADzgxGdGN6WlR3l8iqSp35tLj3XX1Duu8Pj7gyIm7Evfdlvubmdzu6Xobw8PDwOH78+DnfIUIIvMRLvMRLvEQ06IuuwEu8xEu8xGXHS0H5Ei/xEi8xAy8F5Uu8xEu8xAy8FJQv8RIv8RIz8FJQvsRLvMRLzMBLQfkSL/ESLzEDFyYoCSF/hhDyGSHkASHkb17UfV7iJV7iJS4a5CL8KAkhDMA9AD8PYA/A9wD8VSHE3aXf7CVe4iVe4oJxURrlVwA8EEI8EkLYAP4RgF++oHu9xEu8xEtcKJQLKvcKgN2Rf+8B+GrUyZqRE0ahHHGUALiI6CGylFIoIdA1Babt4GWQ04sDIbjU7U8pgcoYXI/D43zm+arCQAiB47qX7rkMXYVCKTwhYNtuoudJjzQPTaAwCsYobNuFSHltRldh2S6ap3tnQoi1sLMuSlDOBCHk1wD8GgDouRK+/Mv/u6lzKKX48lu38NGDPXT7VkxhC9ZlngLI8D+qwuByDs6TvqDlCOmLACGz65Y1NCiMotU1n0ONZoNSgp//2tv4N9+7C9vxXnR1QkEJgaoyuO64oAwzfRECqIoCQgDb8ULPCcAoRS6jw7IdWI6buD7Ev1G86S38mKYqss97HI7rgXO+FF0mEHAEBIxSuHz2u2SU4vUbWziqNdHq9OF6XszAEtLWAJjCwD2O3/5//R+fRl15UVPvfQBXR/694+8bQAjx60KILwkhvqRlcqGFCCHw48+ewrIdyIeM2CZ3QQqvQtYAo7MfUST4jYIQIGfoyGgqhJCduZzPQlVYbDWH9Rv+Q2EUhZwBSkncBc8NBEBGV5HRVQDyHYxuAKCpDNe2VgEIEALkszp0TQk9P2oLg6ow5DL6zHdGKUExl5Ht7d/zD77/aYqBajaSPkewUUKQy2jQVCX0uMc5TMvxP+T4dpB9yoVluzMEmaznerWIrKEneSoM35mBciGD2I4asdu2XXR7FizLAfeSCck035iiUFzdqmJqzA6pi8c57j07QrtnwnGDQSX5dyQAuK4HPqOdL0qj/B6AVwkhNyEF5L8P4D9IW4gQAqbtAJCajqqwkcYYO3Pin1IVL+YycFwPnhUyNUip1I2+SAKKbEaDEIBpOQABrm+vot7s4unBWZLCBqAUqBazsG0Hlscj6hX1EpevmRJCkDV0CCHQt5zpmgiB81YP562efz5QyBromTaskPOjMPkOCcFg0HBdD64XrU0QEJQLWXicw3FdcA70LDvxvS8EvvAxLccf2J8PuBB4sHs8sTf+oycAKPMH5iWPw+mmveFglA77R2xxsv5OCk16XlyIoBRCuISQ/xTAbwFgAP6OEOLjGVeN/D0tAFSFYa1SxFGtAc+b9TIE+qaNQ+s8eqQQE/cIkTmEEBRyBtqdPlRFgaoydPsWuOA4O2/745R8WY/2TrCzUYWAiBBf4Xsd28Ozw/rsjpFKgE4+BwUlxJ/yxbezxzlqzTaAWdMyCc4Fjs6aUkNKVJtwCAH0LRuW7Q6EpsfDtS7X49g9qi18z2XC8ziOa61EbQbIvkUIIrTgC3iqkSK5EGh3+onrOlVA5J70oJQAYrwdVIXh4KQBscQZQjiSl39hNkohxL8A8C+SnTxpiJ/UNuRH3jMtaQ9JiHh1eloLnYSiUJTzWXR7Fgx/OtrrWxAhZbe7Ju4+2g8rOfx+kJqRnIkLkAh7EfH/L+xRIhcwyPgSWCmfQdbQcFRrYLz5oqd+swTqKGZNW5JCCKBUzEBlDJQScAEc15oXes9lYrbgkccJCK5tVtHpmai3uukE1hyPTcj0ZXFmioGt0L9w4eFo8nK/fyoKQ7WYg2k5aHWHgjtraAAAx13uAmlcO88yzf9EROYIIeC6HurNDjifbeua8y5Tm+dxnDU60gRgWmh2eogygXgeB/fSTWUC2wxjFKVCBoyRCZuNgK4ryGV034Y5joyhIZeZtk1pioKvv/sqqN/RTctGu9eHCBTKtNtcF8239U0bXdNCt2+h1zef670vfBNyWrm9VobCKEzLkVpTqneRDuVCFq/f3AZjFNHWwXBboa6pKOSM6MIT1Xl6p6EqeOXqBlQmZ2iW44x9y412F9acXiRp7eIB3nv9RuzxF7bqvSyErxoux3bHOfc/VsDmANxpuxn1O36t0Rna9OLeSUjVhBAo5TPo9MZXkYVfvq6pMG1nbLWUEAICgnzOQMevY7B6L4TA/WdHg6mLaTnSlupDURgIACfkeUIxR4cdVjTd6f0XYGtklIIxkmzFfJG2gNTkmp0+6q0uzCU9a5zG1zUtHJw2xr4TAuD1m1dw/9khXC96hiaEQKWQQ6vTT1wTQPZZhVF/kWT6LI9z1BptWI4DL+T+lp3M5rhMRWnW2sJPvKAMw0UKz0lQghEhNnrfiPtFdJzdo3qoP1qvb6Fv2mNTTQJp2+maFnrW0G0q+GBs18VJvTVWzqgLVFZXsbO5gsf7J7BtF1yIi/PVu6hyl4g71zZx3u7hpNa88Cm9EGJqQEx03ZwNaTsu7MnFDgLUm20wRscF5cQtLMvB3nE9skZRUCjB9noFtuPi4OR86rjjejhvdRM+gX+3Ge+FEGmLT2OaG0W92Yk9fnkE5WRDzJRr6QTfvKMPCYyEEfA8jnqzM3slPg4C8MZWecnooamy87kMSnkDR7UmXDeiY0xUefRDa/X6+OTxAQiAm1fWsH9SD13hngdkrO7J2mAuP9Yl4rOnh4CQds9F7HEDjf55jA5z30JACILTulyMnDVCegl8GSdhux6eHZ6FFH1xA7KhqSjkMmh3TX9WMuNGKetx+W2UkfaZxQw4t69uYKWUT/SJxtk7BODbTVPdPhaaquArb9+CwiiCZ6OE4JWr6ygXsuDcQ69vh05bxioWZSrj0qzAOUffspNPwRNBIGOoMPQ0Y/DQPra5UsLmagmM0ZCjvk2XUuQyGlTGQo8ntcEBMspk4Pu4oIAjlCBjaNB8/860MDQVX3vntnzvC9kqZ10owAWHEPHal/SPVed6Fs+T/WvsW5mzeSuFLK5urkBTo/uU63I5sxOTMzssxdZ7KQRlWL0VSqEpCsbWMGZ2nuS969lRTY48MZKSUoKMrodO2xdxrI5C1tBgaCp0leHjR3vjB4nA0VkD3b6Jbt9CvRVosXNufv32j8+lVrpAUWPFAmCM4o1bV2Y+L2MUhawx1r6njZa/aMZBGUGllJ26Lp/TsblahqrSuYQbZRg4+e9sVrFWKUCQ+HIoITB0FSzokCHPnjM0bK2VofvO+kHVchl94Bwf13a24+KDz55h6LMQ3Y8NTYUxEGILSoEI/NzXPucP1kOM9uuMrkFXwwMNRqEwOnx+HwRykDI0FYxS6JoSaR7rWjbqrQ50TRlTHkb7suO6aLV7sC1nruaY1Y8uhaAEprWCYj6LYj4DmiCyZqSQGY00PGA7DnqmFTvKqQrDtc2qXD2eE0mFKCHA7WsbyGV1XN9eRd+0x5yuORfo9q0la39Aemk4u7h2x8T7d58MTpcLT9NFqYqCzdUS6IgG5brc15YFNEWujo5W09BkXO6D3WN0+/ZcwlxXVGyvV0Apwf0nR3h2WJvpEUAZxZW1CjK6NtUMqsJgaCraXRMPnx2j3TGH5whgo1pELqMNJuaAjI6R/Wp4E+67wLlu/GIGpUApbyCf1UPdWhhd3JhBKcE//t3vo9MzQ/ssJUClmPW9LuL7xdXNVZTyWVBCQAlBuZgDIQTFbAaVYg6qwrBaLvrCdLrxbduBbbtYKeVlxNiC48KsWUZoe6S/zfNBq9tDo92D6/GZD6OpSrQwm/GtK4xGhA8CjuNh97g+ZSBe1sLQaAfkXODR3ima7T6eJInumRPS6J3umvHRPp1U0hXFD++cnjbZloPHe6fwXA7G6NQ7tGwH7999PPbOqsUcNqrFhQavvmXj/pMjuCkGHc/1cHh6jr4Z2L+GAq+QM7Bayfuzn+k2OKm3UC3mMfogqsKwvlKEwujA+TwpOBc4PW/jvDVtG6eUYKVcQMb3RYxC3ACuqQqqpfzAtBHaHlzgqNbEeasbqWwojIFRAoUNhwhDV/H6jS1oqoJ6q4OjWgO24+Ks0YIzOUCMNKPrutg9OkN7hF8gCJudhaDPJxWKYbi0gtJxPcmcMvJgYSMBIcDGShEZI4UtxW/8IMwxG2gJY3JS2nG6fdOvg9xyGR2lfGZqSrIMtLt9OK6LvmVhe60ifd8STvGTTPUJIagUcyjkMonrRAiwvV4dmealA6XwI1CmVTYuOFzPA6MEpXxm5OMealjSDju85uisgf2TIJJpluYbfkzGXs+KC56+pmfaU54JQgg0Oz0c15qR7W87LlSFjQkUz+M+QxCFoatYXymlalcuBLwR35up2HNK5u8nkI8tEG/D9Dwe4iUg24tSgmoph1zWwMPdYzTaXQghQ04f7h7j9ZvbUBiF4PI9W2E+paOlCoxF5KkKw5V1OdubZZO+cWUNqjrDbjxDfl4eQRnX3yNQyBrYWCn51FXJjPdjtxRSIIdqFhH1kdofT/ihzgch/MiJkU6oqgqub6/OFFixH4evuaaxnQoBdHom+AzDfxT6loN6sxMbu80Fh+24/gprfN24EBHx/kMojOL2tQ1sr1fmqnNaeD6LTlSNbMfFZ08Ox6/hMgzTcV0ILkLD9ZIMjIQAt3bWUS5KWy7nUoNtL8DsZNoOTs9b+OKbt2L624x+LwQs24HrurJ9HBcQcpZmWjb6lhX7DiklMDR1LNBi9JtmjMD1eKLZHecCggPFXAZffPNmOHnNDFwS96Co2obEXo20i+N6sBwH3b4FO8ZJNUxYMiqnep1eHxAYrLLGriRD+jWGhg5Ova+0wnJYgOdxHJyej68Y+uwz804dAClkGu0epIaUsFaE4Oy8lWrFUlGkpuAljNWV9lcTCpOMTz1rxor+DBBCUMjqM1d105bJKIU3sZKbBKrCsFLOD2LiJ2HaNsy6k7hcTVWQNTQ02j0QEHT6JpyF6eWm731Sbw3Vy5SXcyHQCiLZJtDpmXjw9HhEO58+iRCKSimHbl9GxE3C9ThK+Qx2j0eDMIByPgfbdYe0jAJ4dnAGAQFVZfA8gWoxFxkaG4VLIiijENLKQtpzCAhs2x04TI+BjP8Z9poNXcW7r13Hd358H4IIbK6WIITA4WljqrxEpvE0304Cgovxj0aOnqf11sIO0WMRGlGx4hPnp7klIQQ5Q8fWWgWfPNqffcHgPoCiDaOQ3ARmhCiB7zgu7j7c911gFmuvACvlPAxNxWm9NeB9jIrPn4SqMGR1PbKx5e5k9SSQC1qBfc7jHCeJyTjStcXjvRP4c/ClIBjkhQC4iBfsrufh9Hy6vwd91nE9PNg9lmHDPiiVZgwuRuss/DII+qaNuw935+oTl2fqnRCECGQNDZVSFq9e38RqpYDp+bH8DyMEhVxGTh8m2sa0bPzwk8eDqJTjWhMnEYJoVOXXNAX5rBHippACqWfs8iCfP1h7CqrC8PNffweGPp/tMbKmQqDTN/Fw9yj1tabt4LzVmUmbVS7mcPvaRiQHo/DLWiaJ73mzi+N6c7DgwBjF27d3sF4tzrzWtGw8OzpbgryRoqZnmjhrtDG0nybtF6FFxpqZksuU8YsVRpHPGtBUNvh20sJxPTmz8IstZDN4/cYV5DM6IDDFBcA5x+l5C+1ub+qZr19ZxfWtVbiuN5fnyKXRKIVIthpLIP3ZFMbw7PAMlhMWUTIcubKG9PUyLXusr3ge0PeG186cuvh1y+gaGKPomtakDpio7vJMMfbvlMXMuklIedMaN6UEP7j70LfPLnJTeUNVYbi1s457T4+io4VmgHMBnqAurU4PluXAnuFGs0x4nGMs2o8LPNw9GQkPDJk+guBzd67h7sM9/+OcY0AN2ekGWtSSND1AuipVSnkIwVFvdscEGyUUuqrA9TwoCvMjYHqx0VyqqsAwtCHreiqBG45u38TTgxN/AAxRaISI9GQ4ODlHKZfBaze3ce/JoW9CSV6xSyMoAfjM1iIiUF9+kJ4QqDXi4zIDcCFXScORttPK+zeSxKhGCHzKZDmB7Y4xAoDA812gwoqhlIJA0syNnkMibqRQ6ZPoCRFNdCzkh/7lt2/j97/3cSo3mfHnIaCEwvM4FEZ9W6aAwhi44HPH3Q7KpxSUULjeuEDkXKBvxaQGeQ7gQgwIU6JAKHBUa8iFB3eJUm1mUen1NwEiCYdllxy7ByEC1XIehayOx4dnMG0bThCkEIFe30JvLH3L4s8v/UyTEYlQQqAwBtvPOeQ4rrR1EkBVGbgdaOLJcGmm3tL/K4+djZWIM4Yq9jwuD7qqzO3iMnn/mdOaiMObK+XBVJcQgs3VMjZXSiAxbyFraCgXc1CUiQgJjJsEpKsUwVqlhK31CjSF4c71TZTy05EtgFyJ/a1v/Qh9c/4Y7/VqCTubVVBKYFo26s0OCOQqrKYmTIsRs+mqgls766MPPNcWfDQBS37U+1ECX86kZc+AEAKntSYsK2Uum6jyEv/8pHcxIX+T4IKj0++j05sg9BXSZ/K41sCD3RNYloN2tx/ixbBgYy2AKF/QL739ykD5EkLAdly0O33ks0ZqWXBpNEqp8ndSZ3QbHfyCFxy4DIxO51+9sYWMruFHnz65gOgWILwzkLHDMg+KDLES8LVdEbgChV4ip5eEJFrE4YLjtNEcuADd3x1Og8N0jOk1sGRez/msjlzGwEmtCUqp7y4lj1EC2LazhFVYn/nGb69ZiFuYymV0VIp5NDtdrFWKePBsMnWCFCxbaxW02j00E9KKhS0UBvtUhWFjpTTlvRCG4N0ojGGjWsJJvQknxp0qCTKGhptX1vHRg934E0dlYkQ1KSEo5rPQFDa2ci8XsyYKuWAkXYixHRd//MH9KYHuetJ7ZMq5fQYukaAUE5Wf/dFSSlAu5NDrW4PcOgBQLeXR7vZhO+5AWD7aPQalkiNP+Ktgk0hiIw0SW3VNK4EwGH+pp6PUZwLR/psjdbMsN7lGQoYck4EzvaYqOKk3Ey1sJJmwERBoioLt9QqOa014fFi3jKYhY2h4dnjmkxOkx6iw9jyOJwens68hwOZqBaf1ZqjZpte34fG2H2Y4PmUfrsQSnNZbU0EOUWCUopTPotXpwfUH90CgWLYDx/Wws7kiBWVCQcIYwZXNKo7qjUTnByAAivkMTHs4EDuuh3tPDmbIsGT1EkIunLTcIeGEDEXMSuLdhPyRceXPW7coRClcrueh3e2nLv/SCMppRD0IGfmL+LG/49NHmbJhaD8SAiHpbuM1LMUP4p/05yP+SrrjekvRmsKR5CXKaSTnfOivOGZXIsjoGjK6hjPSBkbdMRaIwBQQqLU6qLWm7cSMSXKDSJ+shOWnBSEyZeuZH6UxCe4b+R3fiXnyHEplDLJp2yn8S6VwanVHtE+CwaKHrin4+MFuKkd9y3bx3Q8fDHckbQoiV4Q9rwvLv8ZeYoIzIcQY8XNwT11Tp/eHgFIyoaQ8TyS834zTLo2NMs60QQkZ8dAfHvS4h6NaA0JgjJbr4OQc5oKj3Dt3rkNT2JTtQyaRar4QJu5RUErwhbdu+jHScqo3GrPuOB72jmq4//RwesRPa+dLiG7fxHG9GTqahzHILAucczzcPRqbrmqaMhjs8lkdm2slWLaDg7PzqesZpXjrlZ1UlGKux3F01hjjEZUx2C10eyYooSgVsuOkLktr9/EThRA4rbcm4tAvFpwLHJ81IzN1jm6MUnz+9Rtj32hSKD7hSDIkaMQ5+/jlEZRREHLkevv2tdAHFFxge72MXEYb7Bxy7c1vUN47qoVOKYQfmuV5PCa0DJHbsiA4x9P9U7ie55MY5Aa5uP0zfJeWqLZIsU09SPRpg7xBIxsBQSGbQbmQk8b1BaoStfEgDFBIwbdWLqCUywBC+jHWzjvSFBOSo8Z1OfZP6nKGkOKepuXndZmsh5CrvuVC1g9sCV7+srZpWI6T2r6fBHF92eNDzsm4RVXH9fD04DR1tBUhMjHe5moZyfppgtPmxKWZeguIyMUE13VxcBJOSS+jac6lP9tkQyQOK5y+70k9WYgTYzIawDSd2JCsYX3jy0vKTCSAgeuT40nXh0UWqbKGhtdvXcHTg9ME7lfpepwklJCmDx6zSCFZoCgsxx2sXDJKErOvy2gVAsd1UW90BvZK2/EibbTVUh7FnIHDCe1wEhsrZagKw3G9EWlyoYRA11S4Hofruniyn144xCHoa33TWdj1KsDzmAoLIRKGDI7XRQig0zUn3IymTlsYScw9l0qjjHJ2cFyO2nk78rpuTy6sEMiPJY5cNbxNFhh+hMC1zdVBis20qJbzyGeHme7mYQfyPGlDCj5KRsOnuYHRP58Jz6yXNlwxDSzbkZn1Ys5hlGJjteSzowu8dmNzEKqYZDM0Fe+9fn2Eud2NPZ9ACvCT89bMaKB606c1i8kpbxgaVisFaBqDyzna3bS5s+NBCMHtq5uhzFWKwkL3L8I2dXFI+L0J2W/6pj3zszQ0FdVSPnZBNtqZajYuh0Y52gDE1+9G4miTPg6jFF979w6+8+P7cO2hDXFKU03cP2apfwAXwKeP92IEjFwoiHLvMVQVlBD0+lYiF6AknfuNV67g4Ph8wIIeaKnEX8xwXQ70x8vqmTZ+9OkTf1o13E/ga7kJ45rjMHTbCi+nb9l4OrLK/fHDvVTaQ7PTG18MmQEBJM6E6CQIfZt2sl6mxkbguh5+/NnTKZccQgg+9+pVPD04Q63RHuxb3r1TlnNB8jdKChAQKCpDxtDQ7JDYzJIjhaXCpdIoAQACMHQN+awxYPhJCpdz/OEPPoHlhHf+wYeaYiSJg6GpWCnlw81PPjSV4Vd+7ss+H970iYdnDZmvfImj+8f3d3HeHpK6aqqCciEHADg9b40dCyDp48K1jEoxB11TIjWRJHRgQgi89/oNrFUKodeEQVcVrJYLqVju5xUOSZ8h7ZYWYSS+hEjTSKWY88ucbv8fffoU5yOeCOVCdiZ5b3Iy5hBcgB1wWHQ6zU9AZrY8PD0PF5JLqOMlEpTDJxBCIKOrMhXD1ip0VUXcIsLoJhdZhqVSSpDLasjndFzdqvphg8Edk//C4HkcVzdXwxverw/3OH7z934Ad3KhIDiNc3CPY3zFZ3qjlKCYl36RCqOoFnORzOzcF3qjKOSMgTCcPBYHAeDKRnWKL1FVGHKGnmog++Czpz4j9vT9w4SM43K8cm0D17dXxohoJeGCPphqvgiBdhGQZLQVbK0OeTQpIVivlpA1tAh+TfneOfdGyJHh87R6iPxQIHBlo4JcIEwTfFvzCpo031kqBWakPoKLMQKN4bach7pEgjKAgGnZOKk1YdsO2p0+hpIv4QOPCFWFMayUi8gZOrpdE+ViTmooKV+6qjA/jHD4Qm3XxfufPIp96Y7nwbLd8eksIdD9KXdEE0xtBMB6ReZeoZTg1rWNiHOnBa5l23h2eBrKMp5k++DeU5j2eApQXVNRLecT0/ED0tl3Ksd0DBxX+hW2Ov0x44mhS3uUwtilEXLRGG9LQmSkUFhiMEpkFsdsVoeqMmQzkoDFsiXn6kf3n2HYv8f7+WQf+PTRPhzbjX21/b6NrKEldqtNK/AWEXyztwuQ6DG4HDbKCHAhRrINpgOjBLquwbIdPPPtXqrC8PqtHTTbPYylK/aLJ35KgnbXHK5WEknOUMhnYNsOOp4314g6Cl1VsVop4Oy8BU1V4TguTNvBWqWIWrMNzuVHkzU09Cwb3B8tH+4ey7YgwA8+ejS7XdI2W6IvRhba6YbEBc8N309WYMoE4XoctcZ4H2h3TXR68QzZyfF8BS0lBKVCFpZlw7JteXe/Cpbj4OHuMSAEclkdhWwG9WbHz0opll7Vvmlhe73qU7ZN99MoEMhvRfBUYhD+TS7q5NTIGhpcjycavC+JoBRjiw5jR+b8GFSVoZDVfSp6WYbjerj7QBJ3+rSvY9colKGYy6I/mjNbSLW+b9q++8hEUHGMcAkWQsTEc3DBkTU0rFaKUstiFLbrolzKodXtw7RtEEqRzRpwOZerxWKkDJG8U6dCqsE/5MOdM+JH1xRUijIUtdUNOAZH7hUxVb80EBgsQo4KvjEQKSSFEDg5awx4UCfL4X6/6/Ys9Pt26tQd0VWcLsO0HTQ7vZn28cnFSMoobu1s4Kzewnl7gk0rQVWD72JYZvxFxP+/Zb9yQ9f8NB7uTHPUpZp6L9OuZDsuzlvdsfhxIWQ0z5WNqmS3mYDHOY7OfJ/MEfVdCD4IiZTx0yMLM1HTHwDXt9exUi7gzVd2xhytHcfFSb0F07Il0WivDy44nuyfDCjFXM/DuZ/LuJjPjGWRW9Zv6Ug/q5fx0bkMirmsXFVOaIu+VBsARihKhZxMZxsCXVGwXi2hWswjm9FjP0wBMfDJvXl1HdmMdiHv2uM82l/YfzZNYdhaq4yxKgku0OtbMhJqrC2SNZiqMqxWClCVgPw6GoQAuawxSHebBEnlR6PdlbyyCWz2l0SjHIeiMFAiHYfj5OGksKSU4HOvXsfHD3fheRw2D1ep89kMakp7SCrqg3MOO8aRVwgBSsIUp5BKCqDV6cLzOGrn41T9QozzWgZWgMm8P47jwlMVaYuLrNX8SCssKaGolqSNd4zgY5E6iKFbTT8h1+BlhIAMmBj9mEfbVxCfdMU35SRpew6BRqsbOzWkhODG9hrqrY6fDylRZRNDCEBTFJSLOWiKgqOzc3AusH9cS15IyP0TrwEO6kpg6HLl/6zR9mdZi30VnscTBwRcSkGZ0VWsV0s4OmuEkFlMQ2FskJXt8LQOShCacyWY2r9+awePIlIVECLjkj1PTE1JbMeF63mJRiAAkp+RyBzlQVGUEGiqAnug7kf3GCGATs9PZjZyT0oIKKPwvOjMfxcCn53Idq2FNdLRZG5dX1snlIJgmvklKf0bpcS/PrxuF6JF+/C4QKPT8RXi4X3yGQM9y4JlOziuR6e0DS3T4wP7YRQIIai3OkN/0JSPOJ00bbwAx3VxcFqDrmowUw5kQbplgIBRAsaotAm6Ls7O25Ehl7qmwvNkKmMBDPoHoxSdvrnUXEhJcYmm3kPVvNMz8WT/dDp0KQSESLeZ7bUqhJDkAFHhaoF7zG/+3vfQt+xQdZxSihtXNpDPGqFTfJ7SgD1YgPah6yq216vI6EG+l/hpihgwhY9PXd4aTOfFyDazuLmnlvLZOVqd3lI0v3fuXB9MU4Pb7GxUsbVWHktRKo/P/hEC3LiyjmxWB8hzMjVMYNKeSCnBW7d3oKlyhdtzObgnQuPh5908j6PR6kkWn1SuMHLTVIorGxUUcxlESVnLdtHu9gZhqEnAGMW1rVWsVorY2ahipVzA1c0VlPMZCCGkEIyYEl/ZWEEhL+sz6tLmuB6arW5sxtXkiOnoIbg8GuVIPYUQ8IIJaawyIY3j7W4frU5vyu6zsVrG9loFH957NkbgOUnLNgrP83BwXBs7Z5HRa3KByrId7B3VQolZpe2Kz6SuchwXTw5OQmKTlycMVEWy71iWM0YVNnoHhTFoqgLLdqGqkkE8qRB9vHc8DE0DACL5OgXG/TwVJiMu+qYdyt+paQp0TUW3b+Hw9HxorhmtaFgfEhPHFmg63Wcq6pv2SAy2XBX+0SeP4QR+snNAYQzvvX4TnzzaG2hWy4TjeDitN2dGHQW1D/sWVIXha+/ewffvPkLfV248z8PR2Tk8j0tTAxcyv3sCPoKn+ycgNLzJpnelbNeR03NZHbmsIes1gzLxEmmUEYgZECmRaUQZJVIQTvgPntab+OTRnq9NzFCXgtsJoNO3ZOdeRvUnRkvP47AcZ4p1BZBTi9tXt8b2GZqKYi4zRlHFhUCz3btQHenmzjo2VkqxmthatehnIZRRN9ogvny2JtNodX1B7+8TAn3TgmnaIEKmzVAZA4HAq9c3R6aF4xslFN/88lsgkOSyjuNOaVa6qqJazEuzB2PQVXUgHzXGoCmKbzNLr2pnDQ2vXt/CWqU49mUzKn0ibddbiMCCc467D3dhhybRS4cw7c3jHN2+NcitFLYRIhnTWUSElBACP/zkMWxrVLmQhMmW7aJv2jD9WP/ZDEcCruf6sfcz2n+eWdQIrqxX8TNffAtGgpQZl0ajHP0gE9ukCEFG02BZDiBGOpJfFPc4TG9Cw0nMKBR5wYVACJkw/sP7T8f265qKjKHBtO3ECaqSMhDF4f7TQxiadIr3IrShw9Pzwd/f+fG9he8ZgFCCbEZHs9NF37Tx/sePIs8VgqPWaINSAh6huKmK9Eltd/vYWC3B8ThOfOalfM5ANqPLAIcUzvD+3aEqDO1ufxCjTn13MIVRvHPnOr7/0YNh1sQ5wIWInPIu005HfB+csDIZpXjntRt4/+7D4crjCFyPo9NLou3OUd8L1AY+e3KAz54cJDqXLDitfAKgDdl8rhDiS4SQKoD/D4AbAJ4A+CtCiPOoMgCgsLIl3vulvz74t6pI+e267sQMikghIIZiVYa2LanTpJQvxF/GvCjDMgGQzRrQVQWtTlhCpyXea0K4qgrDnRvb2Duqyex1M68HltkWjFLf1zC+PEIIFEaRyxpwXS/0gyVErth7nIMQuTIdaDYB4bHtzMe+PeooryoMuYwB07JhOe7YfWYh7N5B3PfS+ncEAtZ+AkS+60CblAspcaVdLmGYxjb9rb//f/qBEOJLYceWMfX+WSHEeyM3+JsAfkcI8SqA3/H/nQrFfAalQmbKh4BSgp3NFSgjSdU9zuXLi/ilQko1XlWCLIFh+v0cc4GQ6gRT9ItWbMNMBJ89PhhEhczaNEXBFT8WOa0/bNjmel4oKezkxjmH43rwPD5GLTd+znDxgHM++BuQGnsuY4zFkqfZPI8PVozlQB7E2vOxBYvgXUeVEwZGKbbXqqnCROd998H0OqqfcsGRy+g+iS6mzwnMXmmnwqk/0XS/ZeEi3sAvA/im//ffBfB7AP73M68KnonI0CoymjZ0REjoqpJqdJ3VWEmn+aFlCwFDHyEVgBTmlWIOrsfRnPRrC71VfP36poX+YOr1fMwAAAbs6EnBhYCqLpIOeH4EZgtALoiVCzkcnZ7PjDgRQkhfVY8PImIWgeU4sJrhtkRFYagUcuj0rMSrxwJiQGa8GGZ/L812F83JKBsfgcapMObP9sSFaYEX6p0g4Kd0LoAL4Ow8uS/wMt7AvyaE/IAQ8mv+vg0hxKH/9xGAjeRFyVGp17fQ7ZkYpDDwD3kex4Nnh7Mp+xOADXK4pBuhCAE21yrIGBocx8Xd+8/G7kkgTQGhkRcTdVQYG4vYURmTizaRzzYycsc0AKNkJA94+DnFfAa5rD6xiDE/bMfFk/0TqApbio10bgRtn7AOpu2gZybjAo1CFFHyKFTGUC7kIrTD8HfkeR4e7x2jZ5qR50xuhEhqPzqcs0+dRgDcuLKWOC0HJQSbK2W0u33JmzCjqdJqfXNrgKk110A7FiMLbMkEx6KC8k8IIb4A4JcA/A1CyM+MPcfonGMChJBfI4R8nxDyfceU2kDgMDyN4cNMr37Gnh4pSA1NxTc+/3rqj5pSMnB4FRCYnPS7XIYetrq92BevMoYr69UBK7O0R+p489ZOsorEPJ/CGL755bcl03tEMwkhoDDm2+9IBGWbJOcgITeU143vr5ZyuLWzDmnOWlz4zoO+aeGk1lhaqoRpTLfDSiWPq5urflROlNDjOG930LeskONxdxNTt5XvhITeihKCm1fWsVouTJUl+ZelXd3zOBSWLNmb53EcnNTheV6iUNqlIcF3nO4i2e9rjbbP3Zm8rgst5owVRMh/DqAD4H8D4JtCiENCyBaA3xNCvBZ3bWFlS3zxz/6vUMxnIATQiJgCLBOMygWAVsJk9wEIxUgkw/z3J36EjnSkdUEAMIVBVxR0I6ZmSc0EgRP+1NR/BJTKhTHP48hkdGyvVfDw2Xi0UrmQRaWYx+HZ+ZRvZyFnIGNoqJ23B5EwisKgKgymlTzta7IHWlI5FyS3VVXODOL8XwkJ+k20TTIJKCW4vr2Ow5P6WC77wX0gU1K4I4zsApIEZnO1AtOSRBjMj+xKqkkTAMVCFllDH/N2mIm5HvX5D7AA8K1/8F8sfzGHEJIjhBSCvwH8AoCPAPxTAL/qn/arAP5JkvICo3wUGJW5mwN/QoVR6b5C478iQgiubq5gtVIc0x49P8pkctQxdBV3bmyjmM+GTpMEl1n7IhMbJoQQMptjQNohALiuFykkA65CRaEzR3QuOBrtbuzoHlC3AYDruKFCuN01cVRrhL4Xj4upME/X9dA3lywkgUgF4Wvv3EEhm0kx9UqGSjGH29c2wRjFaqWIWzsbkbJa11QQkJlBAkJIN5pASKoK869NCSGFrss9MEag64p0zg7ePaQ7ke26YxqeAAD/Oi58xpxUtv7pHPeDA34xGU2VvrQJtT5CpLO+piqY60U9Rywy9d4A8C1CyI8BfBfAPxdC/CsA/wWAnyeE3Afwc/6/YyEgFwPa3b5M4BSyKkgpwefuXJcfht/IN3bWsbFSjl1BJETGXHd6owTA0XAcF0dndfyFn/2SPzVJ+hX6m0i/UUKQ0bWhIA8plhDgzs0tVEsFzPSNHmvb2T/bdXH/2cHUfo9zn15u2qug17dQa7QHJohl/Qghvp0zvvYA8MG9p2NaU5Jf0HcU3y0o7JxO38JJvQlCCDq9Pg7PziXNV0gd8jkD3/zK26mf84tvv4K1lSJESlYoT/g5zF0PCmO4trWG9WopUdc8PD1Hu9uPOD67n3a6Jg5Pzsf3B20Kgr/0i1+PNT+EddCdjRVc3VqZa9KwDO+KWZ4HAZY29V4E+ZUt8d4v/urM81RG54qYiYpSe6GLDiPQFAVv3L6Ke4/3/dSsSd7JBdX9BTeJpiq4c2Mbj3aPU8UWp7qHomClUoBtu6g140knnhuW/hlGFSgHIjnAvPhvPwxJfWcXSc8chm//w//zhfpRPhfkswbeee1maErOWYhS6GNHlwQDSCmfRT6jzzxvFhzPw90Hz/z48qSdN6WmmxQpimSMolTIyqlTymqslAtj3I3SuVpmGnx2eAZrCSF7UXA8D+fNDto9mWKCURo9aKZt5gTPT/wFMkYpVkp5qGxo4okKE1z0fQfO66oiM5WG8bEuCwpjKOQyY6ar0YWjtNrcJAxNxbuv35x6Z5qq+G5Myxdrl0hQRr90TVVQKRVwXG+CpMjIt2htgHhV/cbOOlYqRXlmyIsvFbKRRK5j9xKSGWWZmRhD7rL0TXCB9UoJxWwm+XX+IHT76iY8LuPzCYBC1kAxlwEXAq12F547ErufeMPUvlJ+5B34+wTnMC0btuMiY2i4srGCoh+ZMlXmktuMQhIV5wwDEMD6Shkbq6VB0rjNtbL/occLvjQgfvqJXMaAEMD3PnowlzaWeBoLgYyhIZeVTEGGruG9N25K+3pEH9dVBfmskYic1+McH3z2ZKqsXMZAIZcZ0XEWG1xGcSkEpa4q2NlYCbe9Qa70ba6WoSkMIlGCrMXxyrVN6KqKq5urkrOP0Sl3ik8f7qGQy4QXIATKhRw0f1Rdhp0kQC6jY6VcGHAJJnXzWDY457j/9ABnjXHHXVVhMZqRxB9/cG+MLotS4ruHzff+CJGZB4Pw1wDlQg66qiBshkAg7cMKo1AYDbU65LIGVkJcbRLVCbIfjS04EunnShnAhYdPHu5ipZwHpUMu1DBZofjpF+ath0z9TOF6nu+V4KsCS7LhjcJ1PRyfNVDzHboVRnyW9Og+oesqMroWuzhLCLBWLYEQMpWFABA4b7VxcFKTA/BEP1IVhu31arx9PwaXQlAqjMpY05EeYugashmZDtWyHPzw7kM8PTiRNE0zH26x0QOQ0+psRke1VACjFPmMgbVqcexjshwXdx/uhnYiAWD38Aytdi9VJ0vSYVVFGbRNxtCwuVpObFqklCQatecFIQR3blwZcXifDSEkG1Kj1U1i8Yi6M7IZfcrxe/fodBC1M31foNu38GT/GI1WFxlDG7tersqqyBo6dE2Vq9Qp2o4QgnIhNzaQcS6Zk9pdc/CsH99/Bst24bgedg/PQmP6FYWhWMjOvGdYf/E4R73R9gmk5b5KMTdXP2CMwtDHmYSkgGeDOPrJQbLbt/Cv/vD9WHrDdtdEo90dWVkP0cYJQT6jx6eQiPjcVaagkM3MvS5xKQRlz7Rx78nBWDRLPmvgi2+9goKvvrt+PO8UUmjV46NJ/NTqx58+HrEZypSvvb4JMTr1FPFU8jIOfTka7iga7S4OjuvwuAfLdtDp9QcEEnEbIZK+LNCQlr1qGGyNdldGT6XAPLaqyeufHpwMfAuDz4Hzaab6yeuCfiddwtSRY0Cj1cHBSR2awpDPGmP2r2jNy09eJ3w+StedElyjceyOO4xrl8emy7RtFx98+mSqnZJqfaMEI4QQrJQLM6OJwqD5U2Rl5NpcxsCNK+tQGEU2Y+DmRNAB59yniYtSWAQYBW5f25S2UyFCT+Wc49nhKbo9K7X+Y9o2Hjw7nIqYE/5vlvJ0KWjWhBBTo2ir28ODZ4fo27b/IBLzxmZrioLXb+3g00d743RaEe3jeRymaWP36GxASx/m4Jtuqr8cTW60vWzHTUwPJoRAwyc4XkQozbrH3tHZXNe+duMKLMfB04PTueoXZNtklCJr6LBdN1aLGbuWc5y3unCd8RzsQZk9U9o0PW82y9BX37kD23FxfNZIFHOetH4BCDBwJ+uZ86TtFTg8PU/BRjUs33FctHnP9/+V+y3bxtl5Ex7nsG2ZMC80XcrELkqlWxwXArbj4uj0PNZ2KkSEL2dszeVNo9KDBPjae6/j2/8o+vilEJRhsGwHByf1qf2qygZsMGkgIPDk4ESO7hCJBG4wZRkFpQSKwuC63swUlwGCKZzreXDdYb7wJFAYA6FyNXj2BxFdaC6jY32ljGcHp/A4R7WU953ul5Wbe34QAJVSHpbjoNZo+/HKqi+UJEFwu9OPdMafhIBAsZCF5ZPFJrpGiBHykSGK/mLQca2RuM99+ngfjFI4jptqGE0KQglyWR0QMlun4zjpnMcFpF9xCBiloIzCjaCdc11viqXccTw0HBlNZ4/mySbAlfUVtDo96b85+gwEqJYL2FgpY/fwDKZpo97sDOt4IS03DUoIVFXB/WfxvJSXYuoNYDgVjtyk2v+nvvautH8lnXL77W07Lprt7qBDiZEfoRhzKKaMDGwZYuK3vb6CK+vVgd1p8njYjxCCP/W1d8ZtNwnqTSAXI25sr0sKrJnXRLQd5MDS7ffllEgI2LYDgpF47ZntP13msjbi8zY2Wp0Bn+RPf+lNGL5xn/MgfWuy8jgXODypo9ZoL1gvKUBtx/X7Q7LO1mh1UWu00epKk0jqtp2xcc5x3myjb1mDvDRJH4sAYEH8/sQxSmRQx7WN1bEFpSR9PLTfA3LhCEHO8+EPRGqn0obcneJMCMVC3Sy8LQkh+Pp7r6PXjR+EL4fDeXVTvPsL/1GCM2U2t3hC1/QkF9VyAZQQnNSbUBjF23du4NnBKeq+MzIhQ0JaAjmiJ9Um5TXDeqcZ+aUgkyQGC0/ffPIL2T/E4N88sAe9SJChb6Fk/vFTuvrPPCDAXUY9R7vHrPIm63UBCMRvWqyvlHBlo4oP7z0dzlJmQECglJcMRufNjgyp9O/OKEXZX+BptLsyTHfBBifwvx0g9HulJOAAXeQui78X5qcP/tY//L/85DucS4iRtJrR50yO8HH7hfBjnv0e63GBTx/uSm4+Aaiqgm98/nVp+PYHIp6C2l9RGErFLLyUQjKosSdk2Nq8o/poYbqqYnu9gnxOxkdzfgmEJCDblY8kFRMyjYfgAiKISV9SPXc2V7FeLSazdU/Wa8lgjOKN21f9WOdkCN5pvdHBx/efDQRa0r4gIDVkj4/3Dy4EXJfDctwxATov8jkD2xtV6Joqv1f/syvmszA0ddD/oj+JeK19eM7i8DjHe6/fij3nJ0JQ5rLGlH+cojAYo/HRISCEwNAU351gErLRhRA4b7b9HCpi4IwcJL5yHQ8/HORsGXlZIaaBsI0A+JNffluuloYcZ5TC0NVxouIlfpejH4ppO6g1OrB9u13cR6WpMrthWIzzT/Lv5KyBTs+a+fzz/AiRycYCP9JiIRtLSwZgYDNOeo+g3o7nwnLcwb6k6PZMf8Ae77OCCzTbXXR7JrKGDpqQq3K4jX8PvZ6FeqMjtV2BwYlfe/dOKG1ftDCchqJQaaNdIu4+fBZ7/NIIyjh3k53NFRj6OHt2KZ/FWqUY66BKKUG5lEchN+1/RilN5EgsBEfP9LP7xZ8ZujmOi3/y238cklVOQlEY1iolqJNO46k6acgWAi44+pYFy3FmfmDlUg6lYjYp/+1MaKr0/Vyrli7Uj3MWLMdBt2+OpeBdFgghWKkWkc3oYArFzZ0NkJj+6Xoe2r05cyHN2S8Ypbi+vRYZ5scoxdZaRTrphxRACEa+OblPUQI/6GE5nHP0TQuOOwzLZYzit//tj9C3kueFD+gAR6GrCq4EzuMJMculbRb706Vc9Q6o1AJXgPuPDwaG9QDtbh8905LMNiP7x6jUPI5GqxtK4qqrCnY2V1Fvthe0kSRB2A3kPsd2UWu0/I8laUWSTRsXQbPdg1w4EMOyIm4rMw+KyHYkRM4KClkD6ytlnLc64EsmNLgM8DjH/nFtMNX86N6TpZoN0mP6xo7r4u6DXf/bmj7uui4e7x1FuuEUcxkYhobTemtgktA1Fesr5cH3GAZKKdaqJfT6VmQAwCQURrGxWsFpfTxDZt+y8fDZoa/ILqdxZ5VzKQVlpZgHANQbbXAhQnO3RLl9TD5wmMsHIFfB7z54lsiYfJEsQ1xw9PppWXIu/sszw9ot5LaKwlAtFdDu9iPbWgiCdqePXs/C4WljZrL5y4vZ7e6NDAAXRrK+AKRNPrr9uRBAjA2eQ/i2xeHD9fsW9o7OYm25Qgh47tDJPlFd/f9OKkPeAul/58WlmXoHIASD0S5MQFFKsL5SwvpKaea0kBCCnc1VbK1Xp465ngfbcRNpk2kjU9Lgzo1trK8kD0G8bAg+PDExlc1mdLz16rWBPcp1XVi2DScyQkNO4TRV8kTqmuqHQS5qg1jmlhxba5UxM0PG0HHnxpUXanYA5NT682/eGqQjzqVkv2p3+jitNwfeE0E0kWnZsVkzOec4qTcSa5OA9NncOwoP6XzeuESCUnZGGdLlwNA1nwpqvJMKAfRNG47rIZ/NxK4YEgJ0++aUs2sUKCVYrRQHcdTzgBAyiA1OIlTPzluwLHswrWWUYmOlPNe9XwQ8j+O82Zmy8QguJClCinbc2VjxhQtQKmSeazsEqTlmMeYnRadnyugV//k5l6zzcTbLAFlDRylBXHcSEAzjsAH5nMe1phRegkPzWdbnCVVdBhijA1PbxWKxgfDyTL1H6tntWTAtZ8K+I/z/F+j0+tBUBeViDkIEcaTTkAQE0ts/cHgdx3inVZmcRl7dWsOj3aPI9J1xYEzm4rFsB2YCo3UQiRL0O1VlqJTyOKk3LswtJSmSmhzC3J76lg3TTpcWYv+4DgEB1+N+uzw/7UtVGd64dRX3nuyjbyZfbIhCu9sf87+1bMfXxOKFDPUHWk1T0Gx1FzayVEp57Gyt4pOHe3AcF47r4sC3o5qWI12wlmzKSeobqmsqNlbLPlt+C7577+BLT1daBGZdmrD4SyIoJ7VGMeFEOxYmgIAkoHbenmksJxTIZgx4ridX28bOHb/Q8Vw82T8GJWQsljW0HqE3JYNY4CsbVQghRvLyhJUhyxn9bizbxcOQ4P0XgUW1hrSXjw54k2FyFwlDly499x7vwbSduZ6bMYqMocs8SP7Cw2Q5ScrlQgwyBC6jBzRaXXR65lh7BvU4PDn3/w3Mt5A4fg2lFK9c24Lreni6fzLTs8C2bRye1AaDByEE+awBz+PomRYIIchlDbiOO3NVOhnG66upKrIZHa1Of2bWzksz9RYTf49v07+VcgGbq5Xp0KgJEEGwVimhVMghnHFwCO4J2I4D07aHkSAjMlxhFLqmICKzKyCEP+Vs4tGzo2E+5pEyZPZHfSxR2lgduBfDtDJr2rC4bW1REAJ8/s1XRphphM8xuVg9dE1N5ZidBquVIhSFoW/ZsQMUITKk9OvvvTaVrE4IgWI+i2J+8Smz63lwQ1eP0753Ob22x4g+hse21is+X2rEM4fcSlGivwHOOZ7uH+HoVM4MRo8FmrLChsnHhJBx4gNfSy6gUMkb+e5rt6Ayhq3VSuhi7iQUhQ78quO9UIeP5nGO29e3oKps5rB0aQQl4NPFJ5xu9S1L5kieeIKw5glIS6OOj/4ICHRVgaSQHWliISnor26uohjilzkKzgVMy5aruxNvR1cVvPf6LXz9vdfH824vsMmEXNRPeh92jki+zVkJSonvm0dQO2+BMQZGgBsFB9/YaGMz640IzHQbIcBapYivvvva4o0Vsu0dneK82faFSdwmn6DV6ePf/YVvDBKgAXKx4uD4DLXzZrr7p3k3Ie+dMZniIb5/ABlDhR6kf/D3d/umTIOBiGsnIYCsruPq5iry2RDCagHYtp+Jc2TAEZDa4o0rG1hfKSHq2+MQqLfa2DuuodZswXFd3HuyLwV9xA+Q0XNr1RJef2UHcXzRlJBBEAUgB6TvfXAfpjk505zGJYn13hBf/DO/iu31Kg5O62PM15EYjVWedWpAcJHg3Kyh4/b1bdx/ejDl7kJABnaneSZG8vphfZJMryklMDQNHueRLlEZQ8PGahmtdm+MgWVh+KxHtuPKjk9G4t4nqp4xNKyUC9g/rg3ie+9UBf7WzyuoGA7+6ImNv/0D4ElbgxDpbI+EABurFVzZWMEPPnqwvOebA5RSECoDIGPNAzFttSwEOWJ6phVrV9U1Fe+9cRP1ZgcPnx4OvhlCZJ+M5escqXwQuw3M9w0wSuXK+IzzktRrtMxXb2zjpNZEs92N5QQoF3N48/Y1fO/De6F8qd/5H/7LyFjvS2KjBITg8Dj3O9/kk4Z8WCJ8ESG87InzYr7TvmXj00e7oS4JAout9snrgVHjXRBzHGo2AEE2Y+D29S0ojOHug2fo9s2pOGXLdrB/VIuYrsVjdCFpEoxSXFlfweFpHX3TBiMEO1trODiuTfEG2o47EPycCzB4KNonONwHnGIGrxeBX77l4e9+nMO5radaDRcCqJ23cFpvpn6+RRBm5+ecAwmaWWUM25sr2Ds4hbegMhK13uB5HI7rRb6/oE9xzvHg2SFM05aEMv7+dLZJX9EcaN0RdQ1IREIqlWQKHdRrMHWfAc/jePDkECCYSZzSbPfwvQ/vRTrbx+HSTL0lDf5phFF1fBqWNTTkMvrY1Gex6c7IIS4Xirg3PdWZe4sBUyhWq8UxxujhUwuYto3He0cQQgxsdJPTD49z2K4LnpI8gxCCUj6LYiEbfoYQODytD6Y+AMFatTT42EZ/rudh/6Q2OKYQDztGG4wSeIKAC4Kfukbx9W0TCk1TS/mzXReu56W+LvrZpYZRKeVDWlRqz9mMgUopP1f5XAisV8sA4mxmCepJCYqF7MCONvpzuYdmp4u+ZYVeG8B2XZydt9AZMPTPiQT9vFzIYWu9OkesOKa/y4QXup4L13VnXiMEh+O4MxduwnBpBGUaaJoCTVNnn5gY80tBhTHsbKyM+V3qmopKKT9cfIgpgoJIPzeE2xcd20Wz1cUPPrrvr4YuE0LaEiMMOx6Xq4+Bdu1yD9/78N5MB+C3Xr0GQ6O4VlFkLnbXQ9/2QAjFX3yD4tXSpCfAfNBUZbCgNw8CH76omlBGEvn45bMGqqXCmJLseh6++8FncLk3qOvWejW1yxOBDP+c21F9pC+NJchL0c0ZpXj1+rYMAohpD+LnLaqW8gmJL1JoFEsCgYwmm5X8bhI/cYJS11T0+hbanXRJuy4K1Cf9pWzEoi5GyQPiO4fj+hT4MaQbi6yzMD/hU9gxzgXqzTbOm52l9VVCCEzLATfbWMtJm263b6Pds+BwYEUX+A9et2B4HaT2H5qAqrAp7YD56QU0VQmd3QfJr7gQqDfaODtvTZ8EQAjpr3t2Hj/dJ/7/hZE3TN4XQkw7tM94fx7nOKu3YNnufGP5CFSm4ObOBjQlyuIWpghQaAqD63lYrRTxxitXIxflhOA4qzfxePcoNkwyrG1UlS2NgCUOmYyO7bUqDGN2GulRXA4bpUAioccYxSvXt/Dw6UEsxf/zdFR2HBeHp/WxD9ZyHDit5OGRFzmavn7rGh7vH6PXN0OPTy0oLVgVIQQe7x5hhTjQ/JG73e/DcV0YmgqaUfHGBsN/+EWKv/3Dxe7XN225wjpSRi6Twa1rm2i2u9g7Opsy2mcMDTtba7j/ZH/mYlo876gY/H+3a6LXs2KndJbt4PiskX7aJ2ROmmXAdmwcntTlNDVhw69UClAVBftHZ6CUoNPrx16ZJv1GAF1TsVYt4bjWSHXtPIqSadk4rTclg1aK6y+1RhmEAwY+eZ7H8fjZ0cwMf/OEYumqAkNTU49qAkHQ/vj+i0zglQYn9YYfX/38IDwXq2ofuipD5zyPo92z0ej04XCp3f7stouvbQZT8Pk27mczHN3X75t49OwQx2fnI0b74WY7DmrnzUhXIENXfUq/5KpaEMsc2yYJzgHkGtfVrTW8+7qMx/7pL72N6hx5xQmkf+hrN3cGWiznYpCxc/yeZOB0P4nzZgf1RhuO68G0nEG0GqVymv3Nr76zsO+o7bo4qTdg+87+STaFMWmSSBlyGjizex6HqjBkDD2RYnWJBOV0ZyREUt5vrlUGZ3X75gUEyQsU8lmsr5ahKkpqQRu3UZ/fL8hNE/rSFYZ8LjOwhy3r3gBwWm/6K6PT9ZLT0+Vo3wpjgwUpSoCbBQuKQiFAYDku/vB+C5/st9Fsd+ASBVmV4K+9ZaFMl2t35ZA+rH3TDhVMjuvh7Lw1NYgF/KQ/8+W3Xwhxxeg7Oztv4v6TPQgAP/zkIZqtLqL6TuQGmcn04KQ2Vn7G0JHLGmMuc1lDx50b28AgNcNwsx13kOlxrG8JYH2ljN3DU3gh/StVPxUiYfK8sRbDV955bY4Zyfhg9ye++KbMRzUDl0hQToNzjif7J9g9PMPCBpoYCCEFytP9k9TThllQGcNqtQRKaORUfHO1gr/wp76KvJ/DfFmYFJqjUBjFV955DZqabGAY9f8MO/6Ft2+jVMgB8CNYcioUSsAFh+NxtLKv4FnXQL1jotsz4XJgo6DgL78pI3fmH4jGbYPFfA6v3rgi86CEaCEBr+nkxhhFtVzAH37/o1DBQAgGAjTpIJV04BslreBcoNe30O1bcBwXna4k9k3bLYQQsCwHrU5vjCMyo6so5jJgdBiH3un18eNPHw+Y75P0IY9zPN49woOnB+j4Zp2gfdPizdvXcW17PRFhiF8rOK6Hf/OdH/thkvPJBMd18T99+33fLBXfwJdHUEY+m/A3pJV/cYVGbBhZDVuO7dB2HNy9/3Sw+jkJTVNx48oG/vW3f5iKgmrxern4ox/eTTwwGJomWawjjn/vg3s4b8joFkYEXil6oJTAdlwctDi2r93Ex911fHxgo9E24UIKsm/scHx1vQOSxDlxAoxRvHn7Gq5trw0YcM6bbXzycNeP1R/Hf/JXflFGZoTAsR08enYE07RDB5YrG6u4eXUzcRilqjD8ys9/HUqC1dX/+C/9PDL6tFajKgzFfDbUdWxenDc7OKk15/K5jYPCGP7DX/7ZyPYdYmLWCOmmNdTqkgq7BN/nEvWqSxGZU6huird+9n+R+jpVZTA0DaZlw/G85HItwoNXURiubK7i6LQOK0kQ/pTUCHaEV6RcyMP0WXUG92QMlFKZb3yhd7H4dFH6qEq7zyQ13fb6CkzLwnmrE6vdEALcrgL/61f2sFHJodO38bufNvD7Z1tosSpW+An+2ust3FzLYKVchEI8PK7Z+H/+gOJht5yqvrevbePkvAHbllyXs5pP11Rwzqec5ZOAUgrVz+eeKPaYUSiKkqhehqbC42JKuDNK8dqtHewenKIdkYd7iBf3HTPGZLaARgum5SRImyJBGUHWkGYA23F8G2XKmy9ILjSKP/4f/6tLnoWRACLiF4eMrqNYyIIFgfYJsbMZnjOEgKDXMyPp7KcwMSIFpBlRPpE7m6tD47O/z3U92UGClAtzbyJ2o4TIVecgFCdsg/QJ3FytTN3g4OTMT5sRXxEhgCLtQ1MpmMJguS7qTgZ1W4XjODjhK/isxnHWNtHp9sEFwa01DX/hnQwyLJ2W0+r24LqezB2d4P1btpNSSA6fi3MPlm3D4x7inj/YXM+DaVkz2wuQAQOlfHbqfXieh7v3n6Ld7Q32EUJwZWNFznzGzk9UrQvZCORqsmnaiYUkIN2V1qolZA0Ntp3MS2QSmqKk0rgn5QshQEbXpIN8DC6FoBRxFPIhP0oJKCNodXrYP67JDpnwRwhwbXsNTJmOdHBcF2fnTandRfyiwBhFRtdQKeYH9qZyMT82Ffn4/lMZkztRDCEy/cWsl7UIdE3F2kopdpVQcIGTWgMPnu4v8OFwlKxdaAoFAYFpuTA9AspUUCpJHH7vZAM/2PVw3u7DdDxwTvBqzsIXVztQhDNT6Afbaa2Bft8a/FtVGFYrxcTXh22MEqxWS/MJH8AnYElBdgI5M1qtFhO9R0qAnY3VSMdvgnGi3ucB1/VwetYIHYTivkXP89Du9Pw49aRf8Pi3nM3oKOZzsedtrFZAKQn9fgkhWFsp+Wz60bgcgjLFuYrCsLZSwnq1PDTMJ7guWIzgQuA7P/p0LFnRsB5inDAgpLNFvYxcxsDNq5syHag/Nctl9EEEkSybh4YZMkpRyGcS2bPmhet56CbQlqXhfrF7vbKqQGHSqbttCXS4gXKlgo2VsiQGya3j905WUe+YaHZ6cDyBzQLFX3xD4JVqcq1ycpFB11Tks8bYOVHvMQqaqqIQxoyTABldw+ZqBRkjRXoFAViWg3uP9xOd7nkc3//oXqgNFpAmgp2tVZSLuRRV8JUAgtj0urFCLyJ8NvZZBEe3b6JvWWMuSyQIkphVbyFTMFtOvJ9pMZ8JdX0C5IJxu9ufqQlfCkEJxI3yGNtK+Szu3NhJHYq1Ui4inzVAIBNA8YDwN2IzdA0b1XLiJ7BsGye1Blrd3qCTnNQb6Jmz42s9znFca8DxojXZRX+246DWaKWOB0/7A4Ctgox+cVwPXVvAhQbOhVzJ9Tyct9o45zn8xo96OD7vodO3wUFxq8rw5S0XWWW+mG7TsnBwUve7hfzdvnEFpWIOST9/x3FxcHw21/1dz5MpIGJowaKu83jyZ5bnh79HLjhanZ4kIE74A6SAzGcNFPLZC53ZYOQb455c4bcnoo5uXNlAIZeNpn8baOMEqqKAkfi48mcHJ3ACzssJ+SKEQL3R+skh7o3G+FObloOne8dod4MQxmSbrqmSYzKhtsQ9Dl1XExVPQPDG7WuSi3Bkv227Q07KmC3gr0zKhjQvDF3F5moFuYwx89xrW2tzxSbrXhu6wqCpkp6tZQFNh6FvWWi2x1MHP8JN/MGuglqzh57pAILiT1/j+NJKEypJbksMHKaFEHLxBMN2PKk1YFk2krYsYYCYYwUekJ4EjXYH1hId/Kn/bHEx1qPgQqDWaEVGYkXC78eUEJC0Jock3wgheOv29Vh7IiGArqtodXojGvOMgoWAOiMJnWnZ8Lzk0Uhh+AkQlOPo9U0cnJyh2+unWiU+OqlLo3jCN2vZDvaOThOdKwTHk70jUBpvVGZ0PJESo1SS9yJxtRbaHGdk+h11no+z8xY63X5qv7gV2oShMTAmXYNaNsHW1Ru4vr0+fiIhWNvcgbf1Jey2CNpdE5bjopwh+I/eZSjPluUjRRFsrFbwyvXtqWPtbi/RKnVg2/uzP/NlKOw5RvbOeGeMUrzz+i0YupZMIAlAV9XIBcXRTWVswDguhMzz02x1E9OhJX4ICFAAx6fnkYxfhEh3qJVSATubq+j3zcjvW9dUSWYiBNrdHurNdmztFEZTk2BMYubVhJC/Qwg5IYR8NLKvSgj5nwgh9/3/Vvz9hBDyXxFCHhBCPiCEfCFpRZI48PpnIp/LYntjNdUUweM8lWAFkHz1G0Cj2YFlWYjrLLmsjvfeuAVApmbdWCujUsrPTxeXcvM8D51uH2ZcPf0pSa9vot3p+yYKkXi7WbCgMgpKGSzHQ6vn4I8/eIh7j/emzj06Pce3PtrHb3xo46DegelwcBCs5Bh+eqsHBcnuzT0PuwcnuP94f+qYrqp485VrUiOLKUNRGG5f28bv/tGP0OuZqZ55oW3GO3NcF+9/eA+drpnoHTOF4r03bk1kMA3fXr1xBaV8dtD/hG9DT9+3ZsPjXLKWR9gCZfrcHTTaXXz42ePYUr/09p1BlgEhxMyY/Xwug9s3rqSQMdNIImn+WwB/ZmLf3wTwO0KIVwH8jv9vAPglAK/6268B+K8TlD8TA1Zx/2E45zLFK+IffFaDJGmgpEjSZfqmhd2DUwBSCDdaHbRTasaLQiTs2MHZabGS16AqFAIyQRxRDChGPtQGJASHJwgadB3f3qU4a3bQtzlAKP6X71B8Y7WOYn56YYVRinzWgDri/D2M+x6HZdv46N6TmcnKPM/D0VkN3V64TXl7fQXXr2z4OWaeL2TIbkKB5Hn44ScPxhZ7KCHIZXQY2rhT+70ne1NBDpqmIGvoMm3yagU/943PJ64npQQ3dzZxdWstfCE0pp+7roeD0xosOz4/uBAC3/3g04FfaZJvvNu3cHRaT/wcoc826wQhxB8AmLzLLwP4u/7ffxfAXxzZ//eExHcAlAkhW/NWjhCgkMuMrSJyLsPNivlsrFtRHHJZY2wKPG/4XNrNdlwc1879UZCj2zMHkSDBVsxnkc3oYIyhXMhBVRhyGQP5bGZswHieW1II7uHzWzJ/D/c4GqaHo74qp42R75jAIgbumyt4VPPQ7PRguxyUEvyNr2v4C9+cnpQICNy8uoVsgtVlyQI+rsVQSpHPTverRsy08/jsHCe181SzjKl6P4d3xbkMXZRELf5+AJtrVZ+keFiPgMS2kMsM8tC/cnVLulgB+Kt//mfx8f2nY/1AYQy6Ll29gv2UUqxWSyCEYv/4DGc+E32aenuco3beShTzbTuuXDCLOS/IigkAtu0M0lbPi3mHxw0hxKH/9xGADf/vKwB2R87b8/cdYgKEkF+D1DqhZfKhN6GEIpfNwHXdMeN07byFbs9EzncF6Vt24vSmhBC8+/otvP/xg+eaEjUpsoYus/C5HnI5+Xz5XAaO60qH6SVqwUmR9H5upw5AQGEKTMtB1+Lou9JPsG/Gl/G4QfH7XKCaMZHPZsAYgaYyuB/9FgxqwPQYCKUgRAq1Dz59NPfzMJ/5xnbcxIserufB7V++/pIEnHM8eHoQeowyinwuA845TAu4++DZ4Nhvf/sHYxqnEAK6rqJcyKPebKHXlzmlKCUoF/JotbtL50qYF6qi4LWbO/jg00dz2FynsfA8QgghCCGpv1whxK8D+HUAyFXWRdjUwuMcjVbbt0EIANLtRFEYNlcrKPsj5LODE9Qb8Qbdkfviex98NlcY20VA19SRXEHAce1c+nt6HIcndWyuVaGqCk7rTZSLWXR65qCDXjasFSh0jUHTVPTaPbQtgd0Wgcg5MvzP8yKTRimagQe9Mr6z30LB6GBnrYCsRvGX3xTo9lv4Z08ryBoaNE1Fo9VdaLCQDELNuVICpMVQ87rwW80Fx3ZwdFqXtugJ3L3/FGIiNLbXM9E3x7k3XdfD491DP2opHYifQnLZ7SNNLo99Ibl44fMKymNCyJYQ4tCfWp/4+/cBXB05b8ffF48IAx+l0rfL9bhkNvE98Qu5DJ4dnODgpAYC2fGnWjpmtdZ+zvyMcaiU8rBtx4+hHjdMe56Ho9O6zDbJOQQyIwsAwDLiu5eJnUwfhqKCUgrLcdGxAZvlsVbMSyacXh9Xt9fxZO8odApr0jy+W2O4Uaojb3ShVQpQKMGvvK3je6cumkJMJVabB0IIeLGkvMsBIQSvXNvG0/3j0ACHi0Wy5xNAJBu5F2La4kIAE6cLIeYi2VAYQ7GQG7AkpbKfzziVeyJU+M+LedfM/ymAX/X//lUA/2Rk/1/zV7+/BqA5MkVPDyGjFuTCDQAh4Lou2p0eOJfC07KdcM0g7cLdRW4x6HT7/nQl/MSgE2uqgrN6E72xFLryBpqqYH2ljEIu80IfdLMAqL7t17I9WB4AZqDnR1+IMSEfXkbTM/DH+8DBuYl234InGIo68O+9biPHm6g1muAJ460vyyb487sXJcCbt68lyvWTGhfwXQztqCL1tc8TMzVKQsg/BPBNAKuEkD0A/xmA/wLAbxBC/hMATwH8Ff/0fwHgzwJ4AKAH4K8nr8rw6RllIJRId5aJVeFhru3LpU3FIubFdrqzWGGAYj4rFz2i2FX8AcR1UzAopUSlXIBju9J/dfKg/yqulwg0hcJxXPRsD0dtaVTv9odhnQ8jbGWj+KRTQfFZE8VMDypjyGdUfOOqi1rPw9//dLnPtVSMNAxlFFe31vDo2eFCC0CTIESuLD89OA4tlxCCXpDSOKIvEADZrAFdUxObrC4Cruvh/MLun+xDIERG98zSiGcKSiHEX4049KdDzhUA/kaiGk5e6/9XvkQda9Uyjk7r6EYa28cbIogPTZJgfRmgREYDTNHqzxDgBMOY86RotbsQCMlv48N1vamol6VDCGiaAtNm8CaJZAWgel2sZAQMQ4PtuuhbHmo9ge3tFfRNCye1RuJbOSSD79UVvHe2j6zOoGtF6ArBe1vAvz1w8Lg5zXlIKYUQ4yk5pH0Q6e2ZRL7fqPYkhAwWlaKgsHCi5iibnIxJj77n6Hmex6EoLFRQepzj2cFJbN0IIdBUda5BlRCM2ZmjpsuUkpn+jZcBlBBcu7I+0+5/eSJzfJVaAOj2TOwenk5MM+Px2s0drFXLz4U1hVKCSimPQkiuEDHjp+kq1lcrfprUZD/bdWMZjTIZDSuV4oAh5SJ+jXYX3V4fpWIO+Vxm6rhKOXRNgcIUOK6HjiOwZxdxcFJDrdFKfT9TKPiNjz2cNk20Oj1wouBGmeBXXrWwbphj51JKcGVzBTd3Nsf2Xb+y4eeYTtcuq5XiYPoafrzge1xEl2H7MePeSC7yUjGHN29fgxLCXFUs5FAu5WbW1eMceydnkpov5pzYMgRHq9PFebuTql0EBAxdQ7mcH+QZD4OqMpSLeWQzKchBXhC4P7CcnJ3Hnnd5BCUCSSng+Ys3kqNx1HYhIrdHzw7RaHeGax0hG6MM2Yzhc/lNH1eoJAKeFf4luGxgSsjswP0QmwylvlE8zvYya9/IMUJkiBYP47SEjHrIGno6+i8hHbszhi6ZgDiH7bjwPC7tXxPnvpprIqtKElfbdtG1PJz3PNiOIx2m09if/O1YuYH/4VOKWquPbt+Eoij4+lWKL27LbpvN6Hjt1g4A4LTWxLPD08G1nHMcnNRwWm/Ilc8U76jR7ODR7oHU7kLOUVXVH7hi+ogQPhnzcF+r3cWDZwfT9YFsa+7xyHuOnuu6bnT/mdgqpTzeevXG8J35cD0v/F4hoIRgtVLC1a21wTVkpC8xJgMAgjBB7gkILsAom+u9J9m21laws7Eaz6/qa70/+7X3fA16+rgQ0u90lhfMJRKUoxh/c4zJvL9xsB0HpmlBxIRg7Wyu4PbVrcG0eXKjlODnfuoLqBTzkWUAAkJwnDc7aLV7UAY5khO8XSFgWTb2j2rgXviChK6qvnP5sEwKAnXAMThdZrvTw8FxbSojoTwOZAwdX3z7Vb8jJ++Na9USXr2+DU2R0zTOOc4bbZw3OlPnlnMKDEVOSS3XQ7vvIVNYGemw82wEn/Q2sFvrodkxYdkOVMbw77wm8PYaBxHAs/1TeB5H3zT9bJPBOxIwTUt6S6S5vy/ghnR50+fsH50NeTBjypk8zjmHaVrwXG/q3Np5C+fNzsiC14xyEz5Tq9WTPAQ+FTgjFCoLyKWnz2eEQlMU6Koqr5GfIEzLxvHZOUzLRu28NYiMAwQqxQI+/+argzI9z0Oj1YXZt0Zy28uoH5qyD0Zt9fOmz90wA0Lgez/+BCxmoqmqbGY2x0sqKOUoFcRyG7qGt+/cWLjMp/vH+OCzR3DccPcg23Hwm7/7R6g3WwlKE9A1BW+9et1PCJ8G0effubWDP/ezXx1btVQUiutX1pGLnMoEHSj8WKvdxR9+78NIDsMoHJ3W8cGnj9AzR5Mvjd9L5qrheH2VIKszyQTucDxtK0tJaO9x4B98SHBwbqLbt+GBYjsn8NNbXaheB31zdmKoMCxWt+T3S36f6Hco04UQv7yU6Vm5h26vP3j366tlbK1XoERIjo3VMjZWK3j9lWvIZDQEAr7T7Y0kHxuv51m9gT/83gdwvWH/yhgqVldKfupf+T3fuLKBtWqy1LthOYRGYTlOYrt8p2eib4VzVjJKsbla9ZWjaFxaQbmxUsVatQTGKEzLwcHR2Yuu0hQs2/G5C6MhDf/JO/dnD5/hn/3OHw0M9YRIo/jZefPSRD0EyBg6ioU8YHcB7kFhDKbtwrI91LtL8hskBHXtGv7FQwWnAR0bYfj52yrervbByDz+exTFfC4dwe4cIIRge30lcUKyMFBKcW17HSvlIlRFwfpqZaFkY8endRye1CNXeVudLs4bLXzy4MmIh0l62I6LZrsz6LOcS4atUz+8MQ6GruGnvvy5ue8dIMlXxwXHebMds2gs8fwj/EMhplYmj87qUBUGzmW+kqMZxtYXAdfzcDyjXqVCFlwAnW4v0Sqg5fuGAvJDy2UMKAq7+FXtOVDIZaAqCmBQ5HQGXZeJ3nq2i0MzA6Gn1/TC4IHgUa+Ijw8OkTVUqEoOhkrxV79g4ODbHh41k/OMAlIf0jUVmqqm521MietXNnF8dj53JJEQAgcntYFGuFouon7enLs8z4+rjsJkUrl54fjx2AGEELDsZP3X9Tz84Xc/GFw3D2QgxzAneRSEQKIp/KXVKD1PJo26bMIhLSgJOCfnmOsJPz8QDexJ80FTFWyvrySi108KRilqjRb2j09RVfvIa3KxzHY8dEwXTWdW2tJxKIzFOkk3bYbf3c3g/omNTt+Bxwm2sxz/7qsWNGGmah/X9QYkF8sEIQSKwgaLGkIIfPsHH6UOlx1tC845+qaF69sboITgo3uPlxLlk89lsL5SXrici4DjuH7CuPn7fC6bkd4FS4qNvLSC8jKAEjJMOTEnzludubVBAaDV6aHeaC/Efs4Yg6YpoSYAQ9di2X0mEdjL5IdWASEEK1kgq1IQQuC40kbJleR5ZwiR7larldJgONG1CdYhquDQW8H3jxScNDro2S5AKL5+XcWfumrHjkMBA1MSEwijFOUZ9qq459hYrQzIWuYto1zK48rG6th+RWFSe58DjNGxZyeEQFdVmb30f6Y4PKktTTsGLpGgJBfBP+9vwz6S7rpqWbItMxqy2pxwE4JHrp4mvX56JT8dTMvG7uHp1JSLUoJiPjuVkCsKjDG888YtKApD37LRanfBucBGniCrE7ieC8vx8PhcQDNSJLcSAp1e34/3lfUqFXIo5Mb9VF0o+PiM4Ue7JuqtHhxOwCjBX3pbxVuVaXvaQOjqKirlfKKwPl3X5ta0OOdotrswIxYOkkAIgW6vj0arO7b//pO9yAWJmNJQKmRx+/q2n0pi2KcarTaOT2uYt18utmHMq+MiNs/zEoS6TtYhGpdCUBJCUMhllzo1DEApQdbQYWjppoIA0O70JMvMpaN+SS9svRB3JMGla5Gk0pIDSj5jRLJjc+7hye6hn4vcRqfXA+MmVjIcGU2GgVmuh72Gk7qOfdNCt9+X9QDwhbdf9VdRx89r8By+fZLH/WNr4Ih+UPgy/twX1lEVw9QdQSpTQ1dhmQ7q5+1Il6zRzXEc7CdMARK2dbo9n3Rl/o+8b1podcZdsCQxsay/wihuXt3E9np1hsCRqVMOj2tTxBce5y+sX2uqgkIuu3B6hnlBALx950YkwXAYLoWgBICrm35OlSUPLgplKBfyw5Atf9NUBa/e2IlNpGTZjtRywhy5F9woCNarZaxXy6kdwefdCIDVSgmVYgEQgBDyo5S+hrJNdrbWBs+bNXSsVkrDvCpcSDPASM4dhQhojEi6ONdD3/LwzFlbqJ6cC3z3R5/4KRkmjhMF+1YZx5aKZtfCP3N/Ab9NfxHfXfvr+Oqv/MfI6jq21qQ9tpjLIZ+RJgApFGa3s+NIH8rn8T7m3TzPw/FpHc1WN8HzeHJAuYA+PLm9efsGVDbbyVzXFKyWi/O714ZsjNJx38+IjYBgtVpGu9tH/byFsYCWGFyKVW8hBO4+eDpkEZkTBGQQYyogKblcj+PwpDYWm0pAoFAGy7L9uGs+VgajMif1ovWJg65ruHl1C52e6Yf4PQduREIHVG6UEukg7Mcsc8FhWQ4+e7Q7eG4uBCghvj+rF9oW23oXVc2FoijoWzYsx4PpAkJdoN0EBmQNYff0QPD3v9+B87UdnGy+DhsGQIAOex3Vm0dwTh+CgODotAYBYH2ljO2NVXx878mAMzHoK0JMx+tfegiCvmkPtMnRPn+RCHgKApe10fsRSDIORVFgx/jrEhC0O310uubSvi8COWvc2lhBrd7CWSPaBSmI4W+1uzKVtAjvY5O4FIISwJiwAvxE7CnbkFKC1WrJn7r0sLFWwZWNVXzw2SN4o64KEOhZFnaPTqY+EsYo3nvzNh7vHqLWSOJ4Ph/6poXvf/gZgCAnysXD4xz1RgsCwO3r23A9GYZ5cFJD37QgMO6mZZqWz1gU3aEzOkNWl0Oy7Xqoty3wzJWF6pmk47LSDv7Afg/XuA5FkQLPdR3Uj3bRbzYgRshRjmvnUyYUSgnKpQKE4C+UQWcuEOl2BgLomoZCLoP9o7NU3AjzwtA1ZDI62p3e2Oq7gMDTg6OZ32zwbpfJ0i8g0O710XmyNzM9jMc91Jut1GTKl0ZQDno1AXIZHaViHvVGG6aZ3IDN/fAp6sdyH5+do95oSfvMRKMEuUQm4XGOjz57DCeIMhAAoTI5k8eFHyY51xOO3x94IakoAmHxaPcQqqLg2tZ6JD2bACBi6KeE4NgpAqt5NiBQftRk02UFZqBlKjyEoLP/CfDWN+BwD4QQ1Hc/Rbd+BFVVYGQM9EwTnssBguFClpCDYTZjwNC1oa/fJVEqKSXIZoxB6o+wekmmITlbOq01UG+0h4OtCC/P9bxU39L0TWWAgUwbIYax3iPgi5Ahj5oK5ygm6nsOwzz1vDyCMnhMQeC6HI1mxw+ZSv5QgotBYqPg33ZCJ9fRa0zLgqqo0A3pkKwyBVvrK77gDlK9/mTDcz14rod7j3dnnxwBbpuAY0JhWf/D5jhtj4e5KUyBpimwbWepmjMRBIwxiEe/jXuHHRjFNZw9/jEAoFIqoFou4MHjPQACqqLA0HR0+31wIbXocjEHIYCTs8Wy8y0buqZhZ3MNe0cnsKzw/s+5QH1kthPFUC4hCS0IIXi8O5sLNBKCoJjP4cbOBr7zw09C67UIKGHIGBpM04YnLkeallFcIkEZQMCyFxj5lgRNZdhcW8HDZ/uwHQf3H+/Bj2x+0VUbA/Wd2RdxzA/slWmTMOU0gnJW8fP+CJiOiyM7h1FlQ9dVlIs5mWFviYJSQK4wv//RPQBApzYUAidn9TEBmDV0VMsl9A9NcA44rou9g5NL9iYl+qaFTx8+XVp5MnRw/iQDQwgcndb8tK/LbzlGCTbXKtg7PIVnj/cTQ9dQyGfR71vo9JbnGzkKOmMF/tKseqcBJeRiqO5HYNo2zs4b8n40oOO9fJ9WPpvF9SsbC/FwaqqK9968nfq6guqiYgxJZHuWC5JdHet03V4fRyf1heLUZ3XiWWh1etg9HGcEX9abTBvL/5OPi/kGXM/Dab05ZY4ihCBj6NhYraBYyI31BQIgY2gLxb4H0Ge4D15CjXI28rkMctkMjs/qC7EoxxmUXddDo9UBpRSFXBYKYzg7nx3Q/7zhuK7P3Tk/s7vjzhcylteBcoaAEQLH5bAdDyZXBwnsAyyiSWqqgu2NFewf1VKzHwWQK/0X41VQyGVQKuSwe3h6IeX//wuEEGj6ubdHBx4hBFqdLnqPTbnSPrLoyxSGL7/zOh7vHWH34GSqzDRwZoSFXgqNUlEYdrbWfMouufKaMXT86Z/6Iiql/DABkc/DZzsuun0TgouxY2m3JJAZ5rwZdiAJVWEwdO25ahi9vom9o9OFxnnX9fDRvcepr6tmKdbzkk7NcT08Oedom95S3W04l+970iviRYAAMDQN6ogG43p8aamPKSUwdG0ps6VcxsBX3ntjCbV6/pj8Tl1X8j7YjuMLSzFwczs4rklf5wXkgBAC786YUV0KQcm5gKaqICPkmVc2VnD7+pUplVhA2nFa7Q4EpHCaJ6KHMTrW4aMgQ8pMNNvdmedmMgZu7GxeSITRZcRJl2K/LdDq9tHp2/i0xpbuDep6Ho5Pz0PzwyiM+aF5zweEEJSKeVzZHEZ0pMkHRAiQyxqRseCaquILn7uzlP6jKAx7h8m1LEVhC5s4njc459g/PltK+umn+0exxy9FywghfIE47CCPd4/w9/7H34qkMRO+N/5rt66hXEpGBjqKrKHjq++9OZPZOKhfEi2p3enhwdP91Isi6bGMcIbFy3lcd/AvP+nh7l4b//wzBw/sdYjBgtfyNi8kZpcQ4KvvvYFiPrv0+0VtXHCc1s+xe3gyiL+XsfxRoZHjbRystkfV2bIs/OCDT30Tw2J1ZZTixs4WkiCXMfDarasL8Wa+CFBCkM8aKOST8wpE4XTGYEeW6fg5L3LlNfG5b/4l34E1eX0IkVqhzP8x2jlngzEKxiSrzO3rV/DhZ4/Skxm8+KZLDwJ882vvYe/oFA+fHkQ76CbwaxOQeaQZAbgAFFVFNmOg3e1J/8WLBJH2S8/jwxw0F3APAKFlKyrDL33za/hXv//HcGx3/HzIuv3lP/ez+O9/83fhOH40EJHEIlwIGQaapg6T9Yh7P0Rq24QSOLYLyggyug7P4zBte3g+8fMpZQw4jgfTsoa5eGKgqAy6psGy7HBf4Oc0oSI+u1fgPz1HCSN/C/zwX/ydHwghvhR25qXRKAe5TVIMnILLJGQy9njGtYObyc1zOWzLQaPVwQefPpJTu9lKQXiZP0kQwB/94GPs7p8gznN+Y6WC1Uo5Nv0uAYEQBC4nuL6zDUPXcevq9pjwTZvlL8kPgE8EK30zg8ihZf4IAbKGAYWOZ0wMwD2B3/32+xAjzssEPs1aJgMhgH/8r/4Q3shxIaQ9OLGQhOQEyBqGjKEegaYoMHQ9/P349wkEuOBAtVKUWUNHXrmuqchlDNy4sgXbcaAqCnRtto1dcIFbV7dQKUXQ0QmZymGlXJQcC/6+pJuqKHj7zs3I6zRVxbXtDQgh4DiuzEGU4tsPNoXJQJKRtD6RuBSCcn6kmYqwEbaS4X5JiiGmEz5BQFEYDFWLZWgJIh8ISZcSlRBA09QRO6nwGVUuLuVs8DNtG6Zty3j2iJ/tuBCC+1oQnZleV2oqAu9/dG8sXeqyoakKdF29cKVF1zV840tvg07YQIPn8riHTq8HxxumEaZ+DDSBT67R74NzDxld87kfJ/sAQTajQwaSDffpmirzlEOAMoo7t65C17TB/QmRgu/N29cT9TsuOJ7tH+Okdj62/wtvvQZVVfHBpw/heR4q5QK21quh6XRHf67n4YNPH+Kk1vDf8fQWkHBI0mmEnhO1KZSi1zdHXPLGpZ7KmLRLRpY7DkYZlBD7K6UM3/ji55DRZ6cEuTSCMuyFANIOIR2Yw89Iuq1WSshnM1MEMpmMjptXt6D6i0ajx9ZXyri+swFCaWS5uqbi7ddupV6pVBSGKxsrqFYKPtsA8NNffQeaT1arKmwp/mHz4rzVQr3RAhcCGUNDtVwMP9FvCEmU2vM9ExC6BaQKwIj/YZqXKICdzXXcvLo1CFNdaMOEH+TIsb5p4Xe+9f0Bs1KSzeUezuoNdPr9Qf8VAO68cg257PTHSBnBW3duQNe1Qf4eTVNwfWcTWf98x3Xx408eoNPvDcoEJLPV473DMXeZUSiMIZ8bkidrqjJFQm1aFjwuyU64EDg6rePJ3lHqxZEwMeW4LpodmTMn7FsFhn1h8php23i8d+QP5JPHBXqmhePTeowMGJcQGUPHarU8JT0cx8V3fvixTBcx0rZhuDSCMgyEAFe2VqGl4JKMmjVQSuToPzHd7Pb6ePB0D47rTDXV0WkNj3YPY33w+paNH929F20jifiobMfF7sEJzurDAP1//Qffg207YIyiXCygkM1OlwMMconPyUccuw3MchwDVwx7NB46rMcDEFzM9GlVFAWvXN0GYxSGruHGzmbs+WF4sneI+4/3xu417yIxAcFqpYhKMT89hRXD50+M4JoJSrOPP3vsk45gbD/3OH589yEoodhcqwJCkiw/fLovc/mMnDfK3SwgQ1Bte9hnx/oCgNdeuYbb169IliRCUS0X/UWkYTk//vQhWu3h4CZC6h71vuPAFIZquYDrO5vDe06CSHLmamV6AOYz/F65SM6lKQBYtj3mUTOoJ6O4urWWSCG5PIIy5KUIAIauJ/7IKSFYGeFPHN0Ojs/Q7U7zDAohOfs8d9pG6Xkctm1D8MAGOr0JzmEFuX3CzomqrJD5o13XHZzrOA5c14XgkuW62+ujmM9hpVwcTP91VcWNnS1QSvD6K9d929VypGTALD5WppC5yGVI4GLlC8FRyGdBQSA4RzGXSV0G51y2kS85VspFlIv5ORmzgUqxgLWV8hgD+OSmKlI7U+Zs67WVEhTKpvoD5xyW7cBxXRyd1Abt7bpuqClo0I6co9nugFGKrbUqGKXIGDpyGd/tSEgb4eFJDcV8DgqjfpI+MTad9VxX5kKP66dhH0yCjVKKfCbj81NOHpdFUUphaNqyum/kZtt+FteJ/VwIPNs/9r9BxOJS+wMILnDv0Shpw4ynAUGlkJeEpjPPnQ1K5aq67ThLYQxKCs65tNEQggzkFCywN0v6Of+DgZhbmwoDoxSFfHYpmQmZn1tnNOLBdT38+JP7UBXFn1I+XPg+gLTrzReFL3B4WoOmqrGaI6UEGV2D67qYx7dcQEBRKHRNheO4E9qQHIjmgxj0CdXPqdPrS57HH3z4GQxDQ7mYh2kFyddSashzwnUlsfDJaT3inch61J5rpNv4h7KxWoFpO2i1u4na5BIJSlnZwOl1npCzwAHVdV0/eyFdiMpsY7WCQi6Lh8/2Qx2e04JSCgIk9rMMwreCvwFpm3q6fwQhBB4+2V84NC9wr+Ccy3jbWgNcCDBG535mQghu7Gyi2+tPpWolhOLVm1dxeHKG2vnifJ/1ZtsnX55DTAqBTrcPoB/7sdiOi0arMyD9HYWqKHLIEiKyvY5O6sgaOq5ub6B23sR5czn8l612F62OnDp3ev0BMXPwbJZpo+a14DguzlvtmVyNy0aauxEimaYCL4ZlI+CHcFzXT4LnTmjS8bhEglI+zGqlCEVRcHhSm6vBAm2omM9hfaWCR7uHc/pYAbXzJpqtzlKEJCCjMm7sbOLDTx+lCqEM+zdjFG/euYHPHj6DOydhK2MMm+tVCC5kGKTvbrG+WgHnHCe187li6YUQ2D86hefxKUHuui4ePTuQJo0lfBCLlpHkeiEE7JABiVL5Do5OZQ76o5N6aDy6EAI908Te4Qlcd3mCQMjCAQTcphN5cYSA52uraekGAcx0E1omVEXBm6/exGePnqF/AQTEWX/R9uP7T8C5TK4mLQnJ3sWlEpRCCPT6FhhzFu5M3b6J/aPTwYeqaQoooVNO5XH3sWwHFtKtACqMQVUV2I4zJWAt00atPn/y+lFwznFWb8wM5o+CqjDouoZ6oz1Whud5gynRIoQjcZ292+vD0HUwxhbKWPi8oCkKGGOwHBucyyiyv/gLP4Pf+Oe/i3uPd2V8tu+mEwXOxYUIgIvE8wxGcT0P9x/vLsQyFQfTtvHo2cHASyBt374UizmEEAQMxZ1efxBXvUiQu+MTZwyYbATBu2/ehqYqY+fNV18gmzGQNaZdPrY3VvGFt+8gLDzBchzsH5/Ndc9JCAHsHZ7C9TwZYZQ1oKoKklq4r1/ZRLmQh2VZY1kDBeQA4boeVCXIBR5dDiHEdz2Z9FGVtj1NVfww0dEpjsAv/MyXB2VTSpHLGNAD5+TUFnvp/iL9ZJNfk/S8W9e3cfvmleE1hOBf/t4fQQjJeG9oGvL57FhaYkalTTKq/Yr57MAlaJGVCkIIVFUZidMe9mlGqeRQuGTcAwpjKBVyY1kYPY+j0+vDCwIIEmyA5KrMZozE8kCIaLkSh0sjKIGhn90iQiwKlm3jO+9/nGrEIhG8l5RQVIoFFAu5kU4oO+6zgyN8+/sfwhtLtTrqwjL/RxG1aaqCSjEPw/fBTIIHT/ewd3TiU6CJkaoJQAioKsMr17dlzu+Q2xIik7AxSnDtygaqlRLGZAIEysU83rpzU7ahf4ugGf7lv/mO1LAEoDCKcimPXNbADLkcuknS1yqubCXM/ggpxClJ4osp8NnDp7h77zGEkILJsiy02l1AyFDEg6NTfPbg6Vh2SlVV8NX33pQDiH9PxujAd3S1UkKpkEUCqoFYlEt5bG+sQtfV4cP5/81mDdy+ERDLLNLHMBIHHjfwJEPG0HFjZ2smB+QsUEJQLORQScj1sIhcuRSCUnAOSik211fw2q2rF3qvac0neru6vY43bl/3heVwv8c5js9qMtJh4OA29kRj/6JUss5kM0MNdJjdMFFVYjfLsnB0WkOnMzt96SwBEsC2HDx8uu8vdkyjXMjj1ZtXkc0YeLZ3jFq9Mb5YIIBGs4O79x7D9WOdNVVFtVwEJQR//ud+aqBR2baDo5MaGq3OXAsOnsdxdFLDQUJOSFVRcGN7E9e21xOdnzF03Ll5FSvlIgrZ7CAFbhxs28Z3f3x3sJiYNXS8dvMaVnzH/Sd7h4OFs0XeWbPZwcHRKSx/0BkV8N1uHw+f7MlV9ZTvfxSaquDP/9xPzSDNSN7JVqslfPPrn/f9o+evmMc5audNHJ/Wl+r9EYZLQYqRLa2K177x56FpCjzXS0n0OuIYMuNRJO/l+sCoPgtMYdAURdqmUiQkolTmF7FsB33LAiUEaysVEABHp3UQSlApFkApGcvx86IxKzohAAGBosiQ0DRckSpT8O5bt/HB3QcDTdxaAkVWWjBKoSoKXNeDO7GSPcmDQSBnFYxSuB5H2dfcD07OUmknCpPuO7bjTHk9xMXTXwYQQqDrqnSPWgLZiWHI6XKr0x0MonPXjVKsVUvo9kx0+ybSarej+PFv/b1IUoxLsZhDiHRAtiKyxG1vruKs1hhLjwnI7Ihr1TJanZ6f9GvGfUCGIWkJ4LkebC5w8+oWnuwepaBPkxpku9ND35SMLMen9ZGj8N1xhhVJKqQuAwSks3xaEed4Lr7/wacXUqc08DiHF5GXKeBlDEw0QWxzMHjXYnJGx2G0jEmoKsNqpYyDk8Xs1wpjuHVtGw+e7KV2l4oT1tIWu7xFN9O0l1eeEHBcT2amhLgwf+eZU29CyN8hhJwQQj4a2fefE0L2CSE/8rc/O3Ls/0AIeUAI+YwQ8otJKjGLQLdSLAxZSEYrT6TmpqsqKCVTsayTcFwXhye1VBorZRSr1QroDFKI0Z/HPTzbP0K92QIhklFmNF7d8yMrWu3uYN+yEBi3U09FUk75dE1FIZcdhFP+z2VTFQU7W+vP756Q9sxScTFORV1TUSrksLFaxTw8Z0n79ry/i4IQAo1mG92eeaFBIUk0yv8WwP8DwN+b2P9fCiH+r6M7CCFvAvj3AbwFYBvAbxNC7ggRn38yymk6aOC7D54MWFJG4XGOR7sHEJzj2pVNdLt91Bqt2KlgmukSAYHrefj+B5+kduwORnRVVVDIZ3F63gBGNMjIEX+Bl00o8e1+xF8oubieo6kqshkdvZ4JL3GahherNRNCB4tFUX3Edhx0Oj2M1/UCp8ZCalifPXw2d/MQIvkmGWP4zg8/TtdXL+jRgkXZABctLNNgHlPHTEEphPgDQsiNhOX9MoB/JISwADwmhDwA8BUAfxR3ket5sQ0ZKUjFkLDzvNnyw8MSdJKE7apqDCuVEk7O5nO8BuT0ff/wdMiZeZEQ0kdRkhtcLHFu3zRh27bf3j8ZZoOf/+kv40d378GyHLQ73dDBSlMU3Lp+Bcdj+b4v9vnmDYgIIIQMtOj2TRm7neripCcmFy6qqmBtpQzbdnFWb6Srz3y3TIV5hPYiq97/KSHkA39qXvH3XQEwGpy95+9bHiKmMM1WF73+5Mrf+FYpFrC9vjrm8kMIgaapoQwiQghkDX0QWz3P5nEPPdN8Ls67AQHyvNkKA2iqAk2L53x0XU9SaF2CxcCk+MCPM6eURH4qHhe49+jZQvfRVCXUVHRxELAdB47jgFKKbMbA59+6s+TUDgKaJhnsdV313ZrC+zznHK12F+1ON+KcxLdcimljGZhXUP7XAF4B8B6AQwD/t7QFEEJ+jRDyfULI9107XsDN9+DTF/f6ffTNcUJQQgTu3LqKn/rS56bcgBzHxZMZNGuLglIy5ngLDCnul5FkSvo6DstnjMYmkSKEYGt9FbevXwFbIh/mrPsuGwEBrqoMhcXRaR3HZ3V0etGx3ZZtx5I1SLKP8eeQ9vHhPl3T8OV33/ADAJ4vCAHefPUGXI+DKbPvT4j0YJjllK5rKt554xW8fvsaPvfaK7Hne56HTrcHK2LB7LlKwITFMRLfN+d6k0KI4+BvQsh/A+A3/X/uAxh1hNzx94WV8esAfh0AssUV8Tymb5bt+MzdI7ZCLvDp/SfYWK1grVqaim9Okygsva0EuLq1Cct2cFKrw/P4QEiur1ZgWtbC7kPFfBZv3LmJb3/vAzBKcfPqNmzbwbOD49D6CiHwzM9It6y0s5QQbK+votc3UWu0nosWqioKvvLum2h1uvj43uOBlp3Rdbzz5m382+9/mHoApJRipVwEpQRHJzUISEFTLuaRzRg4OD6D53G0uz382x+kL38Z8DwP73/0GQAylV99EoQQGJqG6zubePRsf8yrZPI6y3bw/oefDZQMb4Sr88VE/iy3D436OIdhLkFJCNkSQhz6//wVAMGK+D8F8A8IIf93yMWcVwF8d557XBTCOo7reWh2uhAiGVkrAQahNot89AJAq9OB7QzzqAgh4Houmq3OUj60Ts/Ed394F4C09T7ZlazYk/WW9G3y72Xm5Q7Ka3W6kmh2gbIzug6mMPT80NQ4uJ6Lj+8/hjeRk73XN/HdH348lw1XcCkEZVoMf5/PT+C44+8wEXWXn9ir1zNT+g7H1FFIot8/+fUv4DvvfxQbSy9JmR2cnTfgeRwZQwchJJRmb3Q9IOxYFEiC7+RFCNpyqYDVSgl7hycwLRumFe9eOFNQEkL+IYBvAlglhOwB+M8AfJMQ8h7kt/4EwP8WAIQQHxNCfgPAXQAugL8xa8V7Fp6XDazd6aU6P+fHl3ZjuBsr5SJM04olQ6g3pim3OBdod9PVZxqy3cqlPM4bbcA3YTvutLFfVVWsVko4XNCPLw6N1qLUYgS5XAZXt9exu3/sp6mIYcHmArXzBiqlInRd89+BHISQuEcSZDI6VMbQ7shUDL3+dKRS3zQxD4VnMZ/D1voK7j/eBTwZ857xbeKzOUFj/B4BfOf9jxLpXAG1HiUE66sVZAwdj57uD/xIc9kMKCGx5oo4aKrkC7BsO3It83nauQPu0n7fxJ5pDZ7TmRGAkmTV+6+G7P7bMef/LQB/a1a5Idf5KQJ02AHT9yVdJyCUIp/PgHsyf0fYiyYEWCkXUW+0YFpWzLPEP6Sua2CUyoWqkHMpoQAJ2FCmj2+sVtDt9UIFZABNVbC9uSoF5ZxtXq0U0e50B6lZU4FgXJsJrYPAWe0clMhp0nmLyKF4Vr3KBTRaHZhmej87QoGcocMwdDlwLXkmfd5ootluD7Rjmac6A88nbo4CpdQPyY+u0CwNaQqEwjRt9E1rjE0qlzWgKIpPKBHegITIRF3c88ZeHSEEuWwGmYyO45P60rTmtZUKzputubhmA/KdtKxVlyOEsbgibn/1l6DrGrbWqjhvdtDqdJYqKIPVbduyQ79DxigooXC9ZAKaMUmSGjcFXISEGJA6w/paFbqmYe/geMpFiVCCaqkIx3WlxhNScVnP+DpQSgBCUqVRHdSBSI3082+/hs8ePkGj2ZmjDILtzVWoiord/fgIqCAZWNI2pZSAgKSyNU9eH9j7os+hUBQGx57OuxSHjKGjVMyjdt4cCCfmp22IE0r5fBYZXcfJ2flgv2FoMnVr1DsMUUAD7SowtSj+4o/rDolSiM9JIMsNr5OqKNjaWMXRSc1nohreTFUVUEKWSp/20195Fx9++hC24y6FjT/Ah7/99y93Xu8AlmVj7/BU+gLOJSSjl7U0VcFateSnHx0/RogcOYuFbDhVl5jevMAmFbOSxj0+85y4TQigVmvi0CfVDX1cYEBNFYYw8txJcC5ZcOaJuGCMYaVSxCf3H6PT7c9VhvBDPB3HQbVSjE3BygUfZA5M8vM4h5vi/LDr4+4HSCFVLRdBWXya19DnCfrIyPuKU16EAFzHBaNDjwRCCNaqFWQMI/olh/QvQ9Nw59Z16YDtl+s6QQ4neY7gfirnGPIOz/P85HJBX5edlwBYr5Zh6FpE6lkRXbmY7Y/e/xC24+LK1po/kF08LkWs9yim1fPlqJWCc7Q7veHLHLuDgGO7Q6Hy4pXsASbbQ4xUTgiBs/PGc67RODjn6PZMmJY1t/+mgIwd71uWXKm9TC9gBgRkMrBev5+YHCRAf4b9Ogo900LPHGdKane60lw10XZxUSjcT4y36KySc4Hdg+OQIzKDpxcElAjgy++9iQ8+uQ/LcgYuXEIISY4SVY2JR5Aar4NGI0glkqb+8wnWSyQo53hZKS6xHRce78vEVhMuQhDxjNzLwKIfPyUE1XIR3b55qZiyPc6XZiapnTdjk4QpjA3znVwiSLez55uAbhRCCDRa7dB2m+x3jFIwhcF1XFi2jce7B1PnUEIku5LnDUwW84T9CQGc1s7H9h2d1KRdHfJ9Xt3egO04eLZ/DAEBVVXAOR8zISiMghA6tm7hOi7OorIFxFZ1vpd0eabe80xPU2J9pYwbO1tQlPRTpEV/i4JQAlVVh87jCduI+D8IgAY9aI62JkJ+QGHHRnNOY2TqNc8mIvYTANVSATevbkkSlQXusbwtrg2m30MyouD5NpHw22CM4k98+R2oquqfKk8O7JWAJIJ5+41XkMkMp/LL+g52D4/RtywISDNKq9NFuys9CggFvvn1LyCjj/C2UoKN1Squbq1PuRFFasLztmMMLs9izlf+TMwZyeo46yxGKRTGYLtudCOnxYU233jhqqLA47NtjqMo5LLIZg2c1ZvY2VyHolA8fBoaAxCLcrEAyigazdZCuXQWAWMMjJIpur2fBGQzBq7vbOGkdo7aIvHPC4ISAl3XxzwxCrksXr99HR9++gCm5YAQGV1kO276hciUimfAfh9orhlDh22Pc3bOmkkkv2X8mR/+zj+43Is5wcgWp4/NNSBMnOBxDsvxnZ4XGsHlEK6rakj6hWWqCuNw3PQdV1VVbK5WwX1W9qd7R5HnGrqGN+/cHAv7GxwzdJ9s48UNrJ7n/UQKyddeuQ5VUfB073DErvZiwIVAf8A9ILdOr4cPP30I11/ZFkLAtCxw7iF1n02gfY99k64/zfb/3TctKSRHPgHX8/z0snKfoat4686tAZVg8trNP+u7PDbKiXpSJvOxyPSecxaS8jClQxbqoY8agWFosG1pfKaE+EZz4E985T28/8EnMC1LcmH6xxYBoxSqpsKybCiMgovofNGDx4p0JSH4xT/5Nfzjf/374JzPtG3ajoO9w1NJGjFR5uHxdJoF6mvo3ogt6ycRjEnGcxACx9dmCCEwdC00m2YcCPFzffsDs6IwnNTO4Xke+n0TAjIiBwJLz2E9OTVlgduS44L5+duD/c7IrCoQjBeLhM85dppMTue5fCSnOgGlDLsHx0NzwzIwQy29FBplGMqFPCrFAtiAjHcunXImRjuXqlC89+Zt5LJD24yuKfgTX34XmqaiXMxjY7XquxgBv/Ot7w4iTvK5LK5uJcvBElUHIWQq1G988XPQVBUbayvQNW1wLGn2uNEkbf/wn/wW+gl9zTyPo9XuJPZ5Y5Ria30FudzsHDKzIP0jk5+7TCiM4dWb1/DKtSt+Hhc59QzeQ1pc29lEPp8F8aNdqqUiOr0euJCuaJVSAeVSfimkJ6OY7BOVchE3r12BoipYXSmjVJCRQFsbq1AYi+1D80BO65MnuJsFXVPwxu0bE0nIBPo9E612B9OG2fTygUhq9Jni41LYKDPFqrj95T8DSukg5Wc2kwEgcyEL35i/7JqqjKFQyMGybXR7JqifVXA0nClgV/E8DkPXoCoKWp3uVOfSNRX5XDY1/97aSgWOK2O7hZBpTqXTMUc+l0Wn108dgbC2UoFpmmhHJAaLwrD94zEQagIoFfPom7NjZafK8P8rIAVVoZCFbUs3m7guqSgM5UIB3X5/aav/lFJs+OaJs/MmPM8DpRSrlRIA4GRi5TYKgQBfrZbQ7kj2HEVRwSc07qy/SNKPiOpaFoK+5HocuawB1/Vg2w4Yo7KPCxk2uehsIHjuYj6Lne0N3L33OH0ZCP++A4F+ETMWVVGxtbGKw+NTOK6Lj373H13unDmA1Oa2NtZwcHwK1/XGYmpXKiX0lugWM5hyQGoOlBB0un2fQGH6XNuWL6nbixY8MrA+fR6QlUoJ5802gm4iHZzl/Rqt9FEuQZn7h7PaarxbqqqKtWoZx2f1GUSyMiytmM/i4PgM5835bG6GrqNQyOHkrA7GKLKGAc/r+UIyWngILpDPZ+BxD31zOVEZnHs4PBk3LTBKsLWxCsu2cVKrR1w5ej5DqZiHZTtjLjGOM90nwuLFp7G4tjnal0a5DIJ9hqZha2MVT/YOFxLYwbWW7eDeo2epy6qUCpLfYIJMmRCytLDHMAjBsbO5ipOzGmaZvi+NoAzoxcKmVflcJpHGkvYFOa6LgxDb2/PEpw+epLxi+hlVRUExn4Np2+j2+vj0QbIRnQB49dY1PHq6DwLfX23WbYmQHJpBhNOc3xehBIV8FiendViWjf3Dk9jz8/ksViolPN09xJNnB+Nl+WGUcqBbjuYhhECn25OO1AmfkVECSjDI3RTMBAK+x3T+nxdvfCOU+ITVy7nXPIoCAGQMA7Y7Hf6Z9Hue1xTjeh7+7Q8+THTupbFR2o6LJ7sHoYshB0dn6PlB+WlsdZcXaW0r8XYWSokM5QraYOT0UjEvfS9DiiK+XyRjDLbt4PD4DBlDD/eXHKl6u93F3sHJQt9Xv29hdz8smiMcCqWRseiEUFTLRT+x1nLguh4eP90fI4iIg+d5qNWb6HT70FQV13c2B0z6N69tw/BtzbNeLQuUhaWZ5KMLMC0Lj3cPkM9lfPvw4jb/eXBaO8d5oz33NzxLLgQbgVy4m0ewXhqNUggBb0LtDvbbLyD3s1+riymKROyfE7btonbeCrUvXt3cwOPdfXR709NUzgUejqQ2JZRifaWCfccN1Q4CF4plDEpCiBgG7Gk0O100251QNw6PezhvtiSrzhLfmZfyOYPzTcvC8dn5YIp7fFpHz4xiRRpCYQyr1TKC2PeZWPBRhRBQNIqttRU8Ng9DBoWRG8TKlrQrCOOFPa9IK8lLUILH+VTE0CxcGo1yFG/duYXPvf4KctnMVJqESRBCUK2U8Pm378w8d4j5tbdUl4cUpSoK8tkssoaBOzevYXtjNX1q2cnqCElIQAjxc5r4K+UQeLQruQWjfMdc7oELGV9tOw4OTk5huw4+//ZreOPVGyB0SFDxIsE5jyUUNi17qUwyi8DjHG1/wU9AoNnuDBLohf2yGQPf/KkvwvM8NFptnDdbEBBQFIZMxgCl0kWtWMzjm1//IgxDhz5IS7xY57FtF/tHZ7MTnPn9uJjP4bVb11EuFgAh7Zy/8kvfjJ+FRBU2sRECbK2v4L23Xo0tIGPoMPR4RvIwyPbtoNlOb/u/lILy8bN9mJaNQi4bOZIxKqdbuYyBft/EvUfPfF+r5yAAF4CqKbi6vQHHcbF/dOIv5ERXaCCkiOSmnJVfPJ/PYLVaGrCqdHv96IgGQnDr2s7AwTxg6/Y8jk8ePMbT/aPhdP45QGHSj3UZTjOMUqxWy4MV5hcNRVGwtlLB1a0NrFbLY8e44Pj+j+76zuBDMlkhBN66cxOFXBbvvPEqCBH48d174MLDarWEXM6I7QtJnKolqUk/cbSVaUp7skwcJuv4u9/6Xvj1ab8nIck97j98FluAEByvv3JtxFyQDNz3F7XnoHy7HO5Bhap45Us/DxA/nnhAROtjsopE5pUulwrodHvohUwrF8WL1qAmkc3oeOP2LTx48gzNdje8f0xKmBmPwBjFzatXsH986vtazmMfIFMcipRSQIjUbDrvvPkqbNvBo6f7C5tbFMawulJBr28OtLukIISCUhLLwZgO0nl9tVqGqipod7ojbmTxw0IwqQ0UtlAkqeIco4/koiRLWyAbr86LyLMTj49/7zcudwhjMHsgILh1/QrefePOgMghtBMIwHFcnNUa6I8IybSja5qRdybmmY6HbiJ06/Us/Ojjz9Bq9xDpaDt5XeiwPays53l4vLsP05wUkpMPFA1GCd589SaubK7C5/9FuZjD1770uZQNCHz0yQM82T3Ez/3MV1Eu5lNfPwrX83B6Vk8tJAFgc62KL7z92lha48Ug7bGHJ2d4tn80keUxvkOIgRY1+m6Dw2HvOWHfmBFaCAD5bAbf/NoXx6oZCM+pqqdukbBvTsQeX8p3OicuhUZZXdsS22//DAgkI7IQsw28z62RlnabF9vOjBJkMhkZSrdEmraAOd5zvYHPm66p4JzPzEMSXk8KRVHgOM6ETx2QyRhglKHTDWdzXxYYo9BUFZbtzM1On80YUBSGTre/9GyMQdmtdncJpYXPHIL3Kv2Ih99itVJENpPBwdHJfOQoMYqkJNDOgIAsIWfU6C2Taa9xGuWlWPW2HWcwPlgRUy5GKcqBY2q3myhahRCCfC4Dz+PD6TmRBule3wwtgzKKjM+uMtPAfUGgVIaCKZTBXZpgIygVcmCMDdLRLmv12pyo37z+dIDUBMOcjCllKBXyUBVlKYJS9o0sCIBWZ1zguK43Vz6WUaytVAb5rQcmCf+eAhjY+KLqFn0M0DQN17Y38MEn9yPP0zUVxUJ+5uquwihKxQLOR1ihVFVBNmOg2+1DAFhfreC01oAQMnxwuDo+58wrUnARZDMZVEpFfBblXzzHjH0ZStWlEJST4UmKwmDoOkzTGnw0lFKsr1ahaSo+e/AEruuBQKad7PVNWNa0gKWUoFwowHFcmKYN4ts+d7Y38OjpXqjPpsoYquWin2fbGylLOmM/D/IHSiiK+Rxy2Swcx4lgj04n6FzPw/7RKSAum/U1GTzPw8HR6Vha3SjIZFezc8+UCnlQSqYE5TLwdO8AwAS5CCEo+SaFOEEpxLhmV8hn4TiuH/IINJotNJqtMRvi5DW6pmF7c21KUAbx56e1BgAZIbW5XkWr3YGmKj6VHUW1VPTZz4GdrXWc1X1BaVnAggQahEi79mQKEyEETs/qODmrw7fMhjROkhuE79ZUBeViAaf+s6TBpbBRTj68pqrIZnR/UUAed10PD5/sYm//WK5aCfkx5LJZqIo6UZjcuMdxcHSC47MaVJXh9s2rUBSG07N65MqX47o4b7bGfMoIAarlIm5ev7KwK08cAsdYj3N0un3UG03JXL0kB/vLZz5PB+onFpuFUrGAq1sbYzbGYKALwLnA4ckp9g6PB6xQy4Q0AY6/I845Do5PsTcjCmkU1M9kqI0QQwTvX1UYNlZXUCzkAAwTgQkh0On18eEnD6b6DGMMxbwcIF65sQOmUNx7+Awe5zAMHVlDR980sXtwDMt24DgOPrh7f6nmg0I+ixtXtwbJ90YxsMXKf4VudNROOgKCYUK/sEsZYyjkcyNEO8lxKTRKAGPC0rJs2LYzlg0uiLmu2Y3BeZxzHJ+c+VretOAQEPD86YTrejKWlHux0xHOOToT9hEh5HRSTqPGy78I9Sz4OBzHXZIdStrdSoU8PI/P5Ucm8eJ0UeprOUxhOK2dx364juOg0+v7kTySYKVcLOC0Vh+zfQczBk1VcWVzHftHJxcU3DD8qNOuIHMhcFZvhF7nepITIRjU11YqsCwLrXY30mzkuUEEnIdOpwvTtAbPHGi5njuedjbVLIrMtgk6joNWuzu38DUMDVsba3j4ZA+jfbKQzyGbzaBWb4SucZimhWd7h35erAiNNQKXR1COPHBS26AQIrFXfzB1S1ruJNqdbux0aZngQkgbqv8BXN3egOdxHJ2cxTpdx0EIMcJ/+IIE3gK3FZyjZ5pglEry4Jiyut0+uiPMSYJzOK4jbXAh13HOBw7hoeVOfE9v3bmFx7sHKRzc539wIQQcZ8iNOgrP42N+uJxzqVzEBMrwkUi3g+OzsWOBXZZRitduX8fjZ4fpiUfEbJtgr2+i1zenZgfE9xXmHo8laPY4R73RwORDmpYNgWi3tGl5IUAJxc6VDViWjbsxdb5EgvL54UWt9KuKgkIhi/p5POMO9yM7AtTOmwNiYKQUdIrCIIQcKIIy87ksXDckTDGi2Gq5iFanl4qUmI0khFoGhECsv2zG0CGAqYUl+PviVrBd18PB0Um0tjfRLnsHJ4MEWUsZcxIrNvE3qzeaE2aZ+cwJhBCcnNZHyHLjQQmRngojZMCEyKkuD9LYhkAeH01tQvAnv/ZF/OF33o8VlJZlwwpZMLQdJ3RGoCqKT4A9/TyESCXIm7F4d2kF5WVwW4pHsvoJIbtrt12HoWko6EC3eZbwaulL2m34f5MECzFkdOJDkM3oqJbL2D+sB4eRUQRc24E5YUoYLSMAJQAtGXDNDvoxq9mTSky5WIDn2eh3e376iAUD3MnEHyPFMUKgMw4uCPq99vQlUZizi1m9tvTSEBikxoguKu4mMyoQKu9mTWxFyF/RIL7+RzD+Dlvnp8DIsUxBEo6EmXKLhTyu72zjk/uPYfu0coVcHtVqCWf1BjqdaVeffC6L1165jo8/ezgYsIUQ+J1vfTcxEcnUs4Qs9BFC8DNf+wL++IcfTZnUAKmdnjda+Orn38Z34sq+DAIpU6iIm1/400spi1KCn/vpr+Jb3/3RgrG/87o+jP5TwOl30Dg9hOM6fuzv6LnDL2Gq/00KhikEAkNGMtFgn2/QZ4yBMQWMEQB0eA4h42X6PZ9Mljv2z8lzMPHFCF9rlTmcXc8D9/jQJXjsWaefIfQJIx9b1p/4f0t6Pn+1lrGBMZ+MtN9sET314iLPkY8i03N4ngPXlc88ZhKZ1PrFVAkRxU8eI8H/BgMgpRSUMjBGh9wGZNiSse0twmoQZm4YX4kW/mhfWNmCohlTvZUxCl3XxoiIKZW+qJqmwnXdKZJiSuQ1lm1P+WNmDB1/7d/7d/Dr/93/N3EEbXCvUU+ZAPlc1hfGHIZuwHFcvPfWHdQbTTx8ug/OObKZDH7wW//d5fajXCY4F/j977zvv8qUwm5JY4bsXBxmr412/QSuY2O0L0wJpSRCMUSSDoXFcJ8UkspAUBKCAd39UCxNC8rB3UcE4+T9yeS+4RP7xBoAF/IdCAJAkJgykz7n5Clk5An8vwgB8QeH0fYIE8NTQnPsnYvpk8YO+0z7RID5uhgVAlwAhHPfxinGByMBgIiI+oxIUAJE8fgL/10LwF+c9Pz0sgKEUnl06h7jZQ2PivHHG9w77Hnhs9lLYdk6O4KezSNbXPH3y5LG/JR9cI/D5BZeub6Dk1pdKi0jN+GCD1OUTLQ35xz/7//+N30hGfVRjl+kMMlS3+2bOD2rjy1Aua6HzbUqDk/OkM9lwBjDjz7+zE+XIs+bRab8Ey4owxvRivLzeg7KsxACdr+LduMUltn3R1ESjPUzhOL0tDLYMa3NTQg/goGQVBQpKAnIyDc7qolN3oQMi52swEyNUkAI2eGEkIJyeNOEWmQalXJi7kcplcKCUMAXGnJcICHPOQLh618j8mzsYMif8tzABicQUN4GkYDjmuSokCJT+wYFhgpSEXL68Dk4B0A8gBAo8AfC4F37glqMXTMqHGX5YuLfU600JkClsKRCwO514LkOsoUKFE2X7R4FAXz82cPo46P3GsGU/TH0FY5f1DNNNNsdFPI5aJoG27YHwlJTpZiTC6K12fUJwSUSlEuQYksXhCksTwLwXBv2/6+9b421JbnO+lY/9tn3nvuc8TAZj03sAUexf1ixicCAZQFRCLaQBqQo8h9wUCRLkEhEAglDJBT+BSQiBQklMkokB0XYIQmKJYiw4zghMcR2Yo8fw2Q84/eM78x47tx7z2uf3d1Vix/1fnX3PnPnnj1oL+mc3V21qmpVdfXXX73XKxwf3EZnQVJLdp7eCFP0VPIMLwRKC5J60rD52pPPbmJwzaThsDIC0iKj9NgkdLmQmbM49UBmMMhInzy6SCYtqkBVjYqqaG7eSOQRrQqgzO8fyURBzIpVVgQGqQ8EFIti0qwygKa4szHHJP2Ec8AalpWUDCIBodm02UzG7J6RPCvdfLbAl3wB/K6FmKE6N6oBOXQ4uvUClpeuYnHhEqqqTspp/LEq34rI9vWOyoxqBAAvvnQLtw8Ocd+1KzhZqeNdAHUcr1oSORbRuMVbBJQZeQUZ4N1cn8LM6NenOL5zE6cnR4UpPDm2OBMUA2cFkuadMCDZNA2aptUgGeqFcXguMRh6ceZsyoGo2XPRMCvrH2BFHD7K4iZoSRSkT4ZRVhWorjMMZyrukL0ZlplvirpWgSJvjKZRq2/UMxfeCK5r+pWBMPQNAJIBpkw4L6hphsP001YVEnDV4dkCqG9Dhlna5+b5c5g0kTrwa3V4C2Lo0O5dRLu8GE73iUZWYsZekVqSXNVVdl7zRrsLeZGLQfWPCyFs+snO+IGZZvXUOB5sD1C+TNx6xRfmFaLvuxXWqxN0q2OsT9W6XvuQw84yJC9tiWVGYUIwpcC1qis0TauA0jBJW9F9VhnGEbyccVoJWMYc1IWRwjW5VatPr3qgwseiWP9TjzKeepYYRlnXoKoGlTZvLlYPztyWewt9R9VHSJDmY2Gb4BH8JR9OzvxEzX2Km8P57gDJDBISgiSoIgWWlJgagbEBQNWkzuTSs90EZliYVa171HWNfr3C0K0x9B32Ll5CXbfqpMyrlzEIgSuX9nFweJQclGc+rqXNtjd9n02NZj1Bf1R01Bcu7KGiCqvV6eT85O0BygmJC+7+69ew3FvgxvMvbrzvYSGBjaU7PcHq6Ba69SnAjMZfSpmww9y51aRaNGGgbHifAYYsUe2daCaUw9MLASVijwEep4yzFC4GUtbpKTabqU5z2ePMJYRxWREA6KWuarJ1YT7cRkA58qJ6QGpuGQSqGrSLGo1fFzm5iKa9xYDpg2F6zZkwANuBFSEYLP1VNR5yw7/MADfn4k/1OJsngf7gFvr1KfYu7OPS1euK1Q1C7QIlY5tVObx0+06WZD/8PQ/gpdsHG20GMwWsOYa6f+EChkFgRWQPhCvJVgHlJl+Rddep/Q8rAuZu9DIRvVnz60bM8gH69SnWJ0dgOQAssViobelTNhaOSMfsLRuGooeajEqrezUNR0IKga4XnkJwkY/D2jHCaF0O/DaXFyXZLdEMo80BbmhSnJ4fX9mGohDUqhsxQOjJxlJw+MJv8gXMgkPgmdhIGqgqPYjW1E243jgIZ4AmAssYKCMdB65jumrKkpquNERTZDLhinkOqWiqn2fHxMAwHKLTDLPvroP0lnhpOPKDJter1Rpt0+AU601XGhYlhy23Dw/BekL8njn8rSBbA5SbUu3jkxVOVqfFk/ni2OfIom3x1rd8H5546mvZyakAMHSnWJ8cQAq1e1FVuSL0gWwa+ChqiTrwooyuD7hCDpBSL1mTpi+Gsl9nALaPKbj021CRThI09jPMtq5VU6uN2aSftxi0nVuIjXPfhqhBSgKCGQMJMEi3LjZtHqSMp+CgnRUwudFlACwB1G6aUsA5M2zOskQfmArzTs38Ww8wUz0GVaYFQJBQ67pjYIvTS/MVaRd1QzdzJYcew+FtrLtTXLp6HXvLfaQy/nxuHxxqc+LnsgFizlD1J7afduPsdWuAcn7d1hWMfXCdF1ht37bA6rTLLmcahMDjf/ZUsrSPmSGFwOroNkTfqSkpfl/hbEaYYXK+fgEY/TBSDhgGCSH1vo3sx5dnbGmzI2h32zQC9hjHRwDITOYm1HWtBhDaBnXdRn2jMXPNp50F0iTpvL9xlcQgqQdxqAJVEsFHoygZhmOCFZcAavdgsIpgAEUt12NwTR6r9NIh7548ELDXnh97QGvTm9IDmnah9HsCgTCIQc/Z0sxP58mtuYnLIbxXcyhzfmVhllifriD6Dpeu3Y8L+5d1XPOArrxZxmatg6ZpcGG5p8+BGm92Tm3QMQmURPR6AL8K4EFt6QeZ+ReI6D4AHwHwBgDfAPBjzHyLVGn8AoD3ADgB8OPM/LmpdM7MAsZtR9PUEEKtJyUQ3vWOv4RP/OGng4IzXy4hBE6iPfIA1bw7PT6AHHpUlT9gotPJAGC+TzDt48vpkffi+nEKoUFSCPS2P7IKgucZqRdPjtFl+jSzembVCwh106BuWjRNi8qbjmQCZ+dkTqYfS8Y9wl4WAJEEUYWKKnDl9TXxpu0U9ti2c8tqmjXNAEAVGHqyvU5TDTI5ECU/Ls8usgzP6OWa3NADQo5J0khcbUtuQjgRxDCETJIY5AOs+R8Ar3Zl8yzDZgUH93lGPgiBg1svYn1yjCv3PYAq14ftSQlIm6ZGRYS+HzZ6nlcuXcSb3/QIPvP5x4G6QlPXapPw3MDNRMRzGOUA4J8x8+eI6DKAPyWijwP4cQCfYOafI6IPAPgAgH8B4N0A3qT//gqAX9S/91zqusLrX/sgTtfq5Liu7/Hff/cPi/oJQDJjdXgb/XqFuqm9fewyoDMFFP5ATDSNwrGvvI5iK6pPchACgxT2BQ1kYuBltk6GWZqXDoBlk2piu+mTy+QzSc/5jANpxj3rokGyqkAVqT+O8noWZpl1G9HVmZF6mpCUDD0T3PuIeiBEDhwdhLIagc4xSXvtt6DIhEI85QdUQ40rumemiEHcDI9yE2JhVk85cApqOiwHIQhSMk5PT9A//wyuXH8A7d5SzbvMSA7A6rrCa+67jr1Fi2duPL/RNnUv3T7Apz77GC4s99C2e3jkex/Gl554CkJsSspmACUz3wBwQ18fEtETAB4G8CiAv6HVPgTg96GA8lEAv8oq139MRNeI6CEdzysufmGLQeDZG9/deN87KQWWiwVOju5A9Gu0bYs8I6PgxS41l5MmbQCOViNgdrbrkAFm1cy2nfSmKXRXgDEFxTCYvqi8ZrcZwGlbzbCrBODmr/KJ04/C5LLhu7NmblRDrVYJXwKaRMoQCDnjltMNwMkwR6ipQkKDZdMoP3I00AM+ZJrQzt1NMvPQywCsntTOIA9cDUTppY3UoGnJgnWPTg/wsI3S5oZdlUqy7eG8/Sno2XLxPlasszkIgds3X8ByucT+1ftRN+2sprgQEjdfup3dyR2ICUpe/u4Pvwu/83ufwlNf+2aABZWed6resfF6slEfJRG9AcDbAHwawIMe+D0H1TQHFIh+2wv2jHY7M1CWt/On0Qyac3w3ESkE1qsjHN06RVWRGqTwwUwlbC68CheD47ReGJ/RcYCs5icKe4bLYL6mpb7MnH32NmZuG+hQZS/rukFTt6jbFlXV6uWDKRhSLh7fTgpKKgmfk5wraYCs9DJGovCjaCZalyXqk0vcKdFTY2eRu2b3aq27hGDWpNLMa2QbnZprGzNG/WuB0LiTDRsAY6AD2wT3gbOqG7S2CQ6g79XxCwq+vA+uN2fTOnsA7WN1SS/7PWHvu6Xe1dVqhb6/gf2r92GxWKIOTifIy9hmynM29fno//wDfdhduCvRtSuXwKw2Ar45MfdyNlAS0SUAvwngp5n5wH85mJkpN3N1PL73A3g/ADR7FzbeVq2qKtx37YrdeTyWWvdJDELYc6fHDoySQkAMPYZ+DTl0bv9HxKAHBIzQ85ujl4AjvK9iBBJsQFIICClsp3oCalMj7QF9iNJM/N19MMhE6qV304FauwmFtiqTRmRnVD7BVYKCBbCMnZntQI79m4yl1LR2nDIN6cKEYBojiO6nlGqzjNo8L7+JEFxrZuiBHjNrkuoYp+u/JGunr6NicVSR9QYeVJFiliCo/aM6SCnU2vykXBTAknednZ6TY6S5gi405YdB4PD2TSyXF7Hcv4x2as34y5QSWRJCoq4qXL9yWc3pHJFZQElELRRI/hoz/5Z2ft40qYnoIQDmIJBnAbzeC/467RYIM38QwAcBYHnp2sadBnWltmkq7XC+aFtcu3oZtw8OcXl/H13X487hkd0txBcpJbr1Cfr1Sn8gVSUxTzr36oRuc/XMVzh+MYwO2RCS1aCNkOrVrapat34nAGlscCj295lpEKXnbhgiqZHctm1QNw2IKr0jEuvwhg25PKjdgzhfINY8rR/UALJlk0hUUxTuuGWcFJ+lkq1ZqSP7V4l3nGjcF+c8COo8F7XTNqNiCUYVxOsGVnQspXmS3vSYYIYHWw1Px5s/msStpiwZAinEYLfBS/Mc2xRn2t8qMK8TTpzPlxMArNcrSCmwf+kK9i5chPS6EJzkP3V3Q24fHKKuKhyvVpNHK88Z9SYAvwzgCWb+ec/rowDeB+Dn9O9ve+4/RUQfhhrEufNK9E8OQuC551/UfQ5pxe/6HodHR+r8lONj1b8R9VUyM7r1CdarE/R9p/yzfR5jLM7CVQgyBcAJ/Sh00oBUVzWqukJVN1g2VRpHNm5nUzwKn6zBjXRyA09pntShZ2qb/h7rroODC78sCqAclVZ4WVjZO4NlVpUqr2axwN5y6XYeT6QEdhkdH2iKyvFmDiE4MEtIKbBedxiksHWPLZj48ebAxwO8GJgjUAtB0jcqmuCuB2GatkW1qILlgzFY5ya/pzqRnYEtGT9XAoHT0J1CigF1u8z0XRY+mFk5A6gS8ObvewR/VjoeV8scRvnXAfwDAF8iose027+CAshfJ6KfAPBNAD+m/f4H1NSgp6GmB/2jTW3PS1hTmRnZwSvtJgaBoyO1x9zQp18LZsZ6dYzjozsY+j5PPABkQRJImNuZdBJ/taU+qELdNGibRvfBeVBigC7pp/SBNxy0cQBbBu4k/ghsJUvIrsMgBwxDHx44lRSez3RDt/B2AkiNy0j9b5oG1FZoqcZisUBdNTmDEhkDwBCP8uwoPV8nBA+WAv3Q23PKgyk6Y2CGCf9ZLK4AaNDHNqDRa+MrNHrKTpHlJhPj2Yuao/Iq6YQ2xZPYmdXkeCGOQVRhuX8lBMuxxxnUjdmKVqRkfOvZGy9/CSMz/1ExFSDZllyPdv/kVLy+VFWFN7/pDXjyq98sj1BP1/3Z0q2OcXJ8gF7vTl2WjfYwmRZGsSSpUiDZNgs0TatA0m7Kmge4GADTye/haHqRaRbDu6ax2ZFFCJFduzsu8wDxbKLzr/t8w23lgAhCotQj5qLdwrFvgp9Rs12GqxkcuetYqhp1zahribqWaiK6FC7JBPDCdEbzW8zRWJz6OXp7Apj60zS1mlZFOgd2+o/XY6vrrvG3VYVJbS0X6MDWdfKufUtU3MqRdEBmhpQCxwcvYXnxCpp2eqBndh0sDKEQEf7c/fdNHiG8NStzXrp1Rz2cGRk/005BOsj69BhHB7f16F9Z0tUyyjV8v+NpPcbZAZv9n+0bVL5um7SFAstFq5qQZwQ439+lPcUenU0+WwWpjWKFkKrpbZfThUBhdFMAyuXZ5d0LmrrPESJnP1GUDsPLbSr2w6Vf1JHU2X/xc9FEH0GGmjHRtqoJLoUAWLqd7slFStoGtlOEKEg0us0aZablsJ8BPXBkgUnHLyVj0NuzGVFr9aEmthuQZC9tY7Y3GBWXSQyqNk33mF0eTIvay5t5fMzA+uQOurrFhf0rLrCx1SxwYLjTM22xEuq60nOPx1o9SioiHBweQYrxc3q2AihZSn1Wc5qbu7F9GjOj79ZYHd5B33cQauJUpEXhfwrdg6cKH5w8vSgMRYCZxKGnbqhdyRdo2gaLtgVVaq2w3yc51vS2/jmw1CAS5ysb3o9fu5uvvBBqBJ7t22H2Lizs/xiXTXKbA7ACpI3gprJTM2+QHj3NQ17aQ2Caiv6zV29s2BymcBTagqoHxB4rMm5qzmmLepCoG3WuDkmpy9WkaNg5OWYFOOSAs9FMLHFmhOHNNJ/EPwivbGT2Z4GYeuhmMag8GHu8wyjITclzAEmahXr3pmzIa/773wCioDzTtAgsBhwf3sLy4iVUdWPr7I/8zb+GLzz+FVy7ehlPPPX1oFVYVRUeevABHB2d4M7BIeKx/aTWMXD96hXUhUnwRrYDKLHhIeszI1V7BDKGbo1jDZK2EgIIii0CsQBUcsAZ6fhf/kAnHlkOmFuFuqnRtn5zu/FGmr24MgMs8eBOGD8hYZZZe8m6x+FNf9Ig1Wqg8FgLL66ELVLknEG6pLxHdCPnEIPDPJYmIKe9Hoxojzvnw5kseREoBpkq5cLVdY22bSBko1ox7C1pJBfOp6Rx3MGrnkkjZppBSCZN/l371zFUCTVpxJVDHS1HjdlynMcwadJTm8LCios5KT9mu0mxX6WY1QbB65NDNO0S7XIJgPCxP/g/qKsadVUlXXWSJU5WK5x26wQkVdqhm5AST339W5PHKm8FUM6W2eRSFYfoe6yOD9QCfWYvPEXvaQn8fJAs6ySgBkS1OfYntclqrUCybcwOPLUN6zO8GCCV8xhAemnRmH8eII2/YZNyUE1HV3wjAGeLJgeO4+Eil1QnJ+T1S1JYRkkMNgNpO9r3CgDedDUU7MuF84GLUKk18WZbPGY3+yLXtC42vU0zNkKt2N+ELzbLHWCqKBhCDO4jo5kl2bDjeSR9YxiiAVmfFZuy9xcfgXRfcKCj+y29bKhoKwz9KaQUaBdLdAAIPb75TDqZhplx86U7oeNI9ZEsZy1K2V6g3AAUgzvW66L7DuvVMU5PV65zGTE2TgBkopcByFgnC5Chv2lut22r2WSj17/OYYkurrGBnlz/5WyGqoUBDFLqCe+5TpAYHMPwaXmUwo3U5GzQKDUPLOPRUk7Cptwy9nLA4L3EpmkYterdfNhw929f1PzTVh/jq7Y+C446t4Dnp5EDTAOGJX9zm0e0PGDGfZbqr65NdwZcC9+UTS56jxX6/Y/23dMYz16ebfeOr6P92HYbqGdaEYHlgPXqCO3eEnWzAHkrxkhvYJ1dC57Dkg2qHLBNQDkJjNPIab6Oq6NDPZlVepO5PYlBMga3M+n4aFGOQ4Fkq0GytSCp+itNPBMg5unZ9DYBwQJL9f3ZTHjXMwM46dedALtCE/hMIBmXYy6+RI/DR2JBM47HyxPZYBpMCDEyuj43T1m/7QRO+t4Adzpm2zQQQ6N2siJkJnWrOJWftcKzw++nzPnH+ePUL9ePyWrzWtX8NOEbDyzJAislIKZZoAkZ+wMAOTZMGpyLOtpm8tkl1G5HRDUYjP50BdkKtVlyuwAR4crlfdx//Sqe+c7zhSWP0XOfTcRMaWyNbGh5JEPfo+9W6NYd1uuZLLIIfnP1NgVJNW+tbVVTu2katUWZD2Tk9G18JZAb88+FN6AZ+/v3ugKrj47AMAyZKVtnALsI0GaFGwXINO6woz4GDPvOJd37gZ4BK3hgGQTSoGHb414l0yuR2AtjQE/NbGhRN+rDI1moQJlBG5UsF8FyHEz96BzwqOy58KHtBiw56qszxx57eQ1AzAEd23KOQC5hw3kgDAazPJYK8zTIlKfqGmAxYBgGsBRoFnto6hqXL+2jbZsCUMYfktxHqixbBJRnl6HvcHJ0gG69VkenZgEyuEius+AGZAAlEw9N+5t10qY/0py9HfZHmnhmgKRpWk+BIMVxx/5p/AyymwgIKRGufaLwu5CUV8YtAdacTinsiOhOrOkQ3stBUQoc5821Md3Ytv/SK73shrY+WHpAooBNLSNUrFJ9fCT80fVSGr7tc8FSpa2iy/vn41Z9dvHARl3XqPy0iNwewtpmtUGNi0vhX8w+kdcJygruQ2TquLbZDkqxqdcMoVfU3bx5E50+DqQkVVVhuacYaNf36PvBunX9/0+DOZGIYcDp6hjd+hT90KunwNGLMBv8Ir0Nm+Nj/lVV24Ebd6ys2XUnAsEpJumnOQsEnVsp/he+/uWoGaj3vewHCKmmtZiy3QTHNlTeXEgtX6xr91dVFS5euR/3PfwXJ+xx4GbhIu5ZMCBntKNmbNgM13HGYAkDIqzrQKvOtZESUnAIHoU0AkAOwBITYOr7F5rxSTeCBst4XmGtNs+15RQ3xf2BHB8MI3ZJXl/uWDMbEagCSPVAACqwlFidHKHr1mgWe8X9Li/vX8Tb3vpmfPvZ5/DcCy8qoCS1mfcffebz2TBGthYozUtt5nzFIsSA9eoYQ9eBSG2CkfCVBCR9OjHFEH2dnL/HZLxmqwdJTk83u0n3/fV9h2HIhCH3mxtMokgna0MGYIPceH63v/MURHeKoU9H/RQ2MnwY2EaRACQIg8ewJVXYOzlWCpm6UxbT/5e6xR7BVU6fgdyyQmZGVdfY21tiwdIUdCG+eHkje9GHevk4/PyYgByFD220TWhosGe2B5a57LF36WzLpZO6++GjPCTlFpVFFD5+Hn3foerW2FvuZ1f1nK7XuPH8d8HMOFmdWvdvf+e5l38UxL2SGAzbtsW7f+id+NRnH8OL+oB0A5rd6QlYDqipQr23HAEN7Z5jXoFeDE45BoaI1cUAZ9KyIcFQrHfQfX2DWaFho/PSL9p+N/w9Hd9//wGs7jypl9a5vFl29gpufXW3hPVAhJRq5RCzBE5OcOvWrVhzk0hzju5/DDDFsF4YqOddEaFtWyyXS+y1C3UOuY7Papv7DMD44JTuOhTtKOTHMeWv3WN/009t/0SvD/RLwcq3Jc5L0X+k3BK9XFy+ExGANbrTU+xfvop2b6mdVZ1frzt89evftvUFAAYx4Ctf++boFozAFgFlLFIKfOz3/zfWnepzMKOu3XoFSKmX+FUJ6MXNS3ebAcsEKH09E7YEpvl4dFePGwwxFU0ISBk+DDeLg126/tQOp3lG/3AZWuxPRLj02u/XXX2VXUq5XF7EcrnnThMMPjDhhyHIfgDYfpg4XC5sJo6SmOfJqh+1W3fo+x7doH7ZO0xrvsTM0LnHpCerWGCuvjPruMxsAm5bkH4c9kiHwiR4k6q/xHC8rDLPPmMbeX2yuehUHfCAFWqXcc6soiuCYKpS9j+Dnr3UhT2IAQd3XsJyucTFy9cBwDb5ByHw2u95AMzAd557AcxA15X7NY1sJVDWtQLB0/UaZiVDtzqGGHrUjdr1BEDULMUE8Kl/IbBO6E00Z30df6KtGzE2IDk4JpnrQ8wywil/X8qMMvWPQpoBkYrUbjJti7b153WWGHWa/pi9ecBM8xIT5JIeADtQEDPzMXCIYph4Dzl7marNAWUHIlJ/PPt+wN6CoParTEXnToW12EgIB2R8ZdM3inF/26cI179pR7DDARgQ9FHEXqEyMPCg2Lsp7NIgkv2o6YDkPggBECbxINWL47J1iZ23iYIlVqtT9N3zuHT1OppmYRd5DINA13W2SOfI1rStKm/D1QvLJe67fgVNXUMKgW59ApYSdd2AMAWS6nIU/Lym+OYg6auGLygzwDJqrpRA0lnrCmEjENwEEMaFtEW13nbLLGMrxl+qXQVGFnr54JQyP/WhMZU91osSYI6c5gIWZ8KO2L1xGrFekHvLbAYxuIGyGTL7cXsfolzd9RVz31XK/K+qGk1Tq8FIfbBcfrloGjpNO0NQrBdl9SiOKzdu4Nmv6pHEMAgc3LqJ46PbEMOA5V6Ly5f2VbkDyNarjGwFoyQi/PnXvRbfeuY7arfxrgOBsT49wXq1UuehVD4bVP/c/QiLKvU3gsIHlQXJKM4RHTP4IaQBSVEEycDMgkyDYBTflNqIv9mcw7HJTb+f6uvuWm9+V0DOhriJ6dOeMFafSIS6qQVGragxuzVeavTNb3KXo2bHKvsefd1gsVggewjaDMrjWKHHOkcD+qzTT8fEQS5P2t00z6uqQRNt9zkM5vhXP03NLNljxLnMTeo4PZU9Kj4Zm779IQ2Ban7o6uQEQ9dDiB4nJ6cgqqJSGC/orQBKgHHl0r7tmD85OcbRoQCLQa9aAbJgBaTgZ3RHGGf+S5sf4Q7xrTAKjgmQjABgukk9EwRjyaDruIuas0kEVPoI2rouMQVfYqALvRJ33a+WxcwkcGpnAJiB1xQr4DTamcLli5xW1r3oywwJ1Zc2DL06t9rW801kBpJatQhMPVBNgNO6efGTcrVg6ZXtIAYwZGRKBgh9EC7pAEU9Z/5IfAFYejWDGf3Q4+jObXTrNZbLi6jbNmSqI7IVQMnM+OLjTyqq3HeQfQcAcGs5Z4AkeQxxFkjaWxfnBOgGDNS7Nksng4Ebf0Pg7LPY+K2YlsxAT+LEHkUj2IGcVu+o7tikGQzwWIKdQKz5I7t4wqZqFM6kC/2+muqe0Br/coTPsSl3Y6eXN/93UgppzGlqzxoZz4S35a8Gdfq+R1M3WOxVNl82ntSYSftzkofTyDUHpp6nAVP7UfXBldTsDgmZlovFuFcSLP2yiPPg4mRWmxbL1QlE32NveQGL5cVZLaitAEow0K1PwXIAm4mrNNI0BlJQy0SbZVMUX2/K2iLT9ZroflDnbosYJIsyza/myVg8PnWgxNm41LUCyqapHUbpC9PZ70ZHVSV063sZwRsTh4upoFepOUTwlChaKYGomWPr2TACYMXopx08r7kAmYnHAjkszAzDgF4PUqpjGQogmUzdQV4vk+6c78ZMbqp0idQ6az8wAAyDXpo5FesYdPt+sV4MlqXoHFjmmvZSSnR9p8YS+h4X9tV+l2OyFUDJzJCig1nmlAdJ/32LWJ3/IurfvG4eUrJMkVKNWKQUurkt9DEJ5miJu8UW8wBnbrLTP7LThzJxaalIHUFhTlZUWgzXtNYu7IOlcQ/BLmSZIcMEUGSZvoX5j1cBRD3ggZ62At3Y2gT7JiGiiDQbAqTvr0dzpT5bp+nV/orBUsAkPIdx5M0ZtbXEdDftajWH4MFvhgPAwOnkbYdv9j4BtkChpDfiXmqG26swkGKXA+SpwDB0WOxdKGUVwLYApX4ZzR6LKXB594VRsRRN87qhUsxUirfJvZRuCpCQbpedu4aRWSmxx3FW6QDVUUnLJptGj2TWXggP9MhVsKQprhztbcgy/fJ24a1/8ix81fTtyIGoY1gcYEn+pd8QCZB5caO4JgHS2pfTYc0SCWIY0Ne93uk+Gi3x8S0CuZRNclZvvkzoE2Cb5/DAsmbzfVL/go1UNkC7LF5mWKWllJHf3Oa6J5IZ/TBgEEejWd8KoCRoDhgTOeNZAklk9Atu83VzgOruzeogIQYMg8Qg1aDNIMRdxMgMuFmvkDEWJ60Dnl6mCU1qk4BGD+LYA+hNX6IOT7biqUrHFrE4SsID1FDFe1IcEdLSS+QkwMb4xWAL6XDAA5TjHAO/kkQxTgFjNo0csLrnJc2c26ZHU+uzkrSyU/fZUJR+rOuneRYwLZRRnsiRnmcZRTEMyfHQSaWIWWUu4ayzD3sFIN1ASsukfdkKoAQQgaTPDkcAMQa0MVDN6iYRpWn5GAUfJNXxCEI3u4tGzpUxUjgRKAuWMxKoaz0vrqk9HTi9ZPDGlKPTc5g8xTTZC2qewXSFNif1hfZ5/v6cSIMAm7YjM6kGV5PgiJE0OePtWCWzWq8uhh5DX6GvXF9xDiQDRuq5jQPquI1zS6uoR4SqrlHbh9CAGRDwwHIE8FKnTcEvBmBff5pVjmdOydYAZQCHOYD05+lQ4mvDUKBb0B/RzU3dMUUbrrjR/ZJySPTPJhmQs05nQtGEfZp4CIRKb/tWNy3MuoMiO7VTewxbigETmWa1Ce/02GO4UXCkz9K3uZS9GCw2PXdpBDjmgOMoQy2xyHwcQkr0QqARvV6ZFn1wtG65/9J3z7mN5KTAgDcRtWChsYhiPmBijK0VWGUCaCXwtKPlUWQlsIT5yczHnHi9tgYoc0wvBUlEzNNXLOf05UEYeSA5eCCptiALZC6elQAwADa2bGq8aT3BKmOw1Ndmb8Sm9k8tNFGn04JMzCbNZGpPMnjD0WNx8YRxBS7B5XhR2s8X2OxByiaNGcxpFAsmgGKjwZ1pBqqWFErFKuta9RsHW4X5rDQFSfs/8ZoG1HwzfgxQy2Wj+iwboA61hNDLHZXWaBzlyMvBymQ1AktHK8fZZUa2CCh9mYM2qc4cl42F1VYG/tptxSTTKUAOoqYBMD4HuqQ7r2ltXiTN6orNYAW8VU12ORoFOo4pWsBMGGIMmurOAaerie69mgbOnARRJvnmCBMZ0IyyHO0ZXtDRCCPgn4PEubj0oI4U0o2ALwjm6N3pPkkOymEeoJZYZx5QSxL7EJmNNBDYJATD9SEHIaJYNgTSMeArMNazyHYBZQ4DppYr3AUs9MWHItaVTup1uY5JTjXxIqNyeSjZXV58OxJohi4BZv12XdX6rB49HUj3Reaa1nmGmDYL/YnkgGvi+G3sGDgDS8eyljS7jLsEw+3pOMV4zizF92vqxcwFKXwgNIiIYcBQD6ibBnXlaqL7KbPILLCdgXWWdTNJZR3N0Rd65yEdWPVZmjfMfLyRPl97G+V/cnJ5ELh87zXD57LKrQBKZrWrssqzz0iiSkVmN2TAvoQe87Esx2NQBD8K8sqWvGdBXvz2VYdaJ6omlEuz1yEBlXdCnY3X+zEXwaoiX9eqU+Cc1S3FkwxAUaSS6hPpCty2ajoQ62YrLGWF/V8A3HyePLcgS/7N2VoJY8FZVRzrX1UEUI2qiir+DACd7Zso5kAvdE81cgCnSouIIFlCDAOo8Wt2ASwzTeoUANW/nP4oW00mt0dpsdVM88xspw9xw/ZDrd4jB+gu+QwSR7blbArtjm1I3UpxT8mWAKXE6nQF7zUtv6gjgJC+xBndLEilYSpSAx5UVWrgo62BxV6SbrrE0os9sifZqDe2Jc4zRZZm2aYPcPk8+wNUzOqYh9Xpqe5wN9V1CmgjW0uSs2ckvonIRlVJv3yGvezp81BmRq4ky8hK3rmXas6LiQBI/P/pC6tsl1JgtVrp89QlZKyXAxFzVdRVNyk4pgCoNaM8jKcVRuvSNlPSzI5DFbnt+6btLncxZG3KfBTDndRdPPnnkJctAUrGYA73KQBkjlIUAaYQJtWPbxRY1ETgqgE1ldpFXW9aUOmllXHY3Hp0dxnqJ2AeA0sEprlReBcmZX5j+uoYCom+7zH0ateXUidC+kHKxJs4jzDCbNlvKmHgisieQ2POJaqr2s0JBfCymOSZpvzETmPgou5j4FLHL0hIKbDuOw2W7AUfY0gZexLAGQkzAZJJ/DP160pvqFHX6r2q62B39zCoY5zJByEA4RllaxVS+8rhUtkKoAQwUik9/3jDB/07b2DEY05pVDoiQO30rfbeq5sada12/rZsMEgrk3DslBmcCZSC6Te+vnIZm7KjHrABNFORDGB6DEUPSJnlcmLw+lnjcrdf+rjy+HnIlF+pH1Erh020uZIkEtxJzUykZJh+yqCZmpVN7QgBh3PugVNOowxk6YRwJbWug/1AkMIBqH1e2XAceeVsLISJmdpd1heSwDzAjMAxgBpIjxuZwgGCq2u5/t6RapiNylxNzOndHqCcIyVAnK/gdDKqpk+lbtR2Y01ttr+iNPwMFwd+SDEuCBXrzwPLYACGQiBLN7MAxKA2AXBHUvig4hkI5JAwvGPPn4z1vpCvHHnNpZVzajwDkB7xmALK+WmMMsYMUHCsUADHMO4C8OkVL3VdQwwDIP1R/pzN8QdgHoCNgv8G/YGTaTBDkv44D15Nrlhv2n32psYG2Hj2NKaW7twLIaLvAjgG8OJ527KhvAY7m++VvBrt3tl8b+Ru2fy9zPxAzmMrgBIAiOhPmPkHz9uOTWRn872TV6PdO5vvjdwLm7fmzJyd7GQnO9lW2QHlTnayk51MyDYB5QfP24AzyM7meyevRrt3Nt8becVt3po+yp3sZCc72VbZJka5k53sZCdbKecOlET0d4joSSJ6mog+cN72lISIvkFEXyKix4joT7TbfUT0cSJ6Sv9e3wI7f4WIXiCiL3tuWTtJyX/QZf9FInr7Ftn8s0T0rC7vx4joPZ7fv9Q2P0lEP3JONr+eiD5JRP+XiB4non+q3be2rEds3vayXhLRZ4joC9ruf6Pd30hEn9b2fYSIFtp9T98/rf3f8LKNMNugn8cf1OT8rwJ4BMACwBcAvOU8bRqx9RsAXhO5/TsAH9DXHwDwb7fAzncBeDuAL0/ZCeA9AH4Has7uOwB8eots/lkA/zyj+xZdT/YAvFHXn/ocbH4IwNv19WUAX9G2bW1Zj9i87WVNAC7p6xbAp3UZ/jqA92r3XwLwj/X1PwHwS/r6vQA+8nJtOG9G+ZcBPM3MX2PmDsCHATx6zjZtIo8C+JC+/hCAv3d+pihh5v8F4KXIuWTnowB+lZX8MYBrRPTQPTHUk4LNJXkUwIeZec3MXwfwNFQ9uqfCzDeY+XP6+hDAEwAexhaX9YjNJdmWsmZmNqd/tfqPAfwtAL+h3eOyNs/gNwD8ENHs5WBZOW+gfBjAt737ZzD+4M5TGMDHiOhPiej92u1BZr6hr58D8OD5mDYpJTu3vfx/SjdTf8Xr1tg6m3XT7m1QTOdVUdaRzcCWlzUR1UT0GIAXAHwcit3eZmZzFotvm7Vb+98BcP/LSf+8gfLVJO9k5rcDeDeAnySid/merHj+1k8heLXYCeAXAfwFAD8A4AaAf3+u1hSEiC4B+E0AP83MB77ftpZ1xuatL2tmFsz8AwBeB8Vqv/9epn/eQPksgNd796/TblsnzPys/n0BwH+DeljPm+aT/n3h/CwclZKdW1v+zPy8fjkkgP8E1+TbGpuJqIUCnF9j5t/Szltd1jmbXw1lbYSZbwP4JIC/CtV9YTb28W2zdmv/qwBuvpx0zxsoPwvgTXr0agHV8frRc7YpESLaJ6LL5hrA3wbwZShb36fV3gfgt8/Hwkkp2flRAP9Qj8i+A8Adr9l4rhL13/19qPIGlM3v1SObbwTwJgCfOQf7CMAvA3iCmX/e89rasi7Z/Coo6weI6Jq+vgDgh6H6Vz8J4Ee1WlzW5hn8KIDf0+z+7HKvR7AyI1rvgRp9+yqAnzlvewo2PgI1+vcFAI8bO6H6PT4B4CkAvwvgvi2w9b9ANZ96qH6bnyjZCTWa+B912X8JwA9ukc3/Wdv0RV3xH/L0f0bb/CSAd5+Tze+EalZ/EcBj+u8921zWIzZve1m/FcDntX1fBvCvtfsjUMD9NID/CmBPuy/1/dPa/5GXa8NuZc5OdrKTnUzIeTe9d7KTnexk62UHlDvZyU52MiE7oNzJTnaykwnZAeVOdrKTnUzIDih3spOd7GRCdkC5k53sZCcTsgPKnexkJzuZkB1Q7mQnO9nJhPw/xsbx71/JRokAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from torchrl.envs.libs.dm_control import DMControlEnv\n",
    "env = DMControlEnv('acrobot', 'swingup', from_pixels=True, pixels_only=True)\n",
    "tensordict = env.reset()\n",
    "print('result of reset: ', tensordict)\n",
    "plt.imshow(tensordict.get(\"pixels\").numpy())\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0e93b95-fa48-48a3-9acc-9e8d8594103b",
   "metadata": {},
   "source": [
    "___\n",
    "## Transforming envs\n",
    "\n",
    "It is common to pre-process the output of an environment before having it read by the policy or stored in a buffer.\n",
    "\n",
    "In many instances, the RL community has adopted a wrapping scheme of the type\n",
    "\n",
    "```\n",
    "env_transformed = wrapper1(wrapper2(env))\n",
    "```\n",
    "\n",
    "to transform environments. This has numerous advantages: it makes accessing the environment specs obvious (the outer wrapper is the source of truth for the external world), and it makes it easy to interact with vectorized environment.\n",
    "However it also makes it hard to access inner environments: say one wants to remove a wrapper (e.g. `wrapper2`) from the chain, this operation requires us to collect\n",
    "```\n",
    "env0 = env.env.env\n",
    "env_transformed_bis = wrapper1(env0)\n",
    "```\n",
    "\n",
    "TorchRL takes the stance of using sequences of transforms instead, as it is done in other pytorch domain libraries (e.g. `torchvision`). This approach is also similar to the way distributions are transformed in `torch.distribution`, where a `TransformedDistribution` object is built around a `base_dist` distribution and (a sequence of) `transforms`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cf9ae717-2f7a-4722-9ce1-01484d53b984",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reset before transform:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        pixels: Tensor(torch.Size([240, 320, 3]), dtype=torch.uint8)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n",
      "reset after transform:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        pixels: Tensor(torch.Size([3, 240, 320]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs.libs.dm_control import DMControlEnv\n",
    "import torch\n",
    "from torchrl.envs.transforms import TransformedEnv, ToTensorImage\n",
    "# ToTensorImage transforms a numpy-like image into a tensor one, \n",
    "env = DMControlEnv('acrobot', 'swingup', from_pixels=True, pixels_only=True)\n",
    "print('reset before transform: ', env.reset())\n",
    "\n",
    "env = TransformedEnv(env, ToTensorImage())\n",
    "print('reset after transform: ', env.reset())\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0fdb760-bd1b-4688-ba54-da156a63c36b",
   "metadata": {},
   "source": [
    "To compose transforms, simply use the `Compose` class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f0a5081d-2afc-4f0a-ad8a-4df681cfc917",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        pixels: Tensor(torch.Size([3, 32, 32]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torchrl.envs.transforms import Compose, Resize\n",
    "env = DMControlEnv('acrobot', 'swingup', from_pixels=True, pixels_only=True)\n",
    "env = TransformedEnv(env, Compose(ToTensorImage(), Resize(32, 32)))\n",
    "env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "566b0c94-6022-477a-9e2c-32f9009bcaaa",
   "metadata": {},
   "source": [
    "Transforms can also be added one at a time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "22da21c4-e9d6-44bc-996d-268bc37e4909",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
       "        pixels: Tensor(torch.Size([1, 32, 32]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torchrl.envs.transforms import GrayScale\n",
    "env.append_transform(GrayScale())\n",
    "env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef5a2176-20d3-4270-b3ff-a1d8b5c75fcf",
   "metadata": {},
   "source": [
    "As expected, the metadata get updated too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "734c07ec-ff03-4df8-844e-acca466d19e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original obs spec:  CompositeSpec(\n",
      "    next_pixels: NdUnboundedDiscreteTensorSpec(\n",
      "         shape=(240, 320, 3), space=ContinuousBox(minimum=tensor([[[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]],\n",
      "\n",
      "            [[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]],\n",
      "\n",
      "            [[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]],\n",
      "\n",
      "            ...,\n",
      "\n",
      "            [[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]],\n",
      "\n",
      "            [[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]],\n",
      "\n",
      "            [[0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             ...,\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0],\n",
      "             [0, 0, 0]]]), maximum=tensor([[[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]],\n",
      "\n",
      "            [[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]],\n",
      "\n",
      "            [[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]],\n",
      "\n",
      "            ...,\n",
      "\n",
      "            [[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]],\n",
      "\n",
      "            [[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]],\n",
      "\n",
      "            [[255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             ...,\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255],\n",
      "             [255, 255, 255]]])), device=cpu, dtype=torch.uint8, domain=continuous))\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "Input image tensor permitted channel values are [3], but found240",
     "output_type": "error",
     "traceback": [
      "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[0;31mTypeError\u001B[0m                                 Traceback (most recent call last)",
      "\u001B[0;32m/var/folders/zs/9lq15k8x61l1g0c_sf__63c80000gn/T/ipykernel_13887/2654911180.py\u001B[0m in \u001B[0;36m<module>\u001B[0;34m\u001B[0m\n\u001B[1;32m      1\u001B[0m \u001B[0mprint\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m'original obs spec: '\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0menv\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mbase_env\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mobservation_spec\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m----> 2\u001B[0;31m \u001B[0mprint\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m'current obs spec: '\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0menv\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mobservation_spec\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/transforms.py\u001B[0m in \u001B[0;36mobservation_spec\u001B[0;34m(self)\u001B[0m\n\u001B[1;32m    338\u001B[0m         \u001B[0;34m\"\"\"Observation spec of the transformed_in environment\"\"\"\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m    339\u001B[0m         \u001B[0;32mif\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0m_observation_spec\u001B[0m \u001B[0;32mis\u001B[0m \u001B[0;32mNone\u001B[0m \u001B[0;32mor\u001B[0m \u001B[0;32mnot\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mcache_specs\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m--> 340\u001B[0;31m             observation_spec = self.transform.transform_observation_spec(\n\u001B[0m\u001B[1;32m    341\u001B[0m                 \u001B[0mdeepcopy\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mbase_env\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mobservation_spec\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m    342\u001B[0m             )\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/transforms.py\u001B[0m in \u001B[0;36mtransform_observation_spec\u001B[0;34m(self, observation_spec)\u001B[0m\n\u001B[1;32m    604\u001B[0m     \u001B[0;32mdef\u001B[0m \u001B[0mtransform_observation_spec\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mself\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mobservation_spec\u001B[0m\u001B[0;34m:\u001B[0m \u001B[0mTensorSpec\u001B[0m\u001B[0;34m)\u001B[0m \u001B[0;34m->\u001B[0m \u001B[0mTensorSpec\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m    605\u001B[0m         \u001B[0;32mfor\u001B[0m \u001B[0mt\u001B[0m \u001B[0;32min\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mtransforms\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m--> 606\u001B[0;31m             \u001B[0mobservation_spec\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mt\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mtransform_observation_spec\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mobservation_spec\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m    607\u001B[0m         \u001B[0;32mreturn\u001B[0m \u001B[0mobservation_spec\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m    608\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/transforms.py\u001B[0m in \u001B[0;36mnew_fun\u001B[0;34m(self, observation_spec)\u001B[0m\n\u001B[1;32m     76\u001B[0m             \u001B[0;32mfor\u001B[0m \u001B[0mkey_in\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mkey_out\u001B[0m \u001B[0;32min\u001B[0m \u001B[0mzip\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mkeys_in\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mkeys_out\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     77\u001B[0m                 \u001B[0;32mif\u001B[0m \u001B[0mkey_in\u001B[0m \u001B[0;32min\u001B[0m \u001B[0mobservation_spec\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mkeys\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m---> 78\u001B[0;31m                     \u001B[0md\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0mkey_out\u001B[0m\u001B[0;34m]\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mfunction\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mself\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mobservation_spec\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0mkey_in\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m     79\u001B[0m             \u001B[0;32mreturn\u001B[0m \u001B[0mCompositeSpec\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m**\u001B[0m\u001B[0md\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     80\u001B[0m         \u001B[0;32melse\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/transforms.py\u001B[0m in \u001B[0;36mtransform_observation_spec\u001B[0;34m(self, observation_spec)\u001B[0m\n\u001B[1;32m   1204\u001B[0m         \u001B[0mspace\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mobservation_spec\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mspace\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   1205\u001B[0m         \u001B[0;32mif\u001B[0m \u001B[0misinstance\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mspace\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mContinuousBox\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m-> 1206\u001B[0;31m             \u001B[0mspace\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mminimum\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0m_apply_transform\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mspace\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mminimum\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m   1207\u001B[0m             \u001B[0mspace\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mmaximum\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mself\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0m_apply_transform\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mspace\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mmaximum\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   1208\u001B[0m             \u001B[0mobservation_spec\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mshape\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mspace\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mminimum\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mshape\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/transforms.py\u001B[0m in \u001B[0;36m_apply_transform\u001B[0;34m(self, observation)\u001B[0m\n\u001B[1;32m   1197\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   1198\u001B[0m     \u001B[0;32mdef\u001B[0m \u001B[0m_apply_transform\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mself\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mobservation\u001B[0m\u001B[0;34m:\u001B[0m \u001B[0mtorch\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mTensor\u001B[0m\u001B[0;34m)\u001B[0m \u001B[0;34m->\u001B[0m \u001B[0mtorch\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mTensor\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m-> 1199\u001B[0;31m         \u001B[0mobservation\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mF\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mrgb_to_grayscale\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mobservation\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m   1200\u001B[0m         \u001B[0;32mreturn\u001B[0m \u001B[0mobservation\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m   1201\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/functional.py\u001B[0m in \u001B[0;36mrgb_to_grayscale\u001B[0;34m(img, num_output_channels)\u001B[0m\n\u001B[1;32m     34\u001B[0m             \u001B[0;34m\"{}\"\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mformat\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mimg\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mndim\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     35\u001B[0m         )\n\u001B[0;32m---> 36\u001B[0;31m     \u001B[0m_assert_channels\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mimg\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0;34m[\u001B[0m\u001B[0;36m3\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m     37\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     38\u001B[0m     \u001B[0;32mif\u001B[0m \u001B[0mnum_output_channels\u001B[0m \u001B[0;32mnot\u001B[0m \u001B[0;32min\u001B[0m \u001B[0;34m(\u001B[0m\u001B[0;36m1\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0;36m3\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;32m~/Repos/RL/torch_rl/torchrl/envs/transforms/functional.py\u001B[0m in \u001B[0;36m_assert_channels\u001B[0;34m(img, permitted)\u001B[0m\n\u001B[1;32m     22\u001B[0m     \u001B[0mc\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0m_get_image_num_channels\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mimg\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     23\u001B[0m     \u001B[0;32mif\u001B[0m \u001B[0mc\u001B[0m \u001B[0;32mnot\u001B[0m \u001B[0;32min\u001B[0m \u001B[0mpermitted\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m---> 24\u001B[0;31m         raise TypeError(\n\u001B[0m\u001B[1;32m     25\u001B[0m             \u001B[0;34m\"Input image tensor permitted channel values are {}, but found\"\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m     26\u001B[0m             \u001B[0;34m\"{}\"\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mformat\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mpermitted\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mc\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
      "\u001B[0;31mTypeError\u001B[0m: Input image tensor permitted channel values are [3], but found240"
     ]
    }
   ],
   "source": [
    "print('original obs spec: ', env.base_env.observation_spec)\n",
    "print('current obs spec: ', env.observation_spec)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff001409-5c34-46be-95e2-47b2f78114ac",
   "metadata": {},
   "source": [
    "We can also concatenate tensors if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cd294681-b15c-4735-9215-ea754b395fb0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "keys before concat:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        orientations: Tensor(torch.Size([4]), dtype=torch.float64),\n",
      "        velocity: Tensor(torch.Size([2]), dtype=torch.float64)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n",
      "keys after concat:  TensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([1]), dtype=torch.bool),\n",
      "        observation: Tensor(torch.Size([6]), dtype=torch.float64)},\n",
      "    batch_size=torch.Size([]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs.transforms import CatTensors\n",
    "env = DMControlEnv('acrobot', 'swingup')\n",
    "print(\"keys before concat: \", env.reset())\n",
    "# make sure to work with \"next_key\" as this is what step will return\n",
    "env = TransformedEnv(env, CatTensors(in_keys=[\"next_orientations\", \"next_velocity\"], out_key=\"next_observation\"))\n",
    "print(\"keys after concat: \", env.reset())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81b62090-d878-4cfb-8e83-dbefddaf3405",
   "metadata": {},
   "source": [
    "This feature makes it easy to mofidy the sets of transforms applied to an environment input and output.\n",
    "In fact, transforms are run both before and after a step is executed: for the pre-step pass, the `in_keys_inv` list of keys will be passed to the `_inv_apply_transform` method. An example of such a transform would be to transform floating-point actions (output from a neural network) to the double dtype (requires by the wrapped environment).\n",
    "After the step is executed, the `_apply_transform` method will be executed on the keys indicated by the `in_keys` list of keys. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34fb4aa3-6193-44a5-bd79-2ebf087155e8",
   "metadata": {},
   "source": [
    "Another interesting feature of the environment transforms is that they allow the user to retrieve the equivalent of `env.env` in the wrapped case, or in other words the parent environment.\n",
    "The parent environment can be retrieved by calling `transform.parent`: the returned environment will consist in a `TransformedEnvironment` with all the transforms up to (but not including) the current transform. \n",
    "This is be used for instance in the `NoopResetEnv` case, which when reset executes the following steps: resets the parent environment before executing a certain number of steps at random in that environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ede057e5-11da-41b7-9635-bcf90ff10711",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: \n",
      " TransformedEnv(env=DMControlEnv(env=acrobot, task=swingup, batch_size=torch.Size([])), transform=Compose(\n",
      "        CatTensors(in_keys=['next_orientations', 'next_velocity'], out_key=next_observation),\n",
      "        GrayScale(keys=['next_pixels'])))\n",
      "GrayScale transform parent env: \n",
      " TransformedEnv(env=DMControlEnv(env=acrobot, task=swingup, batch_size=torch.Size([])), transform=Compose(\n",
      "        CatTensors(in_keys=['next_orientations', 'next_velocity'], out_key=next_observation)))\n",
      "CatTensors transform parent env: \n",
      " TransformedEnv(env=DMControlEnv(env=acrobot, task=swingup, batch_size=torch.Size([])), transform=Compose(\n",
      "))\n"
     ]
    }
   ],
   "source": [
    "env = DMControlEnv('acrobot', 'swingup')\n",
    "env = TransformedEnv(env)\n",
    "env.append_transform(CatTensors(in_keys=[\"next_orientations\", \"next_velocity\"], out_key=\"next_observation\"))\n",
    "env.append_transform(GrayScale())\n",
    "print(\"env: \\n\", env)\n",
    "print(\"GrayScale transform parent env: \\n\", env.transform[1].parent)\n",
    "print(\"CatTensors transform parent env: \\n\", env.transform[0].parent)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bd8908e-a0b9-4844-8bc4-c95657acd07b",
   "metadata": {},
   "source": [
    "___\n",
    "## Environment device\n",
    "Transforms can work on device, which can bring a significant speedup when operations are moderetely or highly computationally demanding. These include `ToTensorImage`, `Resize`, `GrayScale` etc. \n",
    "\n",
    "One could legitimately ask what that implies on the wrapped environment side. Very little for regular environments: the operations will still happen on the device where they're supposed to happen. The environment device attribute in torchrl indicates on which device is the incoming data supposed to be and on which device the output data will be. Casting from and to that device is the responsibility of the torchrl environment class. The big advantage of storing data on GPU is (1) speedup of transforms as mentioned above and (2) sharing data amongst workers in multiprocessing settings.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a7538009-c098-47ee-8129-c7535aa9eb97",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torchrl.envs.libs.dm_control import DMControlEnv\n",
    "from torchrl.envs.transforms import CatTensors, GrayScale, TransformedEnv\n",
    "env = DMControlEnv('acrobot', 'swingup')\n",
    "env = TransformedEnv(env)\n",
    "env.append_transform(CatTensors(in_keys=[\"next_orientations\", \"next_velocity\"], out_key=\"next_observation\"))\n",
    "\n",
    "if torch.has_cuda and torch.cuda.device_count():\n",
    "    env.to('cuda:0')\n",
    "    env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288f91d7-6736-46db-8e06-4eca34711d0d",
   "metadata": {},
   "source": [
    "___\n",
    "## Running environments in parallel\n",
    "\n",
    "TorchRL provides utilities to run environment in parallel. It is expected that the various environment read and return tensors of similar shapes and dtypes (but one could design masking functions to make this possible in case those tensors differ in shapes). Creating such environments is quite easy. Let us look at the simplest case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ef7cbd08-e0c3-41af-b367-cf08cae9adc0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs import ParallelEnv, SerialEnv\n",
    "from torchrl.envs.libs.gym import GymEnv\n",
    "env_make = lambda: GymEnv(\"Pendulum-v1\")\n",
    "parallel_env = ParallelEnv(3, env_make)  # -> creates 3 envs in parallel\n",
    "parallel_env = ParallelEnv(3, [env_make, env_make, env_make])  # similar to the previous command"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6d4f2ae-35da-41c7-94e0-4e6fd7311918",
   "metadata": {},
   "source": [
    "The `SerialEnv` class is similar to the `ParallelEnv` except for the fact that environments are run sequentially. This is mostly useful for debugging purposes.\n",
    "\n",
    "`ParallelEnv` instances are created in lazy mode: the environment will start running only when called. This allows us to move `ParallelEnv` objects from process to process without worring too much about running processes.\n",
    "A `ParallelEnv` can be started by calling `start`, `reset` or simply by calling `step` (if `reset` does not need to be called first)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c3e4766f-9975-4cc0-96fc-fd4a7344337d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LazyStackedTensorDict(\n",
       "    fields={\n",
       "        done: Tensor(torch.Size([3, 1]), dtype=torch.bool),\n",
       "        observation: Tensor(torch.Size([3, 3]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([3]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "parallel_env.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5ecee3d-5e87-4351-bd03-a979e6e8bc79",
   "metadata": {},
   "source": [
    "One can check that the parallel environment has the right batch size. Conventionally, the first part of the `batch_size` indicates the batch, the second the time frame. Let's check that with the `rollout` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a764b5c2-a17d-49ff-9cbb-b77903b89cad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorDict(\n",
       "    fields={\n",
       "        action: Tensor(torch.Size([3, 20, 1]), dtype=torch.float32),\n",
       "        done: Tensor(torch.Size([3, 20, 1]), dtype=torch.bool),\n",
       "        next_observation: Tensor(torch.Size([3, 20, 3]), dtype=torch.float32),\n",
       "        observation: Tensor(torch.Size([3, 20, 3]), dtype=torch.float32),\n",
       "        reward: Tensor(torch.Size([3, 20, 1]), dtype=torch.float32)},\n",
       "    batch_size=torch.Size([3, 20]),\n",
       "    device=cpu,\n",
       "    is_shared=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "parallel_env.rollout(max_steps=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a18c530-d8ac-4d00-bcd9-02e8350005f1",
   "metadata": {},
   "source": [
    "### Closing parallel environments\n",
    "\n",
    "**Important**: before closing a program, it is important to close the parallel environment. In general, even with regular environments, it is good practice to close a function with a call to `close`. In some instances, TorchRL will throw an error if this is not done (and often it will be at the end of a program, when the environment gets out of scope!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cb805b61-b29c-485c-b224-94ad8bdba05f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "parallel_env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fcdb552-3c82-4be2-b5d2-20d87362669b",
   "metadata": {},
   "source": [
    "### Seeding\n",
    "When seeding a parallel environment, the difficulty we face is that we don't want to provide the same seed to all environments. The heuristic used by TorchRL is that we produce a  deterministic chain of seeds given the input seed in a -- so to say -- Markovian way, such that it can be reconstructed from any of its elements. All `set_seed` methods will return the next seed to be used, such that one can easily keep the chain going given the last seed. This is useful when several collectors all contain a `ParallelEnv` instance and we want each of the sub-sub-environments to have a different seed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2c10bc47-c386-4c00-b07e-97ee1db28316",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3288080526\n"
     ]
    }
   ],
   "source": [
    "out_seed = parallel_env.set_seed(10)\n",
    "print(out_seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52c84cdb-f024-4c88-a462-7f50524d80ac",
   "metadata": {},
   "source": [
    "### Accessing environment attributes\n",
    "It sometimes occurs that a wrapped environment has an attribute that is of interest. \n",
    "First, note that TorchRL environment wrapper constains the toolings to access this attribute. Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "3f317630-6ee7-42b4-89ee-01f5bee14f5e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "from uuid import uuid1\n",
    "from time import sleep\n",
    "def env_make():\n",
    "    env = GymEnv(\"Pendulum-v1\")\n",
    "    env._env.foo = f\"bar_{uuid1()}\"\n",
    "    env._env.get_something = lambda r: r+1    \n",
    "    return env\n",
    "env = env_make()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "16d5621a-14e3-427d-b3d5-1ffa497a117d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'bar_542ef942-3257-11ed-b93c-aa665a2328e0'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# goes through env._env\n",
    "env.foo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1ddbb4f0-7418-4b91-97dc-808c0cb268af",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Aargh what did I do!\n"
     ]
    }
   ],
   "source": [
    "parallel_env = ParallelEnv(3, env_make)  # -> creates 3 envs in parallel\n",
    "# env has not been started --> error:\n",
    "try:\n",
    "    parallel_env.foo\n",
    "except:\n",
    "    print(\"Aargh what did I do!\")\n",
    "    sleep(10)  # make sure we don't get ahead of ourselves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "702838c3-7aa1-4af3-974f-e312629940e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "parallel_env.start()\n",
    "foo_list = parallel_env.foo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "965b8d5f-f549-4e38-80a9-fe82a571b209",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torchrl.envs.vec_env._dispatch_caller_parallel at 0x15e42f070>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "foo_list  # needs to be instantiated, for instance using list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "15e0e5f5-d1f9-4f55-8437-8e393b4754f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['bar_5cdf70ee-3257-11ed-acfd-aa665a2328e0',\n",
       " 'bar_5cdf70da-3257-11ed-8393-aa665a2328e0',\n",
       " 'bar_5cdf7102-3257-11ed-8191-aa665a2328e0']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(foo_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da844a71-f313-4e42-b352-0ec54a1e3b58",
   "metadata": {},
   "source": [
    "Similarly, methods can also be accessed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bce02ca2-b0fc-47fb-b57d-125410d8979e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 1]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "something = parallel_env.get_something(0)\n",
    "something"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "cefbe2dc-9906-4afc-950a-3039f8eebdca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "parallel_env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "521d423e-6468-4ea6-b1b6-ac4befca8d05",
   "metadata": {},
   "source": [
    "### kwargs for parallel environments\n",
    "\n",
    "One may want to provide kwargs to the various environments. This can achieved either at construction time or afterwards:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "3787fd8a-dfee-4006-8870-6be019d8dfc3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]\n",
      "A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAChCAYAAAC8o8hrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlzUlEQVR4nO2dW6wkx3nf/1/PzLnvLndXyu5Gy5tD0gEVwZGj2HREBIIFAY7sSH4QBClCQCcC+OIEMmDAppy3IA92HmwJQRCAkGLoQYDkyIYkC0YMm5IixEBora7OakmLlHfDpbjcJbXLc/Zc5tZfHrpnpi9VXZeu7q6ZU7/F2ZnurstXX1V9XVVfdQ8xMwKBQGBViboWIBAIBJokGLlAILDSBCMXCARWmmDkAoHAShOMXCAQWGmCkQsEAitNLSNHRL9ERM8T0QtE9JQroQIBILSvgBvIdp8cEfUA/C2A9wC4DuCbAD7MzD9wJ17guBLaV8AVdUZyPwfgBWb+ETOPAHwOwPvdiBUIhPYVcEO/Rty3AHgpc3wdwM9XRSCi8HjF8eU1Zn6zQXjj9rW51ecTJwcgG+kM6fcibK33ERGhH8nHCsPJFHtHI2CFWj6DAJwB8xaANQC93FXMayAG0Y9BOGhABuTq+earR9L2VcfIaUFETwJ4EgAiEE7hRNNZBjzkNnavNZFutn3tnBzgQ0/8g3KgtEcUO4aQWaCKwAzg3MlN/Oz9b8b2+gCnt9cRMzCZxgAl0SIi9CLCizffwP967mXEXJ2ml0jkZe5jMv0w4vhnwHwfgJMAjgBMkEwOIyTGb4jB4D+iF32nnKaGLuZBZHJkTv+X/3xZ2r7qGLmXAdybOb6YnssLwvw0gKcB4E10mt+Hd9XIMrCs/CG+aBrFuH2dO78pHi9R7qMaKnxWBMlyOJrgxhv7ICKs9XvYWuvjzPa6OqIRplayplXVjhojoucBeg3ADsDriPmnAAzkaWqkTaUv4uuqQXIdI/dNAA8T0YNIGt+HAPyrqggRCJvIVzyB0Ecv/Zan/Rtf3RxbboQta0iUW4MSGLcvmVxFGTk9YyW3pMDj6RRvHIwQRYTNQR8RASi09fqYStxW22CAXkdE18F8D5i2Ab4IoZFTp4RkQky5c7K6hOC4iLWRY+YJEf07AH+OZFL+35n5clWcCaa4hdu5cwP0cRan0MvN6xOowS4kTrluXm03wnL8Js2eqD6aysumfcnkKk55cuatSmEGPWrvaIznXrmNfi/Cqc01XLhnGxdObeuKu+RM0Yu+iSj6FpjvBfNZMD8A5kL5NXSdXM7XD2UObdpbrTU5Zv4zAH9mEicqdZIqsZvrxD6PnxgAb20g3lgrXYv2DxENx9K4ruRq5iZgKINF+5IiE92s+SUI5kdxzBhOpoiZMZ7GmMSxoYCF9IV56y5oKcI5b8AM4BDAHoB9AJsABOW30bWOnIr5auOOh3xmfbwZZ3LnCEBksJPFh3XbNmQ4euuDGP70/fmTzNj6P/8X6y+Wlqac44OerTBxMlQnIccmYROBpOF0F7QU4eqsqDjwEjupm2wiisRaNXIEpOtvugVdEXeURXheGyDe2kjG6kQAMxDH4H6rVVaUCqrydV5jWSeDxiKizRpPHbmkmVZfaAiNkZ5IOTUNnaiEJUkkdUeahi1LZ8+u6snYfIWb1VeLm52mMWg8AZjBUWLkaDoFsXga1I5k6vro+paU04NIGKo8bAflSK0tHIz0HFHKUlF3XPisosthgQUL8159z9O/I+qsbOiF1rmuHz66e4j+a3fAgz641wNNpsB0CjoaOci5YgSZu9L5uMwYqbSyoij2wylHFylr/R7O7mygH0U4sTnA1lq/sfmy7oqcfQBTIjBfAPPDYD4Pxj0QeZYrszWsH5MBnedGrlhCEnwTYdrl8y5ru1TcsvH8Nay/eD0VIp0jMECjstNBJr8cedhqo748Rq+kE5m1qiiO1ugi5e+d2sR73novQISIgB5ZblFR5KNxWZ2vhWDVNT/AZPoRYDpCYlIiJM6H/A1Zf51TXj/ltq5uk54bObNOZtsFTZvjbIjcRHefpz2eJNNVDWy7k/kK23IYuIQqWettTmIkTzfcHY4xjdNdd5LE9oeT0pRKdxzdPOL81OuUDKI7ILqRXtmRpH8EwpGFRNV1V31cpjMjlxRGts1PVtn1x28mzaiLEV71NER/xKZTTvsVNnm9mY4pm8Kk7EKPXfFaIe7e0RiXX/4JelH1s6tH4wmKL/rRHUc3jzi/qlljUpQYUfTXiOgygA1AsMcVaTiil3LxRYkaOEqtNNS6kZsVKBFWNkpobppkNqnzobsuMJHHreRVI7r8d9VtqkmmzNgby/cQuqQ3iXAUx4goeU5VxngaJ4auFalaggnAGwAOwfPpqTAgCAcAtVMnMlo3cuYTBbvNJhbrzCuFbvn19DG7h5sY2fY1fTSN8fztvVbyovlaqdoJsJq/bbyH7HBFToxWdyUIaNXITTHFT7CbO9dDhB1soye9G+h1lCTUomPJYpl3cBOaMeA2qemUH0aTS1M527+VMBgTLYPiQO88/08/1WW7wyrcoc0Vx23KrRq5uzjEs/ib3LkT2MI/wVszD+5nZ/+mDgH1dMlk/m9Ot4bAdCRrt35mUj/+9Oq8JM2MM5Xp+aEKfRTyNuV4kz2cLwutolUjl4zatnLntrBZeJ5VzwSpPEB6I5km6XbCrFN+O+nE9SPeoO7PqmZVW2msRkw22nmJxuMhjnOpqqcyesK0auS2sYmfx9ty5xavWjLDRtdNbv0o51RlZppv7LZltZVKFK+dktanMflKCfuuiSJUedhQLs5p+dlVwpriHVO1d3NX5t80ed+xWpLmTEDdVF3VQ5vdukeEEwPPt34GWse7FqFrHvzE/8V5XZaxHtZ7PTx0SrYxNbDKfLXimndGLhCwhQAMKjbmBo4nwcgFVgofR5iBblkKI+fP4nX5YbTmcsl/6xI/pFBDAKji6YPaFB5DWha9tEVJL54oyNuxfXab5azDd79vXL3D3V0ubeRUDaf/F7db2KTiDbbiCF6M4UH/9YqSXmYWzwaHzaZ1I6cre7kBud1zpSsHFz5N4spT0IthE9ulSRGZdN06WMjRkSmQKaK8mU8P2aZCk7RqVo7p02G1nyYzLY8ovHmDqY5nUabWjZy6zO3c+bObOKryF921zbqt+X1fb2t0+ZEis1zqUl1PXY1y5lLJK1gcTnSVS2fymO1crTeyAaSvc3IVPofOVFOj/Cw8ECiBpFfyF1ThBHg4XZV35ybMX7ndUUU+rm/VduklZ3UffbGnWu9+TtaYGfvjCfbHE8TMUjGrp+DpAgkVzxQzMxVOlpBm3Dbj2Rhkue1KLpPorGhpSiJPJrCJGjsxcqa6a3oNRP+G7Hr3mF16pgMIW2z13uUq3DiOcePgCK8eHmEa680nZQaMMt9Lg5CZE0JR2Nxly8VNlgqpARnWh8TiKG/HWa9DJgJnvlPme0FEDWGUgaV04l31bQxgfoNtz7/qPnTzdClLRITNfm/+I2d5JJKJFFhYMygNQlIDp5oSKkcmKjkAUM0KlsZXlVt9enG9qI+sk0a11iMtn5uWtBRbSJrGXJXt+Vfdh15t+kQ4v7UBwEAvlgoUGTjR9hJtGyULVLeCa6YrKovQPtYYbTaJh2ty3ePZpocSvsvXJUSEKP2rvWfOYjFYZABc9OHstK9SpkIYF22lWBbjgWWTi+oaLKWRa1pXVa4HH2haPr9Lb0DdgogWJTXStDIAOmnqWE7ZNNuRHFVZS9NztahuWZ9eG7niDaCpzldenvZ7AliUr2m9dHwjtqM4x6oKWt4xUkpqjmqbhCm6BpQtMlLF0XJr6mVTQsNpMW9XupsMLLfgeG3kijeAprysbXkrXdG0vG3pvVF0Frtnl9POk+tD2esaWen2vUp/b1VGpCGIaZwKQ2RSHqM9ApkbyszbSipDX3Pu36mRa2JksFSjjQ45VrqXdJyigcl1WIPOZLJG1cQNyvW6m+i4Kp5R/hmva3FrjHSrTM0CdmrkTCpYt5xLNdroEF09mbQvb3WvuS1CNOKokbwYrTmxGbX07kAe2/JrG9aaDUtp5IjoXiL6GhH9gIguE9HH0vNniOgviOiH6efpeqJU67UJg9gEttOWNnFluJyMIFpsXzO01ho1GpxV+SscBqKpcmWGpseF89KRU9X5CrT0UbGm2dTar85IbgLgN5n5UQCPAfh1InoUwFMAnmHmhwE8kx7XYrFOXK+Y2SG07VpDnbxdhrNFVu6a+0rTNJLUHJWhtfY1w9Vao6Zfwzg9bcNjelw476oNZv0CNjS99qs0csz8CjN/O/2+B+AKgLcAeD+Az6TBPgPgV10J5eJ9IzKFeTulcoys3C7K7/J9MK20LwMrZOPEdKYNk6FMnWmm4yGTafmNddzmmhwRPQDg7QCeBXCOmV9JL90AcM4sa2+XqQO1sK9XJ+1L5Z2ripd6WEln20UTzVf3TlQ1HNfxBDTpLtfQD/HCs6qlx5rOCG0jR0Q7AP4YwG8w824uP2apuET0JBFdIqJLhxhmr+hLaUwwoNU0qR+7enXSvg6n9eZMMwORpiFdq1PM+ZVra7bozgtdz6ML+VcmW6EfFoWpO8fVQMvIEdEASQP8LDP/SXr6VSK6kF6/AOCmKC4zP83M72Dmd2xiXV+yWpSfCeD0/7qOgbbMZ938ZeV1uprmCGfta7Pi93u58jAjjPCrNEzlZddqttkn10D+ymR1vNmSMKV6UYyqdfqDjneVAHwawBVm/v3MpS8DeCL9/gSAL2nk1xriNSm91STVbCAbziXZ9OrMRhbxy+X1y7y12L6o8tCOJhuA6LzEUEtniKqpY1vyG1Cql6qKIr161HkLyTsB/GsAf0NE303P/Q6A3wXwR0T0UQDXAHxQIy0lWQPDYKeL3Lp05SH1xTMrI1sfLry0Ka22ryrybc9+xGKN6s4qMdTSJTbV2lsHDdhYxw5QGjlm/t8VsrzbrTj5jGwMXFuKW3Zs9JStD2fbD1puX1WQ5Ls2qVKt22Dbjdcyv3k0i/i1dWyB18+u2tCs4tp2aDSXX7gRNECN/WeVG3ObguxaWGNrjqZoCr9yRq4ps9DNgn1zL1UK/ucW0FRyl7MP3TVewCRgS2gqzTMjV0eLRTPkska6e/nSYgXMFS71VNdX3Samz74YWCjZcXakJAvHir7qSnVV+9YK8ojilUaapurUDmiSsF6mnhm5OqbEnR+xrLqux+X5/Ou1e5f+Vt9dJVlMy60psyJZ6dRO5RQwFEOJbj4SOZVeT205TQuscK9q0LmRc+XdlnvLzU2CD12yCjv5WHhkq2fd851jPeJwnL+3ClLQlfwO661zI+f6RlVOz3eT1RYkPGproNAZXW8UdK3oLG0Y8Cbl18lXdmxAB0ZOrya67jS+3ni7lku/XrqW9BjQtQFfEjowcstRE75K6atcZZZHUm/oekq4onQ+XQ0ElhWe/2cbuUDXU0KgVnm6spmqfIORCwRsmG39oPlh4YsCXwe6unJx7mPxHGkHlk4lckdGjiFeNTXZE+Non9Oxw/U+JdN6bAOWfFcEVYde/HxecauIrtvZd2+rYr9ckVK5Z0bfTO0a+jCo0wI6D+g3gMj2utw/o3M9YVWeddUvR9P7lHzQpsETkrJ9bbLgqofdVerrylupi6NySPUkiMa8jjh+HMynAXoDhH1E0fdBdFcSw0x5HRg5F2bFnWnyta2Z4rYcftVRPfJyqKWahWhAfl9Uoksj8or0u4HJ9F+C+SEQXQXRqyC6VjBy9sIsoXd12VrKMmL0RGNFGj4g3h+oCm/8IJ/OKoAvKtFFpxkYNhNWDQErhbGTYQkdD8vWUpaV461n49LLIkjW8IyWoFxiulZGsguF65o00qoUiS6hkVPh64qubwQ9tYJszcoyXm3Mlii143WGRjNeQSPna234RtBTDsc2f1l9/63L3cKqyAoaOV+bj28EPeVwbPPd+P7bp3W5XSzRK1hBI+dr8/GNoKdWkHTCpVmT04zXGas5kvNV26vG8dazcelVC/MsPi3FlzU5ieOkdF2TRlqVf95VF5Pw490Bm8fVPjkfEL9HTxXe+MXzOvM8X1Sii04zMPau2j7yURFeIUMHm4Fd3KLc3eaWcfuSCLfl8KuO6mG3T64R+X1RiS6N7vfIJn6Efu9Pc088EL3hTJiOHusSjYF1xsXF8FUF1+v2y9buZOiXw1RvVeFt6rENsjIrylu4rNQOSx5Zkj0oITv29e7qqBxSPQmSIxqiFz2j0IdBnRboaE2OIH4YrkpwUXhVHoEypnpT1YlpPbYBSb4rgqpDLzpu8cF+WUSZOrtWkQyZfBJ5S+VO9WLy7GpV+uIAZspbQsdDIOABlPZnnh8WvijwdX1OVy7KfQCcRu3AeKtEDkYuELCE5v/ZRi7QtuET5VejPF0NTlX5LqF3tR18ldJXucosj6Te0LaV8HXK7Bhv30LSdRfxtf67lku/XrqW9Big+WLL407n01VX9SLffRNqPkG8X8y1/r2ja0PgWtFZTH1xNjQpv06+smMDOjdyuk4p3XTK8cxr3tsOm2Inn3i/mK2edc93ThuGQCd/bxWkoCv5HdabtpEjoh4RfYeIvpIeP0hEzxLRC0T0eSJa00mHJf9mV+1xZ/rL+uza7OXzr9feXA5tdOOqw7lqX/oyOHrfhiJZVoVr67UfuvlI5CxFt25GpgWuCq+XqclI7mMArmSOfw/AHzDzQwBuA/ioKoF9HOISLuf+LuNFjDFJQ9TpvklcLhy7wfghH2fMHjJyh0s96cbVCle7fZnJ4GifZVWyXLG1JDNCqmxbrqpesTFXtT+O5gGrw9USxDhhvUy1jBwRXQTwywA+lR4TgF8E8IU0yGcA/KoqnSFGeAEv5f5ewo2MkatPU6Pqomng2R8BTLQ4dpqr8Uu4tfFp9uSqfXmHgZ3s8ibqzD61jabSdB/r+gSA3wJwIj0+C+AOM8+s03UAb1Elso1N/FO8NXduDWtYw0BTDDXNPi2zSHly4U0YPfj3ge0N0Mlt9K6+gv4PXwINx8Bw5EiG5kri2VNFn4CD9tU5qVJtdDsfKbVZKYpHryqidSNvEc28lUaOiH4FwE1m/hYRvctYDqInATwJACewjYdwX2X4/BNqDDLUYlM6L940pqd3MHzkXtCZU6BzpzGIIkQv30LEDBqNkmf3GpLFBTayZevDVft22r5O1rtZ2j8dORMm92FO14v7ptHqGEi0ZyN1RnLvBPA+InovgA0AJwF8EsA9RNRP77YXAbwsiszMTwN4GgDO0VnlADP/hFo3ZkKk/PjUDibnzmAyHGF8cIT+7gG2/+r7mJ4/i8n95zG98ToOCYgunEV08j4MXnkd/ZdvGZVAt9K7uoFm68Nh/u7a1/nNWrM+46cjXVeELD3Jw/DF06XoqpcBtCV/BmMdO0C5JsfMH2fmi8z8AIAPAfgqM38EwNcAfCAN9gSALzUmpQViHw2XzosQrZHEWxsYnz+D0ZtO4fDUFuLhEGsvXMfg6o8RvXQTvHsX44gwPrWD0f0XML1nx1jmwpq1EBNDKCpv137iIq21L5nXsw6ue6lqfw5Vny5FV23/aEt+A5Re3MI1nXqs86ql3wbwOSL6TwC+A+DTNdJyTHmaS+n/tv7A3p09rD///3D2vh5O/bMBjm6fxd2bF0Fbm+id2MLmmSm2Hh5j/5U93L10G7037grT1aVuu5SV1+1ks1Hcti+JgSiRUY1USwr1NbZmZZpeQ/krk5UEyJ2WhJEaahGkVzwjI8fMXwfw9fT7jwD8nEn8Qmpo2hfqkuhwiOhwiJP3bePiA6fw2qktHGyenw+FT5xcx5nTA/Ru3sXo2uvO83dLkwbOvl6dtC/b7GdDgsz0rtLAVeRTZ82qEt3FrNn1hvLXMnACGUsGjosXDDCo545emgkswUhCyO6Phrj6xdsYjg4wPVwYs73BCKP1EYa3Rh1K5wMd16vt2lNm9FY5QlBNAeugYUDneddZZ9PNxwYN/czeQaft2RXJaSB3h0ZOjo1XtZyGuB7r1uvw9QmGr08AHAK4szif/vmCrNwu2rWL+mkVA1EbHlToZa6TWJ31DMeG2rT8xtnWlLPzZ1ezLEavbgwcUOsGYJ23y3C2yMotcqqYp71Y2VtGWPJpm46rNsWlL7IAlseF867qLzuLtsFVfcjwysiplhlcpNM0to6NNjFdu3aRjk8ovZKAVoOzKn+F4VGu5anu2Lp39Ozamq5B1UBLH4J0teqjBp1OV02GubbT92htDYMTp+Zn49EQ4703tGXsrW+gv3Nyfjw9OsRkf087vq/o6r5LZ54zdLx9xWODghiVu84000X+uvm2UH7tpaSaDatTI9dEh5g/Sk8ARYxoLcJgZ3O+yjk5iDE5YDCzcuGTGYjWexic2FhIS2NMj2IwA4izCfjQvTO3yai6fLXX5WLKZEdO0mwMyQI9UfnYZvuHyVqn6zXiWf51sZXLWP7iNpRMAlK91yygl46HGcXFcp3FcxpMMTgxxNrpA+w88iqitQi9tY359XjEGO9NgIgRDaYgYiDKGAcmcEzgKYHHPUQbEQY7vUX8yRjxeISjV05i/+pZTA8HmB7UfAuQIVUNsr8zRLQxxomHb2Ht7H4j+cejHvaeO4/x3gYmu+vgaU8dqW2ySlF0HC4auOJ1QXRZVioq+29VQjYLgKo4RQMjk6sCHQMv0us8HiH/84WSG1JlXSrw2sjZzNUpYvQ2xhicPsDOQ68h6se56/EowuBgDVE/RrQ+AUWcG/VwDHAcgccRpsM+orUpeptj4ajo6NWT4EmEqW0BLaFCTeen51P0t8bYvHgHWxfvGKSq33qmh30c/fgU4nEPk7traF0BOmR7n6JYJOno2aTmVO3/sqHKS1Y3E1UcR0NK1RRTpVflVpIaBg7w3MjJqNOoqMforScjOYoYoMJKKCEZ3fVj9DAB9bKjvJqZO6L2Bg5hOSpS9KTcxtSek6M8VNFI08kanShNnRFNIQyTZhau1+BEBrRuO7KMu5RGrhbEoH6c3uFZrDhiUASgF+ensp50dFsxOGvPs9/TjsAFez+/7Em5W8dkCpEi6s/O1t10loALYVyv11nZKws9usSrLSStQAD1YlAUVzeUKDGGpZFcK5hlZBQ6JvA0ykUqqYGRrElOj4l1s6xX0U1B1J+1tehwO4fLdEVlEQ4i626Ua4hOjFxrtkIAEUBR+jc7psJ1yoTLXm+tz5tlZLQWHVPiFWYqVER2h2h6PY6sKqvL+q1GNlRVBOXMIS8+Z4vmFjmKL0gqkmu2O2l8VbnVpxfXZ86DjH7mH7IZRJUcWrnq0YmRM62zgt6aE6R2BqYJKJuO1lmTXONxD9NhHzyNEoM3Z+Z5SbzL8aiPeNSz6mFejP+EShFLJgua3SFDme/zz+I2FFWOLLugkUaNUZJRfRTX0gSnhdEka5fZaTZnvhdE1BBGGViKh9PVcimcDKQEyimdInE4fUwlNHF/Fc9y7lh7xJBOVxMDVxzNpSOTdLoaTyJw3WFEy2QNU/6EJBxEWk6uZn1SBjZTTp32ZVsN1gu4FnEVNmmhz7wSJHZVnBApwglo3ciphWuoUwmSFebkPHvzcahodC83/aIjyQVORnJxOpLLb2aehUmMYDKS65fDaBajqynrXFounlCEE12l0pk8pkPqmh4I0zUv6zUyQM8ga5SfhAdya6hsx6pwAlo3crrClfWl91Zf/ZQ5fyjPuCbm41DdhV4beBqBJ8lIjrPzh6xa5o6HqNxRNAVZBOvI3MnktCyP1lqS3cBcG1Mvd22vuGl5DNb4lGnJ4lmUycPpakJ5COvq5T6iRQNJxh3Bgm+uiMc9TEeZkVxxGsAEjtOR3LBfXpMzFskDhWapM+0r3Bf9dbB0Q0kvdUauDpuNt0YuS93y2i3vdwcJvtUhV76YwJPEwM1+1TXvZE0MH8dRutUkI4PvP0FWA602ULgv5vaRSyYGq4poeW2mj6pZqTQdN9IIWQojVxfV1GlF+y0WY+AZVFqTK9ktRupdTcNln4pbsV3Bls5OcSTJxGBVERoyi8JrOR2MUilzLIzcgrI3UarZlbgll6eaPI0W3lWRx5lp4XyYRos0OPO3LCg86tbOThbE89whY4q2nEXvvFHkfDylwbNUnndGrvGpZdUanOrWrszcVLr2m/xinxwttpHkRKLMSK632EsnCOolijoUVr9pNZAgnqgdKbyNIueaHnInHBumIzuq9CZXlJukkavF0FoalylPUdxWn11lMI4a/iWECBPE8Qg8HePgKAb1YnUkCw5GUxzGI0yYMIE/rxqa8gj9eISD0RR8mC87x4SD4RTj4RQ8jDEYxOhFeR1NjmLEI8LhMEY8JvSOYvQLb3KZDmMcTicYxiOMeAD26TUkkt0Jlf3O2HinKcriCVziDOQfsBdmqyuI3AlHhulU5s7IP+CvcvXn9uUYKFURVJhabppcbeVaNXJ3sIcvJ7841xg0YtBPYkR3JxjcOGps9DE92sPk7svJ1M+jAXG0F4MOYvT/aohobVK6Pj3cBU8iRFeTN6wUn+GdPfY1Hd4CmNC7Oi7dKDgmTHb3EE964GnPrymYoEe4bwLVKUq7OFVdVLOIqkokuV5rW15FZHW6bjUuHVVqejhaNXJjTPBj3Go2EwYwSv/q/b6zghGAZl5KWYtJ+icdMKc/maj7Bvdd2QWffpssg0n/mnWUmpt0dZIklE+aZqvvdTcbzwnlSQtR9DU1oDK7RA0y92cIEgi0zcz3ojFdMkhSuMYkDesCR0NpbZkNdxIpxdOsB1uCkauAiDAYDLC9vY319fXctcFggJ2dHayvr4OIQCu2veI4UOkhze7/qopb9C4KRmuVCbmghXSL5Sg9CVOxT7C4RUmcaHNuuOP30kxN1tbWsL29jUceeQSPPfYYLl++jG984xvz629729vw+OOP48qVK7h06RIODw9xeHjYocSBEhmDM/+aOUeZYEBhakaS2VPR51Bc/2tyPdB0juhwTlnyfkrcoVW+By7Gy+xikuk5l4xlecJITkIURej1etjc3MSZM2ewvb2dG61lz/d6PURRUKV3iDyZkk4i2kpC5VPanaxiF4k9ph3coYU1Lo9kLa94Uipi0WgK0tQljOQkDIdDTCYTfPvb38aLL76I/f19jEaj+fXvfe97uHbtGvb397G7u4s4bmarSsAOPf9jQpUBlE1VK/tbZsHeLL7T5XwNxPmJzuZ0pRCzKr7oZPaaSb3pEoycBGbGZDLB7u4udnfLLsa9vT3s7S3/j0yvLlXdod5PASljKjyo8vhtr+uK81MaZoWYdUqh3iIjejaiOscwxwqsJCUjJvMAVMy/Kh0T2nJYoMin9lNBHZWjMlvN+infnNSSaY3kiOgeAJ8C8I/SbP8tgOcBfB7AAwCuAvggM9/WSS8QyOKqfcUARlOPnr4IeIHudPWTAP4nM3+AiNYAbAH4HQDPMPPvEtFTAJ4C8NsNyRlYbZy0r6PpFM/dCUsIgTxKI0dEpwD8cwC/BgDMPAIwIqL3A3hXGuwzAL6OYOQChrhsXzEzDiblkVzby/nmdC1h1/lXU1c6nTW5BwHcAvCHRPQdIvoUEW0DOMfMr6RhbgA4V0OOwPGl8fZVt/tK15Kc7Q/J79hr+llgLn1zZOAkgtctT13pdIxcH8DPAvhvzPx2JA9sPpUNwDz7jSeBgERPEtElIrpUU9bAauKsfU2OYlkwAfpdT7WXyx2kmWw91wMJvjmhag+iNu7rT8fIXQdwnZmfTY+/gKRRvkpEFwAg/bwpFIX5aWZ+BzO/Q1uqwHHCWfvqb2Re8qmkma7XDirZ/Zl6muvNff0pjRwz3wDwEhH9dHrq3QB+AODLAJ5Izz0B4EvauQYCKe21r8JDk4a9zx+zsVwY643n/2VP1ELXu/rvAXw29Xz9CMC/QWIg/4iIPgrgGoAP1pYmcFxpqH0JnmuYPUDZotXye1lfTavyU/ofZ77XlETLyDHzdwGIppvvNs4xECjQXPtadAi9PtNMd15mAwc0Kb/glS2ie1JNScITD4FjQdWzk4pQgcagykPJKWOCkQsEAkuNatWOuPT2u+YgoltItgi81lqmZrwJ/soGLLd89zPzm5vMfAnaF+B3HfosG2DZvlo1cgBARJd83U7is2xAkG9ZZKjCZ/l8lg2wly9MVwOBwEoTjFwgEFhpujByT3eQpy4+ywYE+XTwQYYqfJbPZ9kAS/laX5MLBAKBNgnT1UAgsNK0ZuSI6JeI6HkieiF9CWKnENG9RPQ1IvoBEV0moo+l588Q0V8Q0Q/Tz9MdythLXz/0lfT4QSJ6NtXh59PHoLqS7R4i+gIRPUdEV4joF7rUXWhfVjIei/bVipEjoh6A/wrgXwB4FMCHiejRNvKuYALgN5n5UQCPAfj1VKankLyR9mEAz6Dw2p+W+RiAK5nj3wPwB8z8EIDbAD7aiVQJs7f5/kMAP4NEzk50F9qXNcejfTFz438AfgHAn2eOPw7g423kbSDjlwC8B8lvC1xIz10A8HxH8lxMK/IXAXwFyRMurwHoi3TasmynAPwd0jXdzPlOdBfaV2hfVX9tTVffAuClzPH19JwXENEDAN4O4Fn488bjTwD4LSS/zwIAZwHcYeZJetylDn17W3RoX+Z8AsekfR17xwMR7QD4YwC/wcy5H1jl5JbRuvuZiH4FwE1m/lbbeWtS622+x4nQvqxw2r7aMnIvA7g3c3wxPdcpRDRA0gA/y8x/kp7WeiNtw7wTwPuI6CqAzyGZUnwSwD1ENHs9Vpc6rPU23wYI7cuMY9W+2jJy3wTwcOq9WQPwISRvfu0MIiIAnwZwhZl/P3Op8zceM/PHmfkiMz+ARFdfZeaPAPgagA90KVsqn29viw7ty4Bj175aXEx8L4C/BfAigP/QxYJmQZ7HkQx3vw/gu+nfe5GsTTwD4IcA/hLAmY7lfBeAr6TffwrAXwN4AcD/ALDeoVz/GMClVH9fBHC6S92F9hXal+wvPPEQCARWmmPveAgEAqtNMHKBQGClCUYuEAisNMHIBQKBlSYYuUAgsNIEIxcIBFaaYOQCgcBKE4xcIBBYaf4/bV59SsKqNxoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from torchrl.envs import ParallelEnv, TransformedEnv, ToTensorImage, Resize, Compose\n",
    "from torchrl.envs.libs.gym import GymEnv\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "def env_make(env_name):\n",
    "    env = TransformedEnv(GymEnv(env_name, from_pixels=True, pixels_only=True), Compose(ToTensorImage(), Resize(64, 64)))\n",
    "    return env\n",
    "\n",
    "parallel_env = ParallelEnv(2, [env_make, env_make], [{\"env_name\": \"ALE/AirRaid-v5\"}, {\"env_name\": \"ALE/Pong-v5\"}])\n",
    "tensordict = parallel_env.reset()\n",
    "\n",
    "plt.figure(figsize=(5, 10))\n",
    "plt.subplot(121)\n",
    "plt.imshow(tensordict[0].get(\"pixels\").permute(1, 2, 0).numpy())\n",
    "plt.subplot(122)\n",
    "plt.imshow(tensordict[1].get(\"pixels\").permute(1, 2, 0).numpy())\n",
    "parallel_env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b7d913e-4456-4ba5-84c1-eb78f0d58933",
   "metadata": {},
   "source": [
    "## Transforming parallel environments\n",
    "\n",
    "There are two equivalent ways of transforming parallen environments: in each process separately, or on the main process. It is even possible to do both. One can therefore think carefully about the transform design to leverage the device capabilities (e.g. transforms on cuda devices) and vectorizing operations on the main process if possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ccd43ff0-b866-4d21-8f7d-4f5e53a051ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]\n",
      "A.L.E: Arcade Learning Environment (version 0.8.0+919230b)\n",
      "[Powered by Stella]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "grayscale tensordict:  LazyStackedTensorDict(\n",
      "    fields={\n",
      "        done: Tensor(torch.Size([2, 1]), dtype=torch.bool),\n",
      "        pixels: Tensor(torch.Size([2, 1, 64, 64]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([2]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAChCAYAAAC8o8hrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAASz0lEQVR4nO3da4wdd3nH8e8zM+e2F++u7cQxtsG5OSSQ0lQRhQalCIoKIQWEUgRBVaARfgNtQFSQUPVF1b6ANxBetJVcaBsqqkC5JYpQI+okSAgRSJoIkjiOTRJjx/Elsdfe27nMzNMX59jZtffq3TmXOb+PdLR75sw585zZZ3/7nzOXNXdHRCSvgk4XICKSJYWciOSaQk5Eck0hJyK5ppATkVxTyIlIrq0q5MzsvWa218z2m9mda1WUCKi/ZG3YhR4nZ2Yh8BzwHuAQ8CvgY+7+zNqVJ/1K/SVrZTUjubcC+939eXevA/cCH1ybskTUX7I2olU8dwtwcNb9Q8AfLvaEopW8zOAqFim9aoKTr7j7RSt4yor7KyoPemlo/YWUt2JJGdaPTDAQ1BkJEgybd74X60NMv1LBcnZiURoBBm7NrwsJGmBp9vVMv3powf5aTcgti5ntBHYClIMh/mj0lqwXKV3owRP/eiCL153dX8XBMa7+s89lsZjznNoBt978U64beJH3D0wSk3A8qRECoRkDFjIUlPnUwRt4YtfvEcRtKas9DGpjRlyGtAgeOpbOTToPmtMqR51oOvuSHv+Pzy/YX6sJuZeAbbPub21Nm8PddwG7AIbWb/NTf7JjFYuUnvXdFT9jxf01uHFbx8ZLT9Wdfz76XkphzKbiad5cOcSHBsc7VU7bBTUIEvCgObpLi7boCK+dVhNyvwKuNLNLaTbfR4FbF3uCGySl8995GnbJ2pBusuL+6qTjyTBPHn8dxShhy9AQA2EN+iXk3AiS5qZpGoIFkDq9H3LuHpvZZ4AHgRD4N3d/erHnBIlTGp+7ge4R1IZDXEfsySwX0l+d9LPJHcQPbaRahqObN3LsqmH+euzZTpfVNtE0RNNOUgQPjaRA1/xOr+ozOXf/MfDjFT3pnHR365K47zKNISOuzJ1mDoVJJ6x2pqZ2u6D+6pCZpEBh0rHEqE8HzDQKnS6pvdLmDgZLuyfczsh8x8NsaWRUR8PzpnfbSuk4g/FrY66+6tCcyakbz//i9Yzu7VBdIj2orSEHCrRlK6ZsqkwQWUpgKakH1NKQ357/N0JEFqHI6VaJMdEoEXtAwVJiD6gmBcjZ8VYiWWv7SE6WwSGYDNn36kVUig2iIKWehDTikKhqKOm6z5bSOKcvhbSYkl5cZ9u6kwR9NIZIC5CUIC0YaUjX7FkFhVzXGnvGSPaNUQNqs3JtsKaA60Y7R5/hfbc+BUDZUoYDo2B9cnaPOfUxo+6tZGudCdEtZ3ko5LpUUIeg3iVdIucJ6sYT49s4HZdJeY6A+c9d2jt+cZsraw9Lmgf/0gBPzh+2Ga/tbe00hZzIBagcc/buvpw9hcu4v7jwKbWF08ZQkr8/VtF081CmpXYkBo321LMYhZzkRhA7lRNJW5aVTBnRdICHrZPVFxBVndKppCtGNGspjey1E/QXESSOtedHsiCFnORGcHqGgZ/8uk0LC+DMgezBIsOZNIWkw7/lWVnsfZ+Rdj7d2xpylkBpYu6bTkOIK8GSfxFEluLupNU+OR1Elq2tIRfONBj+zfE503ywzPjV60iKSjkRWXvt3VwNAnywPGdSUil01TE1IpIvbQ25uBJy8k3r5k40XWpJRLLT3pGctfbKiGTAChHRRZd0ugzphMMLP6S9q5IbyVCJkzdu73QZ0gn3LvyQQk5ywwO0A0vOo5CTXNGlvORcagkRyTWFnIjkmkJORHJNISciuaYdD5IrlkJpIsHNqA8F2hEhGslJvkQ1Z2jPCYb3niSI83cdN1k5jeQkV9IQ4vWDrUvTdroa6QYKOcmVpGicumJgWRd0lP6gkJPc0edwMpvaQURyTSEnIrmmkBORXFPIiUiuKeREJNeWDDkz22ZmD5vZM2b2tJnd0Zq+3sx+Ymb7Wl/Hsi9X8kb9JVlbzkguBj7v7tcAbwM+bWbXAHcCu939SmB3677ISqm/JFNLhpy7v+zu/9f6fgLYA2wBPgjc05rtHuBDGdUoOab+kqyt6DM5M9sOXAc8Cmxy95dbDx0BNq1tadJv1F+ShWWHnJkNAd8HPuvup2c/5u4OzHs2tJntNLPHzOyxuDq1qmIlv9RfkpVlhZyZFWg24Lfd/QetyUfNbHPr8c3Asfme6+673P16d78+Kg+uRc2SM+ovydJy9q4a8E1gj7t/ddZD9wO3tb6/Dbhv7cuTvFN/SdaWc4L+DcBfAL8xsydb074EfBn4rpndDhwAPpJJhZJ36i/J1JIh5+4/Y+Erc717bcuRfqP+kqzpjAcRyTWFnIjkmkJORHJNISciuaaQE5FcU8iJSK4p5EQk1xRyIpJrCjkRyTWFnIjkmkJORHJtOSfoi4hkxgNoDBkegiVgKUQzjiVr8/oKORHpLIPGMCQlCOoQxBDWWbOQ0+aqiOSaQk5Eck0hJyK5ppATkVxTyIlIrinkRCTXFHIikmsKORHJNR0MLCKd5VCYgGj6tTMe1upAYFDIiUiHWQrF057Z62tzVURyTSEnIrmmkBORXFPIiUiuKeREJNcUciKSawo5Ecm1ZYecmYVm9oSZPdC6f6mZPWpm+83sO2ZWzK5MyTv1l2RlJSO5O4A9s+5/Bfiau18BnARuX+oFwmrK2J7JObeRF6sEcXYHAkrPWHV/icxnWSFnZluB9wPfaN034F3A91qz3AN8aMnXqTUIDxydcyscPkWwhqdwtJU1bx60vpcLslb9JTKf5Z7WdTfwBWC4dX8DMO7ucev+IWDLUi+SVopUr902d1oxIO3Bk8umthjTl9cpDde4eGSSgy9cxLpnI8KqE810urqeczdr0F8i81kyXszsZuCYuz9uZu9c6QLMbCewE6A4OMbklh7+aOXMaM2htj7luisPcP3o7/jT4d/wN8Gfc+zAFiw1qPrZ+WRxa91fIudazhjqBuADZnYTUAbWAV8HRs0sav213Qq8NN+T3X0XsAtgcOO2nv21r603pjc50YxRPA2lE8ZTP7+CJ1+3jZ+//jIOvLSRgRSmL3FOXZ1QeSli6GDPvt12Un9Jppb8TM7d73L3re6+Hfgo8JC7fxx4GLilNdttwH2ZVdkF4gokm2vUNiQ0hiCagZG9UPptmT0HL8FOFJrzjaRcsv1V6mNphyvuDeovydpqPg37InCvmf0j8ATwzbUpqTsVx4Fny0xfXueGm59i7/jFHHp5PaXBOpcMT1HeHDNYqPP0wc2c/OUmBsdB26ur0lf9JdlZUci5+yPAI63vnwfeuvYldafClFOYgpmrU/5u84M8NHIZ91fecvbxa0cOc+PQs3zm2K0MP69wuxD93F+SnR7cr9lZlT1l/njmc1g1pDD52nEjvy5dzn9W3kH55RCN4ES6h0JuhQYPO4OH51ttrYPmFHAiXUXnropIrvXHSO7csxFWOtha7fNFpGNyH3JxBaob7GxQRTNQftWXHVSNIaM2ytnnFyahdFIpJ9Ir8hlyBt46pzQpG40Rx4NmMHkYUBwH8+Y8C75EK8fiCnOPefOAwkTzcTszuZsyr/WePFj8/a1qEd363kXmkbuQS4tQX2dUNziFq05TLjZ4faV69vHT1TIndwwSBClRISEIHLPmDSBNA9whjkOSOKRcqfOGdRNnnz/dKDBVKzJxeJih5yOiGShMdslvukF92EgqMLmjzvDGqUwWU6sVCPYMUTgNxVNOEC/9HJFOyV/Ihc3RV2N9wk3b9zIQ1Oc8PpmUODK6jmKQsLE0ScESCrP+k21CQC2NmIpLnKxXGClU2VQ6fd5yHuBNNI6Mdd0veFKCeAC2bD3BjZv2Z7KM4/VhHjp8LUE9gIml5xfppNyF3FIKljBSqFIImuEW2PmjsIIlVMI6FGl+FZGe1XchFwUplbBOYD5vwIWkYEFzdBfW54zyRKT39F3IhaRUwsbZ7xeahwCiBR7vZbU0IiE4O4qdbx0kBMRp8xDKKEgXXE8ivaDvQg4WDreVztOLmgEWEgbpvCPZhIDUjYaHpB4QBbUOVCmydvoy5PrZRKNMLY0YLQA0IJgb6GcCbiIu00hDSkGDMMhn4Et/0GldfaaWRlSTiIYHJPP8+FM3UjdmkgL1NJx3HpFeog7uIwkB03GBU/UKM0mBOA3Pm6fhIbU0YrJR4lStMu88Ir1EIddn6mlELY5IPSClOWo7V+oBjSSkroCTHNBncn1mqlFkql6gmkRUwhDOybHYQ2IPmGwUqcfNzVqRXqYO7jNJ2hylxWk47ygOIHFrzpMEpAo56XEayfWR1I2ZRoFqtTmSO7u5OivrGmlIPY2YqRdoNJqjOpFepg7uM3EStC5CYCTnjOTOHCOXupEkAWkSnDePSK/RSK7P1OOQpBpRSyLieQ4RSTHiNKBRj0jqofauSs9rb8g5hPVsL0vkoRHWIJgO+N3UGOUwm8uETE+VKdcgqEPYWP5FODNlENaMtAAnpyq8OL1hzsP1NGRmooxNhrwyOQhANYnOrqMU49XqIJP1IvFkAasHHJ4aOW8xJ2oDhNXWeq5n/zMVWY22hlw0UWP9Iy9mu5AwhGIBr5SY+OFWJjLaIL9isk5wahxrxFBvZLOQC1Eq4lFIsnuI45Xtcx9LYcdkFWskpAMFPBphIhybs44sdoZTZ3R6EtxJhkY4Xhib+zqJc8Wrr2C1RvO9J7qIgXSvtoacN2Lil4+0bXlZfuDoQLf/as/3/p3XBp1n/r/YfNIl5un29y5yhnY8iEiuacfDQsywMCQYGMBG1uHT0yQnTp59OBgaIhgdwaenSU+dxpMEXJ9NiXQbjeQWEI6OEuy4jEOfejOX//Ao+75wFeHoKOHwMOHwMC9/4lqu/NERnrtrB8FVlxNturjTJYvIPBRyCwkDvBhRH3E+PPY48YYGhEFzx0YYUl8HHx57jGRDg7QYQaRBsUg30m/mAtLxU9hMlcu+dTH/8NNPcvWRCdLxU3ja3CS99L9e4u9/cTtvPDoFLxwk6aY9rCJylkJuAR7HeBzD/heI9r9w3t7E+IUDRC8cyOn1g0XyQ5urIpJryxrJmdko8A3gzTQPs/pLYC/wHWA78CLwEXc/Of8riCxsrfrLUoiq2sMtcy13c/XrwP+4+y1mVgQGgC8Bu939y2Z2J3An8MWM6pR8W5P+CieqjO7el3210lOWDDkzGwFuBD4B4O51oG5mHwTe2ZrtHuARFHKyQmvZXx4nJK+8mlWp0qOW85ncpcBx4N/N7Akz+4aZDQKb3P3l1jxHgE1ZFSm5pv6STC0n5CLgD4B/cffrgCmamw5nufvsUyLnMLOdZvaYmT3WQP/DU86j/pJMLSfkDgGH3P3R1v3v0WzKo2a2GaD19dh8T3b3Xe5+vbtfX6C0FjVLvqi/JFNLhpy7HwEOmtlVrUnvBp4B7gdua027Dbgvkwol19RfkrXl7l39K+DbrT1fzwOfpBmQ3zWz24EDwEeyKVH6gPpLMrOskHP3J4Hr53no3WtajfQl9ZdkSWc8iEiuKeREJNfM23ihRzM7TvMQgVfattCV2Uj31ga9Xd8b3P2iLBfeA/0F3f0z7Oba4AL7q60hB2Bmj7n7fJ+/dFw31waqr1dqWEw319fNtcGF16fNVRHJNYWciORaJ0JuVweWuVzdXBuovuXohhoW0831dXNtcIH1tf0zORGRdtLmqojkWttCzszea2Z7zWx/6yKIHWVm28zsYTN7xsyeNrM7WtPXm9lPzGxf6+tYB2sMW5cfeqB1/1Ize7S1Dr/TOg2qU7WNmtn3zOxZM9tjZm/v5LpTf11QjX3RX20JOTMLgX8C3gdcA3zMzK5px7IXEQOfd/drgLcBn27VdCfNK9JeCezmnMv+tNkdwJ5Z978CfM3drwBOArd3pKqmM1fzfSPwFpp1dmTdqb8uWH/0l7tnfgPeDjw46/5dwF3tWPYKarwPeA/N/y2wuTVtM7C3Q/Vsbf0g3wU8ABjNAyGj+dZpm2sbAV6g9ZnurOkdWXfqL/XXYrd2ba5uAQ7Oun+oNa0rmNl24DrgUbrnirR3A1+As//1cAMw7u5x634n12G3Xc1X/bVyd9Mn/dX3Ox7MbAj4PvBZdz89+zFv/slo++5nM7sZOObuj7d72cu0qqv59hP11wVZ0/5qV8i9BGybdX9ra1pHmVmBZgN+291/0Jq8rCvSZuwG4ANm9iJwL81Niq8Do2Z25vJYnVyHq7qabwbUXyvTV/3VrpD7FXBla+9NEfgozSu/doyZGfBNYI+7f3XWQx2/Iq273+XuW919O8119ZC7fxx4GLilk7W16uu2q/mqv1ag7/qrjR8m3gQ8B/wW+NtOfKB5Tj3voDnc/TXwZOt2E83PJnYD+4D/BdZ3uM53Ag+0vr8M+CWwH/hvoNTBun4feKy1/n4EjHVy3am/1F8L3XTGg4jkWt/veBCRfFPIiUiuKeREJNcUciKSawo5Eck1hZyI5JpCTkRyTSEnIrn2/2o103OBWxCiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from torchrl.envs import ParallelEnv, TransformedEnv, ToTensorImage, Resize, Compose, GrayScale\n",
    "from torchrl.envs.libs.gym import GymEnv\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "def env_make(env_name):\n",
    "    env = TransformedEnv(GymEnv(env_name, from_pixels=True, pixels_only=True), Compose(ToTensorImage(), Resize(64, 64)))  # transforms on remote processes\n",
    "    return env\n",
    "\n",
    "parallel_env = ParallelEnv(2, [env_make, env_make], [{\"env_name\": \"ALE/AirRaid-v5\"}, {\"env_name\": \"ALE/Pong-v5\"}])\n",
    "parallel_env = TransformedEnv(parallel_env, GrayScale())  # transforms on main process\n",
    "tensordict = parallel_env.reset()\n",
    "print(\"grayscale tensordict: \", tensordict)\n",
    "plt.figure(figsize=(5, 10))\n",
    "plt.subplot(121)\n",
    "plt.imshow(tensordict[0].get(\"pixels\").permute(1, 2, 0).numpy())\n",
    "plt.subplot(122)\n",
    "plt.imshow(tensordict[1].get(\"pixels\").permute(1, 2, 0).numpy())\n",
    "parallel_env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d66e276d-bd61-431e-852e-55198595fe34",
   "metadata": {},
   "source": [
    "## VecNorm\n",
    "\n",
    "In RL, we commonly face the problem of normalizing data before inputting them into a model. \n",
    "Sometimes, we can get a good approximation of the normalizing statistics from data gathered in the environment with, say, a random policy (or demonstrations). It might, however, be advisable to normalize the data \"on-the-fly\", updating the normalizing constants progressively to what has been observed so far. This is particularily useful when we expect the normalizing statistics to change following changes in performance in the task, or when the environment is evolving due to external factors.\n",
    "\n",
    "**Caution**: this feature should be used with caution with off-policy learning, as old data will be \"deprecated\" due to its normalization with previously valid normalizing statistics. In on-policy settings too, this feature makes learning non-steady and may have unexpected effects. One would therefore advice users to rely on this feature with caution and compare it with data normalizing given a fixed version of the normalizing constants.\n",
    "\n",
    "In regular setting, using VecNorm is quite easy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "10d665f6-daf8-41bb-9767-5316387db737",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "mean: : tensor([-0.2824, -0.3740, -0.1690])\n",
      "std: : tensor([0.9514, 1.0710, 1.1238])\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs.libs.gym import GymEnv\n",
    "from torchrl.envs.transforms import VecNorm, TransformedEnv\n",
    "\n",
    "env = TransformedEnv(GymEnv(\"Pendulum-v1\"), VecNorm())\n",
    "tensordict = env.rollout(max_steps=100)\n",
    "\n",
    "print(\"mean: :\", tensordict.get(\"observation\").mean(0))  # Approx 0\n",
    "print(\"std: :\", tensordict.get(\"observation\").std(0))  # Approx 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34c31e5c-82fc-4795-bb74-917ea5babc7e",
   "metadata": {},
   "source": [
    "In **parallel envs** things are slightly more complicated, as we need to share the running statistics amongst the processes. We created a class `EnvCreator` that is responsible for looking at an environment creation method, retrieving tensordicts to share amongst processes in the environment class, and pointing each process to the right common, shared tensordict once created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "55f3b06d-d4cf-4d5b-b0e5-eda4d46cfcd9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "tensordict:  TensorDict(\n",
      "    fields={\n",
      "        action: Tensor(torch.Size([3, 5, 2]), dtype=torch.int64),\n",
      "        done: Tensor(torch.Size([3, 5, 1]), dtype=torch.bool),\n",
      "        next_observation: Tensor(torch.Size([3, 5, 4]), dtype=torch.float32),\n",
      "        observation: Tensor(torch.Size([3, 5, 4]), dtype=torch.float32),\n",
      "        reward: Tensor(torch.Size([3, 5, 1]), dtype=torch.float32)},\n",
      "    batch_size=torch.Size([3, 5]),\n",
      "    device=cpu,\n",
      "    is_shared=False)\n",
      "mean: : tensor([ 0.1187, -0.0427, -0.1390])\n",
      "std: : tensor([1.1470, 1.1814, 1.1676])\n",
      "update counts:  tensor([18.])\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n",
      "Discarding frameskip arg. This will be taken care of by TorchRL env wrapper.\n"
     ]
    }
   ],
   "source": [
    "from torchrl.envs import EnvCreator, ParallelEnv\n",
    "from torchrl.envs.libs.gym import GymEnv\n",
    "from torchrl.envs.transforms import VecNorm, TransformedEnv\n",
    "\n",
    "make_env = EnvCreator(lambda: TransformedEnv(GymEnv(\"CartPole-v1\"), VecNorm(decay=1.0)))\n",
    "env = ParallelEnv(3, make_env)\n",
    "make_env.state_dict()['_extra_state']['td'][\"next_observation_count\"].fill_(0.0)\n",
    "make_env.state_dict()['_extra_state']['td'][\"next_observation_ssq\"].fill_(0.0)\n",
    "make_env.state_dict()['_extra_state']['td'][\"next_observation_sum\"].fill_(0.0)\n",
    "\n",
    "tensordict = env.rollout(max_steps=5)\n",
    "\n",
    "print('tensordict: ', tensordict)\n",
    "print(\"mean: :\", tensordict.get(\"observation\").view(-1, 3).mean(0))  # Approx 0\n",
    "print(\"std: :\", tensordict.get(\"observation\").view(-1, 3).std(0))  # Approx 1\n",
    "\n",
    "# The count is slightly higher than the number of steps (since we did not use any decay)\n",
    "# The difference between the two is due to the fact that ParallelEnv creates a dummy environment to initialize the shared TensorDict \n",
    "# that is used to collect data from the dispached environments. This small difference will usually be absored throughout training.\n",
    "print(\"update counts: \", make_env.state_dict()['_extra_state']['td'][\"next_observation_count\"])\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98279e92",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}