DEMO.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\">tqdm</h1>\n",
    "<img src=\"https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif\" align=\"left\" />\n",
    "\n",
    "[![Py-Versions](https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white)](https://pypi.org/project/tqdm)|[![Versions](https://img.shields.io/pypi/v/tqdm.svg)](https://tqdm.github.io/releases)|[![Conda-Forge-Status](https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/tqdm)|[![Docker](https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white)](https://hub.docker.com/r/tqdm/tqdm)|[![Snapcraft](https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft)](https://snapcraft.io/tqdm)\n",
    "-|-|-|-|-\n",
    "\n",
    "[![Build-Status](https://img.shields.io/travis/tqdm/tqdm/master.svg?logo=travis)](https://travis-ci.org/tqdm/tqdm)|[![Coverage-Status](https://coveralls.io/repos/tqdm/tqdm/badge.svg?branch=master)](https://coveralls.io/github/tqdm/tqdm)|[![Branch-Coverage-Status](https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg)](https://codecov.io/gh/tqdm/tqdm)|[![Codacy-Grade](https://api.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177)](https://www.codacy.com/app/tqdm/tqdm/dashboard)|[![Libraries-Rank](https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white)](https://libraries.io/pypi/tqdm)|[![PyPI-Downloads](https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white)](https://pypi.org/project/tqdm)\n",
    "-|-|-|-|-|-\n",
    "\n",
    "[![DOI](https://img.shields.io/badge/DOI-10.21105/joss.01277-green.svg)](https://doi.org/10.21105/joss.01277)|[![LICENCE](https://img.shields.io/pypi/l/tqdm.svg)](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[![OpenHub-Status](https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[![binder-demo](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[![notebook-demo](https://img.shields.io/badge/launch-notebook-orange.svg?logo=jupyter)](https://notebooks.ai/demo/gh/tqdm/tqdm)|[![awesome-python](https://awesome.re/mentioned-badge.svg)](https://github.com/vinta/awesome-python)\n",
    "-|-|-|-|-|-\n",
    "\n",
    "`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
    "\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
    "(*te quiero demasiado*).\n",
    "\n",
    "Instantly make your loops show a smart progress meter - just wrap any\n",
    "iterable with `tqdm(iterable)`, and you're done!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 10000/10000 [00:00<00:00, 1541060.37it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm\n",
    "for i in tqdm(range(10000)):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`trange(N)` can be also used as a convenient shortcut for\n",
    "`tqdm(xrange(N))`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "hello: 100%|██████████| 10.0k/10.0k [00:00<00:00, 1.56Mepoch/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import trange\n",
    "for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Screenshot](https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm.gif)|[![Video](https://raw.githubusercontent.com/tqdm/tqdm/master/images/video.jpg)](https://tqdm.github.io/video) [![Slides](https://raw.githubusercontent.com/tqdm/tqdm/master/images/slides.jpg)](https://tqdm.github.io/PyData2019/slides.html)\n",
    "-|-\n",
    "\n",
    "It can also be executed as a module with pipes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "75.2MB [00:00, 217MB/s]\n",
      "9999999\n"
     ]
    }
   ],
   "source": [
    "! seq 9999999 | tqdm --bytes | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```sh\n",
    "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
    " 44%|██████████████▊                   | 153M/352M [00:14<00:18, 11.0MB/s]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
    "is unit tested against performance regression. By comparison, the\n",
    "well-established\n",
    "[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
    "an 800ns/iter overhead.\n",
    "\n",
    "In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
    "the remaining time and to skip unnecessary iteration displays, which\n",
    "allows for a negligible overhead in most cases.\n",
    "\n",
    "`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
    "Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
    "IPython/Jupyter notebooks.\n",
    "\n",
    "`tqdm` does not require any dependencies (not even `curses`!), just\n",
    "Python and an environment supporting `carriage return \\r` and\n",
    "`line feed \\n` control characters.\n",
    "\n",
    "---\n",
    "\n",
    "## Usage\n",
    "\n",
    "`tqdm` is very versatile and can be used in a number of ways.\n",
    "The three main ones are given below.\n",
    "\n",
    "### Iterable-based\n",
    "\n",
    "Wrap `tqdm()` around any iterable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "from time import sleep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4/4 [00:01<00:00,  3.98it/s]\n"
     ]
    }
   ],
   "source": [
    "text = \"\"\n",
    "for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
    "    sleep(0.25)\n",
    "    text = text + char"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:01<00:00, 96.76it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import trange\n",
    "\n",
    "for i in trange(100):\n",
    "    sleep(0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instantiation outside of the loop allows for manual control over `tqdm()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Processing d: 100%|██████████| 4/4 [00:01<00:00,  3.90it/s]\n"
     ]
    }
   ],
   "source": [
    "pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
    "for char in pbar:\n",
    "    sleep(0.25)\n",
    "    pbar.set_description(\"Processing %s\" % char)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manual\n",
    "\n",
    "Manual control of `tqdm()` updates using a `with` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:01<00:00, 97.81it/s]\n"
     ]
    }
   ],
   "source": [
    "with tqdm(total=100) as pbar:\n",
    "    for i in range(10):\n",
    "        sleep(0.1)\n",
    "        pbar.update(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the optional variable `total` (or an iterable with `len()`) is\n",
    "provided, predictive stats are displayed.\n",
    "\n",
    "`with` is also optional (you can just assign `tqdm()` to a variable,\n",
    "but in this case don't forget to `del` or `close()` at the end:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:01<00:00, 97.85it/s]\n"
     ]
    }
   ],
   "source": [
    "pbar = tqdm(total=100)\n",
    "for i in range(10):\n",
    "    sleep(0.1)\n",
    "    pbar.update(10)\n",
    "pbar.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
    "command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
    "pipes will pass through all `stdin` to `stdout` while printing progress\n",
    "to `stderr`.\n",
    "\n",
    "The example below demonstrated counting the number of lines in all\n",
    "Python files in the current directory, with timing information included."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.80user 2.52system 0:08.86elapsed 93%CPU (0avgtext+0avgdata 3084maxresident)k\r\n",
      "161232inputs+0outputs (0major+913287minor)pagefaults 0swaps\r\n",
      "4104300\r\n"
     ]
    }
   ],
   "source": [
    "! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4043186it [00:05, 595767.35it/s]4.09user 1.23system 0:05.88elapsed 90%CPU (0avgtext+0avgdata 3092maxresident)k\n",
      "20128inputs+0outputs (0major+904021minor)pagefaults 0swaps\n",
      "4104300it [00:05, 698847.16it/s]\n",
      "4104300\n"
     ]
    }
   ],
   "source": [
    "! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the usual arguments for `tqdm` can also be specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████| 4.10M/4.10M [00:05<00:00, 724kloc/s]\n"
     ]
    }
   ],
   "source": [
    "! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit_scale --total 4104300 >> /dev/null"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Backing up a large directory?\n",
    "\n",
    "```sh\n",
    "tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
    " 44%|██████████████▊                   | 153M/352M [00:14<00:18, 11.0MB/s]\n",
    "```\n",
    "\n",
    "This can be beautified further:\n",
    "\n",
    "```sh\n",
    "BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
    "tar -cf - docs/ \\\n",
    "  | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
    "  | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
    "  > ~/backup.tgz\n",
    "Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
    "Compressed:  42%|█████████▎            | 148M/352M [00:14<00:19, 10.9MB/s]\n",
    "```\n",
    "\n",
    "Or done on a file level using 7-zip:\n",
    "\n",
    "```sh\n",
    "7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
    "  | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
    "  | grep -v Compressing\n",
    "100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tqdm?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "        Extra CLI Options\n",
      "        -----------------\n",
      "        name  : type, optional\n",
      "             TODO: find out why this is needed.\n",
      "        delim  : chr, optional\n",
      "            Delimiting character [default: '\\n']. Use '\\0' for null.\n",
      "            N.B.: on Windows systems, Python converts '\\n' to '\\r\\n'.\n",
      "        buf_size  : int, optional\n",
      "            String buffer size in bytes [default: 256]\n",
      "            used when `delim` is specified.\n",
      "        bytes  : bool, optional\n",
      "            If true, will count bytes, ignore `delim`, and default\n",
      "            `unit_scale` to True, `unit_divisor` to 1024, and `unit` to 'B'.\n",
      "        manpath  : str, optional\n",
      "            Directory in which to install tqdm man pages.\n",
      "        log  : str, optional\n",
      "            CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "! tqdm --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Examples and Advance Usage\n",
    "\n",
    "- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
    "  folder;\n",
    "- import the module and run `help()`;\n",
    "- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
    "    - this has an\n",
    "      [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
    "      on how to make a **great** progressbar;\n",
    "- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
    "- run this file!\n",
    "\n",
    "### Description and additional stats\n",
    "\n",
    "Custom information can be displayed and updated dynamically on `tqdm` bars\n",
    "with the `desc` and `postfix` arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "GEN 9: 100%|██████████| 10/10 [00:01<00:00,  8.87it/s, gen=980, loss=0.762, lst=[1, 2], str=h]\n",
      "Batch        4\n"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm, trange\n",
    "from random import random, randint\n",
    "from time import sleep\n",
    "\n",
    "with trange(10) as t:\n",
    "    for i in t:\n",
    "        # Description will be displayed on the left\n",
    "        t.set_description('GEN %i' % i)\n",
    "        # Postfix will be displayed on the right,\n",
    "        # formatted automatically based on argument's datatype\n",
    "        t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
    "                      lst=[1, 2])\n",
    "        sleep(0.1)\n",
    "\n",
    "with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
    "          postfix=[\"Batch\", dict(value=0)]) as t:\n",
    "    for i in range(10):\n",
    "        sleep(0.1)\n",
    "        t.postfix[1][\"value\"] = i / 2\n",
    "        t.update()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
    "\n",
    "- `postfix` also needs to be passed as an initial argument in a\n",
    "  compatible format, and\n",
    "- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
    "  object. To prevent this behaviour, insert an extra item into the\n",
    "  dictionary where the key is not a string.\n",
    "\n",
    "Additional `bar_format` parameters may also be defined by overriding\n",
    "`format_dict`, and the bar itself may be modified using `ascii`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "from time import sleep\n",
    "\n",
    "class TqdmExtraFormat(tqdm):\n",
    "    \"\"\"Provides a `total_time` format parameter\"\"\"\n",
    "    @property\n",
    "    def format_dict(self):\n",
    "        d = super(TqdmExtraFormat, self).format_dict\n",
    "        total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
    "        d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
    "        return d\n",
    "\n",
    "for i in TqdmExtraFormat(\n",
    "      range(9), ascii=\" .oO0\",\n",
    "      bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
    "    if i == 4:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `{bar}` also supports a format specifier `[width][type]`.\n",
    "\n",
    "- `width`\n",
    "    + unspecified (default): automatic to fill `ncols`\n",
    "    + `int >= 0`: fixed width overriding `ncols` logic\n",
    "    + `int < 0`: subtract from the automatic default\n",
    "- `type`\n",
    "    + `a`: ascii (`ascii=True` override)\n",
    "    + `u`: unicode (`ascii=False` override)\n",
    "    + `b`: blank (`ascii=\"  \"` override)\n",
    "\n",
    "This means a fixed bar with right-justified text may be created by\n",
    "using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nested progress bars\n",
    "\n",
    "`tqdm` supports nested progress bars. Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.auto import trange\n",
    "from time import sleep\n",
    "\n",
    "for i in trange(4, desc='1st loop'):\n",
    "    for j in trange(5, desc='2nd loop'):\n",
    "        for k in trange(50, desc='3rd loop', leave=False):\n",
    "            sleep(0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On Windows [colorama](https://github.com/tartley/colorama) will be used\n",
    "if available to keep nested bars on their respective lines.\n",
    "\n",
    "For manual control over positioning (e.g. for multi-processing use),\n",
    "you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
    "for the next, and so on. However, it's best to check if tqdm can work\n",
    "without manual position first.\n",
    "\n",
    "```python\n",
    "from time import sleep\n",
    "from tqdm import trange, tqdm\n",
    "from multiprocessing import Pool, freeze_support\n",
    "\n",
    "L = list(range(9))\n",
    "\n",
    "def progresser(n):\n",
    "    interval = 0.001 / (n + 2)\n",
    "    total = 5000\n",
    "    text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
    "    for _ in trange(total, desc=text, position=n):\n",
    "        sleep(interval)\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    freeze_support()  # for Windows support\n",
    "    p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
    "    p.map(progresser, L)\n",
    "```\n",
    "\n",
    "Note that in Python 3, `tqdm.write` is thread-safe:\n",
    "\n",
    "```python\n",
    "from time import sleep\n",
    "from tqdm import tqdm, trange\n",
    "from concurrent.futures import ThreadPoolExecutor\n",
    "\n",
    "L = list(range(9))\n",
    "\n",
    "def progresser(n):\n",
    "    interval = 0.001 / (n + 2)\n",
    "    total = 5000\n",
    "    text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
    "    for _ in trange(total, desc=text):\n",
    "        sleep(interval).auto\n",
    "    if n == 6:\n",
    "        tqdm.write(\"n == 6 completed.\")\n",
    "        tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    with ThreadPoolExecutor() as p:\n",
    "        p.map(progresser, L)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hooks and callbacks\n",
    "\n",
    "`tqdm` can easily support callbacks/hooks and manual updates.\n",
    "Here's an example with `urllib`:\n",
    "\n",
    "**`urllib.urlretrieve` documentation**\n",
    "\n",
    "> [...]\n",
    "> If present, the hook function will be called once\n",
    "> on establishment of the network connection and once after each block read\n",
    "> thereafter. The hook will be passed three arguments; a count of blocks\n",
    "> transferred so far, a block size in bytes, and the total size of the file.\n",
    "> [...]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "matryoshka.zip: 100%|██████████| 262k/262k [00:02<00:00, 87.9kB/s] \n"
     ]
    }
   ],
   "source": [
    "import urllib, os\n",
    "from tqdm import tqdm\n",
    "\n",
    "class TqdmUpTo(tqdm):\n",
    "    \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
    "    def update_to(self, b=1, bsize=1, tsize=None):\n",
    "        \"\"\"\n",
    "        b  : int, optional\n",
    "            Number of blocks transferred so far [default: 1].\n",
    "        bsize  : int, optional\n",
    "            Size of each block (in tqdm units) [default: 1].\n",
    "        tsize  : int, optional\n",
    "            Total size (in tqdm units). If [default: None] remains unchanged.\n",
    "        \"\"\"\n",
    "        if tsize is not None:\n",
    "            self.total = tsize\n",
    "        self.update(b * bsize - self.n)  # will also set self.n = b * bsize\n",
    "\n",
    "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
    "with TqdmUpTo(unit='B', unit_scale=True, miniters=1,\n",
    "              desc=eg_link.split('/')[-1]) as t:  # all optional kwargs\n",
    "    urllib.urlretrieve(eg_link, filename=os.devnull,\n",
    "                       reporthook=t.update_to, data=None)\n",
    "    t.total = t.n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
    "Functional alternative in\n",
    "[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
    "\n",
    "It is recommend to use `miniters=1` whenever there is potentially large\n",
    "differences in iteration speed (e.g. downloading a file over a patchy\n",
    "connection).\n",
    "\n",
    "**Wrapping read/write methods**\n",
    "\n",
    "To measure throughput through a file-like object's `read` or `write`\n",
    "methods, use `CallbackIOWrapper`:\n",
    "\n",
    "```python\n",
    "from tqdm import tqdm\n",
    "from tqdm.utils import CallbackIOWrapper\n",
    "\n",
    "with tqdm(total=file_obj.size,\n",
    "          unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
    "    fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
    "    while True:\n",
    "        chunk = fobj.read(chunk_size)\n",
    "        if not chunk:\n",
    "            break\n",
    "    t.reset()\n",
    "    # ... continue to use `t` for something else\n",
    "```\n",
    "\n",
    "Alternatively, use the even simpler `wrapattr` convenience function,\n",
    "which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
    "down to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "matryoshka.zip: 254kB [00:00, 334kB/s]  \n"
     ]
    }
   ],
   "source": [
    "import urllib, os\n",
    "from tqdm import tqdm\n",
    "\n",
    "eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
    "with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
    "                   miniters=1, desc=eg_link.split('/')[-1]) as fout:\n",
    "    for chunk in urllib.urlopen(eg_link):\n",
    "        fout.write(chunk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pandas Integration\n",
    "\n",
    "Due to popular demand we've added support for `pandas` -- here's an example\n",
    "for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "my bar!: 100%|██████████| 6/6 [00:00<00:00, 193.96it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1296</td>\n",
       "      <td>1156</td>\n",
       "      <td>961</td>\n",
       "      <td>576</td>\n",
       "      <td>6889</td>\n",
       "      <td>7921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>225</td>\n",
       "      <td>3844</td>\n",
       "      <td>8649</td>\n",
       "      <td>121</td>\n",
       "      <td>8281</td>\n",
       "      <td>4624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3025</td>\n",
       "      <td>729</td>\n",
       "      <td>9025</td>\n",
       "      <td>4225</td>\n",
       "      <td>1024</td>\n",
       "      <td>8836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>25</td>\n",
       "      <td>729</td>\n",
       "      <td>1936</td>\n",
       "      <td>6724</td>\n",
       "      <td>4900</td>\n",
       "      <td>3721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7225</td>\n",
       "      <td>1156</td>\n",
       "      <td>64</td>\n",
       "      <td>9604</td>\n",
       "      <td>441</td>\n",
       "      <td>961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5184</td>\n",
       "      <td>441</td>\n",
       "      <td>2500</td>\n",
       "      <td>5041</td>\n",
       "      <td>1936</td>\n",
       "      <td>5041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>289</td>\n",
       "      <td>8281</td>\n",
       "      <td>8464</td>\n",
       "      <td>1089</td>\n",
       "      <td>7569</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7225</td>\n",
       "      <td>1600</td>\n",
       "      <td>5625</td>\n",
       "      <td>5625</td>\n",
       "      <td>3364</td>\n",
       "      <td>7396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3025</td>\n",
       "      <td>4489</td>\n",
       "      <td>2704</td>\n",
       "      <td>400</td>\n",
       "      <td>6724</td>\n",
       "      <td>3364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>6400</td>\n",
       "      <td>7396</td>\n",
       "      <td>4225</td>\n",
       "      <td>49</td>\n",
       "      <td>7921</td>\n",
       "      <td>4761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>64</td>\n",
       "      <td>256</td>\n",
       "      <td>4356</td>\n",
       "      <td>8100</td>\n",
       "      <td>4761</td>\n",
       "      <td>4761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>3600</td>\n",
       "      <td>729</td>\n",
       "      <td>4624</td>\n",
       "      <td>2500</td>\n",
       "      <td>25</td>\n",
       "      <td>441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>5476</td>\n",
       "      <td>5329</td>\n",
       "      <td>6084</td>\n",
       "      <td>3025</td>\n",
       "      <td>3844</td>\n",
       "      <td>7396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>16</td>\n",
       "      <td>6889</td>\n",
       "      <td>400</td>\n",
       "      <td>484</td>\n",
       "      <td>1225</td>\n",
       "      <td>169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>3025</td>\n",
       "      <td>2209</td>\n",
       "      <td>9801</td>\n",
       "      <td>324</td>\n",
       "      <td>4900</td>\n",
       "      <td>4096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>8649</td>\n",
       "      <td>6400</td>\n",
       "      <td>1936</td>\n",
       "      <td>3600</td>\n",
       "      <td>4900</td>\n",
       "      <td>2116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>4356</td>\n",
       "      <td>3481</td>\n",
       "      <td>5476</td>\n",
       "      <td>7569</td>\n",
       "      <td>9216</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>961</td>\n",
       "      <td>196</td>\n",
       "      <td>4096</td>\n",
       "      <td>5476</td>\n",
       "      <td>8649</td>\n",
       "      <td>6724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1</td>\n",
       "      <td>4900</td>\n",
       "      <td>1156</td>\n",
       "      <td>2025</td>\n",
       "      <td>2916</td>\n",
       "      <td>1521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>6889</td>\n",
       "      <td>2809</td>\n",
       "      <td>9409</td>\n",
       "      <td>3249</td>\n",
       "      <td>7569</td>\n",
       "      <td>784</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>2116</td>\n",
       "      <td>49</td>\n",
       "      <td>225</td>\n",
       "      <td>81</td>\n",
       "      <td>8649</td>\n",
       "      <td>289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>7569</td>\n",
       "      <td>2500</td>\n",
       "      <td>25</td>\n",
       "      <td>441</td>\n",
       "      <td>9025</td>\n",
       "      <td>6561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>4489</td>\n",
       "      <td>3136</td>\n",
       "      <td>3025</td>\n",
       "      <td>3249</td>\n",
       "      <td>2116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>5476</td>\n",
       "      <td>2116</td>\n",
       "      <td>9025</td>\n",
       "      <td>4096</td>\n",
       "      <td>1764</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>3025</td>\n",
       "      <td>961</td>\n",
       "      <td>3249</td>\n",
       "      <td>7056</td>\n",
       "      <td>3136</td>\n",
       "      <td>676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2809</td>\n",
       "      <td>7744</td>\n",
       "      <td>1089</td>\n",
       "      <td>6724</td>\n",
       "      <td>0</td>\n",
       "      <td>961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>225</td>\n",
       "      <td>400</td>\n",
       "      <td>7396</td>\n",
       "      <td>3721</td>\n",
       "      <td>784</td>\n",
       "      <td>7744</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>81</td>\n",
       "      <td>100</td>\n",
       "      <td>400</td>\n",
       "      <td>8836</td>\n",
       "      <td>2601</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>441</td>\n",
       "      <td>144</td>\n",
       "      <td>2704</td>\n",
       "      <td>1369</td>\n",
       "      <td>289</td>\n",
       "      <td>8281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>625</td>\n",
       "      <td>3025</td>\n",
       "      <td>8836</td>\n",
       "      <td>64</td>\n",
       "      <td>3025</td>\n",
       "      <td>9409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99970</th>\n",
       "      <td>9216</td>\n",
       "      <td>8100</td>\n",
       "      <td>1369</td>\n",
       "      <td>529</td>\n",
       "      <td>625</td>\n",
       "      <td>7921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99971</th>\n",
       "      <td>4</td>\n",
       "      <td>5184</td>\n",
       "      <td>3364</td>\n",
       "      <td>4356</td>\n",
       "      <td>5625</td>\n",
       "      <td>9604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99972</th>\n",
       "      <td>484</td>\n",
       "      <td>6889</td>\n",
       "      <td>900</td>\n",
       "      <td>36</td>\n",
       "      <td>1444</td>\n",
       "      <td>3364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99973</th>\n",
       "      <td>7744</td>\n",
       "      <td>3844</td>\n",
       "      <td>4489</td>\n",
       "      <td>5776</td>\n",
       "      <td>289</td>\n",
       "      <td>1225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99974</th>\n",
       "      <td>1225</td>\n",
       "      <td>4489</td>\n",
       "      <td>9</td>\n",
       "      <td>2916</td>\n",
       "      <td>1369</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99975</th>\n",
       "      <td>1849</td>\n",
       "      <td>4356</td>\n",
       "      <td>169</td>\n",
       "      <td>2025</td>\n",
       "      <td>2704</td>\n",
       "      <td>7744</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99976</th>\n",
       "      <td>5929</td>\n",
       "      <td>7396</td>\n",
       "      <td>841</td>\n",
       "      <td>441</td>\n",
       "      <td>225</td>\n",
       "      <td>7056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99977</th>\n",
       "      <td>676</td>\n",
       "      <td>7056</td>\n",
       "      <td>529</td>\n",
       "      <td>100</td>\n",
       "      <td>16</td>\n",
       "      <td>484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99978</th>\n",
       "      <td>841</td>\n",
       "      <td>5476</td>\n",
       "      <td>8100</td>\n",
       "      <td>7569</td>\n",
       "      <td>6400</td>\n",
       "      <td>1156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99979</th>\n",
       "      <td>4</td>\n",
       "      <td>7921</td>\n",
       "      <td>1225</td>\n",
       "      <td>5476</td>\n",
       "      <td>3844</td>\n",
       "      <td>7396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99980</th>\n",
       "      <td>6241</td>\n",
       "      <td>1444</td>\n",
       "      <td>8281</td>\n",
       "      <td>1849</td>\n",
       "      <td>5625</td>\n",
       "      <td>3721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99981</th>\n",
       "      <td>2401</td>\n",
       "      <td>121</td>\n",
       "      <td>169</td>\n",
       "      <td>49</td>\n",
       "      <td>2916</td>\n",
       "      <td>1296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99982</th>\n",
       "      <td>1</td>\n",
       "      <td>1764</td>\n",
       "      <td>441</td>\n",
       "      <td>8836</td>\n",
       "      <td>6561</td>\n",
       "      <td>169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99983</th>\n",
       "      <td>1444</td>\n",
       "      <td>3600</td>\n",
       "      <td>9025</td>\n",
       "      <td>8281</td>\n",
       "      <td>7569</td>\n",
       "      <td>1444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99984</th>\n",
       "      <td>5776</td>\n",
       "      <td>1600</td>\n",
       "      <td>9</td>\n",
       "      <td>9409</td>\n",
       "      <td>961</td>\n",
       "      <td>3249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99985</th>\n",
       "      <td>2916</td>\n",
       "      <td>4356</td>\n",
       "      <td>64</td>\n",
       "      <td>1681</td>\n",
       "      <td>49</td>\n",
       "      <td>1936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99986</th>\n",
       "      <td>5929</td>\n",
       "      <td>1089</td>\n",
       "      <td>169</td>\n",
       "      <td>4225</td>\n",
       "      <td>4225</td>\n",
       "      <td>1296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99987</th>\n",
       "      <td>3481</td>\n",
       "      <td>2116</td>\n",
       "      <td>5041</td>\n",
       "      <td>7569</td>\n",
       "      <td>6724</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99988</th>\n",
       "      <td>3136</td>\n",
       "      <td>4225</td>\n",
       "      <td>676</td>\n",
       "      <td>484</td>\n",
       "      <td>961</td>\n",
       "      <td>4624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99989</th>\n",
       "      <td>2500</td>\n",
       "      <td>7056</td>\n",
       "      <td>900</td>\n",
       "      <td>4096</td>\n",
       "      <td>3844</td>\n",
       "      <td>8100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99990</th>\n",
       "      <td>784</td>\n",
       "      <td>900</td>\n",
       "      <td>2916</td>\n",
       "      <td>4356</td>\n",
       "      <td>7396</td>\n",
       "      <td>2916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99991</th>\n",
       "      <td>3721</td>\n",
       "      <td>1764</td>\n",
       "      <td>16</td>\n",
       "      <td>9</td>\n",
       "      <td>5929</td>\n",
       "      <td>4900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99992</th>\n",
       "      <td>64</td>\n",
       "      <td>2809</td>\n",
       "      <td>3025</td>\n",
       "      <td>9</td>\n",
       "      <td>5625</td>\n",
       "      <td>484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99993</th>\n",
       "      <td>2116</td>\n",
       "      <td>1600</td>\n",
       "      <td>5929</td>\n",
       "      <td>81</td>\n",
       "      <td>625</td>\n",
       "      <td>441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99994</th>\n",
       "      <td>1521</td>\n",
       "      <td>529</td>\n",
       "      <td>2025</td>\n",
       "      <td>2809</td>\n",
       "      <td>3364</td>\n",
       "      <td>5929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99995</th>\n",
       "      <td>576</td>\n",
       "      <td>1225</td>\n",
       "      <td>1764</td>\n",
       "      <td>3721</td>\n",
       "      <td>0</td>\n",
       "      <td>144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99996</th>\n",
       "      <td>625</td>\n",
       "      <td>5476</td>\n",
       "      <td>3364</td>\n",
       "      <td>9409</td>\n",
       "      <td>100</td>\n",
       "      <td>4900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99997</th>\n",
       "      <td>900</td>\n",
       "      <td>1444</td>\n",
       "      <td>1521</td>\n",
       "      <td>676</td>\n",
       "      <td>49</td>\n",
       "      <td>7056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99998</th>\n",
       "      <td>6084</td>\n",
       "      <td>36</td>\n",
       "      <td>289</td>\n",
       "      <td>6561</td>\n",
       "      <td>7921</td>\n",
       "      <td>256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99999</th>\n",
       "      <td>1089</td>\n",
       "      <td>784</td>\n",
       "      <td>3969</td>\n",
       "      <td>2500</td>\n",
       "      <td>2401</td>\n",
       "      <td>2500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100000 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          0     1     2     3     4     5\n",
       "0      1296  1156   961   576  6889  7921\n",
       "1       225  3844  8649   121  8281  4624\n",
       "2      3025   729  9025  4225  1024  8836\n",
       "3        25   729  1936  6724  4900  3721\n",
       "4      7225  1156    64  9604   441   961\n",
       "5      5184   441  2500  5041  1936  5041\n",
       "6       289  8281  8464  1089  7569    25\n",
       "7      7225  1600  5625  5625  3364  7396\n",
       "8      3025  4489  2704   400  6724  3364\n",
       "9      6400  7396  4225    49  7921  4761\n",
       "10       64   256  4356  8100  4761  4761\n",
       "11     3600   729  4624  2500    25   441\n",
       "12     5476  5329  6084  3025  3844  7396\n",
       "13       16  6889   400   484  1225   169\n",
       "14     3025  2209  9801   324  4900  4096\n",
       "15     8649  6400  1936  3600  4900  2116\n",
       "16     4356  3481  5476  7569  9216    16\n",
       "17      961   196  4096  5476  8649  6724\n",
       "18        1  4900  1156  2025  2916  1521\n",
       "19     6889  2809  9409  3249  7569   784\n",
       "20     2116    49   225    81  8649   289\n",
       "21     7569  2500    25   441  9025  6561\n",
       "22        1  4489  3136  3025  3249  2116\n",
       "23     5476  2116  9025  4096  1764    16\n",
       "24     3025   961  3249  7056  3136   676\n",
       "25     2809  7744  1089  6724     0   961\n",
       "26      225   400  7396  3721   784  7744\n",
       "27       81   100   400  8836  2601  1600\n",
       "28      441   144  2704  1369   289  8281\n",
       "29      625  3025  8836    64  3025  9409\n",
       "...     ...   ...   ...   ...   ...   ...\n",
       "99970  9216  8100  1369   529   625  7921\n",
       "99971     4  5184  3364  4356  5625  9604\n",
       "99972   484  6889   900    36  1444  3364\n",
       "99973  7744  3844  4489  5776   289  1225\n",
       "99974  1225  4489     9  2916  1369   100\n",
       "99975  1849  4356   169  2025  2704  7744\n",
       "99976  5929  7396   841   441   225  7056\n",
       "99977   676  7056   529   100    16   484\n",
       "99978   841  5476  8100  7569  6400  1156\n",
       "99979     4  7921  1225  5476  3844  7396\n",
       "99980  6241  1444  8281  1849  5625  3721\n",
       "99981  2401   121   169    49  2916  1296\n",
       "99982     1  1764   441  8836  6561   169\n",
       "99983  1444  3600  9025  8281  7569  1444\n",
       "99984  5776  1600     9  9409   961  3249\n",
       "99985  2916  4356    64  1681    49  1936\n",
       "99986  5929  1089   169  4225  4225  1296\n",
       "99987  3481  2116  5041  7569  6724    25\n",
       "99988  3136  4225   676   484   961  4624\n",
       "99989  2500  7056   900  4096  3844  8100\n",
       "99990   784   900  2916  4356  7396  2916\n",
       "99991  3721  1764    16     9  5929  4900\n",
       "99992    64  2809  3025     9  5625   484\n",
       "99993  2116  1600  5929    81   625   441\n",
       "99994  1521   529  2025  2809  3364  5929\n",
       "99995   576  1225  1764  3721     0   144\n",
       "99996   625  5476  3364  9409   100  4900\n",
       "99997   900  1444  1521   676    49  7056\n",
       "99998  6084    36   289  6561  7921   256\n",
       "99999  1089   784  3969  2500  2401  2500\n",
       "\n",
       "[100000 rows x 6 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "\n",
    "df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n",
    "\n",
    "# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n",
    "# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n",
    "tqdm.pandas(desc=\"my bar!\")\n",
    "\n",
    "# Now you can use `progress_apply` instead of `apply`\n",
    "# and `progress_map` instead of `map`\n",
    "df.progress_apply(lambda x: x**2)\n",
    "# can also groupby:\n",
    "# df.groupby(0).progress_apply(lambda x: x**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case you're interested in how this works (and how to modify it for\n",
    "your own callbacks), see the\n",
    "[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n",
    "import the module and run `help()`.\n",
    "\n",
    "### Keras Integration\n",
    "\n",
    "```python\n",
    "A `keras` callback is also available:\n",
    "\n",
    "from tqdm.keras import TqdmCallback\n",
    "\n",
    "...\n",
    "\n",
    "model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n",
    "```\n",
    "\n",
    "### IPython/Jupyter Integration\n",
    "\n",
    "IPython/Jupyter is supported via the `tqdm.notebook` submodule:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.notebook import trange, tqdm\n",
    "from time import sleep\n",
    "\n",
    "for i in trange(3, desc='1st loop'):\n",
    "    for j in tqdm(range(100), desc='2nd loop'):\n",
    "        sleep(0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to `tqdm` features, the submodule provides a native Jupyter\n",
    "widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n",
    "bars and colour hints (blue: normal, green: completed, red:\n",
    "error/interrupt, light blue: no ETA); as demonstrated below.\n",
    "\n",
    "![Screenshot-Jupyter3](https://raw.githubusercontent.com/tqdm/tqdm/master/images/tqdm-jupyter-3.gif)\n",
    "\n",
    "The `notebook` version supports percentage or pixels for overall width\n",
    "(e.g.: `ncols='100%'` or `ncols='480px'`).\n",
    "\n",
    "It is also possible to let `tqdm` automatically choose between console\n",
    "or notebook versions by using the `autonotebook` submodule:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.autonotebook import tqdm\n",
    "tqdm.pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this will issue a `TqdmExperimentalWarning` if run in a\n",
    "notebook since it is not meant to be possible to distinguish between\n",
    "`jupyter notebook` and `jupyter console`. Use `auto` instead of\n",
    "`autonotebook` to suppress this warning.\n",
    "\n",
    "Note that notebooks will display the bar in the cell where it was\n",
    "created. This may be a different cell from the one where it is used. If\n",
    "this is not desired, the creation of the bar must be delayed/moved to\n",
    "the cell where it is desired to be displayed.\n",
    "\n",
    "Another possibility is to have a single bar (near the top of the\n",
    "notebook) which is constantly re-used (using `reset()` rather than\n",
    "`close()`). For this reason, the notebook version (unlike the CLI\n",
    "version) does not automatically call `close()` upon `Exception`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.notebook import tqdm\n",
    "pbar = tqdm()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# different cell\n",
    "iterable = range(100)\n",
    "pbar.reset(total=len(iterable))  # initialise with new `total`\n",
    "for i in iterable:\n",
    "    pbar.update()\n",
    "pbar.refresh()  # force print final status but don't `close()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing messages\n",
    "\n",
    "This is a work in progress (see\n",
    "[#737](https://github.com/tqdm/tqdm/issues/737)).\n",
    "\n",
    "Since `tqdm` uses a simple printing mechanism to display progress bars,\n",
    "you should not write any message in the terminal using `print()` while a\n",
    "progressbar is open.\n",
    "\n",
    "To write messages in the terminal without any collision with `tqdm` bar\n",
    "display, a `.write()` method is provided:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.auto import tqdm, trange\n",
    "from time import sleep\n",
    "\n",
    "bar = trange(10)\n",
    "for i in bar:\n",
    "    # Print using tqdm class method .write()\n",
    "    sleep(0.1)\n",
    "    if not (i % 3):\n",
    "        tqdm.write(\"Done task %i\" % i)\n",
    "    # Can also use bar.write()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, this will print to standard output `sys.stdout`. but you can\n",
    "specify any file-like object using the `file` argument. For example,\n",
    "this can be used to redirect the messages writing to a log file or class.\n",
    "\n",
    "---\n",
    "\n",
    "[![sourcerer-0](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/0)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/0)|[![sourcerer-1](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/1)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/1)|[![sourcerer-2](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/2)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/2)|[![sourcerer-3](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/3)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/3)|[![sourcerer-4](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/4)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/4)|[![sourcerer-5](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/5)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/5)|[![sourcerer-7](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/images/7)](https://sourcerer.io/fame/casperdcl/tqdm/tqdm/links/7)\n",
    "-|-|-|-|-|-|-\n",
    "\n",
    "[![README-Hits](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif)](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif&style=social)|(Since 19 May 2016)\n",
    "-|-"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Do your own experiments here 👇\n",
    "\n",
    "Try `tqdm` youself by adding your code below and running your own experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tqdm\n",
    "\n",
    "# your code here\n",
    "tqdm."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}