From 6995291183a27f8efdf8000d8a72095343d381d5 Mon Sep 17 00:00:00 2001 From: Gil Wassermann Date: Thu, 9 Nov 2017 10:48:38 -0500 Subject: [PATCH] ENH: Universe Selection Lecture: QTradableStocksUS --- .../Universe_Selection/notebook.ipynb | 63 +++------------ .../lectures/Universe_Selection/preview.html | 76 ++++--------------- 2 files changed, 27 insertions(+), 112 deletions(-) diff --git a/notebooks/lectures/Universe_Selection/notebook.ipynb b/notebooks/lectures/Universe_Selection/notebook.ipynb index d59c691a..14224fa9 100644 --- a/notebooks/lectures/Universe_Selection/notebook.ipynb +++ b/notebooks/lectures/Universe_Selection/notebook.ipynb @@ -463,67 +463,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "###Tradability Concerns\n", - "\n", - "While controlling for sector exposure and turnover are important, perhaps the most important part of any universe selection methodology is ensuring that the securities included in the universe **can** be traded.\n", - "\n", - "Recently we released two of our own universes, the Q500US and the Q1500US, built with the same basic methodology. These universes are very similar to the Lectures500 under the hood and are built with all the concerns listed here in mind. As part of this release, we developed a tradability filter. The filter will return True in the follow cases:\n", - "\n", - "1. The stock must be the primary share class for its company.\n", - "2. The company issuing the stock must have known market\n", - " capitalization.\n", - "3. The stock must not be a depository receipt.\n", - "4. The stock must not be traded over the counter (OTC).\n", - "5. The stock must not be for a limited partnership.\n", - "6. The stock must have a known previous-day close price.\n", - "7. The stock must have had nonzero volume on the previous trading\n", - " day.\n", - " \n", - "Let's add these tradability constraints to the Lectures500US." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "from quantopian.pipeline.filters import default_us_equity_universe_mask\n", - "\n", - "Lectures500 = Lectures500 & default_us_equity_universe_mask()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We have now built a workable universe and have optimized it based on particular constraints!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Q500US and Q1500US\n", - "\n", - "Although the Lectures500 is a decent universe, we highly recommend that you use the Q1500US when designing your algorithms.\n", + "##QTradableStocksUS\n", "\n", - "The Q1500US is defined as follows:\n", + "Although the Lectures500 is a decent universe, we highly recommend that you use the QTradableStocksUS when designing your algorithms.\n", "\n", + "The QTradableStocksUS is defined using the following limits:\n", "\n", - " At the start of each month, throw out \"hard-to-trade\" equities, then choose the top 1500 equities \n", - " by 200-day average dollar volume (ADV), choosing no more than 30% of the requested equities from\n", - " any single sector.\n", + "* Market cap: over \\$500M: This restriction eliminates many undiversifiable risks like low liquidity and difficulty in shorting.\n", + "* Dollar volume: It is important that stocks in our universe be relatively easy to trade when entering and exiting positions. The QTradableStocksUS manages that by including only stocks that have median daily dollar volume of \\$2.5m or more over the trailing 200 days.\n", + "* Prior day's close: If a stock's price is lower than \\$5, the bid-ask spread becomes larger relative to the price, and the transaction cost becomes too high.\n", + "200 days of price and volume: If a stock has missing data for the previous 200 days, the company is excluded. This targets stocks with trading halts, IPOs, and other situations that make them harder to assess.\n", + "* Primary/Common share: The QTradableStocksUS chooses a single share class for each company. The criteria is to find the common share with the most dollar volume.\n", + "ADRs, Limited Partnerships: QTradableStocksUS excludes ADRs and LPs.\n", " \n", "and can be implemented in a single line of code.\n", "\n", - "For more information about these universes, as well as a description of a paramaterized function we have released to enable custom universes like the Lectures500 to be created with ease, check out [this link][1].\n", + "For more information about the QTradableStocksUS, check out [this link][1].\n", "\n", "\n", "\n", - "[1]: https://www.quantopian.com/posts/the-q500us-and-q1500us" + "[1]: https://www.quantopian.com/posts/working-on-our-best-universe-yet-qtradablestocksus" ] }, { diff --git a/notebooks/lectures/Universe_Selection/preview.html b/notebooks/lectures/Universe_Selection/preview.html index ace32e6d..f26f9672 100644 --- a/notebooks/lectures/Universe_Selection/preview.html +++ b/notebooks/lectures/Universe_Selection/preview.html @@ -11767,7 +11767,7 @@
-

Universe Selection¶

by Gil Wassermann, Maxwell Margenot

+

Universe Selection

by Gil Wassermann, Maxwell Margenot

Part of the Quantopian Lecture Series:

  • www.quantopian.com/lectures
    • @@ -11784,7 +11784,7 @@

    Universe Selection
    -

    What is a Universe?¶

    On a high level, universe selection is the process of choosing the pool of securities upon which your algorithm will trade. For example, an algorithm designed to play with the characteristics of a universe consisting of technology equities may perform exceptionally well in that universe with the tradeoff of falling flat in other sectors. Experimenting with different universes by tweaking their components is an essential part of developing a trading strategy.

    +

    What is a Universe?

    On a high level, universe selection is the process of choosing the pool of securities upon which your algorithm will trade. For example, an algorithm designed to play with the characteristics of a universe consisting of technology equities may perform exceptionally well in that universe with the tradeoff of falling flat in other sectors. Experimenting with different universes by tweaking their components is an essential part of developing a trading strategy.

    Using Pipeline, we have access to around 8300 securities to choose from each day. However, the securities within this basket are markedly different. Some are different asset classes, some belong to different sectors and super-sectors, some employ different business models, some practice different management styles, and so on. By defining a universe, a trader can narrow in on securities with one or more of these attributes in order to craft a strategy that is most effective for that subset of the population.

    Without a properly-constructed universe, your algorithm may be exposed to risks that you just aren't aware of. For example, it could be possible that your universe selection methodology only selects a stock basket whose constituents do not trade very often. Let's say that your algorithm wants to place an order of 100,000 shares for a company that only trades 1,000 on a given day. The inability to fill this order or others might prevent you from achieving the optimal weights for your portfolio, thereby undermining your strategy. These risks can be controlled for by careful and thoughtful universe slection.

    On the Quantopian platform, universes are often implemented as a Pipeline screen. If you are not familiar with Pipeline, feel free to check out the tutorial. Below is an example implementation of a universe that limits Pipeline output to the 500 securities with the largest revenue each day. This can be seen as a naive implementation of the Fortune500.

    @@ -11934,7 +11934,7 @@

    What is a Universe?
    -

    Lectures500¶

    Sector Exposure¶

    If I create a universe that only looks at equities in the technology sector, my algorithm will have an extreme sector bias. Companies in the same industry sector are affected by similar macroeconomic trends and therefore their performance tends to be correlated. In the case of particular strategies, we may find the benefits of working exclusively within a particular sector greater than the downside risks, but this is not suitable for creating a general-purpose, quality universe.

    +

    Lectures500

    Sector Exposure

    If I create a universe that only looks at equities in the technology sector, my algorithm will have an extreme sector bias. Companies in the same industry sector are affected by similar macroeconomic trends and therefore their performance tends to be correlated. In the case of particular strategies, we may find the benefits of working exclusively within a particular sector greater than the downside risks, but this is not suitable for creating a general-purpose, quality universe.

    Let's have a look at the sector breakdown of the Lectures500.

    @@ -13636,7 +13636,7 @@

    Lectures500Â

    -

    Turnover¶

    Another thing to consider when designing a universe is the rate at which the universe changes. Turnover is a way of measuring this rate of change. Turnover is defined as the number of equities to enter or exit the universe in a particular time window.

    +

    Turnover

    Another thing to consider when designing a universe is the rate at which the universe changes. Turnover is a way of measuring this rate of change. Turnover is defined as the number of equities to enter or exit the universe in a particular time window.

    Let us imagine a universe with a turnover of 0. This universe would be completely unchanged by market movements. Moreover, stocks inappropriate for the universe would never be removed and stocks that should be included will never enter.

    Conversely, imagine a universe that changes every one of its constituents every day. An algorithm built on this universe will be forced to sell its entire portfolio every day. This incurs transaction costs which erode returns.

    When creating a universe, there is an inherent tradeoff between stagnation and sensitivity to the market.

    @@ -14817,7 +14817,7 @@

    Turnover¶
    -

    Smoothing¶

    A good way to reduce turnover is through smoothing functions. Smoothing is the process of taking noisy data and aggregating it in order to analyze its underlying trends. When applied to universe selection, a good smoothing function prevents equities at the universe boundary from entering and exiting frequently.

    +

    Smoothing

    A good way to reduce turnover is through smoothing functions. Smoothing is the process of taking noisy data and aggregating it in order to analyze its underlying trends. When applied to universe selection, a good smoothing function prevents equities at the universe boundary from entering and exiting frequently.

    One example of a potential smoothing function is a filter that finds equities that have passed the Lectures500 criteria for 16 or more days out of the past 21 days. We will call this filter AtLeast16. This aggregation many days of data lends a certain degree of flexibility to the edges of our universe. If, for example, Equity XYZ is very close to the boundary for inclusion, in a given month, it may flit in and out of the Lectures500 day after day. However, with the AtLeast16 filter, Equity XYZ is allowed to enter and exit the daily universe a maximum of 5 times before it is excluded from the smoothed universe.

    Let's apply a smoothing function to our universe and see its effect on turnover.

    @@ -16090,62 +16090,18 @@

    Smoothing¶

    -

    Tradability Concerns¶

    While controlling for sector exposure and turnover are important, perhaps the most important part of any universe selection methodology is ensuring that the securities included in the universe can be traded.

    -

    Recently we released two of our own universes, the Q500US and the Q1500US, built with the same basic methodology. These universes are very similar to the Lectures500 under the hood and are built with all the concerns listed here in mind. As part of this release, we developed a tradability filter. The filter will return True in the follow cases:

    -
      -
    1. The stock must be the primary share class for its company.
      2. -
    2. The company issuing the stock must have known market -capitalization.
      3. -
    3. The stock must not be a depository receipt.
      4. -
    4. The stock must not be traded over the counter (OTC).
      5. -
    5. The stock must not be for a limited partnership.
      6. -
    6. The stock must have a known previous-day close price.
      7. -
    7. The stock must have had nonzero volume on the previous trading -day.
      8. -
    -

    Let's add these tradability constraints to the Lectures500US.

    - -
    -
    -
    -
    -
    -
    In [9]:
    -
    -
    -
    from quantopian.pipeline.filters import default_us_equity_universe_mask
-
-Lectures500 = Lectures500 & default_us_equity_universe_mask()
-
    - -
    -
    -
    - -
    -
    -
    -
    -
    -

    We have now built a workable universe and have optimized it based on particular constraints!

    - -
    -
    -
    -
    -
    -
    -
    -

    Q500US and Q1500US¶

    Although the Lectures500 is a decent universe, we highly recommend that you use the Q1500US when designing your algorithms.

    -

    The Q1500US is defined as follows:

    - -
    At the start of each month, throw out "hard-to-trade" equities, then choose the top 1500 equities 
-by 200-day average dollar volume (ADV), choosing no more than 30% of the requested equities from
-any single sector.
-
-
    +

    QTradableStocksUS

    Although the Lectures500 is a decent universe, we highly recommend that you use the QTradableStocksUS when designing your algorithms.

    +

    The QTradableStocksUS is defined using the following limits:

    +
      +
    • Market cap: over \$500M: This restriction eliminates many undiversifiable risks like low liquidity and difficulty in shorting.
      • +
    • Dollar volume: It is important that stocks in our universe be relatively easy to trade when entering and exiting positions. The QTradableStocksUS manages that by including only stocks that have median daily dollar volume of \$2.5m or more over the trailing 200 days.
      • +
    • Prior day's close: If a stock's price is lower than \$5, the bid-ask spread becomes larger relative to the price, and the transaction cost becomes too high. +200 days of price and volume: If a stock has missing data for the previous 200 days, the company is excluded. This targets stocks with trading halts, IPOs, and other situations that make them harder to assess.
      • +
    • Primary/Common share: The QTradableStocksUS chooses a single share class for each company. The criteria is to find the common share with the most dollar volume. +ADRs, Limited Partnerships: QTradableStocksUS excludes ADRs and LPs.
      • +

    and can be implemented in a single line of code.

    -

    For more information about these universes, as well as a description of a paramaterized function we have released to enable custom universes like the Lectures500 to be created with ease, check out this link.

    +

    For more information about the QTradableStocksUS, check out this link.