Skip to content

Commit

Permalink
(Partially) Fix oppia#2100: add cache slugs. (oppia#2208)
Browse files Browse the repository at this point in the history
* Added cache slugs for images and modified all references to the images.

* Added interpolation service to get static image url, modified non-page level html files and corresponding directives/controllers to use it, addressed review comments

* Modified /script tags

* Added cache slugs for i18n

* Fixed failing test and added i18n/prod to git ignore.

* Cache slugs new approach POC

* Modified references for images, scripts and i18n.
Added methods to return static resource url.
Modified app.yaml.

* Modified gulp file and references to oppia.css and third_party css and js files.

* Added slugs to extensions

* Fixed test for get_cache_slug

* Modified build.py and deploy.py

* Addressed review comments

* Modify imports

* Modified src to ng-src and other review comments fixed.

* Modified build.py

* Minor change

* Removed feconf.CACHE_SLUG_DEV and refactored

* Added random cache slug

* Refactored code.

* Modified references for extensions, added extensions to build process and refactored methods in UrlInterpolationService

* Fixed bug

* Addressed review comments.

* Fix failing tests

* Fixed bug.

* Added tests for urlinterpolationservice.

* Added interpolation for interactions and logic_proof.html

* Addressed review comments

* Addressed review comments

* Removed unnecessary include from templates

* Modified precommit linter script to fix endless waiting

* Commented script and added assets/scripts to exclusions.
  • Loading branch information
gvishal authored and wxyxinyu committed Aug 8, 2016
1 parent ad46f52 commit a13c5c3
Show file tree
Hide file tree
Showing 213 changed files with 1,590 additions and 897 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,8 @@ venv/
.DS_Store
.idea
.vagrant/*

# Oppia uses cache slugs for various resources and we need separate resource
# directories for dev and prod. Resource directories for prod are generated
# via build.py and should not be committed to the repo.
build/*
6 changes: 4 additions & 2 deletions .jscsrc
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@
"extensions/interactions/LogicProof/static/js/generatedParser.js",
"integrations/**",
"integrations_dev/**",
"static/scripts/**",
"third_party/**"
"assets/scripts/**",
"third_party/**",
"build/**",
"**.min.js"
],
"fileExtensions": [".js"],
"extract": ["*.html"],
Expand Down
39 changes: 18 additions & 21 deletions app.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,34 +15,36 @@ inbound_services:

handlers:
- url: /favicon.ico
static_files: static/images/favicon.ico
upload: static/images/favicon.ico
static_files: assets/common/favicon.ico
upload: assets/common/favicon.ico
secure: always
http_headers:
Cache-Control: 'public, max-age=2592000'
Vary: Accept-Encoding
- url: /images
static_dir: static/images
- url: /robots.txt
static_files: assets/common/robots.txt
upload: assets/common/robots.txt
secure: always
http_headers:
Cache-Control: 'public, max-age=600'
- url: /static/pages
static_dir: static/pages
Cache-Control: 'public, max-age=2592000'
Vary: Accept-Encoding
- url: /build
static_dir: build
secure: always
http_headers:
Cache-Control: 'public, max-age=600'
- url: /robots.txt
static_files: static/pages/robots.txt
upload: static/pages/robots.txt
Cache-Control: 'public, max-age=2592000'
- url: /assets/common
static_dir: assets/common
secure: always
http_headers:
Cache-Control: 'public, max-age=2592000'
Vary: Accept-Encoding
- url: /scripts
static_dir: static/scripts
# WARNING TO DEVELOPERS: Files in this folder may be stale for
# up to 10 mins after a code release.
Cache-Control: 'public, max-age=600'
- url: /assets
static_dir: assets
secure: always
http_headers:
Cache-Control: 'no-cache'
Cache-Control: 'public, max-age=60'
- url: /css
# NB: not minified. TODO(sll): fix.
static_dir: core/templates/dev/head/css
Expand All @@ -53,11 +55,6 @@ handlers:
http_headers:
Cache-Control: 'public, max-age=2592000'
Vary: Accept-Encoding
- url: /i18n
static_dir: i18n
secure: always
http_headers:
Cache-Control: 'no-cache'
- url: /third_party/generated
static_dir: third_party/generated
secure: always
Expand Down
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
Binary file added assets/images/general/background.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
Binary file added assets/images/general/icons-bg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
Binary file added assets/images/general/warning.png
File renamed without changes
436 changes: 218 additions & 218 deletions static/images/library/banner1.svg → assets/images/library/banner1.svg

Large diffs are not rendered by default.

File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
2 changes: 1 addition & 1 deletion static/scripts/README → assets/scripts/README
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ them. To load the examples manually you need to:
- In a browser window, open each of the HTML files in this folder, and ensure
that they do the correct thing. Use URLs of the following form:

http://localhost:8181/scripts/embedding_tests_dev_0.0.1.html
http://localhost:8181/assets/scripts/embedding_tests_dev_0.0.1.html


The features supported in each version of the embedding script are as follows:
Expand Down
File renamed without changes.
18 changes: 18 additions & 0 deletions assets/scripts/embedding_tests_dev_0.0.0.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<h2>v0.0.0</h2>
<p>For instructions: see the README file in /static/dev/scripts.</p>

<h3>Standard embedding of the latest version</h3>
<oppia oppia-id="0"></oppia>
<script src="http://localhost:8181/assets/scripts/oppia-player-0.0.0.js"></script>

<h3>Standard embedding of version 1 of the exploration</h3>
<oppia oppia-id="0" exploration-version="1"></oppia>
<script src="http://localhost:8181/assets/scripts/oppia-player-0.0.1.js"></script>

<h3>ERROR: No oppia id specified</h3>
<oppia></oppia>
<script src="http://localhost:8181/assets/scripts/oppia-player-0.0.0.js"></script>

<h3>ERROR: 404 error</h3>
<oppia oppia-id="fake_id"></oppia>
<script src="http://localhost:8181/assets/scripts/oppia-player-0.0.0.js"></script>
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<h2>v0.0.1 (unminified)</h2>
<p>For instructions: see the README file in /static/scripts.</p>
<p>For instructions: see the README file in /assets/scripts.</p>
<p>
NB: When you complete an exploration the text "Exploration completed" should
appear in the paragraph below it. This checks that events are being sent from
the Oppia iframe to the containing page.
</p>

<script src="/scripts/oppia-player-0.0.1.js"></script>
<script src="/scripts/embedding_tests.js"></script>
<script src="/assets/scripts/oppia-player-0.0.1.js"></script>
<script src="/assets/scripts/embedding_tests.js"></script>

<div class="protractor-test-standard">
<h3>Standard embedding of the latest version</h3>
Expand All @@ -32,4 +32,4 @@ <h3>ERROR: 404 error</h3>
<div class="protractor-test-invalid-id-deferred">
<h3>ERROR: 404 error with deferred loading</h3>
<oppia oppia-id="fake_id" autoload="false"></oppia>
</div>
</div>
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<h2>v0.0.1 (minified)</h2>
<p>For instructions: see the README file in /static/scripts.</p>
<p>For instructions: see the README file in /assets/scripts.</p>
<p>
NB: When you complete an exploration the text "Exploration completed" should
appear in the paragraph below it. This checks that events are being sent from
the Oppia iframe to the containing page.
</p>

<script src="/scripts/oppia-player-0.0.1.js"></script>
<script src="/scripts/embedding_tests.js"></script>
<script src="/assets/scripts/oppia-player-0.0.1.js"></script>
<script src="/assets/scripts/embedding_tests.js"></script>

<div class="protractor-test-standard">
<h3>Standard embedding of the latest version</h3>
Expand Down
10 changes: 10 additions & 0 deletions assets/scripts/embedding_tests_dev_i18n_0.0.1.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<h2>v0.0.1 (unminified)</h2>
<p>For instructions: see the README file in /assets/scripts.</p>

<script src="/assets/scripts/oppia-player-0.0.1.js"></script>
<script src="/assets/scripts/embedding_tests.js"></script>

<div class="protractor-test-embedded-exploration">
<h3>Standard embedding of the latest version</h3>
<oppia oppia-id="12"></oppia>
</div>
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<h2>v0.0.1 (minified) from the JSDelivr CDN</h2>
<p>For instructions: see the README file in /static/scripts.</p>
<p>For instructions: see the README file in /assets/scripts.</p>
<p>
NB: This set of tests depends on an external file hosted on the JSDelivr CDN.
They will not work if you are offline.
Expand All @@ -10,8 +10,8 @@ <h2>v0.0.1 (minified) from the JSDelivr CDN</h2>
the Oppia iframe to the containing page.
</p>

<script src="/scripts/oppia-player-0.0.1.js"></script>
<script src="/scripts/embedding_tests.js"></script>
<script src="/assets/scripts/oppia-player-0.0.1.js"></script>
<script src="/assets/scripts/embedding_tests.js"></script>

<div class="protractor-test-standard">
<h3>Standard embedding of the latest version</h3>
Expand All @@ -36,4 +36,4 @@ <h3>ERROR: 404 error</h3>
<div class="protractor-test-invalid-id-deferred">
<h3>ERROR: 404 error with deferred loading</h3>
<oppia oppia-id="fake_id" autoload="false"></oppia>
</div>
</div>
File renamed without changes.
File renamed without changes.
File renamed without changes.
5 changes: 5 additions & 0 deletions cache_slug.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Cache slug is a prefix that is attached to static resource paths such as
# images, scripts and css. It should only contain alphanumeric characters.
# It will be automatically updated when deploy.py is run.
cache_slug:
default
5 changes: 4 additions & 1 deletion core/controllers/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ def render_template(
values.update({
'ALL_CATEGORIES': feconf.ALL_CATEGORIES,
'ALL_LANGUAGE_CODES': feconf.ALL_LANGUAGE_CODES,
'ASSET_DIR_PREFIX': utils.get_asset_dir_prefix(),
'BEFORE_END_HEAD_TAG_HOOK': jinja2.utils.Markup(
BEFORE_END_HEAD_TAG_HOOK.value),
'CAN_SEND_ANALYTICS_EVENTS': feconf.CAN_SEND_ANALYTICS_EVENTS,
Expand All @@ -313,7 +314,8 @@ def render_template(
rights_manager.ACTIVITY_STATUS_PUBLIC),
'ACTIVITY_STATUS_PUBLICIZED': (
rights_manager.ACTIVITY_STATUS_PUBLICIZED),
'FULL_URL': '%s://%s/%s' % (scheme, netloc, path),
# The 'path' variable starts with a forward slash.
'FULL_URL': '%s://%s%s' % (scheme, netloc, path),
'INVALID_NAME_CHARS': feconf.INVALID_NAME_CHARS,
# TODO(sll): Consider including the obj_editor html directly as
# part of the base HTML template?
Expand Down Expand Up @@ -375,6 +377,7 @@ def render_template(

self.response.expires = 'Mon, 01 Jan 1990 00:00:00 GMT'
self.response.pragma = 'no-cache'

self.response.write(self.jinja2_env.get_template(
filename).render(**values))

Expand Down
7 changes: 5 additions & 2 deletions core/controllers/base_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,15 +237,18 @@ class I18nDictsTest(test_utils.GenericTestBase):

def _extract_keys_from_json_file(self, filename):
return sorted(json.loads(utils.get_file_contents(
os.path.join(os.getcwd(), 'i18n', filename)
os.path.join(os.getcwd(), self.get_static_asset_filepath(),
'assets', 'i18n', filename)
)).keys())

def test_i18n_keys(self):
"""Tests that all JSON files in i18n.js have the same set of keys."""
master_key_list = self._extract_keys_from_json_file('en.json')
self.assertGreater(len(master_key_list), 0)

filenames = os.listdir(os.path.join(os.getcwd(), 'i18n'))
filenames = os.listdir(
os.path.join(os.getcwd(), self.get_static_asset_filepath(),
'assets', 'i18n'))
for filename in filenames:
if filename == 'en.json':
continue
Expand Down
6 changes: 3 additions & 3 deletions core/controllers/library_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,12 +195,12 @@ def test_thumbnail_icons_exist_for_each_category(self):
# Test that an icon exists for each default category.
for category in all_categories:
utils.get_file_contents(os.path.join(
'static', 'images', 'subjects',
'%s.svg' % category.replace(' ', '')))
self.get_static_asset_filepath(), 'assets', 'images',
'subjects', '%s.svg' % category.replace(' ', '')))

# Test that the default icon exists.
utils.get_file_contents(os.path.join(
'static', 'images', 'subjects',
self.get_static_asset_filepath(), 'assets', 'images', 'subjects',
'%s.svg' % feconf.DEFAULT_THUMBNAIL_ICON))


Expand Down
7 changes: 7 additions & 0 deletions core/domain/dependency_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@

from extensions.dependencies import dependencies_config
import feconf
import jinja_utils
import utils


Expand Down Expand Up @@ -54,6 +55,12 @@ def get_deps_html_and_angular_modules(cls, dependency_ids):
"""
html = '\n'.join([
cls.get_dependency_html(dep) for dep in set(dependency_ids)])
# Html content for interactions is loaded using get_dependency_html
# method. This content is then interpolated using Jinja. Interactions
# such as logicproof contain jinja like variables that need to be
# interpolated. LogicProof contains a cache slug variable that is
# interpolated using interpolate_cache_slug method.
html = jinja_utils.interpolate_cache_slug(html)
angular_modules_for_each_dep = [
cls.get_angular_modules(dep) for dep in set(dependency_ids)]
deduplicated_angular_modules = list(set(list(
Expand Down
24 changes: 16 additions & 8 deletions core/domain/summary_services_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,8 @@ def test_get_displayable_exp_summary_dicts_matching_ids(self):
'status': 'public',
'tags': [],
'thumbnail_bg_color': '#a33f40',
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'title': u'Exploration 2 Albert title',
}
self.assertIn('last_updated_msec', displayable_summaries[0])
Expand Down Expand Up @@ -278,7 +279,8 @@ def test_get_library_groups(self):
'tags': [],
'title': u'The Lazy Magician',
'thumbnail_bg_color': '#d0982a',
'thumbnail_icon_url': '/images/subjects/Algorithms.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Algorithms.svg'),
}
expected_group = {
'categories': ['Algorithms', 'Computing', 'Programming'],
Expand Down Expand Up @@ -354,7 +356,8 @@ def test_for_featured_explorations(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_2,
'category': 'A category',
Expand Down Expand Up @@ -640,7 +643,8 @@ def test_at_most_eight_top_rated_explorations(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_3,
'category': u'A category',
Expand Down Expand Up @@ -677,7 +681,8 @@ def test_only_explorations_with_ratings_are_returned(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_2,
'category': u'A category',
Expand Down Expand Up @@ -758,7 +763,8 @@ def test_for_recently_published_explorations(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_1,
'category': u'A category',
Expand All @@ -772,7 +778,8 @@ def test_for_recently_published_explorations(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_2,
'category': u'A category',
Expand All @@ -786,7 +793,8 @@ def test_for_recently_published_explorations(self):
'thumbnail_bg_color': '#a33f40',
'community_owned': False,
'tags': [],
'thumbnail_icon_url': '/images/subjects/Lightbulb.svg',
'thumbnail_icon_url': self.get_static_asset_url(
'/images/subjects/Lightbulb.svg'),
'language_code': feconf.DEFAULT_LANGUAGE_CODE,
'id': self.EXP_ID_3,
'category': u'A category',
Expand Down
6 changes: 4 additions & 2 deletions core/domain/user_services_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,8 @@ def test_get_user_id_from_username(self):
def test_fetch_gravatar_success(self):
user_email = 'user@example.com'
expected_gravatar_filepath = os.path.join(
'static', 'images', 'avatar', 'gravatar_example.png')
self.get_static_asset_filepath(), 'assets', 'images', 'avatar',
'gravatar_example.png')
with open(expected_gravatar_filepath, 'r') as f:
gravatar = f.read()
with self.urlfetch_mock(content=gravatar):
Expand Down Expand Up @@ -198,7 +199,8 @@ def log_mock(message):

def test_default_identicon_data_url(self):
identicon_filepath = os.path.join(
'static', 'images', 'avatar', 'user_blue_72px.png')
self.get_static_asset_filepath(), 'assets', 'images', 'avatar',
'user_blue_72px.png')
identicon_data_url = utils.convert_png_to_data_url(
identicon_filepath)
self.assertEqual(
Expand Down
Loading

0 comments on commit a13c5c3

Please sign in to comment.