Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize and remove dependencies, part 1 #4301

Closed
wants to merge 7 commits into from

Conversation

jisantuc
Copy link
Contributor

@jisantuc jisantuc commented Nov 14, 2018

Overview

This PR audits dependencies, removes a number of them, reorganizes some others, and zaps what it can to make the backsplash jar in particular and jars in general smaller.

Why is it good:

A lot of dependencies are no longer a part of projects that don't need them. This is good for CLEANLINESS. For example, anything depending on awsSdk was a waste, since nothing actually used it.

Also, smaller jar sizes are good because we pay for storage and because they shoud... start faster? Seems like maybe that's not true. Because they take less time to assemble! For developer productivity! I'm looking for the wins I can find, since I couldn't make backsplash small enough for lambda and have other things to do than die on this hill (though I am obviously more than happy if this is the particular hill I die on eventually).

Checklist

  • Description of PR is in an appropriate section of the changelog and grouped with similar changes if possible

Demo

Backsplash size before: ~252mb
Backsplash size now: 182mb
Batch size before: ~299mb
Batch size now: 237mb

Notes

The bulk of the improvements for backsplash and batch came from removing the general purpose awsSdk dependency, which wasn't needed and was huge. Other big improvements were getting akka out of common and zap-ing things in the backsplash assembly shade rules.

Testing Instructions

  • reassemble batch, backsplash, the api server, and the tile server
  • console into the batch container
  • pick some jobs (probably not export, since it's broken right now in dev) and run them, they should still be fine
  • docker-compose run --rm --entrypoint bash --service-ports api-server
  • docker-compose run --rm --entrypoint bash --service-ports backsplash
  • docker-compose run --rm --entrypoint bash --service-ports tile-server
  • start up the api server: java -jar app-backend/api/target/scala-2.11/api-assembly.jar
  • bring up the nginx-api container
  • wander around the app in the frontend -- it should be fine
  • start up the tile server: java -jar app-backend/tile/target/scala-2.11/tile-assembly.jar
  • bringup the nginx-tiler container
  • make sure tile requests still work normally and in the lab
  • copy one of each of those requests -- you'll need it in a minute
  • stop the tile server
  • start up backsplash: java -jar app-backend/backsplash/target/sscala-2.11/backsplash-assembly.jar
  • hit backsplash with the two requests you just copied
  • everything should have worked

Closes azavea/raster-foundry-platform#533

@jisantuc jisantuc force-pushed the feature/js/shrink-backsplash-jar branch 2 times, most recently from b360c1f to 70d4e12 Compare November 14, 2018 20:03
@jisantuc jisantuc changed the title [wip] Reorganize and remove dependencies, part 1 Reorganize and remove dependencies, part 1 Nov 15, 2018
@jisantuc jisantuc requested a review from Lknechtli November 15, 2018 20:52
@jisantuc jisantuc force-pushed the feature/js/shrink-backsplash-jar branch from 229bcb9 to a6ce8a8 Compare November 20, 2018 15:15
@jisantuc
Copy link
Contributor Author

Closing because Jenkins is extremely confused about what's going on in this branch for some reason

@jisantuc jisantuc closed this Nov 20, 2018
@jisantuc jisantuc deleted the feature/js/shrink-backsplash-jar branch November 20, 2018 15:16
@jisantuc jisantuc mentioned this pull request Nov 20, 2018
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants