Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(provider/kubernetes): Advanced targetSize use case. #1920

Merged
merged 1 commit into from
Sep 25, 2017

Conversation

Z3r0Sum
Copy link
Contributor

@Z3r0Sum Z3r0Sum commented Sep 21, 2017

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize. It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0. However, we shouldn't use the targetSize
if the desired capacity is greater than its value.


Validation around targetSize already happens at the Create Server Group level. This is just properly casting the
description.targetSize.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment should just be part of the commit message. It's really only relevant with comparing the diff. You could just briefly mention the intentions of the 3 lines here.


Validation around targetSize already happens at the Create Server Group level. This is just properly casting the
description.targetSize.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

description.targetSize.
*/
Integer targetSize = description.targetSize.toInteger()
if (targetSize < 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if targetSize is null? Does description.targetSize.toInteger() return -1 in that case? Just want to make sure we preserve the original behavior as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

targetSize cannot be null - that's part of the Server Group Validation.

def parsedName = Names.parseName(replicaSetName)
def targetSize = description.targetSize ?: description.capacity?.desired
if (targetSize < 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch from 6780331 to 80eb0fd Compare September 21, 2017 21:59
value of 1, which will happen if the targetSize is represented by a null value at this juncture in the code.
*/
Integer targetSize = description.targetSize.toInteger()
if (targetSize < 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since targetSize can never be null or < 0, I don't think description.capacity will ever be considered.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll need to reason about on this... this will need to change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking the right way to approach this would be the following:

Defer to the desired source capacity, so if the targetSize is less than it - the desired source capacity wins. This still covers the use case of this PR as you wouldn't set a desired source capacity of > 0 if you wanted a targetSize of 0 or you wanted the resource at 0. Essentially, desired source capacity always trumps targetSize. Thoughts?

@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch from 80eb0fd to 6c3e42b Compare September 21, 2017 22:55
@Z3r0Sum
Copy link
Contributor Author

Z3r0Sum commented Sep 21, 2017

Scenarios Tested:

useSourceCapacity: false

Should be noted --- it appears desired capacity sets the targetSize

  • targetSize < desired capacity --- desired capacity value is used
  • targetSize > desired capacity -- desired capacity value is still used (it should be brought in line with targetSize) -- I'm not sure what the correct philosophical decision is here, but maybe we should set the desired capacity equal to the targetSize (seems to be happening).

useSourceCapacity: true

  • targetSize < desired capacity --- Does not matter, original Replicas Count is respected.
  • targetSize > desired capacity --- Does not matter, original Replicas Count is respected.

Edit: Interesting based on the UI interaction, it would seem desired capacity already overrides targetSize.

@Z3r0Sum
Copy link
Contributor Author

Z3r0Sum commented Sep 21, 2017

@lwander I'd appreciate you weighing in on this as you probably have some opinions on what should get precedence...

@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch 2 times, most recently from de7926d to 5f5caf8 Compare September 21, 2017 23:33
@Z3r0Sum
Copy link
Contributor Author

Z3r0Sum commented Sep 22, 2017

It seems like the desired source capacity is always null and just sets the targetSize, is this assumption correct?

@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch from 5f5caf8 to 22b2478 Compare September 22, 2017 00:32
def targetSize
if (description.targetSize == 0) {
targetSize = description.targetSize
} else { targetSize = description.targetSize ?: description.capacity?.desired }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you put this onto a newline?

def targetSize
if (description.targetSize == 0) {
targetSize = description.targetSize
} else { targetSize = description.targetSize ?: description.capacity?.desired }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch from 22b2478 to f54ce05 Compare September 25, 2017 13:30
The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.
@Z3r0Sum Z3r0Sum force-pushed the fix-replicaset-creation branch from f54ce05 to 56ade88 Compare September 25, 2017 18:27
@ethanfrogers ethanfrogers merged commit ab7c551 into spinnaker:master Sep 25, 2017
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Oct 26, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* Merged upstream, updated AbstractEcsCachingAgent and Keys.java
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Oct 26, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* target group is now a string, instead of an array of strigns
BrunoCarrier added a commit to lookout/clouddriver that referenced this pull request Nov 6, 2017
* feat(provider/kubernetes): Add spectator to v2 client (spinnaker#1866)

* refactor(provider/kubernetes): Share caching agent abstraction (spinnaker#1865)

* feat(provider/kubernetes): Fix on demand caching (spinnaker#1870)

* chore(provider/kubernetes): On demand caching tests (spinnaker#1871)

* feat(provider/kuberenetes): V2 deployments (spinnaker#1868)

* feat(provider/kubernetes): Cache arbitrary resource (spinnaker#1873)

* feat(search): update application search (spinnaker#1875)

* add ability to pass search filters down to application searching in
front50.

* Additional fix for issue 1632, not able to find AMI with encrypted snapshots in target/managed account (spinnaker#1846)

* docs(intellij): Describe how to enable lombok in intellij (spinnaker#1869)

I figured we'd add a quick one-line how-to for getting gradle to build your intellij project too.

* feat(provider/kubernetes): Infer relationships from ownerReference (spinnaker#1876)

* feat(provider/kubernetes): Infer relationships from ownerReference

* feat(provider/kubernetes): Cache ancillary resources

* feat(search): add filters to project search (spinnaker#1878)

* feat(provider/ecs): Made EcsCloudProvider public (spinnaker#1848)

* feat(provider/kubernetes): Cache pods (spinnaker#1879)

* feat(provider/kubernetes): Annotate templates as well (spinnaker#1880)

A number of k8s resources use a spec.template to deploy a managed
resource. These need to be annotated by Spinnaker as well.

* fix(aws/tags): ensure tag sync is always performed during allow launch

* fix(search): allow SearchProvider filter exclusion (spinnaker#1883)

* allows search providers to specify an exclusion list for filter keys
to optionally be removed before executing a search.

* feat(provider/aws): Remove ability to automatically remove dependencies when deleting a security group (spinnaker#1894)

* fix(provider/aws): trim whitespace when newlines are detected (spinnaker#1844)

* fix(google): Add statusCode tag to google API metrics (spinnaker#1886)

This also refactors all the tests involving google API metrics.

* fix(provider/kubernetes): Add missing credential props (spinnaker#1897)

* feat(provider/kubernetes): add daemon & stateful set support

*  fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)

* feat(provider/kubernetes): v2 generic on demand caching (spinnaker#1884)

* refactor(provider/kubernetes): v2 include api version in resource id (spinnaker#1901)

* feat(provider/docker): add catalogFile option (spinnaker#1890)

adds the `catalogFile` option to the Docker Registry provider. This is
an attempt to make it easier to use registries that don't support
`v2/_catalog` (which happens to be a lot). This file can be managed by
an external process. Each caching cycle, Clouddriver reads this file to
determine the list of repositories it should cache tags for.

the format of the `configFile` is:
```
{
  "repositories":[
    "library/nginx",
    "library/busybox"
  ]
}
```

* fix(provider/docker): fix client constructor (spinnaker#1903)

Bug introduced in `f70d6ea`. I updated the constructor to include
`catalogFile` but didn't update the constructor for those using
`password` and `passwordFile` configurations. This fixes that potential
issue.

* feat(provider/kubernetes): v2 Cache lb & scg relationships (spinnaker#1902)

* Revert " fix(provider/aws): ensure STSAssumeRoleSessionCredentialsProvider get the right endpoint (spinnaker#1888)" (spinnaker#1904)

This reverts commit 828fbb7.

* feat(provider/kubernetes): v2 version deployed manifests as resources (spinnaker#1905)

* refactor(provider/kubernetes): V2 swap kind & version in cache key (spinnaker#1907)

For consistency with naming & other resources.

* fix(provider/aws): Clone spot price if not explicitly specified. (spinnaker#1908)

* refactor(provider/kubernetes): Segregate k8s by provider version (spinnaker#1909)

* feat(provider/kubernetes): Add proper support for source capacity. (spinnaker#1906)

Fully addresses: spinnaker/spinnaker#1657

  Also accounts for the possibility that the user might want to set
  their own targetSize while ignoring the source capacity.

  Deck work is still needed to expose "useSourceCapacity" in the UI.

  Fix cross namespace server groups

* feat(provider/kubernetes): Register k8s moniker for manifests (spinnaker#1910)

* feat(provider/kubernetes): V2 cluster provider (spinnaker#1911)

* refactor(core): s/version/providerVersion (spinnaker#1914)

Having the account property named 'version' was confusing the concept of
the version the provider was at, with the version the cloud environment
was at.

* refactor(provider/kubernetes): Rename duplicate classes (spinnaker#1913)

* feat(moniker): Adds moniker to all clusters & server groups (spinnaker#1915)

* chore(javadocs): Fix lt usage (spinnaker#1916)

* feat(provider/kubernetes): Attach k8s annotation monikers to v2 (spinnaker#1917)

resources

* fix(aws): CopyLastAsgAtomicOperation instance monitoring.

When not explicitly specified, instance monitoring would default to true due to groovy truth casting a non
null object to true instead of traversing into its enabled attribute

* fix(web): Guard against not finding a ClusterProvider (spinnaker#1921)

* feat(search): add fallback query param to search (spinnaker#1912)

* in an effort to speed up searching, this adds logic that attempts to
pick a "good" value to use as a search query string rather than just
using the empty string and pulling all the keys for a given type.

* feat(provider/kubernetes): V2 Cluster details (spinnaker#1918)

* chore(provider/kubernetes): Use version 0.2.0 of client-java (spinnaker#1923)

* fix(provider/kubernetes): Advanced targetSize use case. (spinnaker#1920)

The elvis operator before was not evaluating truthiness correctly for a value
of 0. A User could and may want to deploy an update ReplicaSet of 0 and Scale
it later. Or perhaps it's bundled in a Spinnaker Pipeline with multiple Server
Groups and one of the Server Groups is scaled to 0. If the User is implementing
'useSourceCapacity' we would want that 0 value respected and not overwritten
with a value of 1.

The proposal is that the desired capacity trumps targetSize.  It's
reasonable to assume that someone who wants a targetSize of 0 would also
have a desired capacity of 0.  However, we shouldn't use the targetSize
if the desired capacity is greater than its value.

* refactor(provider/kubernetes): v2 cache view lives in cache package (spinnaker#1926)

* feat(provider/kubernetes): v2 cache network policies (spinnaker#1927)

* feat(provider/kubernetes): Build v2 creds based on context & kubeconfigFile (spinnaker#1925)

* feat(provider/kubernetes): Integrate moniker into v2 (spinnaker#1929)

* fix(provider/kubernetes): v2 guard against empty cache relationships (spinnaker#1932)

* fix(provider/kuberentes): v2 guard against non-existent clusters (spinnaker#1934)

* feat(moniker): adds moniker to server group view model (spinnaker#1936)

* feat(provider/kubernetes): Cache v2 resources as artifacts (spinnaker#1931)

* fix(all): Broken k8s server group lookup breaking all components (spinnaker#1937)

@lwander I'm merging for short term fix since everyone is affected. However see my question about returning null and whether there are other implications.

* feat(provider/google): Adds support for multiple named ports in load balancers. (spinnaker#1930)

* refactor(provider/kubernetes): refactor artifact gen (spinnaker#1938)

* feat(moniker): Adds moniker to pendingOnDemandResults (spinnaker#1940)

* fix(provider/google): Fix named port defaults in server group deploy. (spinnaker#1939)

* feat(moniker): Add monikers to security groups & load balancers (spinnaker#1941)

* feat(provider/google): Support connection draining for LBs (spinnaker#1943)

* fix(provider/gce): s/it/cacheData/ in zonal svg caching agent. (spinnaker#1944)

* feat(provider/docker): Added insecure registry support (spinnaker#1887)

* feat(provider/docker): Added insecure registry support

Added an insecureRegistry flag to clouddriver-docker registry
support that will skip SSL certificate validation.

This will allow for addressing spinnaker/halyard#680.

* Update DockerRegistryNamedAccountCredentials.groovy

* Update DockerRegistryClientSpec.groovy

* feat(appengine): Deploy from Google Cloud Storage [accounts]. (spinnaker#1935)

* fix(provider/docker): Assign proper Spinnaker/<version> user agent (spinnaker#1946)

* feat(provider/kubernetes): Properly version all versioned resources (spinnaker#1945)

* fix(rollbacks): support for tolerating some instance failures (spinnaker#1922)

* fix(provider/gce): Paginate server groups when calculating next name. (spinnaker#1948)

* fix(appengine): Dont assume storage.gce.enabled=true (spinnaker#1953)

* fix(provider/kubernetes): fix k8s client configuration and image id p… (spinnaker#1933)

* fix(provider/kubernetes): fix k8s client configuration and image id problem.

* (feat/cats) Add support for an error interval for clustered agent schedulers (spinnaker#1895)

* feat(search): cull missing ASGs before returning (spinnaker#1951)

* removes ASGs that show up in the search results from the redis cache
but don't actually exist.

* fix(provider/kubernetes): Fixed an issue with jobProvider failing out other providers. (spinnaker#1690)

* fix(provider/dcos): fix cluster provider to guard against errors from invalid ids (spinnaker#1928)

* feat(core): Add provider version to creds controller (spinnaker#1958)

* feat(appengine): Add AppengineStorageController to list accounts (spinnaker#1957)

* chore(provider/kubernetes): Bump client to 1.0.0-beta1 (spinnaker#1961)

* fix(provider/kubernetes): v1 client call to list pods (spinnaker#1963)

* fix(provider/aws): Return load balancers for all target groups associated with an application (spinnaker#1955)

* refactor(provider/kubernetes): V2 deploy 1 manifest at a time (spinnaker#1965)

* feat(provider/gae): Modify deploy description and validators for artifacts. (spinnaker#1966)

* feat(provider/kubernetes): Feature-flag debug mode (spinnaker#1968)

* fix(provider/kubernetes): Don't store-unowned artifacts (spinnaker#1969)

* feat(provider/kuberentes): Application provider (spinnaker#1971)

* fix(provider/aws): fix issue with VPCZoneIdentifier containing deleted subnets (spinnaker#1972)

* chore(provider/docker): Only log trailing whitespace warning during failure (spinnaker#1973)

* feat(provider/kubernetes): V2 cluster provider finishing touches (spinnaker#1974)

* fix(provider/aws): Allow updating healthy threshold count on target groups (spinnaker#1977)

* feat(moniker): Add getServerGroup endpoint that does not require application name. (spinnaker#1967)

* feat(provider/kubernetes): v2 instance counts (spinnaker#1975)

* feat(provider/kuberentes): Register kind mapping using autowired deploy (spinnaker#1978)

* refactor(provider/kubernetes): Move manifest desc. into package (spinnaker#1979)

* refactor(provider/kubernetes): Move ops w/ kind into v1 only (spinnaker#1980)

* feat(provider/kubernetes): Support resize statefulset operation. (spinnaker#1976)

* feat(dynomite): hashtags & dynomite pipelines (spinnaker#1832)

* refactor(provider/google): Rely on cached instance details instead of calling platform MIG.listInstances() api. (spinnaker#1983)

* feat(provider/amazon): Create app elb specific security group for new ELBs (spinnaker#1982)

- Autocreate elb security group and ingress on application security group

* feat(core/search): Improve search speed: faster validation that search keys exists (spinnaker#1984)

* feat(provider/kubernetes): Support resize operations (spinnaker#1981)

* feat(provider/kubernetes): V2 instance state (spinnaker#1986)

* fix(provider/kubernetes): Fix v2 operation converter (spinnaker#1987)

* feat(provider/gce): Replace instanceTemplate get with list call. (spinnaker#1990)

* refactor(provider/kubernetes): use deployed names in result (spinnaker#1989)

* fix(core): Support the eviction of stale cache key identifiers (spinnaker#1985)

This PR attempts to clean up situations where the `:members` set for a
given cached type contains keys that no longer exist.

In particular, a `CachingAgent` can optionally return a pattern glob for
each authoritative type.

Any `:members` key matching the pattern glob BUT not included in the
most recent caching cycle will be evicted.

This has been enabled on a select few `aws` caching agents.

* fix(aws): Do not do stale key evictions of amazon load balancers (spinnaker#1993)

Seeing some mixed results when application/network and classic load
balancers are being used.

* feat(provider/kubernetes): v2 on-demand manifest caching (spinnaker#1988)

* feat(provider/aws): add executionId to user-agent for cloudtrail (spinnaker#1992)

* fix(aws): Enable stale key evictions for amazon load balancers (spinnaker#1994)

Key structures vary between classic and application/networking load
balancers.

* feat(provider/kubernetes): Re-encode api types (spinnaker#1991)

* feat(logging) compile logstashEncoder for json logging (spinnaker#1995)

* fix(provider/gae): Set path correct when application root non-empty. (spinnaker#1970)

* chore(dependencies): Bump spinnaker-dependencies to 0.115.0. (spinnaker#1996)

* fix hidden exception while pulling docker tags (spinnaker#1708)

* fix(provider/kubernetes): reads correct fields from cache refresh (spinnaker#1998)

* feat(provider/kubernetes): v2 delete operation (spinnaker#1999)

* feat(provider/dcos): Adding proxy for marathon-client to make use of spectator. (spinnaker#1960)

* fix(appengine): fix deploy failures (spinnaker#2001)

* feat(provider/kubernetes): Supply app name in attributes (spinnaker#2003)

* feat(provider/kuberentes): implement capacity (spinnaker#2004)

* feat(cats): Adding cache gzip compression (spinnaker#2000)

* feat(provider/kubernetes): Update deployable's patch behavior (spinnaker#2006)

* feat(provider/kubernetes): Show more in the error message (spinnaker#2007)

* feat(provider/kubernetes): V2 Cache services (spinnaker#2009)

* fix(web): Fixing missing property exception (spinnaker#2008)

* fix(cats): Get bytes in consistent chatset during decompress (spinnaker#2010)

* fix(cats): Remove use of unmodifiable collections (spinnaker#2011)

* feat(provider/kubernetes): cache non-obvious relationship (spinnaker#2012)

* feat(provider/kubernetes): load balancer -> sg relationship (spinnaker#2013)

* refactor(aws): Simplified ELB security group autocreation (spinnaker#1997)

- updated code that allows ingress from elb group to app security group

* fix(provider/gae): Delete gcs target directory in deploy. (spinnaker#2015)

* fix(provider/gae): Delete gcs target directory in deploy.

* fix(storage): Change the storage account endpoint to '/storage'.

* feat(provider/kubernetes): Enable setting of labels on k8s Service objects spinnaker/spinnaker#2035 (spinnaker#2014)

* fix(provider/kubernetes): allows dots in secretNames (spinnaker#2017)

The kubernetes api allows dots in secret names:
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L160-L162
https://github.com/kubernetes/kubernetes/blob/8d7d7a5e0d4d7e75f5a860574346944b8cc0fc43/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L126-L142
https://github.com/kubernetes/kubernetes/blob/e5ac41331119bb8c67004b8efaa0c3d942b31d3a/pkg/api/validation/validation.go#L1488
This allows dots in the secretName field of volume source definitions.

* perf(cluster): Avoid prematurely fetching full server groups (spinnaker#2019)

This is an optimization for fetching target server groups by `CURRENT`,
`PREVIOUS`, `OLDEST` and `FAIL`.

For these strategies, the unexpanded server group is sufficient to
narrow down to a match.

Once a match is found, _only_ that server group will be expanded.

This only works for cluster providers that support loading both
expanded and unexpanded server groups (ie. aws). The behavior for other
providers should be unchanged.

* fix(aws): Fix NPE in elb group autocreation (spinnaker#2020)

- fixed NPE due to missing credentials in security group description request

* feat(provider/kubernetes): Enable annotations, labels and secrets for security groups (aka ingress resources) (spinnaker#2000) (spinnaker#2005)

* feat(google): Instrument individual google API calls. (spinnaker#2016)

* fix(cats): Remove unmodifiable collections from modifiable codepaths (spinnaker#2022)

* fix(provider/gce): Fix broken deploys with namedPorts. (spinnaker#2023)

* fix(aws): Only autocreate app elb security group on Create (spinnaker#2025)

- Updated to only auto create <appname>-elb group for new ELBs

* feat(provider/aws): Support specifying explicit subnet ids for deploy (spinnaker#2026)

Expectation is that a `subnetType` will still be provided and that
`subnetIds` will be a subnet of those valid for `subnetType`.

When explicit subnet ids are provided (or inherited), the newly created
server group will be tagged with:

`SPINNAKER_SUBNET_ID_OVERRIDE`: `"comma-separated list of subnet ids"`

* fix(provider/gce): Paginate instance template list in svg caching. (spinnaker#2027)

* fix(eureka): fix instanceId on eureka caching

The instanceId was intended to be the value in dataCenterInfo.metadata but due to some
Jackson behaviour the top level instanceId from the discovery registration was written
over the dataCenterInfo.metadata during deserialization.

This changes to explicitly handle the top level instanceId value in the JsonCreator factory
method to prevent the unhandled property behaviour from kicking in.

* Updating API for IAM role dropdown to work nicely with Deck
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants