Skip to content
This repository has been archived by the owner on Jan 19, 2022. It is now read-only.

Feature/AppConfig as PropertySource #652

Open
wants to merge 74 commits into
base: 2.3.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
a6e7c73
Fix AwsSecretsManagerProperties.prefix javadoc
sixcorners Aug 20, 2019
d52dae1
Bumping versions
spring-builds Jan 20, 2020
33e1535
Bumping versions
spring-builds Jan 21, 2020
328b130
Updated docs
marcingrzejszczak Apr 1, 2020
9294f1c
Merge branch '2.2.x'
spencergibb Apr 22, 2020
5ac4cdb
Bumps versions to 3.0.0-SNAPHOT
spencergibb Apr 22, 2020
6d84666
Merge pull request #484 from sixcorners/patch-1
maciejwalkowiak May 29, 2020
e540b7c
Update of documentation regarding supported DB engines that Amazon RD…
DJakosa Mar 27, 2019
37b1251
Fix "Sending a message" reference documentation code samples.
luramarchanjo May 30, 2020
c935e72
Fixes SNS auto configuration when Spring WebMvc is not on the classpath.
kingbbode May 1, 2020
5f5d8bc
Remove the misleading property in documentation of spring-cloud-aws-rds.
SenthilPanneerselvam Apr 22, 2020
e805878
Rename config properties in ContextCredentialsAutoConfiguration to ke…
lukasznyrc Jun 18, 2019
4c11014
Add tests for relaxed binding support in "cloud.aws.credentials".
tac-yacht Mar 28, 2020
6e3f32d
Allow ConditionalOnAwsCloudEnvironment to be used on method level.
bsamartins Mar 13, 2019
e72a3d2
Set CloudWatchProperties default batch size to API max.
bsamartins Mar 14, 2019
159200b
Add option to use default AWS region provider chain.
maciejwalkowiak May 31, 2020
f6100dd
Add logging to AWSParameterStorePropertySource.
cschroedl-gov Feb 27, 2020
6da9925
Update documentation on how to hide warning message logged by AWS SDK…
maciejwalkowiak May 31, 2020
8caf811
Fix typo in the exception text
tmnuwan12 May 31, 2020
7a052a5
Bump AWS Java SDK dependency version to 1.11.792.
maciejwalkowiak May 31, 2020
f2ab807
Update reference documentation on how to upgrade AWS SDK dependency.
maciejwalkowiak May 31, 2020
58e636d
Fix S3 object key encoding in `SimpleStorageResource.getURL`.
Jun 1, 2020
deb9088
Fix duplicate error logging when processing exception thrown in SQS l…
sayembd Mar 29, 2019
76df24c
Allow to specify PropertySource name returned by AwsSecretsManagerPro…
cbm-afoulfoin Jun 25, 2019
eb4d65a
Add a dependency to JUnit 4.
maciejwalkowiak Jun 6, 2020
f61178d
Document default credentials chain for auto-configuration.
maciejwalkowiak Jun 6, 2020
211990c
Add CloudWatch integration documentation to the reference.
maciejwalkowiak Jun 6, 2020
2a59abf
Add a method to get S3 URI from a SimpleStorageResource.
tmnuwan12 Jun 6, 2020
647375b
Fix StackResourceRegistry creation when CloudFormation support is tur…
maciejwalkowiak Jun 7, 2020
38c7e6a
Drop XML support in all modules and integration tests.
maciejwalkowiak Jun 1, 2020
8af3aca
Changed packaging to jar
marcingrzejszczak Jun 8, 2020
6a7580b
Remove Autowired annotations
eddumelendez Jun 7, 2020
80391e9
Enforce spring-javaformat plugin
eddumelendez Jun 9, 2020
650cc69
Use DefaultAWSCredentialsProvider and DefaultAWSRegionProviderChain b…
maciejwalkowiak Jun 8, 2020
68799fb
Migrate to JUnit 5.
DarrenForsythe Jun 8, 2020
0c84250
Merge branch '2.3.x'
eddumelendez Jun 12, 2020
11e5078
Merge branch '2.3.x'
eddumelendez Jun 12, 2020
7817a37
Configure SQS to use existing ObjectMapper if present in application …
maciejwalkowiak Jun 12, 2020
ddde60b
Drop dependency to javax.validation in Parameter Store and Secrets Ma…
MatejNedic Jun 8, 2020
dea203b
Merge branch '2.3.x'
maciejwalkowiak Jun 17, 2020
babe84f
Merge branch '2.3.x'
maciejwalkowiak Jun 18, 2020
6148d73
Merge branch '2.3.x'
maciejwalkowiak Jun 18, 2020
fec66b9
Merge branch '2.3.x'
maciejwalkowiak Jun 19, 2020
eea7e8f
Merge branch '2.3.x'
maciejwalkowiak Jun 19, 2020
9e5c01c
Merge branch '2.3.x'
maciejwalkowiak Jun 19, 2020
045fd9c
Merge branch '2.3.x'
maciejwalkowiak Jun 20, 2020
582fdc6
Merge branch '2.3.x'
maciejwalkowiak Jun 21, 2020
53275a2
Merge branch '2.3.x'
maciejwalkowiak Jun 21, 2020
2f63fe3
Merge branch '2.3.x'
maciejwalkowiak Jun 26, 2020
2b26b92
Merge branch '2.3.x'
eddumelendez Jun 27, 2020
82a873c
Merge branch '2.3.x'
maciejwalkowiak Jul 7, 2020
7f01157
Merge branch '2.3.x'
eddumelendez Jul 8, 2020
ff28c4b
Update version
eddumelendez Jul 8, 2020
7244864
Merge branch '2.3.x'
eddumelendez Jul 8, 2020
fcbb893
Merge branch '2.3.x'
maciejwalkowiak Jul 8, 2020
26c5435
Merge branch '2.3.x'
eddumelendez Jul 9, 2020
6f67aeb
Merge branch '2.3.x'
eddumelendez Jul 13, 2020
e72fce9
Merge branch '2.3.x'
eddumelendez Jul 13, 2020
926a6ca
Merge branch '2.3.x'
maciejwalkowiak Jul 14, 2020
10d2b77
Merge branch '2.3.x'
maciejwalkowiak Jul 26, 2020
b506a9f
Merge branch '2.3.x'
maciejwalkowiak Aug 7, 2020
24b2068
Merge branch '2.3.x' into master
Aug 10, 2020
4d0b386
feat(aws-appconfig): add appconfig & starter-appconfig modules
jarpz Sep 7, 2020
7eba07c
feat(aws-appconfig): add aws appconfig component
jarpz Sep 7, 2020
241d6d3
feat(aws-appconfig): add starter appconfig
jarpz Sep 7, 2020
33d2559
feat(aws-appconfig): rename module removing config suffix
jarpz Sep 15, 2020
d21e4dc
feat(aws-appconfig): move classes & fix comments
jarpz Sep 15, 2020
ebd950e
feat(aws-appconfig): remove unused dependencies
jarpz Sep 16, 2020
c373eec
test(aws-appconfig): change tests to use ApplicationContextRunner
jarpz Sep 17, 2020
fd58ab1
feat(aws-appconfig): change properties prefix to spring.cloud.aws.* +…
jarpz Sep 22, 2020
2cbf279
feat(aws-appconfig): use assert when appName is empty or null
jarpz Sep 25, 2020
33d96f5
feat(aws-appconfig): rename accountId as clientId + allow default ran…
jarpz Nov 30, 2020
b2cad8d
Merge branch 'master' of https://github.com/jarpz/spring-cloud-aws in…
jarpz Nov 5, 2021
e3cfc4d
feat: Align code with master branch & fix code style
jarpz Nov 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws</artifactId>
<version>2.3.0.BUILD-SNAPSHOT</version>
<version>3.0.0-SNAPSHOT</version>
</parent>
<artifactId>spring-cloud-aws-docs</artifactId>
<packaging>pom</packaging>
<packaging>jar</packaging>
<name>Spring Cloud AWS Docs</name>
<description>Spring Cloud AWS Docs</description>
<properties>
Expand Down Expand Up @@ -86,6 +86,10 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
</plugin>
Expand Down
6 changes: 4 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-build</artifactId>
<version>2.3.2.BUILD-SNAPSHOT</version>
<version>3.0.0-SNAPSHOT</version>
<relativePath/><!-- lookup parent from repository -->
</parent>

<artifactId>spring-cloud-aws</artifactId>
<version>2.3.0.BUILD-SNAPSHOT</version>
<version>3.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>Spring Cloud AWS</name>
<description>Spring Cloud AWS</description>
Expand Down Expand Up @@ -59,11 +59,13 @@
<module>spring-cloud-aws-messaging</module>
<module>spring-cloud-aws-autoconfigure</module>
<module>spring-cloud-aws-parameter-store-config</module>
<module>spring-cloud-aws-appconfig</module>
<module>spring-cloud-aws-secrets-manager-config</module>
<module>spring-cloud-starter-aws</module>
<module>spring-cloud-starter-aws-jdbc</module>
<module>spring-cloud-starter-aws-messaging</module>
<module>spring-cloud-starter-aws-parameter-store-config</module>
<module>spring-cloud-starter-aws-appconfig</module>
<module>spring-cloud-starter-aws-secrets-manager-config</module>
<module>spring-cloud-aws-integration-test</module>
<module>docs</module>
Expand Down
49 changes: 49 additions & 0 deletions spring-cloud-aws-appconfig/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2013-2020 the original author or authors.
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ https://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws</artifactId>
<version>3.0.0-SNAPSHOT</version>
</parent>

<artifactId>spring-cloud-aws-appconfig</artifactId>

<name>Spring Cloud AWS AppConfig Configuration</name>
<description>Spring Cloud AWS AppConfig Configuration</description>


<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
</dependency>

<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-appconfig</artifactId>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
/*
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.aws.appconfig;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.Collection;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Properties;
import java.util.Set;

import com.amazonaws.services.appconfig.AmazonAppConfig;
import com.amazonaws.services.appconfig.model.GetConfigurationRequest;
import com.amazonaws.services.appconfig.model.GetConfigurationResult;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;

import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.env.EnumerablePropertySource;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.util.ReflectionUtils;

/**
* @author jarpz
*/
public class AwsAppConfigPropertySource extends EnumerablePropertySource<AmazonAppConfig> {

private static final String SUPPORTED_TYPE_JSON = "application/json";

private static final String SUPPORTED_TYPE_YAML = "application/x-yaml";

private final String clientId;

private final String application;

private final String configurationVersion;

private final String environment;

private Properties properties;

public AwsAppConfigPropertySource(String name, String clientId, String application, String environment,
String configurationVersion, AmazonAppConfig appConfigClient) {
super(name, appConfigClient);
this.clientId = clientId;
this.application = application;
this.configurationVersion = configurationVersion;
this.environment = environment;
}

public void init() {
GetConfigurationRequest request = new GetConfigurationRequest().withClientId(clientId)
.withApplication(application).withConfiguration(name)
.withClientConfigurationVersion(configurationVersion).withEnvironment(environment);

getAppConfig(request);
}

@Override
public String[] getPropertyNames() {
Set<String> strings = properties.stringPropertyNames();
return strings.toArray(new String[0]);
}

@Override
public Object getProperty(String name) {
return properties.get(name);
}

private void getAppConfig(GetConfigurationRequest request) {
GetConfigurationResult result = this.source.getConfiguration(request);

logger.trace(String.format("loading file: %s/%s/%s/%s", application, name, environment,
result.getConfigurationVersion()));

switch (result.getContentType()) {
case SUPPORTED_TYPE_YAML:
processYamlContent(result.getContent());
break;
case SUPPORTED_TYPE_JSON:
processJsonContent(result.getContent());
break;
default:
throw new IllegalStateException(String.format("Unsupported content type: %s", result.getContentType()));
}
}

private void processYamlContent(ByteBuffer byteBuffer) {
YamlPropertiesFactoryBean bean = new YamlPropertiesFactoryBean();

bean.setResources(new ByteArrayResource(byteBuffer.array()));

properties = bean.getObject();
}

private void processJsonContent(ByteBuffer byteBuffer) {
try {
Map<String, Object> map = new ObjectMapper().readValue(byteBuffer.array(),
new TypeReference<Map<String, Object>>() {
});

Map<String, Object> result = new LinkedHashMap<>();
this.flatten(null, result, map);

properties = new Properties();
properties.putAll(result);
}
catch (IOException ex) {
ReflectionUtils.rethrowRuntimeException(ex);
}
}

/**
* flatten json structure.
*/
private void flatten(String prefix, Map<String, Object> result, Map<String, Object> map) {
String namePrefix = Objects.nonNull(prefix) ? prefix + "." : "";

map.forEach((key, value) -> this.extract(namePrefix + key, result, value));
}

private void extract(String name, Map<String, Object> result, Object value) {
if (value instanceof Map) {
this.flatten(name, result, (Map) value);
}
else if (value instanceof Collection) {
int index = 0;

for (Iterator it = ((Collection) value).iterator(); it.hasNext(); ++index) {
Object object = it.next();
this.extract(name + "[" + index + "]", result, object);
}
}
else {
result.put(name, value);
}
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
/*
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.aws.appconfig;

import java.util.Arrays;

import com.amazonaws.services.appconfig.AmazonAppConfig;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import org.springframework.cloud.bootstrap.config.PropertySourceLocator;
import org.springframework.core.env.CompositePropertySource;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.Environment;
import org.springframework.core.env.PropertySource;
import org.springframework.util.ReflectionUtils;

import static java.util.Objects.isNull;
import static org.springframework.util.Assert.hasText;

/**
* @author jarpz
*/
public class AwsAppConfigPropertySourceLocator implements PropertySourceLocator {

private final AmazonAppConfig appConfigClient;

private final String clientId;

private final String application;

private final String configurationProfile;

private final String environment;

private final String configurationVersion;

private final boolean failFast;

private static final Log logger = LogFactory.getLog(AwsAppConfigPropertySourceLocator.class);

public AwsAppConfigPropertySourceLocator(AmazonAppConfig appConfigClient, String clientId, String application,
String configurationProfile, String environment, String configurationVersion, boolean failFast) {
this.appConfigClient = appConfigClient;
this.clientId = clientId;
this.application = application;
this.configurationProfile = configurationProfile;
this.environment = environment;
this.configurationVersion = configurationVersion;
this.failFast = failFast;
}

@Override
public PropertySource<?> locate(Environment environment) {
if (!(environment instanceof ConfigurableEnvironment)) {
return null;
}

ConfigurableEnvironment env = (ConfigurableEnvironment) environment;

String appName = configurationProfile;
if (isNull(appName)) {
appName = env.getProperty("spring.application.name");
}
String profile = this.environment;
if (isNull(profile)) {
profile = Arrays.stream(env.getActiveProfiles()).findFirst().orElse("default");
}

hasText(appName, "configurationProfile or spring.application.name should not be empty or null.");
hasText(profile, "environment or profiles should not be empty or null ");

CompositePropertySource composite = new CompositePropertySource("aws-app-config");

try {
composite.addPropertySource(create(appName, profile));
}
catch (Exception ex) {
if (failFast) {
logger.error("Fail fast is set and there was an error reading configuration from AWS AppConfig: {}",
ex);
ReflectionUtils.rethrowRuntimeException(ex);
}
else {
logger.warn("Unable to load AWS AppConfig from " + appName, ex);
}
}

return composite;
}

private AwsAppConfigPropertySource create(String appName, String profile) {
AwsAppConfigPropertySource propertySource = new AwsAppConfigPropertySource(appName, clientId, application,
profile, configurationVersion, appConfigClient);
propertySource.init();

return propertySource;
}

}
Loading