From fc7825e3a00bb24683d150c36bcfad6c5c0bd1d7 Mon Sep 17 00:00:00 2001 From: eight-nines <73156903+eight-nines@users.noreply.github.com> Date: Sat, 14 Jan 2023 09:03:15 +0800 Subject: [PATCH] [ISSUE #2576] Enhanced Configuration management (#2917) * [ISSUE #2576]Enhancement the config manage for project * [1] test code * EventMeshServer config improve * Extract the key prefix of the @ConfigFiled field into @Config * Improve configuration management of SPI implementations [1] config improve of RocketMQProducerImpl and RocketMQConsumerImpl [2] config UT of RocketMQProducerImpl and RocketMQConsumerImpl * Improve configuration management of SPI implementations [1] extensionInstance config init process improve [2] RocketMQProducerImpl and RocketMQConsumerImpl config UT improve * Improve configuration management of Kafka SPI Impl [1] KafkaConsumerImpl and KafkaProducerImpl config improve [2] KafkaConsumerImpl and KafkaProducerImpl config UT * Improve configuration management of Redis SPI Impl [1] RedisProperties config improve * Improve configuration management of Redis SPI Impl [1] RedisProperties config and UT improve [2] New parser ConvertProperties [3] To fix a bug where parser convertEnum could not parse null values * Fix config UT of Redis SPI Impl * Tweak Convert's code framework, for ease of enhancement (#2784) [1] Split the converter classes in Convert into separate files [2] Add the converter field to ConfigFiled, used to specify the converter class of the field * Improve configuration management of Knative SPI Impl (#2791) [1] KnativeConsumerImpl and KnativeProducerImpl config improve [2] KnativeConsumerImpl and KnativeProducerImpl config UT * Improve configuration management of Pravega SPI Impl [1] PravegaConsumerImpl and PravegaProducerImpl config improve [2] PravegaConsumerImpl and PravegaProducerImpl config UT [3] add new converter URIConverter * Improve configuration management of Pulsar SPI Impl [1] PulsarConsumerImpl and PulsarProducerImpl config improve [2] PulsarConsumerImpl and PulsarProducerImpl config UT * Improve configuration management of Rabbitmq SPI Impl [1] RabbitmqConsumer and RabbitmqProducer config improve [2] RabbitmqConsumer and RabbitmqProducer config UT * Improve configuration management of Prometheus SPI Impl [1] PrometheusMetricsRegistry config improve [2] PrometheusMetricsRegistry config UT * Improve configuration management of AuthHttpBasic SPI Impl [1] AuthHttpBasicService config improve [2] AuthHttpBasicService config UT * Enhance configuration management capabilities - @ConfigFiled [1] Add two properties to @ConfigFiled,findEnv / notNull, and the configuration file parsing capabilities that go with them [2] Delete @NotNull and replace it with @ConfigFiled.notNull * Improve configuration management of Pinpoint SPI Impl [1] PinpointTraceService config improve [2] PinpointTraceService config UT * Improve configuration management of Zipkin SPI Impl [1] ZipkinTraceService config improve [2] ZipkinTraceService config UT * Add repeatable functionality to @Config For tagging multiple configuration fields on a class * Improve configuration management of TracePlugin [1] ExporterConfiguration config improve [2] ExporterConfiguration config UT * Improve configuration management of WebHook [1] WebHook config improve [2] WebHook config UT * Add field beNumber for @ConfigFiled [1] If String config field not a number, an exception is thrown * Change the configuration management class CommonConfiguration to a new framework [1] Replace the CommonConfiguration field modifier public with private [2] Use getter/setter methods to access it * Change CommonConfiguration subclass to new framework [1] EventMeshHTTPConfiguration/EventMeshGrpcConfiguration/EventMeshTCPConfiguration * Change CommonConfiguration and subclass to new framework [1] EventMeshHTTPConfiguration/EventMeshGrpcConfiguration/EventMeshTCPConfiguration * Add field notEmpty for @ConfigFiled [1] If it cannot be empty but is empty, an exception is thrown * Change the SPI RocketMQ config to new framework * Change the SPI kafka config to new framework * Change the SPI Redis config to new framework * Change the SPI knative config to new framework * Change the SPI pravega config to new framework * Change the SPI pulsar config to new framework * Change the SPI rabbitmq config to new framework * Change the SPI Prometheus config to new framework * Change the SPI auth-http-basic config to new framework * Change the SPI trace-api config to new framework * Change the SPI pinpoint config to new framework * Change the SPI zipkin config to new framework * Use getter/setter methods to access EventMeshGrpcConfiguration * Use getter/setter methods to access EventMeshHTTPConfiguration * Enhanced Configuration management * remove temporary blocking @Ignore * fix styleCheck * fix UT * rename ConfigService method * Revision review comments * Fixed code review comments [1] save the last accessible status of reflection field and recovery it after custom operation * Fixed code review comments [1] CLASS_PATH_PREFIX/FILE_PATH_PREFIX constants optimization [2] Some other code details optimization * Fixed code review comments [1] add UT for ConfigService#getConfig(ConfigInfo configInfo) Co-authored-by: githublaohu <2372554140@qq.com> Co-authored-by: mike_xwm --- eventmesh-common/build.gradle | 1 + .../common/config/CommonConfiguration.java | 198 +++------ .../eventmesh/common/config/Config.java | 54 +++ .../eventmesh/common/config/ConfigFiled.java | 80 ++++ .../eventmesh/common/config/ConfigInfo.java | 49 +++ .../common/config/ConfigMonitorService.java | 73 +++ .../common/config/ConfigService.java | 184 ++++++++ .../common/config/ConfigurationWrapper.java | 183 -------- .../eventmesh/common/config/FileLoad.java | 90 ++++ .../common/config/convert/Convert.java | 39 ++ .../common/config/convert/ConvertInfo.java | 21 +- .../common/config/convert/ConvertValue.java | 78 ++++ .../common/config/convert/ConverterMap.java | 146 ++++++ .../converter/BaseDataTypeConverter.java | 100 +++++ .../convert/converter/DateConverter.java | 42 ++ .../convert/converter/EnumConverter.java | 36 ++ .../convert/converter/IPAddressConverter.java | 40 ++ .../convert/converter/ListConverter.java | 97 ++++ .../convert/converter/LocalDateConverter.java | 38 ++ .../converter/LocalDateTimeConverter.java | 38 ++ .../convert/converter/MapConverter.java | 67 +++ .../convert/converter/ObjectConverter.java | 212 +++++++++ .../converter/PropertiesConverter.java | 53 +++ .../convert/converter/StringConverter.java | 20 +- .../convert/converter/URIConverter.java | 22 +- .../config/CommonConfigurationTest.java | 56 ++- .../common/config/ConfigServiceTest.java | 93 ++++ .../common/utils/PropertiesUtilsTest.java | 13 +- .../test/resources/configuration.properties | 35 +- .../kafka/config/ClientConfiguration.java | 50 ++- .../kafka/config/ConfigurationWrapper.java | 83 ---- .../kafka/consumer/KafkaConsumerImpl.java | 16 +- .../kafka/producer/KafkaProducerImpl.java | 16 +- .../kafka/config/ClientConfigurationTest.java | 74 ++++ .../test/resources/kafka-client.properties | 30 ++ .../knative/config/ClientConfiguration.java | 50 +-- .../knative/config/ConfigurationWrapper.java | 72 --- .../knative/consumer/KnativeConsumerImpl.java | 14 +- .../knative/producer/KnativeProducerImpl.java | 14 +- .../config/ClientConfigurationTest.java | 51 +++ .../consumer/KnativeConsumerImplTest.java | 5 +- .../producer/KnativeProducerImplTest.java | 4 +- .../test/resources/knative-client.properties | 18 + .../pravega/PravegaConsumerImpl.java | 15 +- .../pravega/PravegaProducerImpl.java | 14 +- .../pravega/client/PravegaClient.java | 23 +- .../config/PravegaConnectorConfig.java | 92 +--- .../config/PravegaConnectorConfigWrapper.java | 72 --- .../pravega/client/PravegaClientTest.java | 11 +- .../config/PravegaConnectorConfigTest.java | 69 +-- .../resources/pravega-connector.properties | 16 +- .../pulsar/config/ClientConfiguration.java | 37 +- .../pulsar/config/ConfigurationWrapper.java | 74 ---- .../pulsar/consumer/PulsarConsumerImpl.java | 13 +- .../pulsar/producer/ProducerImpl.java | 11 +- .../pulsar/producer/PulsarProducerImpl.java | 14 +- .../config/ClientConfigurationTest.java | 52 +++ .../test/resources/pulsar-client.properties | 20 + .../connector/rabbitmq/config/ConfigKey.java | 32 -- .../rabbitmq/config/ConfigurationHolder.java | 49 +-- .../rabbitmq/config/ConfigurationWrapper.java | 67 --- .../rabbitmq/consumer/RabbitmqConsumer.java | 12 +- .../rabbitmq/producer/RabbitmqProducer.java | 12 +- .../connector/rabbitmq/RabbitmqServer.java | 6 +- .../config/ConfigurationHolderTest.java | 62 +++ .../test/resources/rabbitmq-client.properties | 12 +- .../redis/client/RedissonClient.java | 62 +-- .../connector/redis/config/ConfigOptions.java | 51 --- .../redis/config/ConfigurationWrapper.java | 74 ---- .../redis/config/RedisProperties.java | 55 +-- .../redis/config/RedisPropertiesTest.java | 47 ++ .../test/resources/redis-client.properties | 7 +- .../rocketmq/admin/RocketMQAdmin.java | 5 +- .../rocketmq/admin/command/Command.java | 56 +++ .../rocketmq/config/ClientConfiguration.java | 196 ++------- .../rocketmq/config/ConfigurationWrapper.java | 68 --- .../consumer/RocketMQConsumerImpl.java | 27 +- .../producer/RocketMQProducerImpl.java | 10 +- .../config/ClientConfigurationTest.java | 65 +++ .../test/resources/rocketmq-client.properties | 15 + .../src/main/resources/application.properties | 2 +- .../eventmesh-metrics-prometheus/build.gradle | 2 +- .../prometheus/PrometheusMetricsRegistry.java | 13 +- .../config/PrometheusConfiguration.java | 59 +-- .../config/PrometheusConfigurationTest.java | 16 +- .../src/test/resources/prometheus.properties | 2 +- .../test/java/ConsulRegistryServiceTest.java | 2 +- .../etcd/service/EtcdRegistryServiceTest.java | 2 +- .../nacos/service/NacosRegistryService.java | 1 + .../service/NacosRegistryServiceTest.java | 2 +- .../service/ZookeeperRegistryServiceTest.java | 2 +- eventmesh-runtime/conf/eventmesh.properties | 2 +- .../admin/handler/ConfigurationHandler.java | 8 +- .../runtime/boot/EventMeshGrpcBootstrap.java | 10 +- .../runtime/boot/EventMeshGrpcServer.java | 42 +- .../runtime/boot/EventMeshHTTPServer.java | 105 ++--- .../runtime/boot/EventMeshHttpBootstrap.java | 13 +- .../runtime/boot/EventMeshServer.java | 20 +- .../runtime/boot/EventMeshStartup.java | 12 +- .../runtime/boot/EventMeshTCPServer.java | 1 - .../runtime/boot/EventMeshTcpBootstrap.java | 13 +- .../runtime/boot/SSLContextFactory.java | 6 +- .../EventMeshGrpcConfiguration.java | 263 +++-------- .../EventMeshHTTPConfiguration.java | 416 +++--------------- .../EventMeshTCPConfiguration.java | 219 ++------- .../grpc/consumer/ConsumerManager.java | 2 +- .../grpc/consumer/EventMeshConsumer.java | 1 - .../grpc/push/WebhookPushRequest.java | 2 +- .../core/protocol/grpc/retry/GrpcRetryer.java | 33 +- .../grpc/service/ConsumerService.java | 8 +- .../grpc/service/HeartbeatService.java | 2 +- .../grpc/service/ProducerService.java | 6 +- .../processor/BatchSendMessageProcessor.java | 18 +- .../BatchSendMessageV2Processor.java | 6 +- .../LocalSubscribeEventProcessor.java | 8 +- .../RemoteSubscribeEventProcessor.java | 6 +- .../RemoteUnSubscribeEventProcessor.java | 8 +- .../http/processor/ReplyMessageProcessor.java | 6 +- .../processor/SendAsyncEventProcessor.java | 4 +- .../processor/SendAsyncMessageProcessor.java | 6 +- .../SendAsyncRemoteEventProcessor.java | 4 +- .../processor/SendSyncMessageProcessor.java | 6 +- .../http/processor/SubscribeProcessor.java | 4 +- .../processor/inf/AbstractEventProcessor.java | 4 +- .../core/protocol/http/retry/HttpRetryer.java | 8 +- .../tcp/client/group/ClientGroupWrapper.java | 3 +- .../ClientManageControllerTest.java | 21 +- .../QueryRecommendEventMeshHandlerTest.java | 7 +- .../runtime/boot/EventMeshServerTest.java | 114 +++++ .../runtime/client/common/Server.java | 8 +- .../EventMeshGrpcConfigurationTest.java | 93 ++++ .../EventMeshHTTPConfigurationTest.java | 108 +++++ .../EventMeshTCPConfigurationTest.java | 95 ++++ .../runtime/util/HttpTinyClientTest.java | 1 + .../test/resources/configuration.properties | 106 +++++ .../EventMeshMessageTCPSubClient.java | 4 +- .../api/common/ConfigurationWrapper.java | 59 --- .../api/common/ConfigurationWrapperTest.java | 53 --- .../build.gradle | 3 + .../auth/http/basic/config/AuthConfigs.java | 43 +- .../http/basic/impl/AuthHttpBasicService.java | 22 +- .../http/basic/config/AuthConfigsTest.java | 23 +- .../basic/impl/AuthHttpBasicServiceTest.java | 6 +- .../test/resources/auth-http-basic.properties | 4 +- .../spi/EventMeshExtensionFactory.java | 20 +- .../api/common/EventMeshTraceConstants.java | 5 - .../api/config/ExporterConfiguration.java | 98 +---- .../api/config/ExporterConfigurationTest.java | 37 ++ .../src/test/resources/exporter.properties | 25 ++ .../trace/jaeger/JaegerTraceService.java | 33 +- .../trace/jaeger/common/JaegerConstants.java | 4 - .../jaeger/config/JaegerConfiguration.java | 77 +--- .../trace/jaeger/JaegerTraceServiceTest.java | 8 +- .../config/JaegerConfigurationTest.java | 25 +- .../src/test/resources/exporter.properties | 5 +- .../eventmesh-trace-pinpoint/build.gradle | 3 + .../trace/pinpoint/PinpointTraceService.java | 36 +- .../pinpoint/common/PinpointConstants.java | 12 - .../config/PinpointConfiguration.java | 91 +--- .../pinpoint/PinpointTraceServiceTest.java | 9 +- .../config/PinpointConfigurationTest.java | 36 +- .../exporter/PinpointSpanExporterTest.java | 20 +- .../src/test/resources/exporter.properties | 20 + .../trace/zipkin/ZipkinTraceService.java | 36 +- .../trace/zipkin/common/ZipkinConstants.java | 4 +- .../zipkin/config/ZipkinConfiguration.java | 78 +--- .../trace/zipkin/ZipkinTraceServiceTest.java | 9 +- .../config/ZipkinConfigurationTest.java | 51 +++ .../src/test/resources/exporter.properties | 20 + .../src/test/resources/zipkin.properties | 5 +- .../eventmesh-webhook-admin/build.gradle | 8 +- .../AdminWebHookConfigOperationManage.java | 33 +- .../webhook/config/AdminConfiguration.java | 49 +++ ...AdminWebHookConfigOperationManageTest.java | 34 +- .../config/AdminConfigurationTest.java | 48 ++ .../src/test/resources/eventmesh.properties | 28 ++ .../webhook/api/WebHookOperationConstant.java | 4 +- .../webhook/receive/WebHookController.java | 22 +- .../receive/config/ReceiveConfiguration.java | 50 +++ .../storage/HookConfigOperationManage.java | 32 +- .../config/ReceiveConfigurationTest.java | 51 +++ .../src/test/resources/eventmesh.properties | 30 ++ 182 files changed, 4378 insertions(+), 3104 deletions(-) create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/Config.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigFiled.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigInfo.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigMonitorService.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigService.java delete mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigurationWrapper.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/FileLoad.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/Convert.java rename eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/common/EventMeshConstants.java => eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertInfo.java (65%) create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertValue.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConverterMap.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/BaseDataTypeConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/DateConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/EnumConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/IPAddressConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ListConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateTimeConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/MapConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ObjectConverter.java create mode 100644 eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/PropertiesConverter.java rename eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/common/EventMeshConstants.java => eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/StringConverter.java (60%) rename eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapperTest.java => eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/URIConverter.java (64%) create mode 100644 eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigServiceTest.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/java/org/apache/eventmesh/connector/kafka/config/ClientConfigurationTest.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/config/ClientConfigurationTest.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-knative/src/test/resources/knative-client.properties delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigWrapper.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/java/org/apache/eventmesh/connector/pulsar/config/ClientConfigurationTest.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/resources/pulsar-client.properties delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigKey.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolderTest.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigOptions.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-redis/src/test/java/org/apache/eventmesh/connector/redis/config/RedisPropertiesTest.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/command/Command.java delete mode 100644 eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapper.java create mode 100644 eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfigurationTest.java create mode 100644 eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/boot/EventMeshServerTest.java create mode 100644 eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfigurationTest.java create mode 100644 eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfigurationTest.java create mode 100644 eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfigurationTest.java create mode 100644 eventmesh-runtime/src/test/resources/configuration.properties delete mode 100644 eventmesh-security-plugin/eventmesh-security-api/src/main/java/org/apache/eventmesh/api/common/ConfigurationWrapper.java delete mode 100644 eventmesh-security-plugin/eventmesh-security-api/src/test/java/org/apache/eventmesh/api/common/ConfigurationWrapperTest.java create mode 100644 eventmesh-trace-plugin/eventmesh-trace-api/src/test/java/org/apache/eventmesh/trace/api/config/ExporterConfigurationTest.java create mode 100644 eventmesh-trace-plugin/eventmesh-trace-api/src/test/resources/exporter.properties rename eventmesh-security-plugin/eventmesh-security-api/src/test/resources/testpath/test1.properties => eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/resources/exporter.properties (84%) create mode 100644 eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/resources/exporter.properties create mode 100644 eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfigurationTest.java create mode 100644 eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/exporter.properties rename eventmesh-security-plugin/eventmesh-security-api/src/test/resources/test.properties => eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/zipkin.properties (88%) create mode 100644 eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/config/AdminConfiguration.java rename eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigurationWrapperTest.java => eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManageTest.java (51%) create mode 100644 eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/config/AdminConfigurationTest.java create mode 100644 eventmesh-webhook/eventmesh-webhook-admin/src/test/resources/eventmesh.properties create mode 100644 eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfiguration.java create mode 100644 eventmesh-webhook/eventmesh-webhook-receive/src/test/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfigurationTest.java create mode 100644 eventmesh-webhook/eventmesh-webhook-receive/src/test/resources/eventmesh.properties diff --git a/eventmesh-common/build.gradle b/eventmesh-common/build.gradle index 59fe1587d2..c4f6e5e4d1 100644 --- a/eventmesh-common/build.gradle +++ b/eventmesh-common/build.gradle @@ -46,6 +46,7 @@ dependencies { implementation "javax.annotation:javax.annotation-api:1.3.2" implementation "com.github.stefanbirkner:system-rules" + implementation "org.yaml:snakeyaml" compileOnly 'org.projectlombok:lombok' annotationProcessor 'org.projectlombok:lombok' diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/CommonConfiguration.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/CommonConfiguration.java index e0ed6960bb..fcd06da90f 100644 --- a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/CommonConfiguration.java +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/CommonConfiguration.java @@ -17,187 +17,105 @@ package org.apache.eventmesh.common.config; -import org.apache.eventmesh.common.utils.AssertUtils; import org.apache.eventmesh.common.utils.ConfigurationContextUtil; import org.apache.eventmesh.common.utils.IPUtils; -import org.apache.commons.lang3.StringUtils; +import org.apache.commons.collections4.CollectionUtils; -import java.util.Arrays; import java.util.Collections; import java.util.List; -import java.util.function.Supplier; -import java.util.stream.Collectors; -import com.google.common.base.Preconditions; +import org.assertj.core.util.Strings; import lombok.Data; +import lombok.NoArgsConstructor; @Data +@NoArgsConstructor +@Config(prefix = "eventMesh") public class CommonConfiguration { - private transient String eventMeshEnv = "P"; - private transient String eventMeshIDC = "FT"; - private transient String eventMeshCluster = "LS"; - - private transient String eventMeshName = ""; - private transient List eventMeshProvideServerProtocols; - private transient String sysID = "5477"; - private transient String eventMeshConnectorPluginType = "rocketmq"; - private transient String eventMeshSecurityPluginType = "security"; - private transient String eventMeshRegistryPluginType = "namesrv"; - private transient List eventMeshMetricsPluginType; - private transient String eventMeshTracePluginType; - - private transient String namesrvAddr = ""; - private transient String eventMeshRegistryPluginUsername = ""; - private transient String eventMeshRegistryPluginPassword = ""; - private transient Integer eventMeshRegisterIntervalInMills = 10 * 1000; - private transient Integer eventMeshFetchRegistryAddrInterval = 10 * 1000; - private transient String eventMeshServerIp = null; - private transient boolean eventMeshServerSecurityEnable = false; - - private transient boolean eventMeshServerRegistryEnable = false; - - private transient boolean eventMeshServerTraceEnable = false; - - protected transient ConfigurationWrapper configurationWrapper; - - private transient String eventMeshWebhookOrigin = "eventmesh." + eventMeshIDC; - - public CommonConfiguration(ConfigurationWrapper configurationWrapper) { - this.configurationWrapper = configurationWrapper; - } - - public String getMeshGroup() { - return String.join("-", this.eventMeshEnv, this.eventMeshCluster, this.sysID); - } - + @ConfigFiled(field = "sysid", beNumber = true, notEmpty = true) + private String sysID = "5477"; - public void init() { + @ConfigFiled(field = "server.env", notEmpty = true) + private String eventMeshEnv = "P"; - if (configurationWrapper != null) { - eventMeshEnv = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_ENV); + @ConfigFiled(field = "server.idc", notEmpty = true) + private String eventMeshIDC = "FT"; - sysID = checkNumeric(ConfKeys.KEYS_EVENTMESH_SYSID); + @ConfigFiled(field = "server.name", notEmpty = true) + private String eventMeshName = ""; - eventMeshCluster = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_SERVER_CLUSTER); + @ConfigFiled(field = "server.cluster", notEmpty = true) + private String eventMeshCluster = "LS"; - eventMeshName = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_SERVER_NAME); + @ConfigFiled(field = "server.hostIp", reload = true) + private String eventMeshServerIp = null; - eventMeshIDC = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_IDC); + @ConfigFiled(field = "registry.plugin.server-addr", notEmpty = true) + private String namesrvAddr = ""; - eventMeshServerIp = get(ConfKeys.KEYS_EVENTMESH_SERVER_HOST_IP, IPUtils::getLocalAddress); - eventMeshConnectorPluginType = checkNotEmpty(ConfKeys.KEYS_ENENTMESH_CONNECTOR_PLUGIN_TYPE); + @ConfigFiled(field = "trace.plugin", notEmpty = true) + private String eventMeshTracePluginType; - eventMeshServerSecurityEnable = Boolean.parseBoolean(get(ConfKeys.KEYS_EVENTMESH_SECURITY_ENABLED, () -> "false")); + @ConfigFiled(field = "metrics.plugin", notEmpty = true) + private List eventMeshMetricsPluginType; - eventMeshSecurityPluginType = checkNotEmpty(ConfKeys.KEYS_ENENTMESH_SECURITY_PLUGIN_TYPE); + @ConfigFiled(field = "registry.plugin.type", notEmpty = true) + private String eventMeshRegistryPluginType = "namesrv"; - eventMeshServerRegistryEnable = Boolean.parseBoolean(get(ConfKeys.KEYS_EVENTMESH_REGISTRY_ENABLED, () -> "false")); + @ConfigFiled(field = "security.plugin.type", notEmpty = true) + private String eventMeshSecurityPluginType = "security"; - eventMeshRegistryPluginType = checkNotEmpty(ConfKeys.KEYS_ENENTMESH_REGISTRY_PLUGIN_TYPE); + @ConfigFiled(field = "connector.plugin.type", notEmpty = true) + private String eventMeshConnectorPluginType = "rocketmq"; - namesrvAddr = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_REGISTRY_PULGIN_SERVER_ADDR); - eventMeshRegistryPluginUsername = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REGISTRY_PULGIN_USERNAME, ""); + @ConfigFiled(field = "registry.plugin.username") + private String eventMeshRegistryPluginUsername = ""; - eventMeshRegistryPluginPassword = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REGISTRY_PULGIN_PASSWORD, ""); + @ConfigFiled(field = "registry.plugin.password") + private String eventMeshRegistryPluginPassword = ""; - String metricsPluginType = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_METRICS_PLUGIN_TYPE); - if (StringUtils.isNotEmpty(metricsPluginType)) { - eventMeshMetricsPluginType = Arrays - .stream(metricsPluginType.split(",")) - .filter(StringUtils::isNotBlank) - .map(String::trim) - .collect(Collectors.toList()); - } + @ConfigFiled(field = "server.registry.registerIntervalInMills") + private Integer eventMeshRegisterIntervalInMills = 10 * 1000; - eventMeshProvideServerProtocols = getProvideServerProtocols(); + @ConfigFiled(field = "server.registry.fetchRegistryAddrIntervalInMills") + private Integer eventMeshFetchRegistryAddrInterval = 10 * 1000; - eventMeshServerTraceEnable = Boolean.parseBoolean(get(ConfKeys.KEYS_EVENTMESH_TRACE_ENABLED, () -> "false")); - if (eventMeshServerTraceEnable) { - eventMeshTracePluginType = checkNotEmpty(ConfKeys.KEYS_EVENTMESH_TRACE_PLUGIN_TYPE); - } - } - } - - private String checkNotEmpty(String key) { - String value = configurationWrapper.getProp(key); - if (value != null) { - value = StringUtils.deleteWhitespace(value); - } - AssertUtils.notBlack(value, key + " is invalidated"); - return value; - } - private String checkNumeric(String key) { - String value = configurationWrapper.getProp(key); - if (value != null) { - value = StringUtils.deleteWhitespace(value); - } - Preconditions.checkState(StringUtils.isNotEmpty(value) && StringUtils.isNumeric(value), key + " is invalidated"); - return value; - } + @ConfigFiled(field = "server.trace.enabled") + private boolean eventMeshServerTraceEnable = false; - private String get(String key, Supplier defaultValueSupplier) { - String value = configurationWrapper.getProp(key); - if (value != null) { - value = StringUtils.deleteWhitespace(value); - } - return StringUtils.isEmpty(value) ? defaultValueSupplier.get() : value; - } + @ConfigFiled(field = "server.security.enabled") + private boolean eventMeshServerSecurityEnable = false; - public List getProvideServerProtocols() { - String provideProtocols = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SERVER_PROVIDE_PROTOCOLS); - if (StringUtils.isNotEmpty(provideProtocols)) { - return Arrays.stream(provideProtocols.split(",")).filter(StringUtils::isNotBlank).map(String::trim).collect(Collectors.toList()); - } else { - return Collections.singletonList(ConfigurationContextUtil.HTTP); - } - } + @ConfigFiled(field = "server.registry.enabled") + private boolean eventMeshServerRegistryEnable = false; - static class ConfKeys { - public static final String KEYS_EVENTMESH_ENV = "eventMesh.server.env"; - public static final String KEYS_EVENTMESH_IDC = "eventMesh.server.idc"; + @ConfigFiled(field = "server.provide.protocols", reload = true) + private List eventMeshProvideServerProtocols; - public static final String KEYS_EVENTMESH_SYSID = "eventMesh.sysid"; - public static final String KEYS_EVENTMESH_SERVER_CLUSTER = "eventMesh.server.cluster"; + @ConfigFiled(reload = true) + private String eventMeshWebhookOrigin; - public static final String KEYS_EVENTMESH_SERVER_NAME = "eventMesh.server.name"; + @ConfigFiled(reload = true) + private String meshGroup; - public static final String KEYS_EVENTMESH_SERVER_PROVIDE_PROTOCOLS = "eventMesh.server.provide.protocols"; + public void reload() { + this.eventMeshWebhookOrigin = "eventmesh." + eventMeshIDC; - public static final String KEYS_EVENTMESH_SERVER_HOST_IP = "eventMesh.server.hostIp"; - - public static final String KEYS_EVENTMESH_SERVER_REGISTER_INTERVAL = "eventMesh.server.registry.registerIntervalInMills"; - - public static final String KEYS_EVENTMESH_SERVER_FETCH_REGISTRY_ADDR_INTERVAL = "eventMesh.server.registry.fetchRegistryAddrIntervalInMills"; - - public static final String KEYS_ENENTMESH_CONNECTOR_PLUGIN_TYPE = "eventMesh.connector.plugin.type"; - - public static final String KEYS_EVENTMESH_SECURITY_ENABLED = "eventMesh.server.security.enabled"; - - public static final String KEYS_ENENTMESH_SECURITY_PLUGIN_TYPE = "eventMesh.security.plugin.type"; - - public static final String KEYS_EVENTMESH_REGISTRY_ENABLED = "eventMesh.registry.plugin.enabled"; - - public static final String KEYS_ENENTMESH_REGISTRY_PLUGIN_TYPE = "eventMesh.registry.plugin.type"; - - public static final String KEYS_EVENTMESH_REGISTRY_PULGIN_SERVER_ADDR = "eventMesh.registry.plugin.server-addr"; - - public static final String KEYS_EVENTMESH_REGISTRY_PULGIN_USERNAME = "eventMesh.registry.plugin.username"; - - public static final String KEYS_EVENTMESH_REGISTRY_PULGIN_PASSWORD = "eventMesh.registry.plugin.password"; - - public static final String KEYS_EVENTMESH_METRICS_PLUGIN_TYPE = "eventMesh.metrics.plugin"; + if (Strings.isNullOrEmpty(this.eventMeshServerIp)) { + this.eventMeshServerIp = IPUtils.getLocalAddress(); + } - public static final String KEYS_EVENTMESH_TRACE_ENABLED = "eventMesh.server.trace.enabled"; + if (CollectionUtils.isEmpty(eventMeshProvideServerProtocols)) { + this.eventMeshProvideServerProtocols = Collections.singletonList(ConfigurationContextUtil.HTTP); + } - public static final String KEYS_EVENTMESH_TRACE_PLUGIN_TYPE = "eventMesh.trace.plugin"; + meshGroup = String.join("-", this.eventMeshEnv, this.eventMeshCluster, this.sysID); } } \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/Config.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/Config.java new file mode 100644 index 0000000000..b6c7cc1327 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/Config.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Repeatable; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Record information about the configuration class to be converted + */ +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.METHOD, ElementType.TYPE}) +@Repeatable(Config.Configs.class) +public @interface Config { + + String field() default ""; + + String path() default ""; + + String prefix() default ""; + + String hump() default "."; + + boolean removePrefix() default true; + + boolean monitor() default false; + + @Retention(RetentionPolicy.RUNTIME) + @Target(ElementType.TYPE) + @interface Configs { + Config[] value(); + } +} + + + diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigFiled.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigFiled.java new file mode 100644 index 0000000000..d7c398e2db --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigFiled.java @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import org.apache.eventmesh.common.config.convert.ConvertValue.DefaultConverter; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Record information about the field in the configuration class to be converted + */ +@Retention(RetentionPolicy.RUNTIME) +@Target({ElementType.TYPE, ElementType.FIELD}) +public @interface ConfigFiled { + + /** + * @return The key name of the configuration file + */ + String field() default ""; + + /** + * Note : When reload is true, the class must have a reload method + * + * @return Whether to reload. This parameter is used when other fields are associated + */ + boolean reload() default false; + + /** + * In some special cases, used to specify the converter class of the field + * + * @return field converter + */ + Class converter() default DefaultConverter.class; + + /** + * if the configuration filed is empty, try to read from env, by field + * + * @return Whether to try to read from env if the configuration filed is empty + */ + boolean findEnv() default false; + + /** + * If it cannot be null but is null, an exception is thrown + * + * @return Whether the field can be null + */ + boolean notNull() default false; + + /** + * If it cannot be empty but is empty, an exception is thrown + * + * @return Whether the field can be empty + */ + boolean notEmpty() default false; + + /** + * If it's not a number, an exception is thrown + * + * @return Whether the field must be number + */ + boolean beNumber() default false; +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigInfo.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigInfo.java new file mode 100644 index 0000000000..39c4624cdc --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigInfo.java @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import java.lang.reflect.Field; + +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Data; +import lombok.NoArgsConstructor; + +@Data +@Builder +@AllArgsConstructor +@NoArgsConstructor +public class ConfigInfo { + + public static final String HUMP_SPOT = "spot"; + public static final String HUMP_ROD = "rod"; + + private String path; + private String field; + private String prefix; + private String hump; + private boolean monitor; + private boolean removePrefix; + + private Class clazz; + private Object object; + private String filePath; + + Field objectField; + Object instance; +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigMonitorService.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigMonitorService.java new file mode 100644 index 0000000000..36e39d3fa4 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigMonitorService.java @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import org.apache.eventmesh.common.ThreadPoolFactory; + +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; + +import lombok.extern.slf4j.Slf4j; + +@Slf4j +public class ConfigMonitorService { + + private static final long TIME_INTERVAL = 30 * 1000L; + + private final List configInfoList = new ArrayList<>(); + + private final ScheduledExecutorService configLoader = ThreadPoolFactory + .createSingleScheduledExecutor("eventMesh-configLoader-"); + + { + configLoader.scheduleAtFixedRate(this::load, TIME_INTERVAL, TIME_INTERVAL, TimeUnit.MILLISECONDS); + } + + public void monitor(ConfigInfo configInfo) { + configInfoList.add(configInfo); + } + + public void load() { + for (ConfigInfo configInfo : configInfoList) { + try { + Object object = ConfigService.getInstance().getConfig(configInfo); + if (configInfo.getObject().equals(object)) { + continue; + } + + Field field = configInfo.getObjectField(); + boolean isAccessible = field.isAccessible(); + try { + field.setAccessible(true); + field.set(configInfo.getInstance(), object); + } finally { + field.setAccessible(isAccessible); + } + + configInfo.setObject(object); + log.info("config reload success: {}", object); + } catch (Exception e) { + log.error("config reload failed", e); + } + } + } + +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigService.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigService.java new file mode 100644 index 0000000000..a2250a2d40 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigService.java @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import org.apache.commons.lang3.StringUtils; + +import java.io.File; +import java.io.IOException; +import java.lang.reflect.Field; +import java.util.Objects; +import java.util.Properties; + +import org.assertj.core.util.Strings; + +import lombok.Getter; + + +public class ConfigService { + + private static final ConfigService INSTANCE = new ConfigService(); + + public static final String CLASS_PATH_PREFIX = "classPath://"; + public static final String FILE_PATH_PREFIX = "file://"; + + /** + * Unified configuration Properties corresponding to eventmesh.properties + */ + private Properties properties = new Properties(); + + @Getter + private String rootPath; + + private static final ConfigMonitorService configMonitorService = new ConfigMonitorService(); + + private String configPath; + + + public static ConfigService getInstance() { + return INSTANCE; + } + + public ConfigService() { + } + + public ConfigService setConfigPath(String configPath) { + this.configPath = configPath; + return this; + } + + public void setRootConfig(String path) throws Exception { + ConfigInfo configInfo = new ConfigInfo(); + rootPath = path; + configInfo.setPath(rootPath); + properties = this.getConfig(configInfo); + } + + public Properties getRootConfig() { + return this.properties; + } + + public T buildConfigInstance(Class clazz) { + Config[] configArray = clazz.getAnnotationsByType(Config.class); + if (configArray.length == 0) { + try { + return this.getConfig(ConfigInfo.builder() + .clazz(clazz) + .hump(ConfigInfo.HUMP_SPOT) + .build()); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + Config config = configArray[0]; + try { + // todo Complete all attributes + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setClazz(clazz); + configInfo.setPath(config.path()); + configInfo.setHump(config.hump()); + configInfo.setPrefix(config.prefix()); + configInfo.setMonitor(config.monitor()); + + return this.getConfig(configInfo); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + public void populateConfigForObject(Object object) throws IllegalAccessException, NoSuchFieldException, IOException { + Class clazz = object.getClass(); + Config[] configArray = clazz.getAnnotationsByType(Config.class); + if (configArray.length == 0) { + return; + } + + for (Config config : configArray) { + populateConfig(object, clazz, config); + } + } + + @SuppressWarnings("unchecked") + public T getConfig(ConfigInfo configInfo) throws IOException { + Object object; + + if (Objects.isNull(configInfo.getPath()) || StringUtils.isEmpty(configInfo.getPath().trim())) { + object = FileLoad.getPropertiesFileLoad().getConfig(properties, configInfo); + } else { + String path = configInfo.getPath(); + String filePath; + + if (path.startsWith(CLASS_PATH_PREFIX)) { + filePath = Objects.requireNonNull( + ConfigService.class.getResource("/" + path.substring(CLASS_PATH_PREFIX.length())) + ).getPath(); + } else if (path.startsWith(FILE_PATH_PREFIX)) { + filePath = path.substring(FILE_PATH_PREFIX.length()); + } else { + filePath = this.configPath + path; + } + + File file = new File(filePath); + if (!file.exists()) { + throw new RuntimeException("file is not exists"); + } + + String suffix = path.substring(path.lastIndexOf('.') + 1); + configInfo.setFilePath(filePath); + object = FileLoad.getFileLoad(suffix).getConfig(configInfo); + } + return (T) object; + } + + private void populateConfig(Object object, Class clazz, Config config) + throws NoSuchFieldException, IOException, IllegalAccessException { + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setField(config.field()); + configInfo.setMonitor(config.monitor()); + + Field field = clazz.getDeclaredField(configInfo.getField()); + Class fieldClazz = field.getType(); + configInfo.setClazz(fieldClazz); + + Config[] configArray = fieldClazz.getAnnotationsByType(Config.class); + if (configArray.length != 0 && !Strings.isNullOrEmpty(configArray[0].prefix())) { + config = configArray[0]; + configInfo.setPrefix(config.prefix()); + configInfo.setPath(config.path()); + configInfo.setPrefix(config.prefix()); + configInfo.setHump(config.hump()); + } + + Object configObject = this.getConfig(configInfo); + + boolean isAccessible = field.isAccessible(); + try { + field.setAccessible(true); + field.set(object, configObject); + } finally { + field.setAccessible(isAccessible); + } + if (configInfo.isMonitor()) { + configInfo.setObjectField(field); + configInfo.setInstance(object); + configInfo.setObject(configObject); + configMonitorService.monitor(configInfo); + } + } +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigurationWrapper.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigurationWrapper.java deleted file mode 100644 index 3df946709a..0000000000 --- a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/ConfigurationWrapper.java +++ /dev/null @@ -1,183 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.common.config; - -import org.apache.eventmesh.common.file.FileChangeContext; -import org.apache.eventmesh.common.file.FileChangeListener; -import org.apache.eventmesh.common.file.WatchFileManager; - -import org.apache.commons.lang3.StringUtils; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStreamReader; -import java.nio.charset.StandardCharsets; -import java.util.Map.Entry; -import java.util.Objects; -import java.util.Properties; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.google.common.base.Preconditions; - -public class ConfigurationWrapper { - - public static final Logger LOG = LoggerFactory.getLogger(ConfigurationWrapper.class); - - private final transient String directoryPath; - - private final transient String fileName; - - private final transient Properties properties = new Properties(); - - private final transient String file; - - private final transient boolean reload; - - - private final transient FileChangeListener fileChangeListener = new FileChangeListener() { - @Override - public void onChanged(FileChangeContext changeContext) { - load(); - } - - @Override - public boolean support(FileChangeContext changeContext) { - return changeContext.getWatchEvent().context().toString().contains(fileName); - } - }; - - private static final ObjectMapper objectMapper = new ObjectMapper(); - - public ConfigurationWrapper(String directoryPath, String fileName, boolean reload) { - Objects.requireNonNull(directoryPath, "please configure environment variable 'confPath'"); - Objects.requireNonNull(fileName, "please configure environment variable 'fileName'"); - - this.directoryPath = directoryPath - .replace('/', File.separator.charAt(0)) - .replace('\\', File.separator.charAt(0)); - this.fileName = fileName; - this.file = (directoryPath + File.separator + fileName) - .replace('/', File.separator.charAt(0)) - .replace('\\', File.separator.charAt(0)); - this.reload = reload; - init(); - } - - private void init() { - load(); - if (this.reload) { - WatchFileManager.registerFileChangeListener(directoryPath, fileChangeListener); - Runtime.getRuntime().addShutdownHook(new Thread(() -> { - if (LOG.isInfoEnabled()) { - LOG.info("Configuration reload task closed"); - } - WatchFileManager.deregisterFileChangeListener(directoryPath); - })); - } - } - - private void load() { - Objects.requireNonNull(file, "properties can not be null"); - - try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file), - StandardCharsets.UTF_8))) { - if (LOG.isInfoEnabled()) { - LOG.info("loading config: {}", file); - } - properties.load(reader); - } catch (IOException e) { - LOG.error("loading properties [{}] error", file, e); - } - } - - public String getProp(String key) { - return getProp(key, null); - } - - /** - * get key value if value is null then return defaultValue - * @param key key - * @param defaultValue defaultValue - * @return string - */ - public String getProp(String key, String defaultValue) { - return StringUtils.isEmpty(key) ? defaultValue : properties.getProperty(key, defaultValue); - } - - public int getIntProp(String configKey, int defaultValue) { - Objects.requireNonNull(configKey, "configKey can not be null"); - - String configValue = StringUtils.deleteWhitespace(getProp(configKey)); - if (StringUtils.isEmpty(configValue)) { - return defaultValue; - } - - Preconditions.checkState(StringUtils.isNumeric(configValue), - String.format("key:%s, value:%s error", configKey, configValue)); - return Integer.parseInt(configValue); - } - - public boolean getBoolProp(String configKey, boolean defaultValue) { - Objects.requireNonNull(configKey, "configKey can not be null"); - - String configValue = StringUtils.deleteWhitespace(getProp(configKey)); - if (StringUtils.isEmpty(configValue)) { - return defaultValue; - } - return Boolean.parseBoolean(configValue); - } - - private String removePrefix(String key, String prefix, boolean removePrefix) { - Objects.requireNonNull(key, "key can not be null"); - Objects.requireNonNull(prefix, "prefix can not be null"); - - String newPrefix = prefix.endsWith(".") ? prefix : prefix + "."; - return removePrefix ? key.replace(newPrefix, "") : key; - } - - public Properties getPropertiesByConfig(String prefix, boolean isRemovePrefix) { - Objects.requireNonNull(prefix, "prefix can not be null"); - - Properties properties = new Properties(); - for (Entry entry : this.properties.entrySet()) { - String key = (String) entry.getKey(); - if (key.startsWith(prefix)) { - properties.put(removePrefix(key, prefix, isRemovePrefix), entry.getValue()); - } - } - return properties; - } - - @SuppressWarnings("unchecked") - public T getPropertiesByConfig(String prefix, Class clazz, boolean removePrefix) { - Objects.requireNonNull(prefix, "prefix can not be null"); - Objects.requireNonNull(clazz, "clazz can not be null"); - - return (T) objectMapper.convertValue(getPropertiesByConfig(prefix, removePrefix), clazz); - } - - public Properties getProperties() { - return this.properties; - } - -} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/FileLoad.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/FileLoad.java new file mode 100644 index 0000000000..05500a074a --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/FileLoad.java @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import org.apache.eventmesh.common.config.convert.Convert; + +import java.io.BufferedInputStream; +import java.io.BufferedReader; +import java.io.FileInputStream; +import java.io.FileReader; +import java.io.IOException; +import java.util.Objects; +import java.util.Properties; + +import org.yaml.snakeyaml.Yaml; + +/** + * load config from file + */ +public interface FileLoad { + + PropertiesFileLoad PROPERTIES_FILE_LOAD = new PropertiesFileLoad(); + + YamlFileLoad YAML_FILE_LOAD = new YamlFileLoad(); + + public static FileLoad getFileLoad(String fileType) { + if (Objects.equals("properties", fileType)) { + return PROPERTIES_FILE_LOAD; + } else if (Objects.equals("yaml", fileType)) { + return YAML_FILE_LOAD; + } + return PROPERTIES_FILE_LOAD; + } + + static PropertiesFileLoad getPropertiesFileLoad() { + return PROPERTIES_FILE_LOAD; + } + + static YamlFileLoad getYamlFileLoad() { + return YAML_FILE_LOAD; + } + + T getConfig(ConfigInfo configInfo) throws IOException; + + class PropertiesFileLoad implements FileLoad { + + private final Convert convert = new Convert(); + + @SuppressWarnings("unchecked") + public T getConfig(ConfigInfo configInfo) throws IOException { + Properties properties = new Properties(); + properties.load(new BufferedReader(new FileReader(configInfo.getFilePath()))); + if (Objects.isNull(configInfo.getClazz())) { + return (T) properties; + } + + return (T) convert.doConvert(configInfo, properties); + } + + @SuppressWarnings("unchecked") + public T getConfig(Properties properties, ConfigInfo configInfo) { + return (T) convert.doConvert(configInfo, properties); + } + } + + class YamlFileLoad implements FileLoad { + + @SuppressWarnings("unchecked") + @Override + public T getConfig(ConfigInfo configInfo) throws IOException { + Yaml yaml = new Yaml(); + return (T) yaml.loadAs(new BufferedInputStream(new FileInputStream(configInfo.getFilePath())), configInfo.getClazz()); + } + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/Convert.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/Convert.java new file mode 100644 index 0000000000..84fbdc380e --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/Convert.java @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert; + +import org.apache.eventmesh.common.config.ConfigInfo; + +import java.util.Properties; + +/** + * Used to convert Config properties + */ +public class Convert { + + public Object doConvert(ConfigInfo configInfo, Properties properties) { + Class clazz = configInfo.getClazz(); + ConvertInfo convertInfo = new ConvertInfo(); + convertInfo.setConfigInfo(configInfo); + convertInfo.setProperties(properties); + convertInfo.setClazz(clazz); + + ConvertValue clazzConverter = ConverterMap.getClazzConverter(clazz); + return clazzConverter.convert(convertInfo); + } +} diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/common/EventMeshConstants.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertInfo.java similarity index 65% rename from eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/common/EventMeshConstants.java rename to eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertInfo.java index c1e1f95692..c3a867da9e 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/common/EventMeshConstants.java +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertInfo.java @@ -15,10 +15,25 @@ * limitations under the License. */ -package org.apache.eventmesh.connector.knative.common; +package org.apache.eventmesh.common.config.convert; -public class EventMeshConstants { +import org.apache.eventmesh.common.config.ConfigInfo; - public static final String EVENTMESH_CONF_FILE = "knative-client.properties"; +import java.lang.reflect.Field; +import java.util.Properties; +import lombok.Data; + +/** + * Records the information about the field to be converted + */ +@Data +public class ConvertInfo { + char hump; + String key; + Field field; + Object value; + Class clazz; + Properties properties; + ConfigInfo configInfo; } diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertValue.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertValue.java new file mode 100644 index 0000000000..e3870759d0 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConvertValue.java @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert; + +import org.apache.eventmesh.common.config.ConfigFiled; + +import org.apache.commons.lang3.StringUtils; + +import java.util.Objects; +import java.util.Optional; +import java.util.Properties; + +/** + * convert convertInfo to obj + * + * @param obj type + */ +public interface ConvertValue { + + T convert(ConvertInfo convertInfo); + + /** + * @return Whether can to process null values + */ + default boolean canHandleNullValue() { + return false; + } + + /** + * @return The value converter needs + */ + default Object processFieldValue(ConvertInfo convertInfo, String key, ConfigFiled configFiled) { + Properties properties = convertInfo.getProperties(); + String value = properties.getProperty(key); + + if (Objects.isNull(value)) { + return null; + } + + value = value.trim(); + + boolean findEnv = configFiled.findEnv(); + String fieldName = configFiled.field(); + + if (StringUtils.isBlank(value) && !StringUtils.isBlank(fieldName) && findEnv) { + value = Optional.ofNullable(System.getProperty(fieldName)).orElse(System.getenv(fieldName)); + } + + if (StringUtils.isBlank(value) && configFiled.notEmpty()) { + throw new RuntimeException(key + " can't be empty!"); + } + + return value; + } + + class DefaultConverter implements ConvertValue { + + @Override + public Object convert(ConvertInfo convertInfo) { + return null; + } + } +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConverterMap.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConverterMap.java new file mode 100644 index 0000000000..47ee0534d6 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/ConverterMap.java @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert; + +import org.apache.eventmesh.common.config.ConfigFiled; +import org.apache.eventmesh.common.config.convert.converter.BaseDataTypeConverter; +import org.apache.eventmesh.common.config.convert.converter.DateConverter; +import org.apache.eventmesh.common.config.convert.converter.EnumConverter; +import org.apache.eventmesh.common.config.convert.converter.IPAddressConverter; +import org.apache.eventmesh.common.config.convert.converter.ListConverter; +import org.apache.eventmesh.common.config.convert.converter.LocalDateConverter; +import org.apache.eventmesh.common.config.convert.converter.LocalDateTimeConverter; +import org.apache.eventmesh.common.config.convert.converter.MapConverter; +import org.apache.eventmesh.common.config.convert.converter.ObjectConverter; +import org.apache.eventmesh.common.config.convert.converter.PropertiesConverter; +import org.apache.eventmesh.common.config.convert.converter.StringConverter; +import org.apache.eventmesh.common.config.convert.converter.URIConverter; + +import java.lang.reflect.Field; +import java.net.URI; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Properties; +import java.util.TreeMap; +import java.util.Vector; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import inet.ipaddr.IPAddress; + +/** + * Use to map the field clazz and the converter for the field clazz + */ +public class ConverterMap { + + public static final Logger LOGGER = LoggerFactory.getLogger(ConverterMap.class); + + private static final ObjectConverter objectConverter = new ObjectConverter(); + + private static final Map, ConvertValue> classToConverter = new HashMap<>(); + + static { + register(new URIConverter(), URI.class); + register(new EnumConverter(), Enum.class); + register(new DateConverter(), Date.class); + register(new StringConverter(), String.class); + register(new LocalDateConverter(), LocalDate.class); + register(new IPAddressConverter(), IPAddress.class); + register(new PropertiesConverter(), Properties.class); + register(new LocalDateTimeConverter(), LocalDateTime.class); + register(new ListConverter(), List.class, ArrayList.class, LinkedList.class, Vector.class); + register(new MapConverter(), Map.class, HashMap.class, TreeMap.class, LinkedHashMap.class); + register(new BaseDataTypeConverter.CharacterConverter(), Character.class, char.class); + register(new BaseDataTypeConverter.ByteConverter(), Byte.class, byte.class); + register(new BaseDataTypeConverter.ShortConverter(), Short.class, short.class); + register(new BaseDataTypeConverter.IntegerConverter(), Integer.class, int.class); + register(new BaseDataTypeConverter.LongConverter(), Long.class, long.class); + register(new BaseDataTypeConverter.FloatConverter(), Float.class, float.class); + register(new BaseDataTypeConverter.DoubleConverter(), Double.class, double.class); + register(new BaseDataTypeConverter.BooleanConverter(), Boolean.class, boolean.class); + } + + public static void register(ConvertValue convertValue, Class... clazzs) { + for (Class clazz : clazzs) { + classToConverter.put(clazz, convertValue); + } + } + + /** + * Get the converter for the field + * + * @param field The field to be parsed + * @return the converter for the field + */ + public static ConvertValue getFieldConverter(Field field) { + Class clazz = field.getType(); + ConfigFiled configFiled = field.getAnnotation(ConfigFiled.class); + + Class converter1 = configFiled.converter(); + if (!converter1.equals(ConvertValue.DefaultConverter.class)) { + if (!classToConverter.containsKey(converter1)) { + try { + ConvertValue convertValue = (ConvertValue) converter1.newInstance(); + register(convertValue, converter1); + } catch (Exception e) { + LOGGER.error("The converter failed to register.", e); + } + } + + return classToConverter.get(converter1); + } + + return getClazzConverter(clazz); + } + + /** + * Get the converter for the clazz + * + * @param clazz The clazz to be parsed + * @return the converter for the clazz + */ + public static ConvertValue getClazzConverter(Class clazz) { + ConvertValue converter = classToConverter.get(clazz); + if (Objects.isNull(converter)) { + if (clazz.isEnum()) { + converter = classToConverter.get(Enum.class); + } else { + converter = objectConverter; + } + } + + return converter; + } + + public static Map, ConvertValue> getClassToConverter() { + return classToConverter; + } + + public static ObjectConverter getObjectConverter() { + return objectConverter; + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/BaseDataTypeConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/BaseDataTypeConverter.java new file mode 100644 index 0000000000..053580c46f --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/BaseDataTypeConverter.java @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import java.util.Objects; + +/** + * Config field conversion class for base data types + */ +public class BaseDataTypeConverter { + + public static class CharacterConverter implements ConvertValue { + + @Override + public Character convert(ConvertInfo convertInfo) { + String value = (String) convertInfo.getValue(); + + return value.charAt(0); + } + } + + public static class BooleanConverter implements ConvertValue { + + @Override + public Boolean convert(ConvertInfo convertInfo) { + String value = (String) convertInfo.getValue(); + if (Objects.equals(value.length(), 1)) { + return Objects.equals(convertInfo.getValue(), "1") ? Boolean.TRUE : Boolean.FALSE; + } + + return Boolean.valueOf((String) convertInfo.getValue()); + } + } + + public static class ByteConverter implements ConvertValue { + + @Override + public Byte convert(ConvertInfo convertInfo) { + return Byte.valueOf((String) convertInfo.getValue()); + } + } + + public static class ShortConverter implements ConvertValue { + + @Override + public Short convert(ConvertInfo convertInfo) { + return Short.valueOf((String) convertInfo.getValue()); + } + } + + public static class IntegerConverter implements ConvertValue { + + @Override + public Integer convert(ConvertInfo convertInfo) { + return Integer.valueOf((String) convertInfo.getValue()); + } + } + + public static class LongConverter implements ConvertValue { + + @Override + public Long convert(ConvertInfo convertInfo) { + return Long.valueOf((String) convertInfo.getValue()); + } + } + + public static class FloatConverter implements ConvertValue { + + @Override + public Float convert(ConvertInfo convertInfo) { + return Float.valueOf((String) convertInfo.getValue()); + } + } + + public static class DoubleConverter implements ConvertValue { + + @Override + public Double convert(ConvertInfo convertInfo) { + return Double.valueOf((String) convertInfo.getValue()); + } + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/DateConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/DateConverter.java new file mode 100644 index 0000000000..4a864eb6b2 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/DateConverter.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.Date; + +/** + * Config field conversion class for Date + */ +public class DateConverter implements ConvertValue { + + @Override + public Date convert(ConvertInfo convertInfo) { + SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); + + try { + return sdf.parse((String) convertInfo.getValue()); + } catch (ParseException e) { + throw new RuntimeException(e); + } + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/EnumConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/EnumConverter.java new file mode 100644 index 0000000000..513adf101e --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/EnumConverter.java @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +/** + * Config field conversion class for Enum + */ +public class EnumConverter implements ConvertValue> { + + @SuppressWarnings({"unchecked", "rawtypes"}) + @Override + public Enum convert(ConvertInfo convertInfo) { + Class enumType = (Class) convertInfo.getField().getType(); + String name = (String) convertInfo.getValue(); + + return Enum.valueOf(enumType, name); + } +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/IPAddressConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/IPAddressConverter.java new file mode 100644 index 0000000000..ee3ebae8c5 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/IPAddressConverter.java @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import inet.ipaddr.AddressStringException; +import inet.ipaddr.IPAddress; +import inet.ipaddr.IPAddressString; + +/** + * Config field conversion class for IPAddress + */ +public class IPAddressConverter implements ConvertValue { + + @Override + public IPAddress convert(ConvertInfo convertInfo) { + try { + return new IPAddressString((String) convertInfo.getValue()).toAddress(); + } catch (AddressStringException e) { + throw new RuntimeException(e); + } + } +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ListConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ListConverter.java new file mode 100644 index 0000000000..fb7a2b0664 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ListConverter.java @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; +import org.apache.eventmesh.common.config.convert.ConverterMap; + +import java.lang.reflect.ParameterizedType; +import java.lang.reflect.Type; +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; + +import com.google.common.base.Splitter; + +/** + * Config field conversion class for List + */ +public class ListConverter implements ConvertValue> { + + public String separator = ","; + + @Override + public boolean canHandleNullValue() { + return true; + } + + public String getSeparator() { + return separator; + } + + @Override + public List convert(ConvertInfo convertInfo) { + return convert(convertInfo, this.getSeparator()); + } + + @SuppressWarnings("unchecked") + public List convert(ConvertInfo convertInfo, String separator) { + try { + if (convertInfo.getValue() == null) { + return new ArrayList<>(); + } + List list; + if (Objects.equals(convertInfo.getField().getType(), List.class)) { + list = new ArrayList<>(); + } else { + list = (List) convertInfo.getField().getType().newInstance(); + } + + Type parameterizedType = ((ParameterizedType) convertInfo.getField() + .getGenericType()).getActualTypeArguments()[0]; + + ConvertValue clazzConverter = ConverterMap.getClazzConverter((Class) parameterizedType); + + List values = Splitter.on(separator).omitEmptyStrings().trimResults() + .splitToList((String) convertInfo.getValue()); + for (String value : values) { + convertInfo.setValue(value); + list.add(clazzConverter.convert(convertInfo)); + } + + return list; + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + + public static class ListConverterSemi extends ListConverter { + public String separator = ";"; + + public String getSeparator() { + return separator; + } + + @Override + public List convert(ConvertInfo convertInfo) { + return super.convert(convertInfo, this.getSeparator()); + } + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateConverter.java new file mode 100644 index 0000000000..b8c19ee0ea --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateConverter.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import java.time.LocalDate; +import java.time.format.DateTimeFormatter; + +/** + * Config field conversion class for LocalDate + */ +public class LocalDateConverter implements ConvertValue { + + @Override + public LocalDate convert(ConvertInfo convertInfo) { + String value = (String) convertInfo.getValue(); + DateTimeFormatter timeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd"); + + return LocalDate.parse(value, timeFormatter); + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateTimeConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateTimeConverter.java new file mode 100644 index 0000000000..e2423397e6 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/LocalDateTimeConverter.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import java.time.LocalDateTime; +import java.time.format.DateTimeFormatter; + +/** + * Config field conversion class for LocalDateTime + */ +public class LocalDateTimeConverter implements ConvertValue { + + @Override + public LocalDateTime convert(ConvertInfo convertInfo) { + String value = (String) convertInfo.getValue(); + DateTimeFormatter timeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"); + + return LocalDateTime.parse(value, timeFormatter); + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/MapConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/MapConverter.java new file mode 100644 index 0000000000..0902a758f7 --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/MapConverter.java @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; +import org.apache.eventmesh.common.config.convert.ConverterMap; + +import java.lang.reflect.ParameterizedType; +import java.lang.reflect.Type; +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; + +/** + * Config field conversion class for Map + */ +public class MapConverter implements ConvertValue> { + + @Override + public boolean canHandleNullValue() { + return true; + } + + @SuppressWarnings("unchecked") + @Override + public Map convert(ConvertInfo convertInfo) { + try { + String key = convertInfo.getKey() + convertInfo.getHump(); + Map map; + if (Objects.equals(Map.class, convertInfo.getField().getType())) { + map = new HashMap<>(); + } else { + map = (Map) convertInfo.getField().getType().newInstance(); + } + Type parameterizedType = ((ParameterizedType) convertInfo.getField().getGenericType()).getActualTypeArguments()[1]; + ConvertValue clazzConverter = ConverterMap.getClazzConverter((Class) parameterizedType); + + for (Map.Entry entry : convertInfo.getProperties().entrySet()) { + String propertiesKey = entry.getKey().toString(); + if (propertiesKey.startsWith(key)) { + String value = entry.getValue().toString(); + convertInfo.setValue(value); + map.put(propertiesKey.replace(key, ""), clazzConverter.convert(convertInfo)); + } + } + return map; + } catch (Exception e) { + throw new RuntimeException(e); + } + } +} diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ObjectConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ObjectConverter.java new file mode 100644 index 0000000000..421707a8db --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/ObjectConverter.java @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; +import org.apache.eventmesh.common.config.ConfigInfo; +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; +import org.apache.eventmesh.common.config.convert.ConverterMap; + +import org.apache.commons.lang3.StringUtils; + +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.util.Objects; + +import org.assertj.core.util.Strings; + +/** + * Config field conversion class for Configuration class + */ +public class ObjectConverter implements ConvertValue { + + private String prefix; + + private ConvertInfo convertInfo; + + private Object object; + + private char hump; + + private Class clazz; + + private void init(ConfigInfo configInfo) { + String prefix = configInfo.getPrefix(); + if (Objects.nonNull(prefix)) { + this.prefix = prefix.endsWith(".") ? prefix : prefix + "."; + } + this.hump = Objects.equals(configInfo.getHump(), ConfigInfo.HUMP_ROD) ? '_' : '.'; + this.clazz = convertInfo.getClazz(); + this.convertInfo.setHump(this.hump); + } + + @Override + public Object convert(ConvertInfo convertInfo) { + try { + this.convertInfo = convertInfo; + this.object = convertInfo.getClazz().newInstance(); + this.init(convertInfo.getConfigInfo()); + this.setValue(); + + Class superclass = convertInfo.getClazz(); + for (; ; ) { + superclass = superclass.getSuperclass(); + if (Objects.equals(superclass, Object.class) || Objects.isNull(superclass)) { + break; + } + + this.clazz = superclass; + this.prefix = null; + Config[] configArray = clazz.getAnnotationsByType(Config.class); + if (configArray.length != 0 && !Strings.isNullOrEmpty(configArray[0].prefix())) { + String prefix = configArray[0].prefix(); + this.prefix = prefix.endsWith(".") ? prefix : prefix + "."; + this.hump = Objects.equals(configArray[0].hump(), ConfigInfo.HUMP_ROD) ? '_' : '.'; + this.convertInfo.setHump(this.hump); + } + + this.setValue(); + } + + return object; + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + private void setValue() throws Exception { + boolean needReload = Boolean.FALSE; + + for (Field field : this.clazz.getDeclaredFields()) { + if (Modifier.isStatic(field.getModifiers())) { + continue; + } + + boolean isAccessible = field.isAccessible(); + try { + field.setAccessible(true); + + ConvertInfo convertInfo = this.convertInfo; + ConfigFiled configFiled = field.getAnnotation(ConfigFiled.class); + if (Objects.isNull(configFiled)) { + continue; + } + + String key = this.buildKey(configFiled); + needReload = this.checkNeedReload(needReload, configFiled); + + ConvertValue convertValue = ConverterMap.getFieldConverter(field); + Object fieldValue = convertValue.processFieldValue(convertInfo, key, configFiled); + + if (!checkFieldValueBefore(configFiled, key, convertValue, fieldValue)) { + continue; + } + convertInfo.setValue(fieldValue); + convertInfo.setField(field); + convertInfo.setKey(key); + Object convertedValue = convertValue.convert(convertInfo); + + if (!checkFieldValueAfter(configFiled, key, convertedValue)) { + continue; + } + field.set(object, convertedValue); + } finally { + field.setAccessible(isAccessible); + } + } + + reloadConfigIfNeed(needReload); + } + + private void reloadConfigIfNeed(boolean needReload) throws NoSuchMethodException, IllegalAccessException, InvocationTargetException { + if (needReload) { + Method method = this.clazz.getDeclaredMethod("reload", null); + + boolean isAccessible = method.isAccessible(); + try { + method.setAccessible(true); + method.invoke(this.object, null); + } finally { + method.setAccessible(isAccessible); + } + } + } + + private boolean checkFieldValueAfter(ConfigFiled configFiled, String key, Object convertedValue) { + if (Objects.isNull(convertedValue)) { + if (configFiled.notNull()) { + throw new RuntimeException(key + " can not be null!"); + } + + return false; + } + + if (configFiled.beNumber()) { + if (!StringUtils.isNumeric(String.valueOf(convertedValue))) { + throw new RuntimeException(key + " must be number!"); + } + } + + return true; + } + + private boolean checkFieldValueBefore(ConfigFiled configFiled, String key, ConvertValue convertValue, Object fieldValue) { + if (Objects.isNull(fieldValue) && !convertValue.canHandleNullValue()) { + if (configFiled.notNull()) { + throw new RuntimeException(key + " can not be null!"); + } + + return false; + } + + return true; + } + + private boolean checkNeedReload(boolean needReload, ConfigFiled configFiled) { + if (!needReload && configFiled != null && configFiled.reload()) { + needReload = Boolean.TRUE; + } + + if (needReload) { + try { + this.clazz.getDeclaredMethod("reload", null); + } catch (NoSuchMethodException e) { + throw new RuntimeException("The field needs to be reloaded, but the reload method cannot be found.", e); + } + } + + return needReload; + } + + private String buildKey(ConfigFiled configFiled) { + String key; + StringBuilder keyPrefix = new StringBuilder(Objects.isNull(prefix) ? "" : prefix); + + if (configFiled == null || configFiled.field().isEmpty() && keyPrefix.length() > 0) { + key = keyPrefix.deleteCharAt(keyPrefix.length() - 1).toString(); + } else { + key = keyPrefix.append(configFiled.field()).toString(); + } + + return key; + } +} \ No newline at end of file diff --git a/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/PropertiesConverter.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/PropertiesConverter.java new file mode 100644 index 0000000000..e4dbf470df --- /dev/null +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/PropertiesConverter.java @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config.convert.converter; + +import org.apache.eventmesh.common.config.ConfigFiled; +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; +import org.apache.eventmesh.common.utils.PropertiesUtils; + +import org.apache.commons.lang3.StringUtils; + +import java.util.Properties; + +/** + * Config field conversion class for Properties, by prefix + */ +public class PropertiesConverter implements ConvertValue { + + @Override + public Properties convert(ConvertInfo convertInfo) { + try { + return (Properties) convertInfo.getValue(); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + @Override + public Object processFieldValue(ConvertInfo convertInfo, String prefix, ConfigFiled configFiled) { + Properties properties = convertInfo.getProperties(); + + if (StringUtils.isBlank(prefix)) { + return null; + } + + return PropertiesUtils.getPropertiesByPrefix(properties, prefix); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/common/EventMeshConstants.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/StringConverter.java similarity index 60% rename from eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/common/EventMeshConstants.java rename to eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/StringConverter.java index 9049608a96..330775d805 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/common/EventMeshConstants.java +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/StringConverter.java @@ -15,8 +15,22 @@ * limitations under the License. */ -package org.apache.eventmesh.connector.pulsar.common; +package org.apache.eventmesh.common.config.convert.converter; -public class EventMeshConstants { - public static final String EVENTMESH_CONF_FILE = "pulsar-client.properties"; +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; + +import org.apache.commons.lang3.StringUtils; + +/** + * Config field conversion class for String + */ +public class StringConverter implements ConvertValue { + + @Override + public String convert(ConvertInfo convertInfo) { + String value = (String) convertInfo.getValue(); + + return StringUtils.trim(value); + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapperTest.java b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/URIConverter.java similarity index 64% rename from eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapperTest.java rename to eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/URIConverter.java index 6d25baf7cf..bbf951c681 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapperTest.java +++ b/eventmesh-common/src/main/java/org/apache/eventmesh/common/config/convert/converter/URIConverter.java @@ -15,16 +15,20 @@ * limitations under the License. */ -package org.apache.eventmesh.connector.rocketmq.config; +package org.apache.eventmesh.common.config.convert.converter; -import org.junit.Assert; -import org.junit.Test; +import org.apache.eventmesh.common.config.convert.ConvertInfo; +import org.apache.eventmesh.common.config.convert.ConvertValue; -public class ConfigurationWrapperTest { +import java.net.URI; - @Test - public void getProp() { - String namesrcAddr = ConfigurationWrapper.getProp("eventMesh.server.rocketmq.namesrvAddr"); - Assert.assertNotNull(namesrcAddr); +/** + * Config field conversion class for URI + */ +public class URIConverter implements ConvertValue { + + @Override + public URI convert(ConvertInfo convertInfo) { + return URI.create((String) convertInfo.getValue()); } -} \ No newline at end of file +} diff --git a/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/CommonConfigurationTest.java b/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/CommonConfigurationTest.java index f131a85901..755ac1d5e9 100644 --- a/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/CommonConfigurationTest.java +++ b/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/CommonConfigurationTest.java @@ -17,7 +17,8 @@ package org.apache.eventmesh.common.config; -import java.io.File; +import java.util.ArrayList; +import java.util.List; import org.junit.Assert; import org.junit.Before; @@ -25,21 +26,52 @@ public class CommonConfigurationTest { - private CommonConfiguration configuration; + public CommonConfiguration config; @Before - public void before() { - String file = ConfigurationWrapperTest.class.getResource("/configuration.properties").getFile(); - File f = new File(file); - ConfigurationWrapper wraper = new ConfigurationWrapper(f.getParent(), f.getName(), false); - configuration = new CommonConfiguration(wraper); + public void beforeCommonConfigurationTest() throws Exception { + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + + config = configService.buildConfigInstance(CommonConfiguration.class); + + testGetCommonConfiguration(); } @Test - public void testInit() { - configuration.init(); - Assert.assertEquals("value1", configuration.getEventMeshEnv()); - Assert.assertEquals("value2", configuration.getEventMeshIDC()); - Assert.assertEquals("3", configuration.getSysID()); + public void testGetCommonConfiguration() { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + Assert.assertEquals("idc-succeed!!!", config.getEventMeshIDC()); + Assert.assertEquals("cluster-succeed!!!", config.getEventMeshCluster()); + Assert.assertEquals("name-succeed!!!", config.getEventMeshName()); + Assert.assertEquals("816", config.getSysID()); + Assert.assertEquals("connector-succeed!!!", config.getEventMeshConnectorPluginType()); + Assert.assertEquals("security-succeed!!!", config.getEventMeshSecurityPluginType()); + Assert.assertEquals("registry-succeed!!!", config.getEventMeshRegistryPluginType()); + Assert.assertEquals("trace-succeed!!!", config.getEventMeshTracePluginType()); + Assert.assertEquals("hostIp-succeed!!!", config.getEventMeshServerIp()); + Assert.assertEquals("username-succeed!!!", config.getEventMeshRegistryPluginUsername()); + Assert.assertEquals("password-succeed!!!", config.getEventMeshRegistryPluginPassword()); + + Assert.assertEquals(Integer.valueOf(816), config.getEventMeshRegisterIntervalInMills()); + Assert.assertEquals(Integer.valueOf(1816), config.getEventMeshFetchRegistryAddrInterval()); + + List list = new ArrayList<>(); + list.add("metrics-succeed1!!!"); + list.add("metrics-succeed2!!!"); + list.add("metrics-succeed3!!!"); + Assert.assertEquals(list, config.getEventMeshMetricsPluginType()); + + List list1 = new ArrayList<>(); + list1.add("TCP"); + list1.add("HTTP"); + list1.add("GRPC"); + Assert.assertEquals(list1, config.getEventMeshProvideServerProtocols()); + + Assert.assertTrue(config.isEventMeshServerSecurityEnable()); + Assert.assertTrue(config.isEventMeshServerRegistryEnable()); + Assert.assertTrue(config.isEventMeshServerTraceEnable()); + + Assert.assertEquals("eventmesh.idc-succeed!!!", config.getEventMeshWebhookOrigin()); } } diff --git a/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigServiceTest.java b/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigServiceTest.java new file mode 100644 index 0000000000..3900bef9b8 --- /dev/null +++ b/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigServiceTest.java @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.common.config; + +import static org.apache.eventmesh.common.config.ConfigService.CLASS_PATH_PREFIX; +import static org.apache.eventmesh.common.config.ConfigService.FILE_PATH_PREFIX; + +import java.io.File; + +import org.junit.Assert; +import org.junit.Test; + +public class ConfigServiceTest { + + private final String configFileName = "configuration.properties"; + + @Test + public void getConfigByRootConfig() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig(CLASS_PATH_PREFIX + configFileName); + + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setClazz(CommonConfiguration.class); + configInfo.setPrefix(CommonConfiguration.class.getAnnotation(Config.class).prefix()); + + CommonConfiguration config = configService.getConfig(configInfo); + assertCommonConfiguration(config); + } + + @Test + public void getConfigByClassPath() throws Exception { + ConfigService configService = ConfigService.getInstance(); + + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setPath(CLASS_PATH_PREFIX + configFileName); + configInfo.setClazz(CommonConfiguration.class); + configInfo.setPrefix(CommonConfiguration.class.getAnnotation(Config.class).prefix()); + + CommonConfiguration config = configService.getConfig(configInfo); + assertCommonConfiguration(config); + } + + @Test + public void getConfigByFilePath() throws Exception { + ConfigService configService = ConfigService.getInstance(); + String rootPath = new File(this.getClass().getResource("/" + configFileName) + .getPath()).getParent(); + + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setPath(FILE_PATH_PREFIX + rootPath + "/" + configFileName); + configInfo.setClazz(CommonConfiguration.class); + configInfo.setPrefix(CommonConfiguration.class.getAnnotation(Config.class).prefix()); + + CommonConfiguration config = configService.getConfig(configInfo); + assertCommonConfiguration(config); + } + + @Test + public void getConfigByConfigPath() throws Exception { + ConfigService configService = ConfigService.getInstance(); + String configPath = new File(this.getClass().getResource("/" + configFileName) + .getPath()).getParent(); + + configService.setConfigPath(configPath); + ConfigInfo configInfo = new ConfigInfo(); + configInfo.setPath("/" + configFileName); + configInfo.setClazz(CommonConfiguration.class); + configInfo.setPrefix(CommonConfiguration.class.getAnnotation(Config.class).prefix()); + + CommonConfiguration config = configService.getConfig(configInfo); + assertCommonConfiguration(config); + } + + public void assertCommonConfiguration(CommonConfiguration config) { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + } +} \ No newline at end of file diff --git a/eventmesh-common/src/test/java/org/apache/eventmesh/common/utils/PropertiesUtilsTest.java b/eventmesh-common/src/test/java/org/apache/eventmesh/common/utils/PropertiesUtilsTest.java index 11cef76530..b030981104 100644 --- a/eventmesh-common/src/test/java/org/apache/eventmesh/common/utils/PropertiesUtilsTest.java +++ b/eventmesh-common/src/test/java/org/apache/eventmesh/common/utils/PropertiesUtilsTest.java @@ -17,7 +17,7 @@ package org.apache.eventmesh.common.utils; -import org.apache.eventmesh.common.config.ConfigurationWrapperTest; +import org.apache.eventmesh.common.config.ConfigService; import java.util.Enumeration; import java.util.Properties; @@ -44,13 +44,16 @@ public void testGetPropertiesByPrefix() { } @Test - public void testLoadPropertiesWhenFileExist() { + public void testLoadPropertiesWhenFileExist() throws Exception { Properties properties = new Properties(); - String path = ConfigurationWrapperTest.class.getResource("/configuration.properties").getPath(); + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + properties = configService.getRootConfig(); + String path = configService.getRootPath(); try { PropertiesUtils.loadPropertiesWhenFileExist(properties, path); - Assert.assertEquals(properties.get("eventMesh.server.env").toString(), "value1"); - Assert.assertEquals(properties.get("eventMesh.server.idc").toString(), "value2"); + Assert.assertEquals(properties.get("eventMesh.server.env").toString(), "env-succeed!!!"); + Assert.assertEquals(properties.get("eventMesh.server.idc").toString(), "idc-succeed!!!"); } catch (Exception e) { Assert.fail(); } diff --git a/eventmesh-common/src/test/resources/configuration.properties b/eventmesh-common/src/test/resources/configuration.properties index fe3f749a1b..47bbb1c36c 100644 --- a/eventmesh-common/src/test/resources/configuration.properties +++ b/eventmesh-common/src/test/resources/configuration.properties @@ -14,16 +14,25 @@ # See the License for the specific language governing permissions and # limitations under the License. # -eventMesh.server.env=value1 -eventMesh.server.idc=value2 -eventMesh.sysid=3 -eventMesh.server.cluster=value4 -eventMesh.server.name=value5 -eventMesh.server.hostIp=value6 -eventMesh.connector.plugin.type=rocketmq -eventMesh.security.plugin.type=acl -eventMesh.registry.plugin.type=namesrv -eventMesh.registry.plugin.server-addr=127.0.0.1:8848 -eventMesh.registry.plugin.username=nacos -eventMesh.registry.plugin.password=nacos -eventMesh.trace.plugin=zipkin \ No newline at end of file +eventMesh.server.env=env-succeed!!! +eventMesh.server.idc=idc-succeed!!! +eventMesh.sysid=816 +eventMesh.server.cluster=cluster-succeed!!! +eventMesh.server.name=name-succeed!!! +eventMesh.server.hostIp=hostIp-succeed!!! +eventMesh.connector.plugin.type=connector-succeed!!! +eventMesh.security.plugin.type=security-succeed!!! +eventMesh.registry.plugin.type=registry-succeed!!! +eventMesh.trace.plugin=trace-succeed!!! +eventMesh.server.registry.registerIntervalInMills=816 +eventMesh.server.registry.fetchRegistryAddrIntervalInMills=1816 +eventMesh.metrics.plugin=metrics-succeed1!!!,metrics-succeed2!!!,metrics-succeed3!!! +eventMesh.registry.plugin.server-addr=server-addr-succeed1!!! + +eventMesh.server.security.enabled=true +eventMesh.server.registry.enabled=true +eventMesh.server.trace.enabled=true + +eventMesh.server.provide.protocols=TCP,HTTP,GRPC +eventMesh.registry.plugin.username=username-succeed!!! +eventMesh.registry.plugin.password=password-succeed!!! diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ClientConfiguration.java b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ClientConfiguration.java index f185fcd7cf..e8b9ee658d 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ClientConfiguration.java +++ b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ClientConfiguration.java @@ -17,39 +17,57 @@ package org.apache.eventmesh.connector.kafka.config; -import org.apache.commons.lang3.StringUtils; - -import com.google.common.base.Preconditions; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; +@Config(prefix = "eventMesh.server.kafka", path = "classPath://kafka-client.properties") public class ClientConfiguration { + @ConfigFiled(field = "namesrvAddr", notEmpty = true) public String namesrvAddr = ""; + + @ConfigFiled(field = "username") public String clientUserName = "username"; + + @ConfigFiled(field = "password") public String clientPass = "password"; + + @ConfigFiled(field = "client.consumeThreadMin") public Integer consumeThreadMin = 2; + + @ConfigFiled(field = "client.consumeThreadMax") public Integer consumeThreadMax = 2; + + @ConfigFiled(field = "client.consumeThreadPoolQueueSize") public Integer consumeQueueSize = 10000; + + @ConfigFiled(field = "client.pullBatchSize") public Integer pullBatchSize = 32; + + @ConfigFiled(field = "client.ackwindow") public Integer ackWindow = 1000; + + @ConfigFiled(field = "client.pubwindow") public Integer pubWindow = 100; + + @ConfigFiled(field = "client.comsumeTimeoutInMin") public long consumeTimeout = 0L; + + @ConfigFiled(field = "client.pollNameServerInterval") public Integer pollNameServerInterval = 10 * 1000; + + @ConfigFiled(field = "client.heartbeatBrokerInterval") public Integer heartbeatBrokerInterval = 30 * 1000; - public Integer rebalanceInterval = 20 * 1000; - public String clusterName = ""; - public String accessKey = ""; - public String secretKey = ""; - public void init() { - String namesrvAddrStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_KAFKA_SERVER_PORT); - Preconditions.checkState(StringUtils.isNotEmpty(namesrvAddrStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_KAFKA_SERVER_PORT)); - namesrvAddr = StringUtils.trim(namesrvAddrStr); - } + @ConfigFiled(field = "client.rebalanceInterval") + public Integer rebalanceInterval = 20 * 1000; - static class ConfKeys { + @ConfigFiled(field = "cluster") + public String clusterName = ""; - public static String KEYS_EVENTMESH_KAFKA_SERVER_PORT = "eventMesh.server.kafka.port"; + @ConfigFiled(field = "accessKey") + public String accessKey = ""; - } + @ConfigFiled(field = "secretKey") + public String secretKey = ""; } \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ConfigurationWrapper.java deleted file mode 100644 index ecc7d1857c..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/config/ConfigurationWrapper.java +++ /dev/null @@ -1,83 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.kafka.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.connector.kafka.common.EventMeshConstants; - -import org.apache.commons.lang3.StringUtils; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass -public class ConfigurationWrapper { - public static final Logger logger = LoggerFactory.getLogger(ConfigurationWrapper.class); - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public String getProp(String key) { - return StringUtils.isEmpty(key) ? null : properties.getProperty(key, null); - } - - /** - * Load kafka properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + EventMeshConstants.EVENTMESH_CONF_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - logger.error("Error while loading from classpath:", e); - throw new RuntimeException(String.format("Load %s.properties file from classpath error", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + EventMeshConstants.EVENTMESH_CONF_FILE; - if (new File(configPath).exists()) { - try (FileInputStream fileInputStream = new FileInputStream(configPath); - InputStreamReader inputStreamReader = new InputStreamReader(fileInputStream, StandardCharsets.UTF_8); - BufferedReader reader = new BufferedReader(inputStreamReader)) { - properties.load(reader); - } - } - } catch (IOException e) { - logger.error("Error while loading from conf home: ", e); - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/consumer/KafkaConsumerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/consumer/KafkaConsumerImpl.java index 8743ef3342..59393778db 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/consumer/KafkaConsumerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/consumer/KafkaConsumerImpl.java @@ -20,6 +20,8 @@ import org.apache.eventmesh.api.AbstractContext; import org.apache.eventmesh.api.EventListener; import org.apache.eventmesh.api.consumer.Consumer; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.connector.kafka.config.ClientConfiguration; import org.apache.kafka.clients.consumer.ConsumerConfig; @@ -28,15 +30,22 @@ import io.cloudevents.CloudEvent; +@Config(field = "clientConfiguration") public class KafkaConsumerImpl implements Consumer { private ConsumerImpl consumer; + /** + * Unified configuration class corresponding to kafka-client.properties + */ + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties props) throws Exception { + String namesrvAddr = clientConfiguration.namesrvAddr; String consumerGroup = props.getProperty("consumerGroup"); - // Other config props - props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); + // Other config props + props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, namesrvAddr); props.put(ConsumerConfig.GROUP_ID_CONFIG, consumerGroup); consumer = new ConsumerImpl(props); } @@ -81,4 +90,7 @@ public synchronized void shutdown() { consumer.shutdown(); } + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/producer/KafkaProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/producer/KafkaProducerImpl.java index f5c1355b2f..49bccf42f1 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/producer/KafkaProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/main/java/org/apache/eventmesh/connector/kafka/producer/KafkaProducerImpl.java @@ -20,6 +20,8 @@ import org.apache.eventmesh.api.RequestReplyCallback; import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.producer.Producer; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.connector.kafka.config.ClientConfiguration; import org.apache.kafka.clients.producer.ProducerConfig; @@ -27,13 +29,21 @@ import io.cloudevents.CloudEvent; +@Config(field = "clientConfiguration") public class KafkaProducerImpl implements Producer { private ProducerImpl producer; + /** + * Unified configuration class corresponding to kafka-client.properties + */ + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties keyValue) { - keyValue.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); + String namesrvAddr = clientConfiguration.namesrvAddr; + + keyValue.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, namesrvAddr); this.producer = new ProducerImpl(keyValue); } @@ -87,4 +97,8 @@ public void setExtFields() { public void sendOneway(CloudEvent message) { producer.sendOneway(message); } + + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/java/org/apache/eventmesh/connector/kafka/config/ClientConfigurationTest.java b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/java/org/apache/eventmesh/connector/kafka/config/ClientConfigurationTest.java new file mode 100644 index 0000000000..24eacc325b --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/java/org/apache/eventmesh/connector/kafka/config/ClientConfigurationTest.java @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.kafka.config; + +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.kafka.consumer.KafkaConsumerImpl; +import org.apache.eventmesh.connector.kafka.producer.KafkaProducerImpl; + +import org.junit.Assert; +import org.junit.Test; + +/** + * test config of Kafka SPI Impl + */ +public class ClientConfigurationTest { + + /** + * test KafkaConsumerImpl config init when ConnectorPluginFactory get it + */ + @Test + public void getConfigWhenKafkaConsumerImplInit() { + KafkaConsumerImpl consumer = + (KafkaConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("kafka"); + + ClientConfiguration config = consumer.getClientConfiguration(); + assertConfig(config); + } + + /** + * test KafkaProducerImpl config init when ConnectorPluginFactory get it + */ + @Test + public void getConfigWhenKafkaProducerImplInit() { + KafkaProducerImpl producer = + (KafkaProducerImpl) ConnectorPluginFactory.getMeshMQProducer("kafka"); + + ClientConfiguration config = producer.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(ClientConfiguration config) { + Assert.assertEquals(config.namesrvAddr, "127.0.0.1:9092;127.0.0.1:9092"); + Assert.assertEquals(config.clientUserName, "username-succeed!!!"); + Assert.assertEquals(config.clientPass, "password-succeed!!!"); + Assert.assertEquals(config.consumeThreadMin, Integer.valueOf(1816)); + Assert.assertEquals(config.consumeThreadMax, Integer.valueOf(2816)); + Assert.assertEquals(config.consumeQueueSize, Integer.valueOf(3816)); + Assert.assertEquals(config.pullBatchSize, Integer.valueOf(4816)); + Assert.assertEquals(config.ackWindow, Integer.valueOf(5816)); + Assert.assertEquals(config.pubWindow, Integer.valueOf(6816)); + Assert.assertEquals(config.consumeTimeout, 7816); + Assert.assertEquals(config.pollNameServerInterval, Integer.valueOf(8816)); + Assert.assertEquals(config.heartbeatBrokerInterval, Integer.valueOf(9816)); + Assert.assertEquals(config.rebalanceInterval, Integer.valueOf(11816)); + Assert.assertEquals(config.clusterName, "cluster-succeed!!!"); + Assert.assertEquals(config.accessKey, "accessKey-succeed!!!"); + Assert.assertEquals(config.secretKey, "secretKey-succeed!!!"); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/resources/kafka-client.properties b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/resources/kafka-client.properties index 7afdf4c408..3b0c217e0a 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/resources/kafka-client.properties +++ b/eventmesh-connector-plugin/eventmesh-connector-kafka/src/test/resources/kafka-client.properties @@ -16,3 +16,33 @@ # #######################kafka-client################## eventMesh.server.kafka.namesrvAddr=127.0.0.1:9092;127.0.0.1:9092 +eventMesh.server.kafka.username=username-succeed!!! +eventMesh.server.kafka.password=password-succeed!!! +eventMesh.server.kafka.client.consumeThreadMin=1816 +eventMesh.server.kafka.client.consumeThreadMax=2816 +eventMesh.server.kafka.client.consumeThreadPoolQueueSize=3816 +eventMesh.server.kafka.client.pullBatchSize=4816 +eventMesh.server.kafka.client.ackwindow=5816 +eventMesh.server.kafka.client.pubwindow=6816 +eventMesh.server.kafka.client.comsumeTimeoutInMin=7816 +eventMesh.server.kafka.client.pollNameServerInterval=8816 +eventMesh.server.kafka.client.heartbeatBrokerInterval=9816 +eventMesh.server.kafka.client.rebalanceInterval=11816 +eventMesh.server.kafka.cluster=cluster-succeed!!! +eventMesh.server.kafka.accessKey=accessKey-succeed!!! +eventMesh.server.kafka.secretKey=secretKey-succeed!!! +eventMesh.server.kafka.username=username-succeed!!! +eventMesh.server.kafka.password=password-succeed!!! +eventMesh.server.kafka.client.consumeThreadMin=1816 +eventMesh.server.kafka.client.consumeThreadMax=2816 +eventMesh.server.kafka.client.consumeThreadPoolQueueSize=3816 +eventMesh.server.kafka.client.pullBatchSize=4816 +eventMesh.server.kafka.client.ackwindow=5816 +eventMesh.server.kafka.client.pubwindow=6816 +eventMesh.server.kafka.client.comsumeTimeoutInMin=7816 +eventMesh.server.kafka.client.pollNameServerInterval=8816 +eventMesh.server.kafka.client.heartbeatBrokerInterval=9816 +eventMesh.server.kafka.client.rebalanceInterval=11816 +eventMesh.server.kafka.cluster=cluster-succeed!!! +eventMesh.server.kafka.accessKey=accessKey-succeed!!! +eventMesh.server.kafka.secretKey=secretKey-succeed!!! \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ClientConfiguration.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ClientConfiguration.java index 56855b8139..6e741cf956 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ClientConfiguration.java +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ClientConfiguration.java @@ -17,39 +17,35 @@ package org.apache.eventmesh.connector.knative.config; -import org.apache.commons.lang3.StringUtils; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; +import org.apache.eventmesh.common.config.convert.converter.ListConverter.ListConverterSemi; -import com.google.common.base.Preconditions; - -public class ClientConfiguration { - - - private transient String emurl; - - private transient String serviceAddr; +import java.util.List; +import lombok.Data; - public String getEmurl() { - return emurl; - } - - public String getServiceAddr() { - return serviceAddr; - } +@Data +@Config(prefix = "eventMesh.server.knative", path = "classPath://knative-client.properties") +public class ClientConfiguration { - public void init() { - String serviceAddrStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_KNATIVE_SERVICE_ADDR); - Preconditions.checkState(StringUtils.isNotEmpty(serviceAddrStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_KNATIVE_SERVICE_ADDR)); - serviceAddr = StringUtils.trim(serviceAddrStr); - String[] temp = serviceAddr.split(";"); - emurl = temp[0]; - serviceAddr = temp[1]; - } + @ConfigFiled(field = "service", converter = ListConverterSemi.class) + public List service; - static class ConfKeys { + /** + * In keeping with the old way of configuration parsing, the value is taken from the service field [0] + */ + @ConfigFiled(reload = true) + public String emurl = ""; - public static final String KEYS_EVENTMESH_KNATIVE_SERVICE_ADDR = "eventMesh.server.knative.service"; + /** + * In keeping with the old way of configuration parsing, the value is taken from the service field [1] + */ + @ConfigFiled(reload = true) + public String serviceAddr = ""; + public void reload() { + emurl = this.service.get(0); + serviceAddr = this.service.get(1); } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ConfigurationWrapper.java deleted file mode 100644 index a6433e2cdd..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/config/ConfigurationWrapper.java +++ /dev/null @@ -1,72 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.knative.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; -import org.apache.eventmesh.connector.knative.common.EventMeshConstants; - -import org.apache.commons.lang3.StringUtils; - -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass -public class ConfigurationWrapper { - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public String getProp(String key) { - return StringUtils.isEmpty(key) ? null : properties.getProperty(key, null); - } - - /** - * Load Knative properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + EventMeshConstants.EVENTMESH_CONF_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - throw new RuntimeException(String.format("Load %s.properties file from classpath error", EventMeshConstants.EVENTMESH_CONF_FILE), e); - } - - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + EventMeshConstants.EVENTMESH_CONF_FILE; - if (new File(configPath).exists()) { - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath, StandardCharsets.UTF_8); - } - } catch (IOException e) { - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", EventMeshConstants.EVENTMESH_CONF_FILE), e); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImpl.java index 56062779c5..cc41ae4484 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImpl.java @@ -20,6 +20,7 @@ import org.apache.eventmesh.api.AbstractContext; import org.apache.eventmesh.api.EventListener; import org.apache.eventmesh.api.consumer.Consumer; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.knative.config.ClientConfiguration; import java.util.List; @@ -30,6 +31,7 @@ import io.cloudevents.CloudEvent; +@Config(field = "clientConfiguration") public class KnativeConsumerImpl implements Consumer { private static final Logger LOG = LoggerFactory.getLogger(KnativeConsumerImpl.class); @@ -37,12 +39,16 @@ public class KnativeConsumerImpl implements Consumer { private transient PullConsumerImpl pullConsumer; + /** + * Unified configuration class corresponding to knative-client.properties + */ + private ClientConfiguration clientConfiguration; + + private static final Logger logger = LoggerFactory.getLogger(KnativeConsumerImpl.class); @Override public synchronized void init(Properties properties) throws Exception { // Load parameters from properties file: - final ClientConfiguration clientConfiguration = new ClientConfiguration(); - clientConfiguration.init(); properties.put("emUrl", clientConfiguration.getEmurl()); properties.put("serviceAddr", clientConfiguration.getServiceAddr()); @@ -92,4 +98,8 @@ public void start() { public void shutdown() { pullConsumer.shutdown(); } + + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImpl.java index cda05dff0b..3e3c5f0768 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/main/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImpl.java @@ -21,22 +21,26 @@ import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.api.producer.Producer; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.knative.config.ClientConfiguration; import java.util.Properties; import io.cloudevents.CloudEvent; +@Config(field = "clientConfiguration") public class KnativeProducerImpl implements Producer { private transient ProducerImpl producer; + /** + * Unified configuration class corresponding to knative-client.properties + */ + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties properties) throws Exception { // Load parameters from properties file: - final ClientConfiguration clientConfiguration = new ClientConfiguration(); - clientConfiguration.init(); - properties.put("url", clientConfiguration.getServiceAddr()); producer = new ProducerImpl(properties); } @@ -90,4 +94,8 @@ public void checkTopicExist(String topic) throws Exception { public void setExtFields() { throw new ConnectorRuntimeException("SetExtFields is not supported"); } + + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/config/ClientConfigurationTest.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/config/ClientConfigurationTest.java new file mode 100644 index 0000000000..aa13b3a608 --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/config/ClientConfigurationTest.java @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.knative.config; + +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.knative.consumer.KnativeConsumerImpl; +import org.apache.eventmesh.connector.knative.producer.KnativeProducerImpl; + +import org.junit.Assert; +import org.junit.Test; + +public class ClientConfigurationTest { + + @Test + public void getConfigWhenKnativeConsumerInit() { + KnativeConsumerImpl consumer = + (KnativeConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("knative"); + + ClientConfiguration config = consumer.getClientConfiguration(); + assertConfig(config); + } + + @Test + public void getConfigWhenKnativeProducerInit() { + KnativeProducerImpl producer = + (KnativeProducerImpl) ConnectorPluginFactory.getMeshMQProducer("knative"); + + ClientConfiguration config = producer.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(ClientConfiguration config) { + Assert.assertEquals(config.getEmurl(), "127.0.0.1"); + Assert.assertEquals(config.getServiceAddr(), "cloudevents-player.default.127.0.0.1.sslip.io"); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImplTest.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImplTest.java index e5edc1261c..c5690fb753 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImplTest.java +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/consumer/KnativeConsumerImplTest.java @@ -17,6 +17,8 @@ package org.apache.eventmesh.connector.knative.consumer; +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; + import java.util.Properties; import org.junit.Assert; @@ -31,7 +33,8 @@ public void testSubscribe() throws Exception { properties.put("topic", topic); // Create a Knative consumer: - KnativeConsumerImpl knativeConsumer = new KnativeConsumerImpl(); + KnativeConsumerImpl knativeConsumer = + (KnativeConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("knative"); try { knativeConsumer.init(properties); diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImplTest.java b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImplTest.java index 85c459beb5..a4ca490aac 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImplTest.java +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/java/org/apache/eventmesh/connector/knative/producer/KnativeProducerImplTest.java @@ -20,6 +20,7 @@ import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.SendResult; import org.apache.eventmesh.api.exception.OnExceptionContext; +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; import org.apache.eventmesh.connector.knative.cloudevent.KnativeMessageFactory; import org.apache.eventmesh.connector.knative.cloudevent.impl.KnativeHeaders; @@ -42,7 +43,8 @@ public void testPublish() throws Exception { properties.put("data", "Hello Knative from EventMesh!"); // Create a Knative producer: - KnativeProducerImpl knativehProducer = new KnativeProducerImpl(); + KnativeProducerImpl knativehProducer = + (KnativeProducerImpl) ConnectorPluginFactory.getMeshMQProducer("knative"); try { knativehProducer.init(properties); diff --git a/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/resources/knative-client.properties b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/resources/knative-client.properties new file mode 100644 index 0000000000..8fec1c4302 --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-knative/src/test/resources/knative-client.properties @@ -0,0 +1,18 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +#######################knative-client################## +eventMesh.server.knative.service=127.0.0.1;cloudevents-player.default.127.0.0.1.sslip.io diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaConsumerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaConsumerImpl.java index 5c06f6d4ad..5415d328ae 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaConsumerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaConsumerImpl.java @@ -20,7 +20,9 @@ import org.apache.eventmesh.api.AbstractContext; import org.apache.eventmesh.api.EventListener; import org.apache.eventmesh.api.consumer.Consumer; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.pravega.client.PravegaClient; +import org.apache.eventmesh.connector.pravega.config.PravegaConnectorConfig; import org.apache.eventmesh.connector.pravega.exception.PravegaConnectorException; import java.util.List; @@ -32,9 +34,15 @@ import lombok.extern.slf4j.Slf4j; @Slf4j +@Config(field = "pravegaConnectorConfig") public class PravegaConsumerImpl implements Consumer { private static final AtomicBoolean started = new AtomicBoolean(false); + /** + * Unified configuration class corresponding to pravega-connector.properties + */ + private PravegaConnectorConfig pravegaConnectorConfig; + private boolean isBroadcast; private String instanceName; private String consumerGroup; @@ -46,7 +54,8 @@ public void init(Properties keyValue) throws Exception { isBroadcast = Boolean.parseBoolean(keyValue.getProperty("isBroadcast", "false")); instanceName = keyValue.getProperty("instanceName", ""); consumerGroup = keyValue.getProperty("consumerGroup", ""); - client = PravegaClient.getInstance(); + + client = PravegaClient.getInstance(pravegaConnectorConfig); } @Override @@ -92,4 +101,8 @@ public void unsubscribe(String topic) { public void registerEventListener(EventListener listener) { this.eventListener = listener; } + + public PravegaConnectorConfig getClientConfiguration() { + return this.pravegaConnectorConfig; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaProducerImpl.java index a5aa1d2909..58bd0b31bc 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/PravegaProducerImpl.java @@ -23,7 +23,9 @@ import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.api.exception.OnExceptionContext; import org.apache.eventmesh.api.producer.Producer; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.pravega.client.PravegaClient; +import org.apache.eventmesh.connector.pravega.config.PravegaConnectorConfig; import org.apache.eventmesh.connector.pravega.exception.PravegaConnectorException; import java.util.Properties; @@ -34,13 +36,19 @@ import lombok.extern.slf4j.Slf4j; @Slf4j +@Config(field = "pravegaConnectorConfig") public class PravegaProducerImpl implements Producer { private final AtomicBoolean started = new AtomicBoolean(false); private PravegaClient client; + /** + * Unified configuration class corresponding to pravega-connector.properties + */ + private PravegaConnectorConfig pravegaConnectorConfig; + @Override public void init(Properties properties) throws Exception { - client = PravegaClient.getInstance(); + client = PravegaClient.getInstance(pravegaConnectorConfig); } @Override @@ -105,4 +113,8 @@ public void checkTopicExist(String topic) throws Exception { public void setExtFields() { } + + public PravegaConnectorConfig getClientConfiguration() { + return this.pravegaConnectorConfig; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/client/PravegaClient.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/client/PravegaClient.java index 5fd9863200..ad353abca3 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/client/PravegaClient.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/client/PravegaClient.java @@ -57,6 +57,18 @@ public class PravegaClient { private static PravegaClient instance; + public static PravegaClient getInstance() { + return instance; + } + + public static PravegaClient getInstance(PravegaConnectorConfig config) { + if (instance == null) { + instance = new PravegaClient(config); + } + + return instance; + } + private PravegaClient(PravegaConnectorConfig config) { this.config = config; streamManager = StreamManager.create(config.getControllerURI()); @@ -72,22 +84,15 @@ private PravegaClient(PravegaConnectorConfig config) { readerGroupManager = ReaderGroupManager.withScope(config.getScope(), clientConfig); } - public static PravegaClient getInstance() { - if (instance == null) { - instance = new PravegaClient(PravegaConnectorConfig.getInstance()); - } - return instance; - } - protected static PravegaClient getNewInstance(PravegaConnectorConfig config) { return new PravegaClient(config); } public void start() { if (createScope()) { - log.info("Create Pravega scope[{}] success.", PravegaConnectorConfig.getInstance().getScope()); + log.info("Create Pravega scope[{}] success.", config.getScope()); } else { - log.info("Pravega scope[{}] has already been created.", PravegaConnectorConfig.getInstance().getScope()); + log.info("Pravega scope[{}] has already been created.", config.getScope()); } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfig.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfig.java index 2954db4c13..b7c5ffbac0 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfig.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfig.java @@ -17,6 +17,9 @@ package org.apache.eventmesh.connector.pravega.config; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; + import org.apache.commons.lang3.StringUtils; import java.net.URI; @@ -26,87 +29,38 @@ @Getter @Setter +@Config(prefix = "eventMesh.server.pravega", path = "classPath://pravega-connector.properties") public class PravegaConnectorConfig { - public static final String EVENTMESH_PRAVEGA_CONTROLLER_URI = "eventMesh.server.pravega.controller.uri"; - public static final String EVENTMESH_PRAVEGA_SCOPE = "eventMesh.server.pravega.scope"; - public static final String EVENTMESH_PRAVEGA_AUTH_ENABLED = "eventMesh.server.pravega.authEnabled"; - public static final String EVENTMESH_PRAVEGA_USERNAME = "eventMesh.server.pravega.username"; - public static final String EVENTMESH_PRAVEGA_PASSWORD = "eventMesh.server.pravega.password"; - public static final String EVENTMESH_PRAVEGA_TLS_ENABLED = "eventMesh.server.pravega.tlsEnabled"; - public static final String EVENTMESH_PRAVEGA_TRUSTSTORE = "eventMesh.server.pravega.truststore"; - public static final String EVENTMESH_PRAVEGA_CLIENTPOOL_SIZE = "eventMesh.server.pravega.clientPool.size"; - public static final String EVENTMESH_PRAVEGA_QUEUE_SIZE = "eventMesh.server.pravega.queue.size"; + @ConfigFiled(field = "controller.uri") private URI controllerURI = URI.create("tcp://127.0.0.1:9090"); - private String scope = "eventmesh-pravega"; - private int clientPoolSize = 8; - private int queueSize = 512; - private boolean authEnabled = false; - private String username = ""; - private String password = ""; - private boolean tlsEnable = false; - private String truststore = ""; - - private static PravegaConnectorConfig INSTANCE = null; - - private PravegaConnectorConfig() { - } - - public static synchronized PravegaConnectorConfig getInstance() { - if (INSTANCE == null) { - INSTANCE = new PravegaConnectorConfig(); - INSTANCE.init(); - } - return INSTANCE; - } - - private void init() { - String controllerURIStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_CONTROLLER_URI); - if (StringUtils.isNotBlank(controllerURIStr)) { - controllerURI = URI.create(StringUtils.trim(controllerURIStr)); - } - String scopeStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_SCOPE); - if (StringUtils.isNotBlank(scopeStr)) { - scope = StringUtils.trim(scopeStr); - } + @ConfigFiled(field = "scope") + private String scope = "eventmesh-pravega"; - String authEnableStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_AUTH_ENABLED); - if (StringUtils.isNotBlank(authEnableStr)) { - authEnabled = Boolean.parseBoolean(StringUtils.trim(authEnableStr)); - } + @ConfigFiled(field = "clientPool.size") + private int clientPoolSize = 8; - String usernameStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_USERNAME); - if (StringUtils.isNotBlank(usernameStr)) { - username = StringUtils.trim(usernameStr); - } + @ConfigFiled(field = "queue.size") + private int queueSize = 512; - String passwordStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_PASSWORD); - if (StringUtils.isNotBlank(passwordStr)) { - password = StringUtils.trim(passwordStr); - } + @ConfigFiled(field = "authEnabled", reload = true) + private boolean authEnabled = false; - String tlsEnableStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_TLS_ENABLED); - if (StringUtils.isNotBlank(tlsEnableStr)) { - tlsEnable = Boolean.parseBoolean(StringUtils.trim(tlsEnableStr)); - } + @ConfigFiled(field = "username") + private String username = ""; - String truststoreStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_TRUSTSTORE); - if (StringUtils.isNotBlank(truststoreStr)) { - truststore = StringUtils.trim(truststoreStr); - } + @ConfigFiled(field = "password") + private String password = ""; - String clientPoolSizeStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_CLIENTPOOL_SIZE); - if (StringUtils.isNumeric(clientPoolSizeStr)) { - clientPoolSize = Integer.parseInt(clientPoolSizeStr); - } + @ConfigFiled(field = "tlsEnabled") + private boolean tlsEnable = false; - String queueSizeStr = PravegaConnectorConfigWrapper.getProp(EVENTMESH_PRAVEGA_QUEUE_SIZE); - if (StringUtils.isNumeric(queueSizeStr)) { - queueSize = Integer.parseInt(queueSizeStr); - } + @ConfigFiled(field = "truststore") + private String truststore = ""; - if (StringUtils.isNotBlank(username) && StringUtils.isNotBlank(password)) { + public void reload() { + if (!authEnabled && StringUtils.isNotBlank(username) && StringUtils.isNotBlank(password)) { authEnabled = true; } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigWrapper.java deleted file mode 100644 index b3553f9214..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/main/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigWrapper.java +++ /dev/null @@ -1,72 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.pravega.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; - -import org.apache.commons.lang3.StringUtils; - -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass -public class PravegaConnectorConfigWrapper { - public static final String PRAVEGA_CONF_FILE = "pravega-connector.properties"; - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public String getProp(String key) { - return StringUtils.isEmpty(key) ? null : properties.getProperty(key, null); - } - - /** - * Load Pravega properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - String path = File.separator + PRAVEGA_CONF_FILE; - try (InputStream resourceAsStream = PravegaConnectorConfigWrapper.class.getResourceAsStream(path)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - log.error("Load {}.properties file from classpath error", path); - throw new RuntimeException(String.format("Load %s.properties file from classpath error", PRAVEGA_CONF_FILE)); - } - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + PRAVEGA_CONF_FILE; - try { - if (new File(configPath).exists()) { - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } - } catch (IOException e) { - log.error("Cannot load {} file from conf", configPath); - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", PRAVEGA_CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/client/PravegaClientTest.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/client/PravegaClientTest.java index c1e5da5b4d..f4673508a9 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/client/PravegaClientTest.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/client/PravegaClientTest.java @@ -24,6 +24,7 @@ import org.apache.eventmesh.api.AsyncConsumeContext; import org.apache.eventmesh.api.EventListener; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.connector.pravega.config.PravegaConnectorConfig; import java.net.URI; @@ -145,10 +146,12 @@ public void checkTopicExistTest() { assertTrue(pravegaClient.checkTopicExist("test1")); } - private PravegaClient getNewPravegaClient() { - config = PravegaConnectorConfig.getInstance(); - config.setControllerURI(controllerURI); - return PravegaClient.getNewInstance(config); + public PravegaClient getNewPravegaClient() { + ConfigService configService = ConfigService.getInstance(); + + this.config = configService.buildConfigInstance(PravegaConnectorConfig.class); + this.config.setControllerURI(controllerURI); + return PravegaClient.getNewInstance(this.config); } private CloudEvent createCloudEvent() { diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigTest.java b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigTest.java index ef638a6386..4256f8cafb 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigTest.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/java/org/apache/eventmesh/connector/pravega/config/PravegaConnectorConfigTest.java @@ -17,65 +17,44 @@ package org.apache.eventmesh.connector.pravega.config; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.pravega.PravegaConsumerImpl; +import org.apache.eventmesh.connector.pravega.PravegaProducerImpl; import java.net.URI; -import org.junit.BeforeClass; +import org.junit.Assert; import org.junit.Test; public class PravegaConnectorConfigTest { - private static PravegaConnectorConfig config; - - @BeforeClass - public static void init() { - config = PravegaConnectorConfig.getInstance(); - } - - @Test - public void getControllerURI() { - assertEquals(URI.create("tcp://127.0.0.1:9090"), config.getControllerURI()); - } - - @Test - public void getScope() { - assertEquals("eventmesh-pravega", config.getScope()); - } - - @Test - public void isAuthEnabled() { - assertFalse(config.isAuthEnabled()); - } - - @Test - public void getUsername() { - assertEquals("", config.getUsername()); - } - @Test - public void getPassword() { - assertEquals("", config.getPassword()); - } + public void getConfigWhenPravegaConsumerInit() { + PravegaConsumerImpl consumer = + (PravegaConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("pravega"); - @Test - public void isTslEnabled() { - assertFalse(config.isTlsEnable()); + PravegaConnectorConfig config = consumer.getClientConfiguration(); + assertConfig(config); } @Test - public void getTruststore() { - assertEquals("", config.getTruststore()); - } + public void getConfigWhenPravegaProducerInit() { + PravegaProducerImpl producer = + (PravegaProducerImpl) ConnectorPluginFactory.getMeshMQProducer("pravega"); - @Test - public void getClientPoolSize() { - assertEquals(8, config.getClientPoolSize()); + PravegaConnectorConfig config = producer.getClientConfiguration(); + assertConfig(config); } - @Test - public void getQueueSize() { - assertEquals(512, config.getQueueSize()); + private void assertConfig(PravegaConnectorConfig config) { + Assert.assertEquals(config.getControllerURI(), URI.create("tcp://127.0.0.1:816")); + Assert.assertEquals(config.getScope(), "scope-success!!!"); + Assert.assertTrue(config.isAuthEnabled()); + Assert.assertEquals(config.getUsername(), "username-success!!!"); + Assert.assertEquals(config.getPassword(), "password-success!!!"); + Assert.assertTrue(config.isTlsEnable()); + Assert.assertEquals(config.getTruststore(), "truststore-success!!!"); + Assert.assertEquals(config.getClientPoolSize(), 816); + Assert.assertEquals(config.getQueueSize(), 1816); } } \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/resources/pravega-connector.properties b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/resources/pravega-connector.properties index 25fa6f0af4..5bb5e54907 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/resources/pravega-connector.properties +++ b/eventmesh-connector-plugin/eventmesh-connector-pravega/src/test/resources/pravega-connector.properties @@ -15,12 +15,12 @@ # limitations under the License. # #######################pravega-client################## -eventMesh.server.pravega.controller.uri=tcp://127.0.0.1:9090 -eventMesh.server.pravega.scope=eventmesh-pravega +eventMesh.server.pravega.controller.uri=tcp://127.0.0.1:816 +eventMesh.server.pravega.scope=scope-success!!! eventMesh.server.pravega.authEnabled=false -eventMesh.server.pravega.username= -eventMesh.server.pravega.password= -eventMesh.server.pravega.tlsEnabled=false -eventMesh.server.pravega.truststore= -eventMesh.server.pravega.clientPool.size=8 -eventMesh.server.pravega.queue.size=512 \ No newline at end of file +eventMesh.server.pravega.username=username-success!!! +eventMesh.server.pravega.password=password-success!!! +eventMesh.server.pravega.tlsEnabled=true +eventMesh.server.pravega.truststore=truststore-success!!! +eventMesh.server.pravega.clientPool.size=816 +eventMesh.server.pravega.queue.size=1816 \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ClientConfiguration.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ClientConfiguration.java index ae0bc56267..9410debf59 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ClientConfiguration.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ClientConfiguration.java @@ -17,44 +17,23 @@ package org.apache.eventmesh.connector.pulsar.config; -import org.apache.commons.lang3.StringUtils; - -import com.google.common.base.Preconditions; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; import lombok.Getter; import lombok.Setter; @Getter @Setter +@Config(prefix = "eventMesh.server.pulsar", path = "classPath://pulsar-client.properties") public class ClientConfiguration { + @ConfigFiled(field = "service") private String serviceAddr; - private String authPlugin; - private String authParams; - - private static ClientConfiguration INSTANCE = null; - public void init() { - String serviceAddrStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PULSAR_SERVICE_ADDR); - Preconditions.checkState(StringUtils.isNotEmpty(serviceAddrStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_PULSAR_SERVICE_ADDR)); - serviceAddr = StringUtils.trim(serviceAddrStr); - authPlugin = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PULSAR_AUTH_PLUGIN); - authParams = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PULSAR_AUTH_PARAMS); - } - - public static ClientConfiguration getInstance() { - if (INSTANCE == null) { - INSTANCE = new ClientConfiguration(); - INSTANCE.init(); - } - return INSTANCE; - } - - static class ConfKeys { - public static final String KEYS_EVENTMESH_PULSAR_SERVICE_ADDR = "eventMesh.server.pulsar.service"; - public static final String KEYS_EVENTMESH_PULSAR_AUTH_PLUGIN = "eventMesh.server.pulsar.authPlugin"; - public static final String KEYS_EVENTMESH_PULSAR_AUTH_PARAMS = "eventMesh.server.pulsar.authParams"; - } + @ConfigFiled(field = "authPlugin") + private String authPlugin; + @ConfigFiled(field = "authParams") + private String authParams; } \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ConfigurationWrapper.java deleted file mode 100644 index ca2ccd0aa6..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/config/ConfigurationWrapper.java +++ /dev/null @@ -1,74 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.pulsar.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.connector.pulsar.common.EventMeshConstants; - -import org.apache.commons.lang3.StringUtils; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass -public class ConfigurationWrapper { - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public String getProp(String key) { - return StringUtils.isEmpty(key) ? null : properties.getProperty(key, null); - } - - /** - * Load Pulsar properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + EventMeshConstants.EVENTMESH_CONF_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - log.error("Load {}.properties file from classpath error", EventMeshConstants.EVENTMESH_CONF_FILE, e); - throw new RuntimeException(String.format("Load %s.properties file from classpath error", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + EventMeshConstants.EVENTMESH_CONF_FILE; - if (new File(configPath).exists()) { - properties.load(new BufferedReader(new FileReader(configPath))); - } - } catch (IOException e) { - log.error("Cannot load {} file from conf.", EventMeshConstants.EVENTMESH_CONF_FILE, e); - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/consumer/PulsarConsumerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/consumer/PulsarConsumerImpl.java index f2a621ab9c..fba1f8a6d7 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/consumer/PulsarConsumerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/consumer/PulsarConsumerImpl.java @@ -24,6 +24,7 @@ import org.apache.eventmesh.api.consumer.Consumer; import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.common.Constants; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.pulsar.config.ClientConfiguration; import org.apache.pulsar.client.api.ClientBuilder; @@ -45,6 +46,7 @@ import lombok.extern.slf4j.Slf4j; @Slf4j +@Config(field = "clientConfiguration") public class PulsarConsumerImpl implements Consumer { private final AtomicBoolean started = new AtomicBoolean(false); @@ -53,12 +55,15 @@ public class PulsarConsumerImpl implements Consumer { private org.apache.pulsar.client.api.Consumer consumer; private EventListener eventListener; + /** + * Unified configuration class corresponding to pulsar-client.properties + */ + private ClientConfiguration clientConfiguration; + @Override public void init(Properties properties) throws Exception { this.properties = properties; - final ClientConfiguration clientConfiguration = ClientConfiguration.getInstance(); - try { ClientBuilder clientBuilder = PulsarClient.builder() .serviceUrl(clientConfiguration.getServiceAddr()); @@ -160,4 +165,8 @@ public void shutdown() { String.format("Failed to close the pulsar client with exception: %s", ex.getMessage())); } } + + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/ProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/ProducerImpl.java index e49ef3b39e..2d90ebaa86 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/ProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/ProducerImpl.java @@ -36,10 +36,13 @@ public class ProducerImpl extends AbstractProducer { private ClientConfiguration config; private PulsarClientWrapper pulsarClient; + public ProducerImpl(final Properties properties, ClientConfiguration config) { + this(properties); + setConfig(config); + } + public ProducerImpl(final Properties properties) { super(properties); - this.config = new ClientConfiguration(); - this.config.init(); } public void publish(CloudEvent cloudEvent, SendCallback sendCallback) { @@ -73,4 +76,8 @@ public boolean isStarted() { public boolean isClosed() { return !this.isStarted(); } + + public void setConfig(ClientConfiguration config) { + this.config = config; + } } \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/PulsarProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/PulsarProducerImpl.java index 64c08847e0..063c1439ec 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/PulsarProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/main/java/org/apache/eventmesh/connector/pulsar/producer/PulsarProducerImpl.java @@ -21,18 +21,26 @@ import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.api.producer.Producer; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.connector.pulsar.config.ClientConfiguration; import java.util.Properties; import io.cloudevents.CloudEvent; +@Config(field = "clientConfiguration") public class PulsarProducerImpl implements Producer { private ProducerImpl producer; + /** + * Unified configuration class corresponding to pulsar-client.properties + */ + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties properties) { - producer = new ProducerImpl(properties); + producer = new ProducerImpl(properties, clientConfiguration); } @Override @@ -84,4 +92,8 @@ public void checkTopicExist(String topic) throws Exception { public void setExtFields() { throw new ConnectorRuntimeException("SetExtFields is not supported"); } + + public ClientConfiguration getClientConfiguration() { + return this.clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/java/org/apache/eventmesh/connector/pulsar/config/ClientConfigurationTest.java b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/java/org/apache/eventmesh/connector/pulsar/config/ClientConfigurationTest.java new file mode 100644 index 0000000000..493cd167ad --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/java/org/apache/eventmesh/connector/pulsar/config/ClientConfigurationTest.java @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.pulsar.config; + +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.pulsar.consumer.PulsarConsumerImpl; +import org.apache.eventmesh.connector.pulsar.producer.PulsarProducerImpl; + +import org.junit.Assert; +import org.junit.Test; + +public class ClientConfigurationTest { + + @Test + public void getConfigWhenPulsarConsumerInit() { + PulsarConsumerImpl consumer = + (PulsarConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("pulsar"); + + ClientConfiguration config = consumer.getClientConfiguration(); + assertConfig(config); + } + + @Test + public void getConfigWhenPulsarProducerInit() { + PulsarProducerImpl producer = + (PulsarProducerImpl) ConnectorPluginFactory.getMeshMQProducer("pulsar"); + + ClientConfiguration config = producer.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(ClientConfiguration config) { + Assert.assertEquals(config.getServiceAddr(), "127.0.0.1:6650"); + Assert.assertEquals(config.getAuthPlugin(), "authPlugin-success!!!"); + Assert.assertEquals(config.getAuthParams(), "authParams-success!!!"); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/resources/pulsar-client.properties b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/resources/pulsar-client.properties new file mode 100644 index 0000000000..645a5edb6e --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-pulsar/src/test/resources/pulsar-client.properties @@ -0,0 +1,20 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +eventMesh.server.pulsar.service=127.0.0.1:6650 +eventMesh.server.pulsar.authPlugin=authPlugin-success!!! +eventMesh.server.pulsar.authParams=authParams-success!!! \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigKey.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigKey.java deleted file mode 100644 index 9173e5ffc6..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigKey.java +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.rabbitmq.config; - -public class ConfigKey { - public static final String HOST = "eventMesh.server.rabbitmq.host"; - public static final String PORT = "eventMesh.server.rabbitmq.port"; - public static final String USER_NAME = "eventMesh.server.rabbitmq.username"; - public static final String PASSWD = "eventMesh.server.rabbitmq.passwd"; - public static final String VIRTUAL_HOST = "eventMesh.server.rabbitmq.virtualHost"; - - public static final String EXCHANGE_TYPE = "eventMesh.server.rabbitmq.exchangeType"; - public static final String EXCHANGE_NAME = "eventMesh.server.rabbitmq.exchangeName"; - public static final String ROUTING_KEY = "eventMesh.server.rabbitmq.routingKey"; - public static final String QUEUE_NAME = "eventMesh.server.rabbitmq.queueName"; - public static final String AUTO_ACK = "eventMesh.server.rabbitmq.autoAck"; -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolder.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolder.java index 34a7e71895..7ac00ef0b3 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolder.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolder.java @@ -17,49 +17,44 @@ package org.apache.eventmesh.connector.rabbitmq.config; -import org.apache.eventmesh.common.utils.AssertUtils; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; import com.rabbitmq.client.BuiltinExchangeType; import lombok.Data; @Data +@Config(prefix = "eventMesh.server.rabbitmq", path = "classPath://rabbitmq-client.properties") public class ConfigurationHolder { + + @ConfigFiled(field = "host") public String host; + + @ConfigFiled(field = "port") public int port; + + @ConfigFiled(field = "username") public String username; + + @ConfigFiled(field = "passwd") public String passwd; + + @ConfigFiled(field = "virtualHost") public String virtualHost; + @ConfigFiled(field = "exchangeType") public BuiltinExchangeType exchangeType; + + @ConfigFiled(field = "exchangeName") public String exchangeName; - public String routingKey; - public String queueName; - public boolean autoAck; - public void init() { - this.host = getProperty(ConfigKey.HOST); - this.port = Integer.parseInt(getProperty(ConfigKey.PORT)); - this.username = getProperty(ConfigKey.USER_NAME); - this.passwd = getProperty(ConfigKey.PASSWD); - this.virtualHost = ConfigurationWrapper.getProperty(ConfigKey.VIRTUAL_HOST); - this.exchangeType = BuiltinExchangeType.valueOf(getProperty(ConfigKey.EXCHANGE_TYPE)); - this.exchangeName = getProperty(ConfigKey.EXCHANGE_NAME); - this.routingKey = getProperty(ConfigKey.ROUTING_KEY); - this.queueName = getProperty(ConfigKey.QUEUE_NAME); - this.autoAck = Boolean.parseBoolean(getProperty(ConfigKey.AUTO_ACK)); - } + @ConfigFiled(field = "routingKey") + public String routingKey; - /** - * get property - * - * @param configKey config key - * @return property - */ - private String getProperty(String configKey) { - String property = ConfigurationWrapper.getProperty(configKey); - AssertUtils.notBlack(property, String.format("%s error", configKey)); - return property; + @ConfigFiled(field = "queueName") + public String queueName; - } + @ConfigFiled(field = "autoAck") + public boolean autoAck; } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationWrapper.java deleted file mode 100644 index 6a244977fa..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationWrapper.java +++ /dev/null @@ -1,67 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.rabbitmq.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; - -import org.apache.commons.lang3.StringUtils; - -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import lombok.extern.slf4j.Slf4j; - -@Slf4j -public class ConfigurationWrapper { - - private static final String CONF_FILE = "rabbitmq-client.properties"; - - private static final Properties PROPERTIES = new Properties(); - - static { - loadProperties(); - } - - public static String getProperty(String key) { - return StringUtils.isEmpty(key) - ? null : PROPERTIES.getProperty(key, null); - } - - private static void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + CONF_FILE)) { - if (resourceAsStream != null) { - PROPERTIES.load(resourceAsStream); - } - } catch (IOException e) { - log.error("load file from classpath exception:", e); - throw new RuntimeException(String.format("Load %s file from classpath error", CONF_FILE)); - } - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONF_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(PROPERTIES, configPath, StandardCharsets.UTF_8); - } catch (IOException e) { - log.error("load file from conf exception:", e); - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/consumer/RabbitmqConsumer.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/consumer/RabbitmqConsumer.java index 56124ab824..b2da7db84e 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/consumer/RabbitmqConsumer.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/consumer/RabbitmqConsumer.java @@ -21,6 +21,7 @@ import org.apache.eventmesh.api.EventListener; import org.apache.eventmesh.api.consumer.Consumer; import org.apache.eventmesh.common.ThreadPoolFactory; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.rabbitmq.client.RabbitmqClient; import org.apache.eventmesh.connector.rabbitmq.client.RabbitmqConnectionFactory; import org.apache.eventmesh.connector.rabbitmq.config.ConfigurationHolder; @@ -37,6 +38,7 @@ import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; +@Config(field = "configurationHolder") public class RabbitmqConsumer implements Consumer { private static final Logger logger = LoggerFactory.getLogger(RabbitmqConsumer.class); @@ -51,7 +53,10 @@ public class RabbitmqConsumer implements Consumer { private volatile boolean started = false; - private final ConfigurationHolder configurationHolder = new ConfigurationHolder(); + /** + * Unified configuration class corresponding to rabbitmq-client.properties + */ + private ConfigurationHolder configurationHolder; private final ThreadPoolExecutor executor = ThreadPoolFactory.createThreadPoolExecutor( Runtime.getRuntime().availableProcessors() * 2, @@ -92,7 +97,6 @@ public void shutdown() { @Override public void init(Properties keyValue) throws Exception { - this.configurationHolder.init(); this.rabbitmqClient = new RabbitmqClient(rabbitmqConnectionFactory); this.connection = rabbitmqClient.getConnection(configurationHolder.getHost(), configurationHolder.getUsername(), configurationHolder.getPasswd(), configurationHolder.getPort(), configurationHolder.getVirtualHost()); @@ -131,4 +135,8 @@ public void registerEventListener(EventListener listener) { public void setRabbitmqConnectionFactory(RabbitmqConnectionFactory rabbitmqConnectionFactory) { this.rabbitmqConnectionFactory = rabbitmqConnectionFactory; } + + public ConfigurationHolder getClientConfiguration() { + return this.configurationHolder; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/producer/RabbitmqProducer.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/producer/RabbitmqProducer.java index 700c44792d..f6e4e00412 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/producer/RabbitmqProducer.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/main/java/org/apache/eventmesh/connector/rabbitmq/producer/RabbitmqProducer.java @@ -23,6 +23,7 @@ import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.api.exception.OnExceptionContext; import org.apache.eventmesh.api.producer.Producer; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.rabbitmq.client.RabbitmqClient; import org.apache.eventmesh.connector.rabbitmq.client.RabbitmqConnectionFactory; import org.apache.eventmesh.connector.rabbitmq.cloudevent.RabbitmqCloudEvent; @@ -41,6 +42,7 @@ import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; +@Config(field = "configurationHolder") public class RabbitmqProducer implements Producer { private static final Logger logger = LoggerFactory.getLogger(RabbitmqProducer.class); @@ -55,7 +57,10 @@ public class RabbitmqProducer implements Producer { private volatile boolean started = false; - private final ConfigurationHolder configurationHolder = new ConfigurationHolder(); + /** + * Unified configuration class corresponding to rabbitmq-client.properties + */ + private ConfigurationHolder configurationHolder; @Override public boolean isStarted() { @@ -88,7 +93,6 @@ public void shutdown() { @Override public void init(Properties properties) throws Exception { - this.configurationHolder.init(); this.rabbitmqClient = new RabbitmqClient(rabbitmqConnectionFactory); this.connection = rabbitmqClient.getConnection(configurationHolder.getHost(), configurationHolder.getUsername(), configurationHolder.getPasswd(), configurationHolder.getPort(), configurationHolder.getVirtualHost()); @@ -158,4 +162,8 @@ public void setExtFields() { public void setRabbitmqConnectionFactory(RabbitmqConnectionFactory rabbitmqConnectionFactory) { this.rabbitmqConnectionFactory = rabbitmqConnectionFactory; } + + public ConfigurationHolder getClientConfiguration() { + return this.configurationHolder; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/RabbitmqServer.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/RabbitmqServer.java index a6287339d9..ce36c30096 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/RabbitmqServer.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/RabbitmqServer.java @@ -17,6 +17,7 @@ package org.apache.eventmesh.connector.rabbitmq; +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; import org.apache.eventmesh.connector.rabbitmq.consumer.RabbitmqConsumer; import org.apache.eventmesh.connector.rabbitmq.producer.RabbitmqProducer; @@ -33,12 +34,13 @@ public class RabbitmqServer { public void setup() throws Exception { RabbitmqMockConnectionFactory rabbitmqMockConnectionFactory = new RabbitmqMockConnectionFactory(); - rabbitmqConsumer = new RabbitmqConsumer(); + rabbitmqConsumer = + (RabbitmqConsumer) ConnectorPluginFactory.getMeshMQPushConsumer("rabbitmq"); rabbitmqConsumer.setRabbitmqConnectionFactory(rabbitmqMockConnectionFactory); rabbitmqConsumer.init(new Properties()); rabbitmqConsumer.start(); - rabbitmqProducer = new RabbitmqProducer(); + rabbitmqProducer = (RabbitmqProducer) ConnectorPluginFactory.getMeshMQProducer("rabbitmq"); rabbitmqProducer.setRabbitmqConnectionFactory(rabbitmqMockConnectionFactory); rabbitmqProducer.init(new Properties()); rabbitmqProducer.start(); diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolderTest.java b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolderTest.java new file mode 100644 index 0000000000..231703ecc8 --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/java/org/apache/eventmesh/connector/rabbitmq/config/ConfigurationHolderTest.java @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.rabbitmq.config; + +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.rabbitmq.consumer.RabbitmqConsumer; +import org.apache.eventmesh.connector.rabbitmq.producer.RabbitmqProducer; + +import org.junit.Assert; +import org.junit.Test; + +import com.rabbitmq.client.BuiltinExchangeType; + +public class ConfigurationHolderTest { + + @Test + public void getConfigWhenRabbitmqConsumerInit() { + RabbitmqConsumer consumer = + (RabbitmqConsumer) ConnectorPluginFactory.getMeshMQPushConsumer("rabbitmq"); + + ConfigurationHolder config = consumer.getClientConfiguration(); + assertConfig(config); + } + + @Test + public void getConfigWhenRabbitmqProducerInit() { + RabbitmqProducer producer = + (RabbitmqProducer) ConnectorPluginFactory.getMeshMQProducer("rabbitmq"); + + ConfigurationHolder config = producer.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(ConfigurationHolder config) { + Assert.assertEquals(config.getHost(), "127.0.0.1"); + Assert.assertEquals(config.getPort(), 5672); + Assert.assertEquals(config.getUsername(), "username-success!!!"); + Assert.assertEquals(config.getPasswd(), "passwd-success!!!"); + Assert.assertEquals(config.getVirtualHost(), "virtualHost-success!!!"); + + Assert.assertEquals(config.getExchangeType(), BuiltinExchangeType.TOPIC); + Assert.assertEquals(config.getExchangeName(), "exchangeName-success!!!"); + Assert.assertEquals(config.getRoutingKey(), "routingKey-success!!!"); + Assert.assertEquals(config.getQueueName(), "queueName-success!!!"); + Assert.assertTrue(config.isAutoAck()); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/resources/rabbitmq-client.properties b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/resources/rabbitmq-client.properties index b24967fa6c..082fbe5b45 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/resources/rabbitmq-client.properties +++ b/eventmesh-connector-plugin/eventmesh-connector-rabbitmq/src/test/resources/rabbitmq-client.properties @@ -18,14 +18,14 @@ ####################### rabbitmq server ################## eventMesh.server.rabbitmq.host=127.0.0.1 eventMesh.server.rabbitmq.port=5672 -eventMesh.server.rabbitmq.username=root -eventMesh.server.rabbitmq.passwd=123456 -eventMesh.server.rabbitmq.virtualHost=test +eventMesh.server.rabbitmq.username=username-success!!! +eventMesh.server.rabbitmq.passwd=passwd-success!!! +eventMesh.server.rabbitmq.virtualHost=virtualHost-success!!! ####################### rabbitmq queue setting ################## # DIRECT, FANOUT, TOPIC, HEADERS eventMesh.server.rabbitmq.exchangeType=TOPIC -eventMesh.server.rabbitmq.exchangeName=test -eventMesh.server.rabbitmq.routingKey=test -eventMesh.server.rabbitmq.queueName=test +eventMesh.server.rabbitmq.exchangeName=exchangeName-success!!! +eventMesh.server.rabbitmq.routingKey=routingKey-success!!! +eventMesh.server.rabbitmq.queueName=queueName-success!!! eventMesh.server.rabbitmq.autoAck=true \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/client/RedissonClient.java b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/client/RedissonClient.java index 6250862045..42891c26f4 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/client/RedissonClient.java +++ b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/client/RedissonClient.java @@ -17,13 +17,10 @@ package org.apache.eventmesh.connector.redis.client; -import static org.apache.eventmesh.connector.redis.config.ConfigurationWrapper.getPropertiesByPrefix; -import static org.apache.eventmesh.connector.redis.config.ConfigurationWrapper.getProperty; - import org.apache.eventmesh.api.exception.ConnectorRuntimeException; import org.apache.eventmesh.common.Constants; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.connector.redis.cloudevent.CloudEventCodec; -import org.apache.eventmesh.connector.redis.config.ConfigOptions; import org.apache.eventmesh.connector.redis.config.RedisProperties; import java.util.Arrays; @@ -59,39 +56,8 @@ public final class RedissonClient { } public static Redisson create() { - RedisProperties properties = new RedisProperties(); - String serverTypeName = getProperty(ConfigOptions.SERVER_TYPE); - if (serverTypeName != null) { - try { - properties.setServerType(RedisProperties.ServerType.valueOf(serverTypeName)); - } catch (Exception e) { - final String message = "Invalid Redis server type: " + properties.getServerType() - + ", supported values are: " - + Arrays.toString(RedisProperties.ServerType.values()); - throw new ConnectorRuntimeException(message, e); - } - } else { - properties.setServerType(RedisProperties.ServerType.SINGLE); - } - - String serverAddress = getProperty(ConfigOptions.SERVER_ADDRESS); - if (serverAddress != null) { - properties.setServerAddress(serverAddress); - } else { - throw new ConnectorRuntimeException("Lack Redis server address"); - } - - String serverMasterName = getProperty(ConfigOptions.SERVER_MASTER_NAME); - if (serverMasterName != null) { - properties.setServerMasterName(serverMasterName); - } - - String serverPassword = getProperty(ConfigOptions.SERVER_PASSWORD); - if (serverPassword != null) { - properties.setServerPassword(serverPassword); - } - - properties.setRedissonProperties(getPropertiesByPrefix(ConfigOptions.REDISSON_PROPERTIES_PREFIX)); + ConfigService configService = ConfigService.getInstance(); + RedisProperties properties = configService.buildConfigInstance(RedisProperties.class); return create(properties); } @@ -102,8 +68,8 @@ private static Redisson create(RedisProperties properties) { serverType = properties.getServerType(); } catch (IllegalArgumentException ie) { final String message = "Invalid Redis server type: " + properties.getServerType() - + ", supported values are: " - + Arrays.toString(RedisProperties.ServerType.values()); + + ", supported values are: " + + Arrays.toString(RedisProperties.ServerType.values()); throw new ConnectorRuntimeException(message, ie); } @@ -122,24 +88,24 @@ private static Redisson create(RedisProperties properties) { switch (serverType) { case SINGLE: config.useSingleServer() - .setAddress(serverAddress) - .setPassword(serverPassword); + .setAddress(serverAddress) + .setPassword(serverPassword); break; case CLUSTER: config.useClusterServers() - .addNodeAddress(serverAddress.split(Constants.COMMA)) - .setPassword(serverPassword); + .addNodeAddress(serverAddress.split(Constants.COMMA)) + .setPassword(serverPassword); break; case SENTINEL: config.useSentinelServers() - .setMasterName(masterName) - .addSentinelAddress(serverAddress) - .setPassword(serverPassword); + .setMasterName(masterName) + .addSentinelAddress(serverAddress) + .setPassword(serverPassword); break; default: final String message = "Invalid Redis server type: " + properties.getServerType() - + ", supported values are: " - + Arrays.toString(RedisProperties.ServerType.values()); + + ", supported values are: " + + Arrays.toString(RedisProperties.ServerType.values()); throw new ConnectorRuntimeException(message); } diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigOptions.java b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigOptions.java deleted file mode 100644 index f882a866fd..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigOptions.java +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.redis.config; - -/** - * Redis connector related configuration options. - */ -public interface ConfigOptions { - - /** - * The redis server configuration to be used, default is SINGLE. - */ - String SERVER_TYPE = "eventMesh.server.redis.serverType"; - - /** - * The master server name used by Redis Sentinel servers and master change monitoring task, default is master. - */ - String SERVER_MASTER_NAME = "eventMesh.server.redis.serverMasterName"; - - /** - * The address of the redis server following format -- host1:port1,host2:port2,…… - */ - String SERVER_ADDRESS = "eventMesh.server.redis.serverAddress"; - - /** - * The password for redis authentication. - */ - String SERVER_PASSWORD = "eventMesh.server.redis.serverPassword"; - - /** - * The redisson options, redisson properties, please refer to the redisson manual. - *

- * For example, the redisson timeout property is configured as eventMesh.server.redis.redisson.timeout - */ - String REDISSON_PROPERTIES_PREFIX = "eventMesh.server.redis.redisson"; -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigurationWrapper.java deleted file mode 100644 index 0bf743a784..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/ConfigurationWrapper.java +++ /dev/null @@ -1,74 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.redis.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; - -import org.apache.commons.lang3.StringUtils; - -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -import lombok.extern.slf4j.Slf4j; - -@Slf4j -public class ConfigurationWrapper { - - private static final String CONF_FILE = "redis-client.properties"; - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public static String getProperty(String key) { - return StringUtils.isEmpty(key) - ? null : properties.getProperty(key, null); - } - - public static Properties getPropertiesByPrefix(String prefix) { - if (StringUtils.isBlank(prefix)) { - return null; - } - - return PropertiesUtils.getPropertiesByPrefix(properties, prefix); - } - - private static void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + CONF_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - log.error(String.format("Load %s file from classpath error", CONF_FILE), e); - throw new RuntimeException(String.format("Load %s file from classpath error", CONF_FILE)); - } - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONF_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } catch (IOException e) { - log.error(String.format("Cannot load %s file from conf", CONF_FILE), e); - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/RedisProperties.java b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/RedisProperties.java index 7e5d7b20ed..42906b1b08 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/RedisProperties.java +++ b/eventmesh-connector-plugin/eventmesh-connector-redis/src/main/java/org/apache/eventmesh/connector/redis/config/RedisProperties.java @@ -17,75 +17,48 @@ package org.apache.eventmesh.connector.redis.config; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; + import java.util.Properties; +import lombok.Data; + +@Data +@Config(prefix = "eventMesh.server.redis", path = "classPath://redis-client.properties") public class RedisProperties { /** * The redis server configuration to be used. */ + @ConfigFiled(field = "serverType") private ServerType serverType = ServerType.SINGLE; /** * The master server name used by Redis Sentinel servers and master change monitoring task. */ + @ConfigFiled(field = "serverMasterName") private String serverMasterName = "master"; /** * The address of the redis server following format -- host1:port1,host2:port2,…… */ + @ConfigFiled(field = "serverAddress") private String serverAddress; /** * The password for redis authentication. */ + @ConfigFiled(field = "serverPassword") private String serverPassword; /** - * The redisson options, redisson properties, please refer to the redisson manual. + * The redisson options, redisson properties + * prefix is `eventMesh.server.redis.redisson` */ + @ConfigFiled(field = "redisson") private Properties redissonProperties; - public ServerType getServerType() { - return serverType; - } - - public void setServerType(ServerType serverType) { - this.serverType = serverType; - } - - public String getServerAddress() { - return serverAddress; - } - - public void setServerAddress(String serverAddress) { - this.serverAddress = serverAddress; - } - - public String getServerPassword() { - return serverPassword; - } - - public void setServerPassword(String serverPassword) { - this.serverPassword = serverPassword; - } - - public String getServerMasterName() { - return serverMasterName; - } - - public void setServerMasterName(String serverMasterName) { - this.serverMasterName = serverMasterName; - } - - public Properties getRedissonProperties() { - return redissonProperties; - } - - public void setRedissonProperties(Properties redissonProperties) { - this.redissonProperties = redissonProperties; - } - public enum ServerType { SINGLE, CLUSTER, diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/java/org/apache/eventmesh/connector/redis/config/RedisPropertiesTest.java b/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/java/org/apache/eventmesh/connector/redis/config/RedisPropertiesTest.java new file mode 100644 index 0000000000..bded565b68 --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/java/org/apache/eventmesh/connector/redis/config/RedisPropertiesTest.java @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.redis.config; + +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.Properties; + +import org.junit.Assert; +import org.junit.Test; + +public class RedisPropertiesTest { + + @Test + public void getRedisProperties() { + ConfigService configService = ConfigService.getInstance(); + RedisProperties config = configService.buildConfigInstance(RedisProperties.class); + assertConfig(config); + } + + private void assertConfig(RedisProperties config) { + Assert.assertEquals(config.getServerAddress(), "redis://127.0.0.1:6379"); + Assert.assertEquals(config.getServerType(), RedisProperties.ServerType.SINGLE); + Assert.assertEquals(config.getServerMasterName(), "serverMasterName-success!!!"); + + Properties properties = new Properties(); + properties.put("threads", "816"); + properties.put("nettyThreads", "1816"); + Properties redissonProperties = config.getRedissonProperties(); + Assert.assertEquals(redissonProperties, properties); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/resources/redis-client.properties b/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/resources/redis-client.properties index 9151c027ee..08814f488b 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/resources/redis-client.properties +++ b/eventmesh-connector-plugin/eventmesh-connector-redis/src/test/resources/redis-client.properties @@ -14,5 +14,8 @@ # See the License for the specific language governing permissions and # limitations under the License. # - -eventMesh.server.redis.serverAddress=redis://127.0.0.1:6379 \ No newline at end of file +eventMesh.server.redis.serverAddress=redis://127.0.0.1:6379 +eventMesh.server.redis.serverType=SINGLE +eventMesh.server.redis.serverMasterName=serverMasterName-success!!! +eventMesh.server.redis.redisson.threads=816 +eventMesh.server.redis.redisson.nettyThreads=1816 \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/RocketMQAdmin.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/RocketMQAdmin.java index 85857638ad..4e5e918f0f 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/RocketMQAdmin.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/RocketMQAdmin.java @@ -21,6 +21,7 @@ import org.apache.eventmesh.api.admin.Admin; import org.apache.eventmesh.api.admin.TopicProperties; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.connector.rocketmq.config.ClientConfiguration; import org.apache.commons.lang3.StringUtils; @@ -60,8 +61,8 @@ public class RocketMQAdmin implements Admin { public RocketMQAdmin(Properties properties) { isStarted = new AtomicBoolean(false); - final ClientConfiguration clientConfiguration = new ClientConfiguration(); - clientConfiguration.init(); + ConfigService configService = ConfigService.getInstance(); + ClientConfiguration clientConfiguration = configService.buildConfigInstance(ClientConfiguration.class); nameServerAddr = clientConfiguration.namesrvAddr; clusterName = clientConfiguration.clusterName; diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/command/Command.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/command/Command.java new file mode 100644 index 0000000000..1ccf5ef60f --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/admin/command/Command.java @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.rocketmq.admin.command; + +import org.apache.eventmesh.common.config.ConfigService; +import org.apache.eventmesh.connector.rocketmq.config.ClientConfiguration; + +import org.apache.rocketmq.acl.common.AclClientRPCHook; +import org.apache.rocketmq.acl.common.SessionCredentials; +import org.apache.rocketmq.remoting.RPCHook; +import org.apache.rocketmq.tools.admin.DefaultMQAdminExt; + +import java.util.UUID; + +import lombok.Data; + +@Data +public abstract class Command { + protected DefaultMQAdminExt adminExt; + + protected String nameServerAddr; + protected String clusterName; + + public void init() { + ConfigService configService = ConfigService.getInstance(); + ClientConfiguration clientConfiguration = configService.buildConfigInstance(ClientConfiguration.class); + + nameServerAddr = clientConfiguration.namesrvAddr; + clusterName = clientConfiguration.clusterName; + String accessKey = clientConfiguration.accessKey; + String secretKey = clientConfiguration.secretKey; + + RPCHook rpcHook = new AclClientRPCHook(new SessionCredentials(accessKey, secretKey)); + adminExt = new DefaultMQAdminExt(rpcHook); + String groupId = UUID.randomUUID().toString(); + adminExt.setAdminExtGroup("admin_ext_group-" + groupId); + adminExt.setNamesrvAddr(nameServerAddr); + } + + public abstract void execute() throws Exception; +} diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfiguration.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfiguration.java index 8fb2d3f9c0..547e3a6fef 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfiguration.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfiguration.java @@ -17,185 +17,57 @@ package org.apache.eventmesh.connector.rocketmq.config; -import org.apache.commons.lang3.StringUtils; - -import com.google.common.base.Preconditions; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; +@Config(prefix = "eventMesh.server.rocketmq", path = "classPath://rocketmq-client.properties") public class ClientConfiguration { + @ConfigFiled(field = "namesrvAddr", notEmpty = true) public String namesrvAddr = ""; + + @ConfigFiled(field = "username") public String clientUserName = "username"; + + @ConfigFiled(field = "password") public String clientPass = "password"; + + @ConfigFiled(field = "client.consumeThreadMin") public Integer consumeThreadMin = 2; + + @ConfigFiled(field = "client.consumeThreadMax") public Integer consumeThreadMax = 2; + + @ConfigFiled(field = "client.consumeThreadPoolQueueSize") public Integer consumeQueueSize = 10000; + + @ConfigFiled(field = "client.pullBatchSize") public Integer pullBatchSize = 32; + + @ConfigFiled(field = "client.ackwindow") public Integer ackWindow = 1000; + + @ConfigFiled(field = "client.pubwindow") public Integer pubWindow = 100; + + @ConfigFiled(field = "client.comsumeTimeoutInMin") public long consumeTimeout = 0L; + + @ConfigFiled(field = "client.pollNameServerInterval") public Integer pollNameServerInterval = 10 * 1000; + + @ConfigFiled(field = "client.heartbeatBrokerInterval") public Integer heartbeatBrokerInterval = 30 * 1000; + + @ConfigFiled(field = "client.rebalanceInterval") public Integer rebalanceInterval = 20 * 1000; - public String clusterName = ""; - public String accessKey = ""; - public String secretKey = ""; - public void init() { - - String clientUserNameStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_USERNAME); - if (StringUtils.isNotBlank(clientUserNameStr)) { - clientUserName = StringUtils.trim(clientUserNameStr); - } - - String clientPassStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_PASSWORD); - if (StringUtils.isNotBlank(clientPassStr)) { - clientPass = StringUtils.trim(clientPassStr); - } - - String namesrvAddrStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_NAMESRV_ADDR); - Preconditions.checkState(StringUtils.isNotEmpty(namesrvAddrStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_NAMESRV_ADDR)); - namesrvAddr = StringUtils.trim(namesrvAddrStr); - - String consumeThreadPoolMinStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MIN); - if (StringUtils.isNotEmpty(consumeThreadPoolMinStr)) { - Preconditions.checkState(StringUtils.isNumeric(consumeThreadPoolMinStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MIN)); - consumeThreadMin = Integer.valueOf(consumeThreadPoolMinStr); - } - - String consumeThreadPoolMaxStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MAX); - if (StringUtils.isNotEmpty(consumeThreadPoolMaxStr)) { - Preconditions.checkState(StringUtils.isNumeric(consumeThreadPoolMaxStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MAX)); - consumeThreadMax = Integer.valueOf(consumeThreadPoolMaxStr); - } - - String consumerThreadPoolQueueSizeStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_QUEUESIZE); - if (StringUtils.isNotEmpty(consumerThreadPoolQueueSizeStr)) { - Preconditions.checkState(StringUtils.isNumeric(consumerThreadPoolQueueSizeStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_QUEUESIZE)); - consumeQueueSize = Integer.valueOf(consumerThreadPoolQueueSizeStr); - } - - String clientAckWindowStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_ACK_WINDOW); - if (StringUtils.isNotEmpty(clientAckWindowStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientAckWindowStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_ACK_WINDOW)); - ackWindow = Integer.valueOf(clientAckWindowStr); - } - - String clientPubWindowStr = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_PUB_WINDOW); - if (StringUtils.isNotEmpty(clientPubWindowStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientPubWindowStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_PUB_WINDOW)); - pubWindow = Integer.valueOf(clientPubWindowStr); - } - - String consumeTimeoutStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_CONSUME_TIMEOUT); - if (StringUtils.isNotBlank(consumeTimeoutStr)) { - Preconditions.checkState(StringUtils.isNumeric(consumeTimeoutStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_CONSUME_TIMEOUT)); - consumeTimeout = Long.parseLong(consumeTimeoutStr); - } - - String clientPullBatchSizeStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_PULL_BATCHSIZE); - if (StringUtils.isNotEmpty(clientPullBatchSizeStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientPullBatchSizeStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_PULL_BATCHSIZE)); - pullBatchSize = Integer.valueOf(clientPullBatchSizeStr); - } - - String clientPollNamesrvIntervalStr = - ConfigurationWrapper.getProp( - ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_POLL_NAMESRV_INTERVAL); - if (StringUtils.isNotEmpty(clientPollNamesrvIntervalStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientPollNamesrvIntervalStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_POLL_NAMESRV_INTERVAL)); - pollNameServerInterval = Integer.valueOf(clientPollNamesrvIntervalStr); - } - - String clientHeartbeatBrokerIntervalStr = - ConfigurationWrapper.getProp( - ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_HEARTBEAT_BROKER_INTERVAL); - if (StringUtils.isNotEmpty(clientHeartbeatBrokerIntervalStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientHeartbeatBrokerIntervalStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_HEARTBEAT_BROKER_INTERVAL)); - heartbeatBrokerInterval = Integer.valueOf(clientHeartbeatBrokerIntervalStr); - } - - String clientRebalanceIntervalIntervalStr = - ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_REBALANCE_INTERVAL); - if (StringUtils.isNotEmpty(clientRebalanceIntervalIntervalStr)) { - Preconditions.checkState(StringUtils.isNumeric(clientRebalanceIntervalIntervalStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLIENT_REBALANCE_INTERVAL)); - rebalanceInterval = Integer.valueOf(clientRebalanceIntervalIntervalStr); - } - - String cluster = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_CLUSTER); - if (StringUtils.isNotBlank(cluster)) { - clusterName = cluster; - } - - String ak = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_ACCESS_KEY); - if (StringUtils.isNotBlank(ak)) { - accessKey = ak; - } - - String sk = ConfigurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ROCKETMQ_SECRET_KEY); - if (StringUtils.isNotBlank(sk)) { - secretKey = sk; - } - } - - static class ConfKeys { - - public static final String KEYS_EVENTMESH_ROCKETMQ_NAMESRV_ADDR = "eventMesh.server.rocketmq.namesrvAddr"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_USERNAME = "eventMesh.server.rocketmq.username"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_PASSWORD = "eventMesh.server.rocketmq.password"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MIN = - "eventMesh.server.rocketmq.client.consumeThreadMin"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_MAX = - "eventMesh.server.rocketmq.client.consumeThreadMax"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CONSUME_THREADPOOL_QUEUESIZE = - "eventMesh.server.rocketmq.client.consumeThreadPoolQueueSize"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_ACK_WINDOW = "eventMesh.server.rocketmq.client.ackwindow"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_PUB_WINDOW = "eventMesh.server.rocketmq.client.pubwindow"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_CONSUME_TIMEOUT = - "eventMesh.server.rocketmq.client.comsumeTimeoutInMin"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_PULL_BATCHSIZE = - "eventMesh.server.rocketmq.client.pullBatchSize"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_POLL_NAMESRV_INTERVAL = - "eventMesh.server.rocketmq.client.pollNameServerInterval"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_HEARTBEAT_BROKER_INTERVAL = - "eventMesh.server.rocketmq.client.heartbeatBrokerInterval"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLIENT_REBALANCE_INTERVAL = - "eventMesh.server.rocketmq.client.rebalanceInterval"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_CLUSTER = "eventMesh.server.rocketmq.cluster"; - - public static final String KEYS_EVENTMESH_ROCKETMQ_ACCESS_KEY = - "eventMesh.server.rocketmq.accessKey"; + @ConfigFiled(field = "cluster") + public String clusterName = ""; - public static final String KEYS_EVENTMESH_ROCKETMQ_SECRET_KEY = - "eventMesh.server.rocketmq.secretKey"; + @ConfigFiled(field = "accessKey") + public String accessKey = ""; - } + @ConfigFiled(field = "secretKey") + public String secretKey = ""; } \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapper.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapper.java deleted file mode 100644 index af5faceb35..0000000000 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/config/ConfigurationWrapper.java +++ /dev/null @@ -1,68 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.connector.rocketmq.config; - -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; -import org.apache.eventmesh.connector.rocketmq.common.EventMeshConstants; - -import org.apache.commons.lang3.StringUtils; - -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass -public class ConfigurationWrapper { - - private static final Properties properties = new Properties(); - - static { - loadProperties(); - } - - public String getProp(String key) { - return StringUtils.isEmpty(key) ? null : properties.getProperty(key, null); - } - - /** - * Load rocketmq properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - try (InputStream resourceAsStream = ConfigurationWrapper.class.getResourceAsStream( - "/" + EventMeshConstants.EVENTMESH_CONF_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - throw new RuntimeException(String.format("Load %s.properties file from classpath error", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + EventMeshConstants.EVENTMESH_CONF_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } catch (IOException e) { - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", EventMeshConstants.EVENTMESH_CONF_FILE)); - } - } -} diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/consumer/RocketMQConsumerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/consumer/RocketMQConsumerImpl.java index a2630652ad..2456949249 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/consumer/RocketMQConsumerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/consumer/RocketMQConsumerImpl.java @@ -21,6 +21,7 @@ import org.apache.eventmesh.api.EventListener; import org.apache.eventmesh.api.consumer.Consumer; import org.apache.eventmesh.common.Constants; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.rocketmq.config.ClientConfiguration; import org.apache.rocketmq.common.protocol.heartbeat.MessageModel; @@ -36,34 +37,35 @@ import lombok.extern.slf4j.Slf4j; @Slf4j +@Config(field = "clientConfiguration") public class RocketMQConsumerImpl implements Consumer { public Logger messageLogger = LoggerFactory.getLogger("message"); private PushConsumerImpl pushConsumer; + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties keyValue) throws Exception { - final ClientConfiguration clientConfiguration = new ClientConfiguration(); - clientConfiguration.init(); - boolean isBroadcast = Boolean.parseBoolean(keyValue.getProperty(Constants.IS_BROADCAST)); + boolean isBroadcast = Boolean.parseBoolean(keyValue.getProperty("isBroadcast")); - String consumerGroup = keyValue.getProperty(Constants.CONSUMER_GROUP); + String consumerGroup = keyValue.getProperty("consumerGroup"); if (isBroadcast) { consumerGroup = Constants.BROADCAST_PREFIX + consumerGroup; } String namesrvAddr = clientConfiguration.namesrvAddr; - String instanceName = keyValue.getProperty(Constants.INSTANCE_NAME); + String instanceName = keyValue.getProperty("instanceName"); Properties properties = new Properties(); - properties.put(Constants.ACCESS_POINTS, namesrvAddr); - properties.put(Constants.REGION, Constants.NAMESPACE); - properties.put(Constants.INSTANCE_NAME, instanceName); - properties.put(Constants.CONSUMER_ID, consumerGroup); + properties.put("ACCESS_POINTS", namesrvAddr); + properties.put("REGION", "namespace"); + properties.put("instanceName", instanceName); + properties.put("CONSUMER_ID", consumerGroup); if (isBroadcast) { - properties.put(Constants.MESSAGE_MODEL, MessageModel.BROADCASTING.name()); + properties.put("MESSAGE_MODEL", MessageModel.BROADCASTING.name()); } else { - properties.put(Constants.MESSAGE_MODEL, MessageModel.CLUSTERING.name()); + properties.put("MESSAGE_MODEL", MessageModel.CLUSTERING.name()); } pushConsumer = new PushConsumerImpl(properties); @@ -113,4 +115,7 @@ public Properties attributes() { return pushConsumer.attributes(); } + public ClientConfiguration getClientConfiguration() { + return clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/producer/RocketMQProducerImpl.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/producer/RocketMQProducerImpl.java index 1dbca5926f..ca3761ad1a 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/producer/RocketMQProducerImpl.java +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/main/java/org/apache/eventmesh/connector/rocketmq/producer/RocketMQProducerImpl.java @@ -21,6 +21,7 @@ import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.producer.Producer; import org.apache.eventmesh.common.Constants; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.connector.rocketmq.common.EventMeshConstants; import org.apache.eventmesh.connector.rocketmq.config.ClientConfiguration; @@ -36,14 +37,15 @@ @Slf4j @SuppressWarnings("deprecation") +@Config(field = "clientConfiguration") public class RocketMQProducerImpl implements Producer { private ProducerImpl producer; + private ClientConfiguration clientConfiguration; + @Override public synchronized void init(Properties keyValue) { - final ClientConfiguration clientConfiguration = new ClientConfiguration(); - clientConfiguration.init(); String producerGroup = keyValue.getProperty(Constants.PRODUCER_GROUP); String omsNamesrv = clientConfiguration.namesrvAddr; @@ -114,4 +116,8 @@ public void setExtFields() { public void sendOneway(CloudEvent message) { producer.sendOneway(message); } + + public ClientConfiguration getClientConfiguration() { + return clientConfiguration; + } } diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfigurationTest.java b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfigurationTest.java new file mode 100644 index 0000000000..e2a4eba287 --- /dev/null +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/java/org/apache/eventmesh/connector/rocketmq/config/ClientConfigurationTest.java @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.connector.rocketmq.config; + +import org.apache.eventmesh.api.factory.ConnectorPluginFactory; +import org.apache.eventmesh.connector.rocketmq.consumer.RocketMQConsumerImpl; +import org.apache.eventmesh.connector.rocketmq.producer.RocketMQProducerImpl; + +import org.junit.Assert; +import org.junit.Test; + +public class ClientConfigurationTest { + + @Test + public void getConfigWhenRocketMQConsumerInit() { + RocketMQConsumerImpl consumer = + (RocketMQConsumerImpl) ConnectorPluginFactory.getMeshMQPushConsumer("rocketmq"); + + ClientConfiguration config = consumer.getClientConfiguration(); + assertConfig(config); + } + + @Test + public void getConfigWhenRocketMQProducerInit() { + RocketMQProducerImpl producer = + (RocketMQProducerImpl) ConnectorPluginFactory.getMeshMQProducer("rocketmq"); + + ClientConfiguration config = producer.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(ClientConfiguration config) { + Assert.assertEquals(config.namesrvAddr, "127.0.0.1:9876;127.0.0.1:9876"); + Assert.assertEquals(config.clientUserName, "username-succeed!!!"); + Assert.assertEquals(config.clientPass, "password-succeed!!!"); + Assert.assertEquals(config.consumeThreadMin, Integer.valueOf(1816)); + Assert.assertEquals(config.consumeThreadMax, Integer.valueOf(2816)); + Assert.assertEquals(config.consumeQueueSize, Integer.valueOf(3816)); + Assert.assertEquals(config.pullBatchSize, Integer.valueOf(4816)); + Assert.assertEquals(config.ackWindow, Integer.valueOf(5816)); + Assert.assertEquals(config.pubWindow, Integer.valueOf(6816)); + Assert.assertEquals(config.consumeTimeout, 7816); + Assert.assertEquals(config.pollNameServerInterval, Integer.valueOf(8816)); + Assert.assertEquals(config.heartbeatBrokerInterval, Integer.valueOf(9816)); + Assert.assertEquals(config.rebalanceInterval, Integer.valueOf(11816)); + Assert.assertEquals(config.clusterName, "cluster-succeed!!!"); + Assert.assertEquals(config.accessKey, "accessKey-succeed!!!"); + Assert.assertEquals(config.secretKey, "secretKey-succeed!!!"); + } +} \ No newline at end of file diff --git a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/resources/rocketmq-client.properties b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/resources/rocketmq-client.properties index 1261f30e2c..e9e78992d0 100644 --- a/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/resources/rocketmq-client.properties +++ b/eventmesh-connector-plugin/eventmesh-connector-rocketmq/src/test/resources/rocketmq-client.properties @@ -16,3 +16,18 @@ # #######################rocketmq-client################## eventMesh.server.rocketmq.namesrvAddr=127.0.0.1:9876;127.0.0.1:9876 +eventMesh.server.rocketmq.username=username-succeed!!! +eventMesh.server.rocketmq.password=password-succeed!!! +eventMesh.server.rocketmq.client.consumeThreadMin=1816 +eventMesh.server.rocketmq.client.consumeThreadMax=2816 +eventMesh.server.rocketmq.client.consumeThreadPoolQueueSize=3816 +eventMesh.server.rocketmq.client.pullBatchSize=4816 +eventMesh.server.rocketmq.client.ackwindow=5816 +eventMesh.server.rocketmq.client.pubwindow=6816 +eventMesh.server.rocketmq.client.comsumeTimeoutInMin=7816 +eventMesh.server.rocketmq.client.pollNameServerInterval=8816 +eventMesh.server.rocketmq.client.heartbeatBrokerInterval=9816 +eventMesh.server.rocketmq.client.rebalanceInterval=11816 +eventMesh.server.rocketmq.cluster=cluster-succeed!!! +eventMesh.server.rocketmq.accessKey=accessKey-succeed!!! +eventMesh.server.rocketmq.secretKey=secretKey-succeed!!! \ No newline at end of file diff --git a/eventmesh-examples/src/main/resources/application.properties b/eventmesh-examples/src/main/resources/application.properties index 259e49e897..962106f5cd 100644 --- a/eventmesh-examples/src/main/resources/application.properties +++ b/eventmesh-examples/src/main/resources/application.properties @@ -18,7 +18,7 @@ server.port=8088 server.name=orderapp eventmesh.ip=127.0.0.1 eventmesh.http.port=10105 -eventmesh.tcp.port=10000 +eventmesh.tcp.port=10002 eventmesh.grpc.port=10205 eventmesh.selector.type=nacos eventmesh.selector.nacos.address=127.0.0.1:8848 diff --git a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/build.gradle b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/build.gradle index 2257341b38..237a1f8dee 100644 --- a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/build.gradle +++ b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/build.gradle @@ -22,7 +22,7 @@ dependencies { implementation 'org.apache.commons:commons-lang3' implementation 'com.google.guava:guava' - // todo:Can we remove some dependency? + // todo: Can we remove some dependency? implementation 'io.opentelemetry:opentelemetry-api' implementation 'io.opentelemetry:opentelemetry-sdk' implementation 'io.opentelemetry:opentelemetry-sdk-metrics' diff --git a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/PrometheusMetricsRegistry.java b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/PrometheusMetricsRegistry.java index 60e5d587be..99e79923c0 100644 --- a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/PrometheusMetricsRegistry.java +++ b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/PrometheusMetricsRegistry.java @@ -17,6 +17,7 @@ package org.apache.eventmesh.metrics.prometheus; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.metrics.api.MetricsRegistry; import org.apache.eventmesh.metrics.api.model.GrpcSummaryMetrics; import org.apache.eventmesh.metrics.api.model.HttpSummaryMetrics; @@ -36,10 +37,16 @@ import lombok.extern.slf4j.Slf4j; @Slf4j +@Config(field = "prometheusConfiguration") public class PrometheusMetricsRegistry implements MetricsRegistry { private volatile HTTPServer prometheusHttpServer; + /** + * Unified configuration class corresponding to prometheus.properties + */ + private PrometheusConfiguration prometheusConfiguration; + @Override public void start() { if (prometheusHttpServer == null) { @@ -48,7 +55,7 @@ public void start() { SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder().buildAndRegisterGlobal(); PrometheusCollector .builder().setMetricProducer(sdkMeterProvider).buildAndRegister(); - int port = PrometheusConfiguration.getEventMeshPrometheusPort(); + int port = prometheusConfiguration.getEventMeshPrometheusPort(); try { //Use the daemon thread to start an HTTP server to serve the default Prometheus registry. prometheusHttpServer = new HTTPServer(port, true); @@ -90,4 +97,8 @@ public void register(Metric metric) { public void unRegister(Metric metric) { // todo: need to split the current metrics } + + public PrometheusConfiguration getClientConfiguration() { + return this.prometheusConfiguration; + } } diff --git a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfiguration.java b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfiguration.java index eb76580dd4..63e5688302 100644 --- a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfiguration.java +++ b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/main/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfiguration.java @@ -17,62 +17,15 @@ package org.apache.eventmesh.metrics.prometheus.config; -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; -import org.apache.commons.lang3.StringUtils; +import lombok.Data; -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass +@Data +@Config(prefix = "eventMesh.metrics.prometheus", path = "classPath://prometheus.properties") public class PrometheusConfiguration { - private static final String CONFIG_FILE = "prometheus.properties"; - private static final Properties properties = new Properties(); - + @ConfigFiled(field = "port") private int eventMeshPrometheusPort = 19090; - - static { - loadProperties(); - initializeConfig(); - } - - public static int getEventMeshPrometheusPort() { - return eventMeshPrometheusPort; - } - - private void initializeConfig() { - String eventMeshPrometheusPortStr = properties.getProperty("eventMesh.metrics.prometheus.port"); - if (StringUtils.isNotEmpty(eventMeshPrometheusPortStr)) { - eventMeshPrometheusPort = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshPrometheusPortStr)); - } - } - - /** - * Load properties file from classpath and conf home. - * The properties defined in conf home will override classpath. - */ - private void loadProperties() { - try (InputStream resourceAsStream = PrometheusConfiguration.class.getResourceAsStream(File.separator + CONFIG_FILE)) { - if (resourceAsStream != null) { - properties.load(resourceAsStream); - } - } catch (IOException e) { - throw new RuntimeException(String.format("Load %s file from classpath error", CONFIG_FILE)); - } - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONFIG_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } catch (IOException e) { - throw new IllegalArgumentException(String.format("Cannot load %s file from conf", CONFIG_FILE)); - } - } - } diff --git a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfigurationTest.java b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfigurationTest.java index 3afac9f257..0e076422d4 100644 --- a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfigurationTest.java +++ b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/java/org/apache/eventmesh/metrics/prometheus/config/PrometheusConfigurationTest.java @@ -17,14 +17,24 @@ package org.apache.eventmesh.metrics.prometheus.config; +import org.apache.eventmesh.metrics.api.MetricsPluginFactory; +import org.apache.eventmesh.metrics.prometheus.PrometheusMetricsRegistry; + import org.junit.Assert; import org.junit.Test; public class PrometheusConfigurationTest { @Test - public void getEventMeshPrometheusPort() { - int eventMeshPrometheusPort = PrometheusConfiguration.getEventMeshPrometheusPort(); - Assert.assertEquals(19090, eventMeshPrometheusPort); + public void getConfigWhenPrometheusMetricsRegistryInit() { + PrometheusMetricsRegistry registry = + (PrometheusMetricsRegistry) MetricsPluginFactory.getMetricsRegistry("prometheus"); + + PrometheusConfiguration config = registry.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(PrometheusConfiguration config) { + Assert.assertEquals(config.getEventMeshPrometheusPort(), 19091); } } diff --git a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/resources/prometheus.properties b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/resources/prometheus.properties index d4fb5a13c3..dd5ec0b83a 100644 --- a/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/resources/prometheus.properties +++ b/eventmesh-metrics-plugin/eventmesh-metrics-prometheus/src/test/resources/prometheus.properties @@ -15,4 +15,4 @@ # limitations under the License. # -eventMesh.metrics.prometheus.port=19090 \ No newline at end of file +eventMesh.metrics.prometheus.port=19091 \ No newline at end of file diff --git a/eventmesh-registry-plugin/eventmesh-registry-consul/src/test/java/ConsulRegistryServiceTest.java b/eventmesh-registry-plugin/eventmesh-registry-consul/src/test/java/ConsulRegistryServiceTest.java index e4c5f3307d..5d8eab1631 100644 --- a/eventmesh-registry-plugin/eventmesh-registry-consul/src/test/java/ConsulRegistryServiceTest.java +++ b/eventmesh-registry-plugin/eventmesh-registry-consul/src/test/java/ConsulRegistryServiceTest.java @@ -48,7 +48,7 @@ public class ConsulRegistryServiceTest { @Before public void registryTest() { consulRegistryService = new ConsulRegistryService(); - CommonConfiguration configuration = new CommonConfiguration(null); + CommonConfiguration configuration = new CommonConfiguration(); ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.HTTP, configuration); configuration.setNamesrvAddr("127.0.0.1:8500"); Mockito.when(eventMeshRegisterInfo.getEventMeshClusterName()).thenReturn("eventmesh"); diff --git a/eventmesh-registry-plugin/eventmesh-registry-etcd/src/test/java/org/apache/eventmesh/registry/etcd/service/EtcdRegistryServiceTest.java b/eventmesh-registry-plugin/eventmesh-registry-etcd/src/test/java/org/apache/eventmesh/registry/etcd/service/EtcdRegistryServiceTest.java index fb080b71bd..e7406e0a9a 100644 --- a/eventmesh-registry-plugin/eventmesh-registry-etcd/src/test/java/org/apache/eventmesh/registry/etcd/service/EtcdRegistryServiceTest.java +++ b/eventmesh-registry-plugin/eventmesh-registry-etcd/src/test/java/org/apache/eventmesh/registry/etcd/service/EtcdRegistryServiceTest.java @@ -48,7 +48,7 @@ public class EtcdRegistryServiceTest { @Before public void setUp() { etcdRegistryService = new EtcdRegistryService(); - CommonConfiguration configuration = new CommonConfiguration(null); + CommonConfiguration configuration = new CommonConfiguration(); configuration.setNamesrvAddr("127.0.0.1:2379"); ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.HTTP, configuration); diff --git a/eventmesh-registry-plugin/eventmesh-registry-nacos/src/main/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryService.java b/eventmesh-registry-plugin/eventmesh-registry-nacos/src/main/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryService.java index cc94d61619..428400f94c 100644 --- a/eventmesh-registry-plugin/eventmesh-registry-nacos/src/main/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryService.java +++ b/eventmesh-registry-plugin/eventmesh-registry-nacos/src/main/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryService.java @@ -79,6 +79,7 @@ public void init() throws RegistryException { if (StringUtils.isBlank(commonConfiguration.getNamesrvAddr())) { throw new RegistryException("namesrvAddr cannot be null"); } + this.serverAddr = commonConfiguration.getNamesrvAddr(); this.username = commonConfiguration.getEventMeshRegistryPluginUsername(); this.password = commonConfiguration.getEventMeshRegistryPluginPassword(); diff --git a/eventmesh-registry-plugin/eventmesh-registry-nacos/src/test/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryServiceTest.java b/eventmesh-registry-plugin/eventmesh-registry-nacos/src/test/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryServiceTest.java index e1a5fb3f68..d2e361a943 100644 --- a/eventmesh-registry-plugin/eventmesh-registry-nacos/src/test/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryServiceTest.java +++ b/eventmesh-registry-plugin/eventmesh-registry-nacos/src/test/java/org/apache/eventmesh/registry/nacos/service/NacosRegistryServiceTest.java @@ -47,7 +47,7 @@ public class NacosRegistryServiceTest { @Before public void setUp() { nacosRegistryService = new NacosRegistryService(); - CommonConfiguration configuration = new CommonConfiguration(null); + CommonConfiguration configuration = new CommonConfiguration(); configuration.setNamesrvAddr("127.0.0.1"); configuration.setEventMeshRegistryPluginPassword("nacos"); configuration.setEventMeshRegistryPluginUsername("nacos"); diff --git a/eventmesh-registry-plugin/eventmesh-registry-zookeeper/src/test/java/org/apache/eventmesh/registry/zookeeper/service/ZookeeperRegistryServiceTest.java b/eventmesh-registry-plugin/eventmesh-registry-zookeeper/src/test/java/org/apache/eventmesh/registry/zookeeper/service/ZookeeperRegistryServiceTest.java index a1337033ee..99777f8c7b 100644 --- a/eventmesh-registry-plugin/eventmesh-registry-zookeeper/src/test/java/org/apache/eventmesh/registry/zookeeper/service/ZookeeperRegistryServiceTest.java +++ b/eventmesh-registry-plugin/eventmesh-registry-zookeeper/src/test/java/org/apache/eventmesh/registry/zookeeper/service/ZookeeperRegistryServiceTest.java @@ -59,7 +59,7 @@ public void setUp() throws Exception { testingServer.start(); zkRegistryService = new ZookeeperRegistryService(); - CommonConfiguration configuration = new CommonConfiguration(null); + CommonConfiguration configuration = new CommonConfiguration(); configuration.setNamesrvAddr("127.0.0.1:1500"); configuration.setEventMeshName("eventmesh"); ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.HTTP, configuration); diff --git a/eventmesh-runtime/conf/eventmesh.properties b/eventmesh-runtime/conf/eventmesh.properties index 7a0cb514aa..17bfab0676 100644 --- a/eventmesh-runtime/conf/eventmesh.properties +++ b/eventmesh-runtime/conf/eventmesh.properties @@ -25,7 +25,7 @@ eventMesh.server.http.port=10105 eventMesh.server.grpc.port=10205 ########################## eventMesh tcp configuration ############################ eventMesh.server.tcp.enabled=true -eventMesh.server.tcp.port=10000 +eventMesh.server.tcp.port=10002 eventMesh.server.tcp.readerIdleSeconds=120 eventMesh.server.tcp.writerIdleSeconds=120 eventMesh.server.tcp.allIdleSeconds=120 diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/admin/handler/ConfigurationHandler.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/admin/handler/ConfigurationHandler.java index a2b29f2ad0..ee93ca5dd2 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/admin/handler/ConfigurationHandler.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/admin/handler/ConfigurationHandler.java @@ -96,11 +96,11 @@ void get(HttpExchange httpExchange) throws IOException { // TCP Configuration eventMeshTCPConfiguration.eventMeshTcpServerPort, // HTTP Configuration - eventMeshHTTPConfiguration.httpServerPort, - eventMeshHTTPConfiguration.eventMeshServerUseTls, + eventMeshHTTPConfiguration.getHttpServerPort(), + eventMeshHTTPConfiguration.isEventMeshServerUseTls(), // gRPC Configuration - eventMeshGrpcConfiguration.grpcServerPort, - eventMeshGrpcConfiguration.eventMeshServerUseTls + eventMeshGrpcConfiguration.getGrpcServerPort(), + eventMeshGrpcConfiguration.isEventMeshServerUseTls() ); String result = JsonUtils.toJson(getConfigurationResponse); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcBootstrap.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcBootstrap.java index 6db1759b60..51a6ed5ee3 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcBootstrap.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcBootstrap.java @@ -17,7 +17,7 @@ package org.apache.eventmesh.runtime.boot; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.common.utils.ConfigurationContextUtil; import org.apache.eventmesh.runtime.configuration.EventMeshGrpcConfiguration; import org.apache.eventmesh.runtime.registry.Registry; @@ -30,10 +30,12 @@ public class EventMeshGrpcBootstrap implements EventMeshBootstrap { private final Registry registry; - public EventMeshGrpcBootstrap(ConfigurationWrapper configurationWrapper, Registry registry) { + public EventMeshGrpcBootstrap(Registry registry) { this.registry = registry; - this.eventMeshGrpcConfiguration = new EventMeshGrpcConfiguration(configurationWrapper); - eventMeshGrpcConfiguration.init(); + + ConfigService configService = ConfigService.getInstance(); + this.eventMeshGrpcConfiguration = configService.buildConfigInstance(EventMeshGrpcConfiguration.class); + ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.GRPC, eventMeshGrpcConfiguration); } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcServer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcServer.java index 66f99fd7b5..089e66b17d 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcServer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshGrpcServer.java @@ -103,7 +103,7 @@ public void init() throws Exception { initHttpClientPool(); - msgRateLimiter = RateLimiter.create(eventMeshGrpcConfiguration.eventMeshMsgReqNumPerSecond); + msgRateLimiter = RateLimiter.create(eventMeshGrpcConfiguration.getEventMeshMsgReqNumPerSecond()); producerManager = new ProducerManager(this); producerManager.init(); @@ -114,7 +114,7 @@ public void init() throws Exception { grpcRetryer = new GrpcRetryer(this); grpcRetryer.init(); - int serverPort = eventMeshGrpcConfiguration.grpcServerPort; + int serverPort = eventMeshGrpcConfiguration.getGrpcServerPort(); server = ServerBuilder.forPort(serverPort) .addService(new ProducerService(this, sendMsgExecutor)) @@ -168,7 +168,7 @@ public boolean register() { boolean registerResult = false; try { String endPoints = IPUtils.getLocalAddress() - + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshGrpcConfiguration.grpcServerPort; + + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshGrpcConfiguration.getGrpcServerPort(); EventMeshRegisterInfo eventMeshRegisterInfo = new EventMeshRegisterInfo(); eventMeshRegisterInfo.setEventMeshClusterName(eventMeshGrpcConfiguration.getEventMeshCluster()); eventMeshRegisterInfo.setEventMeshName(eventMeshGrpcConfiguration.getEventMeshName() + "-" @@ -185,7 +185,7 @@ public boolean register() { private void unRegister() throws Exception { String endPoints = IPUtils.getLocalAddress() - + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshGrpcConfiguration.grpcServerPort; + + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshGrpcConfiguration.getGrpcServerPort(); EventMeshUnRegisterInfo eventMeshUnRegisterInfo = new EventMeshUnRegisterInfo(); eventMeshUnRegisterInfo.setEventMeshClusterName(eventMeshGrpcConfiguration.getEventMeshCluster()); eventMeshUnRegisterInfo.setEventMeshName(eventMeshGrpcConfiguration.getEventMeshName()); @@ -240,29 +240,33 @@ public EventMeshGrpcMonitor getMetricsMonitor() { private void initThreadPool() { BlockingQueue sendMsgThreadPoolQueue = - new LinkedBlockingQueue(eventMeshGrpcConfiguration.eventMeshServerSendMsgBlockQueueSize); + new LinkedBlockingQueue(eventMeshGrpcConfiguration.getEventMeshServerSendMsgBlockQueueSize()); - sendMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor(eventMeshGrpcConfiguration.eventMeshServerSendMsgThreadNum, - eventMeshGrpcConfiguration.eventMeshServerSendMsgThreadNum, sendMsgThreadPoolQueue, - "eventMesh-grpc-sendMsg-%d", true); + sendMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshGrpcConfiguration.getEventMeshServerSendMsgThreadNum(), + eventMeshGrpcConfiguration.getEventMeshServerSendMsgThreadNum(), sendMsgThreadPoolQueue, + "eventMesh-grpc-sendMsg-%d", true); BlockingQueue subscribeMsgThreadPoolQueue = - new LinkedBlockingQueue(eventMeshGrpcConfiguration.eventMeshServerSubscribeMsgBlockQueueSize); + new LinkedBlockingQueue(eventMeshGrpcConfiguration.getEventMeshServerSubscribeMsgBlockQueueSize()); - clientMgmtExecutor = ThreadPoolFactory.createThreadPoolExecutor(eventMeshGrpcConfiguration.eventMeshServerSubscribeMsgThreadNum, - eventMeshGrpcConfiguration.eventMeshServerSubscribeMsgThreadNum, subscribeMsgThreadPoolQueue, - "eventMesh-grpc-clientMgmt-%d", true); + clientMgmtExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshGrpcConfiguration.getEventMeshServerSubscribeMsgThreadNum(), + eventMeshGrpcConfiguration.getEventMeshServerSubscribeMsgThreadNum(), subscribeMsgThreadPoolQueue, + "eventMesh-grpc-clientMgmt-%d", true); BlockingQueue pushMsgThreadPoolQueue = - new LinkedBlockingQueue(eventMeshGrpcConfiguration.eventMeshServerPushMsgBlockQueueSize); + new LinkedBlockingQueue(eventMeshGrpcConfiguration.getEventMeshServerPushMsgBlockQueueSize()); - pushMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor(eventMeshGrpcConfiguration.eventMeshServerPushMsgThreadNum, - eventMeshGrpcConfiguration.eventMeshServerPushMsgThreadNum, pushMsgThreadPoolQueue, - "eventMesh-grpc-pushMsg-%d", true); + pushMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshGrpcConfiguration.getEventMeshServerPushMsgThreadNum(), + eventMeshGrpcConfiguration.getEventMeshServerPushMsgThreadNum(), pushMsgThreadPoolQueue, + "eventMesh-grpc-pushMsg-%d", true); - replyMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor(eventMeshGrpcConfiguration.eventMeshServerReplyMsgThreadNum, - eventMeshGrpcConfiguration.eventMeshServerReplyMsgThreadNum, sendMsgThreadPoolQueue, - "eventMesh-grpc-replyMsg-%d", true); + replyMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshGrpcConfiguration.getEventMeshServerReplyMsgThreadNum(), + eventMeshGrpcConfiguration.getEventMeshServerReplyMsgThreadNum(), sendMsgThreadPoolQueue, + "eventMesh-grpc-replyMsg-%d", true); } private void initHttpClientPool() { diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHTTPServer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHTTPServer.java index ec375a182a..860b57f265 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHTTPServer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHTTPServer.java @@ -116,7 +116,8 @@ public class EventMeshHTTPServer extends AbstractHTTPServer { public EventMeshHTTPServer(final EventMeshServer eventMeshServer, final EventMeshHTTPConfiguration eventMeshHttpConfiguration) { - super(eventMeshHttpConfiguration.httpServerPort, eventMeshHttpConfiguration.eventMeshServerUseTls, eventMeshHttpConfiguration); + super(eventMeshHttpConfiguration.getHttpServerPort(), + eventMeshHttpConfiguration.isEventMeshServerUseTls(), eventMeshHttpConfiguration); this.eventMeshServer = eventMeshServer; this.eventMeshHttpConfiguration = eventMeshHttpConfiguration; this.registry = eventMeshServer.getRegistry(); @@ -140,47 +141,47 @@ public void shutdownThreadPool() { private void initThreadPool() { - batchMsgExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerBatchMsgThreadNum, - eventMeshHttpConfiguration.eventMeshServerBatchMsgThreadNum, - new LinkedBlockingQueue(eventMeshHttpConfiguration.eventMeshServerBatchBlockQSize), - "eventMesh-batchMsg-", true); - - sendMsgExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerSendMsgThreadNum, - eventMeshHttpConfiguration.eventMeshServerSendMsgThreadNum, - new LinkedBlockingQueue(eventMeshHttpConfiguration.eventMeshServerSendMsgBlockQSize), - "eventMesh-sendMsg-", true); - - remoteMsgExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerRemoteMsgThreadNum, - eventMeshHttpConfiguration.eventMeshServerRemoteMsgThreadNum, - new LinkedBlockingQueue(eventMeshHttpConfiguration.eventMeshServerRemoteMsgBlockQSize), - "eventMesh-remoteMsg-", true); - - pushMsgExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerPushMsgThreadNum, - eventMeshHttpConfiguration.eventMeshServerPushMsgThreadNum, - new LinkedBlockingQueue(eventMeshHttpConfiguration.eventMeshServerPushMsgBlockQSize), - "eventMesh-pushMsg-", true); - - clientManageExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerClientManageThreadNum, - eventMeshHttpConfiguration.eventMeshServerClientManageThreadNum, - new LinkedBlockingQueue(eventMeshHttpConfiguration.eventMeshServerClientManageBlockQSize), - "eventMesh-clientManage-", true); - - adminExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerAdminThreadNum, - eventMeshHttpConfiguration.eventMeshServerAdminThreadNum, - new LinkedBlockingQueue(50), "eventMesh-admin-", - true); - - replyMsgExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerReplyMsgThreadNum, - eventMeshHttpConfiguration.eventMeshServerReplyMsgThreadNum, - new LinkedBlockingQueue(100), - "eventMesh-replyMsg-", true); + batchMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerBatchMsgThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerBatchMsgThreadNum(), + new LinkedBlockingQueue(eventMeshHttpConfiguration.getEventMeshServerBatchBlockQSize()), + "eventMesh-batchMsg-", true); + + sendMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerSendMsgThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerSendMsgThreadNum(), + new LinkedBlockingQueue(eventMeshHttpConfiguration.getEventMeshServerSendMsgBlockQSize()), + "eventMesh-sendMsg-", true); + + remoteMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerRemoteMsgThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerRemoteMsgThreadNum(), + new LinkedBlockingQueue(eventMeshHttpConfiguration.getEventMeshServerRemoteMsgBlockQSize()), + "eventMesh-remoteMsg-", true); + + pushMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerPushMsgThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerPushMsgThreadNum(), + new LinkedBlockingQueue(eventMeshHttpConfiguration.getEventMeshServerPushMsgBlockQSize()), + "eventMesh-pushMsg-", true); + + clientManageExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerClientManageThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerClientManageThreadNum(), + new LinkedBlockingQueue(eventMeshHttpConfiguration.getEventMeshServerClientManageBlockQSize()), + "eventMesh-clientManage-", true); + + adminExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerAdminThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerAdminThreadNum(), + new LinkedBlockingQueue(50), "eventMesh-admin-", + true); + + replyMsgExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerReplyMsgThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerReplyMsgThreadNum(), + new LinkedBlockingQueue(100), + "eventMesh-replyMsg-", true); } public ThreadPoolExecutor getBatchMsgExecutor() { @@ -227,8 +228,8 @@ private void init() throws Exception { initThreadPool(); - msgRateLimiter = RateLimiter.create(eventMeshHttpConfiguration.eventMeshHttpMsgReqNumPerSecond); - batchRateLimiter = RateLimiter.create(eventMeshHttpConfiguration.eventMeshBatchMsgRequestNumPerSecond); + msgRateLimiter = RateLimiter.create(eventMeshHttpConfiguration.getEventMeshHttpMsgReqNumPerSecond()); + batchRateLimiter = RateLimiter.create(eventMeshHttpConfiguration.getEventMeshBatchMsgRequestNumPerSecond()); // The MetricsRegistry is singleton, so we can use factory method to get. final List metricsRegistries = Lists.newArrayList(); @@ -315,10 +316,11 @@ public boolean register() { boolean registerResult = false; try { final String endPoints = IPUtils.getLocalAddress() - + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshHttpConfiguration.httpServerPort; + + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshHttpConfiguration.getHttpServerPort(); final EventMeshRegisterInfo eventMeshRegisterInfo = new EventMeshRegisterInfo(); eventMeshRegisterInfo.setEventMeshClusterName(eventMeshHttpConfiguration.getEventMeshCluster()); - eventMeshRegisterInfo.setEventMeshName(eventMeshHttpConfiguration.getEventMeshName() + "-" + ConfigurationContextUtil.HTTP); + eventMeshRegisterInfo.setEventMeshName(eventMeshHttpConfiguration.getEventMeshName() + + "-" + ConfigurationContextUtil.HTTP); eventMeshRegisterInfo.setEndPoint(endPoints); eventMeshRegisterInfo.setProtocolType(ConfigurationContextUtil.HTTP); registerResult = registry.register(eventMeshRegisterInfo); @@ -331,7 +333,7 @@ public boolean register() { private void unRegister() throws Exception { final String endPoints = IPUtils.getLocalAddress() - + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshHttpConfiguration.httpServerPort; + + EventMeshConstants.IP_PORT_SEPARATOR + eventMeshHttpConfiguration.getHttpServerPort(); final EventMeshUnRegisterInfo eventMeshUnRegisterInfo = new EventMeshUnRegisterInfo(); eventMeshUnRegisterInfo.setEventMeshClusterName(eventMeshHttpConfiguration.getEventMeshCluster()); eventMeshUnRegisterInfo.setEventMeshName(eventMeshHttpConfiguration.getEventMeshName()); @@ -394,14 +396,13 @@ public void registerHTTPRequestProcessor() { private void initWebhook() throws Exception { - webhookExecutor = - ThreadPoolFactory.createThreadPoolExecutor(eventMeshHttpConfiguration.eventMeshServerWebhookThreadNum, - eventMeshHttpConfiguration.eventMeshServerWebhookThreadNum, new LinkedBlockingQueue(100), - "eventMesh-webhook-", true); + webhookExecutor = ThreadPoolFactory.createThreadPoolExecutor( + eventMeshHttpConfiguration.getEventMeshServerWebhookThreadNum(), + eventMeshHttpConfiguration.getEventMeshServerWebhookThreadNum(), + new LinkedBlockingQueue(100), "eventMesh-webhook-", true); final WebHookProcessor webHookProcessor = new WebHookProcessor(); final WebHookController webHookController = new WebHookController(); - webHookController.setConfigurationWrapper(eventMeshHttpConfiguration.getConfigurationWrapper()); webHookController.init(); webHookProcessor.setWebHookController(webHookController); this.getHandlerService().register(webHookProcessor, webhookExecutor); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHttpBootstrap.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHttpBootstrap.java index 94a87cb098..d478400b79 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHttpBootstrap.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshHttpBootstrap.java @@ -17,7 +17,7 @@ package org.apache.eventmesh.runtime.boot; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.common.utils.ConfigurationContextUtil; import org.apache.eventmesh.runtime.configuration.EventMeshHTTPConfiguration; import org.apache.eventmesh.runtime.registry.Registry; @@ -32,15 +32,14 @@ public class EventMeshHttpBootstrap implements EventMeshBootstrap { private final Registry registry; - public EventMeshHttpBootstrap(EventMeshServer eventMeshServer, - ConfigurationWrapper configurationWrapper, - Registry registry) { + public EventMeshHttpBootstrap(EventMeshServer eventMeshServer, Registry registry) { this.eventMeshServer = eventMeshServer; this.registry = registry; - this.eventMeshHttpConfiguration = new EventMeshHTTPConfiguration(configurationWrapper); - eventMeshHttpConfiguration.init(); - ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.HTTP, eventMeshHttpConfiguration); + ConfigService configService = ConfigService.getInstance(); + this.eventMeshHttpConfiguration = configService.buildConfigInstance(EventMeshHTTPConfiguration.class); + + ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.HTTP, eventMeshHttpConfiguration); } @Override diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshServer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshServer.java index 1167d9df69..a6595a70e8 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshServer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshServer.java @@ -18,7 +18,7 @@ package org.apache.eventmesh.runtime.boot; import org.apache.eventmesh.common.config.CommonConfiguration; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.common.utils.ConfigurationContextUtil; import org.apache.eventmesh.runtime.acl.Acl; import org.apache.eventmesh.runtime.admin.controller.ClientManageController; @@ -57,28 +57,26 @@ public class EventMeshServer { private static final String SERVER_STATE_MSG = "server state:{}"; - public EventMeshServer(final ConfigurationWrapper configurationWrapper) throws Exception { - CommonConfiguration configuration = new CommonConfiguration(configurationWrapper); - configuration.init(); - this.configuration = configuration; + public EventMeshServer() throws Exception { + ConfigService configService = ConfigService.getInstance(); + this.configuration = configService.buildConfigInstance(CommonConfiguration.class); + this.acl = new Acl(); this.registry = new Registry(); trace = new Trace(configuration.isEventMeshServerTraceEnable()); this.connectorResource = new ConnectorResource(); + trace = new Trace(configuration.isEventMeshServerTraceEnable()); final List provideServerProtocols = configuration.getEventMeshProvideServerProtocols(); for (final String provideServerProtocol : provideServerProtocols) { if (ConfigurationContextUtil.HTTP.equals(provideServerProtocol)) { - BOOTSTRAP_LIST.add(new EventMeshHttpBootstrap(this, - configurationWrapper, registry)); + BOOTSTRAP_LIST.add(new EventMeshHttpBootstrap(this, registry)); } if (ConfigurationContextUtil.TCP.equals(provideServerProtocol)) { - BOOTSTRAP_LIST.add(new EventMeshTcpBootstrap(this, - configurationWrapper, registry)); + BOOTSTRAP_LIST.add(new EventMeshTcpBootstrap(this, registry)); } if (ConfigurationContextUtil.GRPC.equals(provideServerProtocol)) { - BOOTSTRAP_LIST.add(new EventMeshGrpcBootstrap(configurationWrapper, - registry)); + BOOTSTRAP_LIST.add(new EventMeshGrpcBootstrap(registry)); } } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshStartup.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshStartup.java index 33622a5852..986726c924 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshStartup.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshStartup.java @@ -17,9 +17,11 @@ package org.apache.eventmesh.runtime.boot; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.runtime.constants.EventMeshConstants; +import java.io.File; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -29,9 +31,11 @@ public class EventMeshStartup { public static void main(String[] args) throws Exception { try { - final EventMeshServer server = new EventMeshServer( - new ConfigurationWrapper(EventMeshConstants.EVENTMESH_CONF_HOME, - EventMeshConstants.EVENTMESH_CONF_FILE, false)); + ConfigService.getInstance() + .setConfigPath(EventMeshConstants.EVENTMESH_CONF_HOME + File.separator) + .setRootConfig(EventMeshConstants.EVENTMESH_CONF_FILE); + + EventMeshServer server = new EventMeshServer(); server.start(); Runtime.getRuntime().addShutdownHook(new Thread(() -> { try { diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTCPServer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTCPServer.java index c91c1346f0..cc990db043 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTCPServer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTCPServer.java @@ -207,7 +207,6 @@ public void init() throws Exception { adminWebHookConfigOperationManage = new AdminWebHookConfigOperationManage(); - adminWebHookConfigOperationManage.setConfigurationWrapper(eventMeshTCPConfiguration.getConfigurationWrapper()); adminWebHookConfigOperationManage.init(); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTcpBootstrap.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTcpBootstrap.java index fce57708cc..eecbd1cf21 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTcpBootstrap.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshTcpBootstrap.java @@ -17,7 +17,7 @@ package org.apache.eventmesh.runtime.boot; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.common.utils.ConfigurationContextUtil; import org.apache.eventmesh.runtime.configuration.EventMeshTCPConfiguration; import org.apache.eventmesh.runtime.registry.Registry; @@ -32,15 +32,14 @@ public class EventMeshTcpBootstrap implements EventMeshBootstrap { private final Registry registry; - public EventMeshTcpBootstrap(EventMeshServer eventMeshServer, - ConfigurationWrapper configurationWrapper, - Registry registry) { + public EventMeshTcpBootstrap(EventMeshServer eventMeshServer, Registry registry) { this.eventMeshServer = eventMeshServer; this.registry = registry; - this.eventMeshTcpConfiguration = new EventMeshTCPConfiguration(configurationWrapper); - eventMeshTcpConfiguration.init(); - ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.TCP, eventMeshTcpConfiguration); + ConfigService configService = ConfigService.getInstance(); + this.eventMeshTcpConfiguration = configService.buildConfigInstance(EventMeshTCPConfiguration.class); + + ConfigurationContextUtil.putIfAbsent(ConfigurationContextUtil.TCP, eventMeshTcpConfiguration); } @Override diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/SSLContextFactory.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/SSLContextFactory.java index 3132c094ba..f11bb3aa38 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/SSLContextFactory.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/SSLContextFactory.java @@ -58,9 +58,9 @@ public static SSLContext getSslContext(final EventMeshHTTPConfiguration eventMes try (InputStream inputStream = Files.newInputStream(Paths.get(EventMeshConstants.EVENTMESH_CONF_HOME + File.separator + fileName), StandardOpenOption.READ)) { - protocol = eventMeshHttpConfiguration.eventMeshServerSSLProtocol; - fileName = eventMeshHttpConfiguration.eventMeshServerSSLCer; - password = eventMeshHttpConfiguration.eventMeshServerSSLPass; + protocol = eventMeshHttpConfiguration.getEventMeshServerSSLProtocol(); + fileName = eventMeshHttpConfiguration.getEventMeshServerSSLCer(); + password = eventMeshHttpConfiguration.getEventMeshServerSSLPass(); char[] filePass = StringUtils.isNotBlank(password) ? password.toCharArray() : new char[0]; final KeyStore keyStore = KeyStore.getInstance("JKS"); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfiguration.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfiguration.java index 65142f6326..20916e41ed 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfiguration.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfiguration.java @@ -18,240 +18,93 @@ package org.apache.eventmesh.runtime.configuration; import org.apache.eventmesh.common.config.CommonConfiguration; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; import org.apache.eventmesh.common.utils.IPUtils; -import org.apache.commons.lang3.StringUtils; - -import com.google.common.base.Preconditions; +import lombok.Data; +import lombok.NoArgsConstructor; +@Data +@NoArgsConstructor +@Config(prefix = "eventMesh.server") public class EventMeshGrpcConfiguration extends CommonConfiguration { - public int grpcServerPort = 10205; - - public int eventMeshSessionExpiredInMills = 60000; - - public boolean eventMeshServerBatchMsgBatchEnabled = Boolean.TRUE; - - public int eventMeshServerBatchMsgThreadNum = 10; - - public int eventMeshServerSendMsgThreadNum = 8; - - public int eventMeshServerPushMsgThreadNum = 8; - - public int eventMeshServerReplyMsgThreadNum = 8; - - public int eventMeshServerSubscribeMsgThreadNum = 4; - - public int eventMeshServerRegistryThreadNum = 10; - - public int eventMeshServerAdminThreadNum = 2; - - public int eventMeshServerRetryThreadNum = 2; - - public int eventMeshServerPullRegistryInterval = 30000; - - public int eventMeshServerAsyncAccumulationThreshold = 1000; - - public int eventMeshServerRetryBlockQueueSize = 10000; - - public int eventMeshServerBatchBlockQueueSize = 1000; - - public int eventMeshServerSendMsgBlockQueueSize = 1000; - - public int eventMeshServerPushMsgBlockQueueSize = 1000; - - public int eventMeshServerSubscribeMsgBlockQueueSize = 1000; - - public int eventMeshServerBusyCheckInterval = 1000; - - public boolean eventMeshServerConsumerEnabled = false; - - public boolean eventMeshServerUseTls = false; - - public int eventMeshBatchMsgRequestNumPerSecond = 20000; - - public int eventMeshMsgReqNumPerSecond = 15000; - - public String eventMeshIp = IPUtils.getLocalAddress(); - - public EventMeshGrpcConfiguration(ConfigurationWrapper configurationWrapper) { - super(configurationWrapper); - } - - @Override - public void init() { - super.init(); - - if (configurationWrapper != null) { - String httpServerPortStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SERVER_GRPC_PORT); - Preconditions.checkState(StringUtils.isNotEmpty(httpServerPortStr) && StringUtils.isNumeric(httpServerPortStr), - String.format("%s error", ConfKeys.KEYS_EVENTMESH_SERVER_GRPC_PORT)); - grpcServerPort = Integer.parseInt(StringUtils.deleteWhitespace(httpServerPortStr)); - - String eventMeshServerBatchMsgThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerBatchMsgThreadNumStr) && StringUtils.isNumeric(eventMeshServerBatchMsgThreadNumStr)) { - eventMeshServerBatchMsgThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBatchMsgThreadNumStr)); - } - - String eventMeshTcpSessionExpiredInMillsStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SERVER_SESSION_EXPIRED_TIME); - if (StringUtils.isNotEmpty(eventMeshTcpSessionExpiredInMillsStr) && StringUtils.isNumeric(eventMeshTcpSessionExpiredInMillsStr)) { - eventMeshSessionExpiredInMills = Integer.parseInt(eventMeshTcpSessionExpiredInMillsStr); - } - - String eventMeshServerBatchMsgReqNumPerSecondStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_REQ_NUM_PER_SECOND); - if (StringUtils.isNotEmpty(eventMeshServerBatchMsgReqNumPerSecondStr) - && StringUtils.isNumeric(eventMeshServerBatchMsgReqNumPerSecondStr)) { - eventMeshBatchMsgRequestNumPerSecond = Integer.parseInt(eventMeshServerBatchMsgReqNumPerSecondStr); - } - - String eventMeshServerBatchMsgBatchEnableStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_BATCH_ENABLED); - if (StringUtils.isNotBlank(eventMeshServerBatchMsgBatchEnableStr)) { - eventMeshServerBatchMsgBatchEnabled = Boolean.parseBoolean(eventMeshServerBatchMsgBatchEnableStr); - } - - String eventMeshServerAsyncAccumulationThresholdStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ASYNC_ACCUMULATION_THRESHOLD); - if (StringUtils.isNotEmpty(eventMeshServerAsyncAccumulationThresholdStr) - && StringUtils.isNumeric(eventMeshServerAsyncAccumulationThresholdStr)) { - eventMeshServerAsyncAccumulationThreshold = Integer.parseInt( - StringUtils.deleteWhitespace(eventMeshServerAsyncAccumulationThresholdStr)); - } - - String eventMeshServerSendMsgThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SENDMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerSendMsgThreadNumStr) && StringUtils.isNumeric(eventMeshServerSendMsgThreadNumStr)) { - eventMeshServerSendMsgThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerSendMsgThreadNumStr)); - } - - String eventMeshServerReplyMsgThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REPLYMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerReplyMsgThreadNumStr) && StringUtils.isNumeric(eventMeshServerReplyMsgThreadNumStr)) { - eventMeshServerReplyMsgThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerReplyMsgThreadNumStr)); - } - - String eventMeshServerPushMsgThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PUSHMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerPushMsgThreadNumStr) && StringUtils.isNumeric(eventMeshServerPushMsgThreadNumStr)) { - eventMeshServerPushMsgThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPushMsgThreadNumStr)); - } - - String eventMeshServerRegistryThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REGISTRY_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerRegistryThreadNumStr) && StringUtils.isNumeric(eventMeshServerRegistryThreadNumStr)) { - eventMeshServerRegistryThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRegistryThreadNumStr)); - } - - String eventMeshServerClientManageThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_CLIENTMANAGE_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerClientManageThreadNumStr) && StringUtils.isNumeric(eventMeshServerClientManageThreadNumStr)) { - eventMeshServerSubscribeMsgThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerClientManageThreadNumStr)); - } + @ConfigFiled(field = "grpc.port", notNull = true, beNumber = true) + private int grpcServerPort = 10205; - String eventMeshServerPullRegistryIntervalStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PULL_REGISTRY_INTERVAL); - if (StringUtils.isNotEmpty(eventMeshServerPullRegistryIntervalStr) && StringUtils.isNumeric(eventMeshServerPullRegistryIntervalStr)) { - eventMeshServerPullRegistryInterval = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPullRegistryIntervalStr)); - } - - String eventMeshServerAdminThreadNumStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ADMIN_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerAdminThreadNumStr) && StringUtils.isNumeric(eventMeshServerAdminThreadNumStr)) { - eventMeshServerAdminThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerAdminThreadNumStr)); - } - - String eventMeshServerRetryBlockQueueSizeStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_RETRY_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerRetryBlockQueueSizeStr) && StringUtils.isNumeric(eventMeshServerRetryBlockQueueSizeStr)) { - eventMeshServerRetryBlockQueueSize = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRetryBlockQueueSizeStr)); - } - - String eventMeshServerBatchBlockQueueSizeStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_BATCHMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerBatchBlockQueueSizeStr) && StringUtils.isNumeric(eventMeshServerBatchBlockQueueSizeStr)) { - eventMeshServerBatchBlockQueueSize = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBatchBlockQueueSizeStr)); - } - - String eventMeshServerSendMsgBlockQueueSizeStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SENDMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerSendMsgBlockQueueSizeStr) && StringUtils.isNumeric(eventMeshServerSendMsgBlockQueueSizeStr)) { - eventMeshServerSendMsgBlockQueueSize = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerSendMsgBlockQueueSizeStr)); - } - - String eventMeshServerPushMsgBlockQueueSizeStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_PUSHMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerPushMsgBlockQueueSizeStr) && StringUtils.isNumeric(eventMeshServerPushMsgBlockQueueSizeStr)) { - eventMeshServerPushMsgBlockQueueSize = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPushMsgBlockQueueSizeStr)); - } - - String eventMeshServerClientManageBlockQueueSizeStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_CLIENTM_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerClientManageBlockQueueSizeStr) - && StringUtils.isNumeric(eventMeshServerClientManageBlockQueueSizeStr)) { - eventMeshServerSubscribeMsgBlockQueueSize = Integer.parseInt( - StringUtils.deleteWhitespace(eventMeshServerClientManageBlockQueueSizeStr)); - } - - String eventMeshServerBusyCheckIntervalStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_BUSY_CHECK_INTERVAL); - if (StringUtils.isNotEmpty(eventMeshServerBusyCheckIntervalStr) && StringUtils.isNumeric(eventMeshServerBusyCheckIntervalStr)) { - eventMeshServerBusyCheckInterval = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBusyCheckIntervalStr)); - } - - String eventMeshServerConsumerEnabledStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_CONSUMER_ENABLED); - if (StringUtils.isNotEmpty(eventMeshServerConsumerEnabledStr)) { - eventMeshServerConsumerEnabled = Boolean.parseBoolean(StringUtils.deleteWhitespace(eventMeshServerConsumerEnabledStr)); - } - - String eventMeshServerRetryThreadNumStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_RETRY_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerRetryThreadNumStr) && StringUtils.isNumeric(eventMeshServerRetryThreadNumStr)) { - eventMeshServerRetryThreadNum = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRetryThreadNumStr)); - } - - String eventMeshServerUseTlsStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_TLS_ENABLED); - if (StringUtils.isNotEmpty(eventMeshServerUseTlsStr)) { - eventMeshServerUseTls = Boolean.parseBoolean(StringUtils.deleteWhitespace(eventMeshServerUseTlsStr)); - } - - String eventMeshMsgReqNumPerSecondStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECOND); - if (StringUtils.isNotEmpty(eventMeshMsgReqNumPerSecondStr) && StringUtils.isNumeric(eventMeshMsgReqNumPerSecondStr)) { - eventMeshMsgReqNumPerSecond = Integer.parseInt(eventMeshMsgReqNumPerSecondStr); - } - } - } + @ConfigFiled(field = "session.expiredInMills") + private int eventMeshSessionExpiredInMills = 60000; - static class ConfKeys { + @ConfigFiled(field = "batchmsg.batch.enabled") + private boolean eventMeshServerBatchMsgBatchEnabled = Boolean.TRUE; - public static final String KEYS_EVENTMESH_SERVER_GRPC_PORT = "eventMesh.server.grpc.port"; + @ConfigFiled(field = "batchmsg.threads.num") + private int eventMeshServerBatchMsgThreadNum = 10; - public static final String KEYS_EVENTMESH_SERVER_SESSION_EXPIRED_TIME = "eventMesh.server.session.expiredInMills"; + @ConfigFiled(field = "sendmsg.threads.num") + private int eventMeshServerSendMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_BATCHMSG_THREAD_NUM = "eventMesh.server.batchmsg.threads.num"; + @ConfigFiled(field = "pushmsg.threads.num") + private int eventMeshServerPushMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_BATCHMSG_REQ_NUM_PER_SECOND = "eventMesh.server.batchmsg.reqNumPerSecond"; + @ConfigFiled(field = "replymsg.threads.num") + private int eventMeshServerReplyMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_BATCHMSG_BATCH_ENABLED = "eventMesh.server.batchmsg.batch.enabled"; + @ConfigFiled(field = "clientmanage.threads.num") + private int eventMeshServerSubscribeMsgThreadNum = 4; - public static final String KEYS_EVENTMESH_ASYNC_ACCUMULATION_THRESHOLD = "eventMesh.server.async.accumulation.threshold"; + @ConfigFiled(field = "registry.threads.num") + private int eventMeshServerRegistryThreadNum = 10; - public static final String KEY_EVENTMESH_BUSY_CHECK_INTERVAL = "eventMesh.server.busy.check.interval"; + @ConfigFiled(field = "admin.threads.num") + private int eventMeshServerAdminThreadNum = 2; - public static final String KEYS_EVENTMESH_SENDMSG_THREAD_NUM = "eventMesh.server.sendmsg.threads.num"; + @ConfigFiled(field = "retry.threads.num") + private int eventMeshServerRetryThreadNum = 2; - public static final String KEYS_EVENTMESH_REPLYMSG_THREAD_NUM = "eventMesh.server.replymsg.threads.num"; + @ConfigFiled(field = "pull.registry.interval") + private int eventMeshServerPullRegistryInterval = 30000; - public static final String KEYS_EVENTMESH_PUSHMSG_THREAD_NUM = "eventMesh.server.pushmsg.threads.num"; + @ConfigFiled(field = "async.accumulation.threshold") + private int eventMeshServerAsyncAccumulationThreshold = 1000; - public static final String KEYS_EVENTMESH_REGISTRY_THREAD_NUM = "eventMesh.server.registry.threads.num"; + @ConfigFiled(field = "retry.blockQ.size") + private int eventMeshServerRetryBlockQueueSize = 10000; - public static final String KEYS_EVENTMESH_CLIENTMANAGE_THREAD_NUM = "eventMesh.server.clientmanage.threads.num"; + @ConfigFiled(field = "batchmsg.blockQ.size") + private int eventMeshServerBatchBlockQueueSize = 1000; - public static final String KEYS_EVENTMESH_ADMIN_THREAD_NUM = "eventMesh.server.admin.threads.num"; + @ConfigFiled(field = "sendmsg.blockQ.size") + private int eventMeshServerSendMsgBlockQueueSize = 1000; - public static final String KEY_EVENTMESH_RETRY_THREAD_NUM = "eventMesh.server.retry.threads.num"; + @ConfigFiled(field = "pushmsg.blockQ.size") + private int eventMeshServerPushMsgBlockQueueSize = 1000; - public static final String KEYS_EVENTMESH_PULL_REGISTRY_INTERVAL = "eventMesh.server.pull.registry.interval"; + @ConfigFiled(field = "clientM.blockQ.size") + private int eventMeshServerSubscribeMsgBlockQueueSize = 1000; - public static final String KEY_EVENTMESH_RETRY_BLOCKQ_SIZE = "eventMesh.server.retry.blockQ.size"; + @ConfigFiled(field = "busy.check.interval") + private int eventMeshServerBusyCheckInterval = 1000; - public static final String KEY_EVENTMESH_BATCHMSG_BLOCKQ_SIZE = "eventMesh.server.batchmsg.blockQ.size"; + @ConfigFiled(field = "consumer.enabled") + private boolean eventMeshServerConsumerEnabled = false; - public static final String KEY_EVENTMESH_SENDMSG_BLOCKQ_SIZE = "eventMesh.server.sendmsg.blockQ.size"; + @ConfigFiled(field = "useTls.enabled") + private boolean eventMeshServerUseTls = false; - public static final String KEY_EVENTMESH_PUSHMSG_BLOCKQ_SIZE = "eventMesh.server.pushmsg.blockQ.size"; + @ConfigFiled(field = "batchmsg.reqNumPerSecond") + private int eventMeshBatchMsgRequestNumPerSecond = 20000; - public static final String KEY_EVENTMESH_CLIENTM_BLOCKQ_SIZE = "eventMesh.server.clientM.blockQ.size"; + @ConfigFiled(field = "http.msgReqnumPerSecond") + private int eventMeshMsgReqNumPerSecond = 15000; - public static final String KEY_EVENTMESH_CONSUMER_ENABLED = "eventMesh.server.consumer.enabled"; + @ConfigFiled(field = "", reload = true) + private String eventMeshIp; - public static final String KEY_EVENTMESH_TLS_ENABLED = "eventMesh.server.useTls.enabled"; + public void reload() { + super.reload(); - public static final String KEY_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECOND = "eventMesh.server.http.msgReqnumPerSecond"; + this.eventMeshIp = IPUtils.getLocalAddress(); } } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfiguration.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfiguration.java index f0415a7faa..226827e9bf 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfiguration.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfiguration.java @@ -18,389 +18,115 @@ package org.apache.eventmesh.runtime.configuration; import org.apache.eventmesh.common.config.CommonConfiguration; -import org.apache.eventmesh.common.config.ConfigurationWrapper; - -import org.apache.commons.lang3.StringUtils; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; import java.util.Collections; import java.util.List; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import com.google.common.base.Preconditions; -import com.google.common.base.Splitter; -import com.google.common.collect.Lists; +import lombok.Data; +import lombok.NoArgsConstructor; import inet.ipaddr.IPAddress; -import inet.ipaddr.IPAddressString; +@Data +@NoArgsConstructor +@Config(prefix = "eventMesh.server") public class EventMeshHTTPConfiguration extends CommonConfiguration { - public static final Logger logger = LoggerFactory.getLogger(EventMeshHTTPConfiguration.class); - - public int httpServerPort = 10105; - - public boolean eventMeshServerBatchMsgBatchEnabled = Boolean.TRUE; - - public int eventMeshServerBatchMsgThreadNum = 10; - - public int eventMeshServerSendMsgThreadNum = 8; - - public int eventMeshServerRemoteMsgThreadNum = 8; - - public int eventMeshServerPushMsgThreadNum = 8; - - public int eventMeshServerReplyMsgThreadNum = 8; - - public int eventMeshServerClientManageThreadNum = 4; - - public int eventMeshServerRegistryThreadNum = 10; - - public int eventMeshServerAdminThreadNum = 2; - - public int eventMeshServerRetryThreadNum = 2; - - public int eventMeshServerWebhookThreadNum = 4; - - public int eventMeshServerPullRegistryInterval = 30000; - - public int eventMeshServerAsyncAccumulationThreshold = 1000; - - public int eventMeshServerRetryBlockQSize = 10000; - - public int eventMeshServerBatchBlockQSize = 1000; - - public int eventMeshServerSendMsgBlockQSize = 1000; - - public int eventMeshServerRemoteMsgBlockQSize = 1000; - - public int eventMeshServerPushMsgBlockQSize = 1000; - - public int eventMeshServerClientManageBlockQSize = 1000; - - public int eventMeshServerBusyCheckInterval = 1000; - - public boolean eventMeshServerConsumerEnabled = false; - - public boolean eventMeshServerUseTls = false; - - public String eventMeshServerSSLProtocol = "TLSv1.1"; - - public String eventMeshServerSSLCer = "sChat2.jks"; - - public String eventMeshServerSSLPass = "sNetty"; - - public int eventMeshHttpMsgReqNumPerSecond = 15000; - - public int eventMeshBatchMsgRequestNumPerSecond = 20000; - - public int eventMeshEventSize = 1000; - - public int eventMeshEventBatchSize = 10; - - public List eventMeshIpv4BlackList = Collections.emptyList(); - - public List eventMeshIpv6BlackList = Collections.emptyList(); - - public EventMeshHTTPConfiguration(ConfigurationWrapper configurationWrapper) { - super(configurationWrapper); - } - - @Override - public void init() { - super.init(); - - if (configurationWrapper != null) { - String httpServerPortStr = configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SERVER_HTTP_PORT); - Preconditions.checkState(StringUtils.isNotEmpty(httpServerPortStr) - && StringUtils.isNumeric(httpServerPortStr), String.format("%s error", ConfKeys.KEYS_EVENTMESH_SERVER_HTTP_PORT)); - httpServerPort = Integer.parseInt(StringUtils.deleteWhitespace(httpServerPortStr)); - - String eventMeshServerBatchMsgThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerBatchMsgThreadNumStr) - && StringUtils.isNumeric(eventMeshServerBatchMsgThreadNumStr)) { - eventMeshServerBatchMsgThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBatchMsgThreadNumStr)); - } - - String eventMeshServerBatchMsgReqNumPerSecondStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_REQ_NUM_PER_SECOND); - if (StringUtils.isNotEmpty(eventMeshServerBatchMsgReqNumPerSecondStr) - && StringUtils.isNumeric(eventMeshServerBatchMsgReqNumPerSecondStr)) { - eventMeshBatchMsgRequestNumPerSecond = Integer.parseInt(eventMeshServerBatchMsgReqNumPerSecondStr); - } - - String eventMeshServerBatchMsgBatchEnableStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_BATCHMSG_BATCH_ENABLED); - if (StringUtils.isNotBlank(eventMeshServerBatchMsgBatchEnableStr)) { - eventMeshServerBatchMsgBatchEnabled = Boolean.parseBoolean(eventMeshServerBatchMsgBatchEnableStr); - } - - String eventMeshServerAsyncAccumulationThresholdStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ASYNC_ACCUMULATION_THRESHOLD); - if (StringUtils.isNotEmpty(eventMeshServerAsyncAccumulationThresholdStr) - && StringUtils.isNumeric(eventMeshServerAsyncAccumulationThresholdStr)) { - eventMeshServerAsyncAccumulationThreshold = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerAsyncAccumulationThresholdStr)); - } - - String eventMeshServerSendMsgThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_SENDMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerSendMsgThreadNumStr) - && StringUtils.isNumeric(eventMeshServerSendMsgThreadNumStr)) { - eventMeshServerSendMsgThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerSendMsgThreadNumStr)); - } - - String eventMeshServerRemoteMsgThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REMOTEMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerRemoteMsgThreadNumStr) - && StringUtils.isNumeric(eventMeshServerRemoteMsgThreadNumStr)) { - eventMeshServerRemoteMsgThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRemoteMsgThreadNumStr)); - } - - String eventMeshServerReplyMsgThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REPLYMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerReplyMsgThreadNumStr) - && StringUtils.isNumeric(eventMeshServerReplyMsgThreadNumStr)) { - eventMeshServerReplyMsgThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerReplyMsgThreadNumStr)); - } - - String eventMeshServerPushMsgThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PUSHMSG_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerPushMsgThreadNumStr) - && StringUtils.isNumeric(eventMeshServerPushMsgThreadNumStr)) { - eventMeshServerPushMsgThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPushMsgThreadNumStr)); - } - - String eventMeshServerRegistryThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_REGISTRY_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerRegistryThreadNumStr) - && StringUtils.isNumeric(eventMeshServerRegistryThreadNumStr)) { - eventMeshServerRegistryThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRegistryThreadNumStr)); - } - - String eventMeshServerClientManageThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_CLIENTMANAGE_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerClientManageThreadNumStr) - && StringUtils.isNumeric(eventMeshServerClientManageThreadNumStr)) { - eventMeshServerClientManageThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerClientManageThreadNumStr)); - } - - String eventMeshServerPullRegistryIntervalStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_PULL_REGISTRY_INTERVAL); - if (StringUtils.isNotEmpty(eventMeshServerPullRegistryIntervalStr) - && StringUtils.isNumeric(eventMeshServerPullRegistryIntervalStr)) { - eventMeshServerPullRegistryInterval = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPullRegistryIntervalStr)); - } - - String eventMeshServerAdminThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEYS_EVENTMESH_ADMIN_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerAdminThreadNumStr) - && StringUtils.isNumeric(eventMeshServerAdminThreadNumStr)) { - eventMeshServerAdminThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerAdminThreadNumStr)); - } - - String eventMeshServerRetryBlockQSizeStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_RETRY_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerRetryBlockQSizeStr) - && StringUtils.isNumeric(eventMeshServerRetryBlockQSizeStr)) { - eventMeshServerRetryBlockQSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRetryBlockQSizeStr)); - } - - String eventMeshServerBatchBlockQSizeStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_BATCHMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerBatchBlockQSizeStr) - && StringUtils.isNumeric(eventMeshServerBatchBlockQSizeStr)) { - eventMeshServerBatchBlockQSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBatchBlockQSizeStr)); - } - - String eventMeshServerSendMsgBlockQSizeStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SENDMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerSendMsgBlockQSizeStr) - && StringUtils.isNumeric(eventMeshServerSendMsgBlockQSizeStr)) { - eventMeshServerSendMsgBlockQSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerSendMsgBlockQSizeStr)); - } - - String eventMeshServerPushMsgBlockQSizeStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_PUSHMSG_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerPushMsgBlockQSizeStr) - && StringUtils.isNumeric(eventMeshServerPushMsgBlockQSizeStr)) { - eventMeshServerPushMsgBlockQSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerPushMsgBlockQSizeStr)); - } - - String eventMeshServerClientManageBlockQSizeStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_CLIENTM_BLOCKQ_SIZE); - if (StringUtils.isNotEmpty(eventMeshServerClientManageBlockQSizeStr) - && StringUtils.isNumeric(eventMeshServerClientManageBlockQSizeStr)) { - eventMeshServerClientManageBlockQSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerClientManageBlockQSizeStr)); - } - - String eventMeshServerBusyCheckIntervalStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_BUSY_CHECK_INTERVAL); - if (StringUtils.isNotEmpty(eventMeshServerBusyCheckIntervalStr) - && StringUtils.isNumeric(eventMeshServerBusyCheckIntervalStr)) { - eventMeshServerBusyCheckInterval = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerBusyCheckIntervalStr)); - - } - - String eventMeshServerConsumerEnabledStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_CONSUMER_ENABLED); - if (StringUtils.isNotEmpty(eventMeshServerConsumerEnabledStr)) { - eventMeshServerConsumerEnabled = - Boolean.parseBoolean(StringUtils.deleteWhitespace(eventMeshServerConsumerEnabledStr)); - } - - - String eventMeshServerRetryThreadNumStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_RETRY_THREAD_NUM); - if (StringUtils.isNotEmpty(eventMeshServerRetryThreadNumStr) - && StringUtils.isNumeric(eventMeshServerRetryThreadNumStr)) { - eventMeshServerRetryThreadNum = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshServerRetryThreadNumStr)); - - } - - String eventMeshServerUseTlsStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_HTTPS_ENABLED); - if (StringUtils.isNotEmpty(eventMeshServerUseTlsStr)) { - eventMeshServerUseTls = Boolean.parseBoolean(StringUtils.deleteWhitespace(eventMeshServerUseTlsStr)); - } - - String eventMeshServerSslProtocolStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_HTTPS_SSL_PROTOCOL); - if (StringUtils.isNotEmpty(eventMeshServerSslProtocolStr)) { - eventMeshServerSSLProtocol = StringUtils.deleteWhitespace(eventMeshServerSslProtocolStr); - } - - String eventMeshServerSslCerStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_HTTPS_SSL_CER); - if (StringUtils.isNotEmpty(eventMeshServerSslCerStr)) { - eventMeshServerSSLCer = StringUtils.deleteWhitespace(eventMeshServerSslCerStr); - } - - String eventMeshServerSslPassStr = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_HTTPS_SSL_PASS); - if (StringUtils.isNotEmpty(eventMeshServerSslPassStr)) { - eventMeshServerSSLPass = StringUtils.deleteWhitespace(eventMeshServerSslPassStr); - } - - String eventMeshHttpMsgReqNumPerSecondStr = - configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECOND); - if (StringUtils.isNotEmpty(eventMeshHttpMsgReqNumPerSecondStr) - && StringUtils.isNumeric(eventMeshHttpMsgReqNumPerSecondStr)) { - eventMeshHttpMsgReqNumPerSecond = Integer.parseInt(eventMeshHttpMsgReqNumPerSecondStr); - - } - - String eventSize = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_EVENTSIZE); - if (StringUtils.isNotEmpty(eventSize) && StringUtils.isNumeric(eventSize)) { - eventMeshEventSize = Integer.parseInt(eventSize); - } - - String eventBatchSize = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_EVENT_BATCHSIZE); - if (StringUtils.isNotEmpty(eventBatchSize) && StringUtils.isNumeric(eventBatchSize)) { - eventMeshEventBatchSize = Integer.parseInt(eventBatchSize); - } - - String ipv4BlackList = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_IPV4_BLACK_LIST); - if (StringUtils.isNotEmpty(ipv4BlackList)) { - eventMeshIpv4BlackList = getBlacklist(ipv4BlackList); - } - - String ipv6BlackList = configurationWrapper.getProp(ConfKeys.KEY_EVENTMESH_SERVER_IPV6_BLACK_LIST); - if (StringUtils.isNotEmpty(ipv6BlackList)) { - eventMeshIpv6BlackList = getBlacklist(ipv6BlackList); - } - } - } - - private static List getBlacklist(String cidrs) { - List cidrList = Splitter.on(",").omitEmptyStrings() - .trimResults().splitToList(cidrs); - - List ipAddresses = Lists.newArrayList(); - for (String cidr : cidrList) { - try { - ipAddresses.add(new IPAddressString(cidr).toAddress()); - } catch (Exception e) { - logger.warn("Invalid cidr={}", cidr, e); - } - } - return ipAddresses; - } + @ConfigFiled(field = "http.port", notNull = true, beNumber = true) + private int httpServerPort = 10105; - static class ConfKeys { + @ConfigFiled(field = "batchmsg.batch.enabled") + private boolean eventMeshServerBatchMsgBatchEnabled = Boolean.TRUE; - public static final String KEYS_EVENTMESH_SERVER_HTTP_PORT = "eventMesh.server.http.port"; + @ConfigFiled(field = "batchmsg.threads.num") + private int eventMeshServerBatchMsgThreadNum = 10; - public static final String KEYS_EVENTMESH_BATCHMSG_THREAD_NUM = "eventMesh.server.batchmsg.threads.num"; + @ConfigFiled(field = "sendmsg.threads.num") + private int eventMeshServerSendMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_BATCHMSG_REQ_NUM_PER_SECOND = "eventMesh.server.batchmsg.reqNumPerSecond"; + @ConfigFiled(field = "remotemsg.threads.num") + private int eventMeshServerRemoteMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_BATCHMSG_BATCH_ENABLED = "eventMesh.server.batchmsg.batch.enabled"; + @ConfigFiled(field = "pushmsg.threads.num") + private int eventMeshServerPushMsgThreadNum = 8; - public static final String KEYS_EVENTMESH_ASYNC_ACCUMULATION_THRESHOLD = "eventMesh.server.async.accumulation.threshold"; + @ConfigFiled(field = "replymsg.threads.num") + private int eventMeshServerReplyMsgThreadNum = 8; - public static final String KEY_EVENTMESH_BUSY_CHECK_INTERVAL = "eventMesh.server.busy.check.interval"; + @ConfigFiled(field = "clientmanage.threads.num") + private int eventMeshServerClientManageThreadNum = 4; - public static final String KEYS_EVENTMESH_SENDMSG_THREAD_NUM = "eventMesh.server.sendmsg.threads.num"; + @ConfigFiled(field = "registry.threads.num") + private int eventMeshServerRegistryThreadNum = 10; - public static final String KEYS_EVENTMESH_REMOTEMSG_THREAD_NUM = "eventMesh.server.remotemsg.threads.num"; + @ConfigFiled(field = "admin.threads.num") + private int eventMeshServerAdminThreadNum = 2; - public static final String KEYS_EVENTMESH_REPLYMSG_THREAD_NUM = "eventMesh.server.replymsg.threads.num"; + @ConfigFiled(field = "retry.threads.num") + private int eventMeshServerRetryThreadNum = 2; - public static final String KEYS_EVENTMESH_PUSHMSG_THREAD_NUM = "eventMesh.server.pushmsg.threads.num"; + @ConfigFiled(field = "") + private int eventMeshServerWebhookThreadNum = 4; - public static final String KEYS_EVENTMESH_REGISTRY_THREAD_NUM = "eventMesh.server.registry.threads.num"; + @ConfigFiled(field = "pull.registry.interval") + private int eventMeshServerPullRegistryInterval = 30000; - public static final String KEYS_EVENTMESH_CLIENTMANAGE_THREAD_NUM = "eventMesh.server.clientmanage.threads.num"; + @ConfigFiled(field = "async.accumulation.threshold") + private int eventMeshServerAsyncAccumulationThreshold = 1000; - public static final String KEYS_EVENTMESH_ADMIN_THREAD_NUM = "eventMesh.server.admin.threads.num"; + @ConfigFiled(field = "retry.blockQ.size") + private int eventMeshServerRetryBlockQSize = 10000; - public static final String KEY_EVENTMESH_RETRY_THREAD_NUM = "eventMesh.server.retry.threads.num"; + @ConfigFiled(field = "batchmsg.blockQ.size") + private int eventMeshServerBatchBlockQSize = 1000; - public static final String KEYS_EVENTMESH_PULL_REGISTRY_INTERVAL = "eventMesh.server.pull.registry.interval"; + @ConfigFiled(field = "sendmsg.blockQ.size") + private int eventMeshServerSendMsgBlockQSize = 1000; - public static final String KEY_EVENTMESH_RETRY_BLOCKQ_SIZE = "eventMesh.server.retry.blockQ.size"; + @ConfigFiled(field = "") + private int eventMeshServerRemoteMsgBlockQSize = 1000; - public static final String KEY_EVENTMESH_BATCHMSG_BLOCKQ_SIZE = "eventMesh.server.batchmsg.blockQ.size"; + @ConfigFiled(field = "pushmsg.blockQ.size") + private int eventMeshServerPushMsgBlockQSize = 1000; - public static final String KEY_EVENTMESH_SENDMSG_BLOCKQ_SIZE = "eventMesh.server.sendmsg.blockQ.size"; + @ConfigFiled(field = "clientM.blockQ.size") + private int eventMeshServerClientManageBlockQSize = 1000; - public static final String KEY_EVENTMESH_PUSHMSG_BLOCKQ_SIZE = "eventMesh.server.pushmsg.blockQ.size"; + @ConfigFiled(field = "busy.check.interval") + private int eventMeshServerBusyCheckInterval = 1000; - public static final String KEY_EVENTMESH_CLIENTM_BLOCKQ_SIZE = "eventMesh.server.clientM.blockQ.size"; + @ConfigFiled(field = "consumer.enabled") + private boolean eventMeshServerConsumerEnabled = false; - public static final String KEY_EVENTMESH_CONSUMER_ENABLED = "eventMesh.server.consumer.enabled"; + @ConfigFiled(field = "useTls.enabled") + private boolean eventMeshServerUseTls = false; - public static final String KEY_EVENTMESH_HTTPS_ENABLED = "eventMesh.server.useTls.enabled"; + @ConfigFiled(field = "ssl.protocol") + private String eventMeshServerSSLProtocol = "TLSv1.1"; - public static final String KEY_EVENTMESH_HTTPS_SSL_PROTOCOL = "eventMesh.server.ssl.protocol"; + @ConfigFiled(field = "ssl.cer") + private String eventMeshServerSSLCer = "sChat2.jks"; - public static final String KEY_EVENTMESH_HTTPS_SSL_CER = "eventMesh.server.ssl.cer"; + @ConfigFiled(field = "ssl.pass") + private String eventMeshServerSSLPass = "sNetty"; - public static final String KEY_EVENTMESH_HTTPS_SSL_PASS = "eventMesh.server.ssl.pass"; + @ConfigFiled(field = "http.msgReqnumPerSecond") + private int eventMeshHttpMsgReqNumPerSecond = 15000; - public static final String KEY_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECOND = "eventMesh.server.http.msgReqnumPerSecond"; + @ConfigFiled(field = "batchmsg.reqNumPerSecond") + private int eventMeshBatchMsgRequestNumPerSecond = 20000; - public static final String KEY_EVENTMESH_SERVER_EVENTSIZE = "eventMesh.server.maxEventSize"; + @ConfigFiled(field = "maxEventSize") + private int eventMeshEventSize = 1000; - public static final String KEY_EVENTMESH_SERVER_EVENT_BATCHSIZE = "eventMesh.server.maxEventBatchSize"; + @ConfigFiled(field = "maxEventBatchSize") + private int eventMeshEventBatchSize = 10; - public static final String KEY_EVENTMESH_SERVER_IPV4_BLACK_LIST = "eventMesh.server.blacklist.ipv4"; + @ConfigFiled(field = "blacklist.ipv4") + private List eventMeshIpv4BlackList = Collections.emptyList(); - public static final String KEY_EVENTMESH_SERVER_IPV6_BLACK_LIST = "eventMesh.server.blacklist.ipv6"; - } + @ConfigFiled(field = "blacklist.ipv6") + private List eventMeshIpv6BlackList = Collections.emptyList(); } \ No newline at end of file diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfiguration.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfiguration.java index 82c5f872d4..d89d043e18 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfiguration.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfiguration.java @@ -18,247 +18,108 @@ package org.apache.eventmesh.runtime.configuration; import org.apache.eventmesh.common.config.CommonConfiguration; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.NoArgsConstructor; + +@Data +@NoArgsConstructor +@Config(prefix = "eventMesh.server") public class EventMeshTCPConfiguration extends CommonConfiguration { + + @ConfigFiled(field = "tcp.port") public int eventMeshTcpServerPort = 10000; + @ConfigFiled(field = "tcp.allIdleSeconds") public int eventMeshTcpIdleAllSeconds = 60; + @ConfigFiled(field = "tcp.writerIdleSeconds") public int eventMeshTcpIdleWriteSeconds = 60; + @ConfigFiled(field = "tcp.readerIdleSeconds") public int eventMeshTcpIdleReadSeconds = 60; + @ConfigFiled(field = "tcp.msgReqnumPerSecond") public Integer eventMeshTcpMsgReqnumPerSecond = 15000; /** * TCP Server allows max client num */ + @ConfigFiled(field = "tcp.clientMaxNum") public int eventMeshTcpClientMaxNum = 10000; //======================================= New add config ================================= + @ConfigFiled(field = "global.scheduler") public int eventMeshTcpGlobalScheduler = 5; + @ConfigFiled(field = "tcp.taskHandleExecutorPoolSize") public int eventMeshTcpTaskHandleExecutorPoolSize = Runtime.getRuntime().availableProcessors(); + @ConfigFiled(field = "tcp.msgDownStreamExecutorPoolSize") public int eventMeshTcpMsgDownStreamExecutorPoolSize = Math.max(Runtime.getRuntime().availableProcessors(), 8); + @ConfigFiled(field = "session.expiredInMills") public int eventMeshTcpSessionExpiredInMills = 60000; + @ConfigFiled(field = "session.upstreamBufferSize") public int eventMeshTcpSessionUpstreamBufferSize = 100; + @ConfigFiled(field = "retry.async.pushRetryTimes") public int eventMeshTcpMsgAsyncRetryTimes = 3; + @ConfigFiled(field = "retry.sync.pushRetryTimes") public int eventMeshTcpMsgSyncRetryTimes = 1; + @ConfigFiled(field = "retry.sync.pushRetryDelayInMills") public int eventMeshTcpMsgRetrySyncDelayInMills = 500; + @ConfigFiled(field = "retry.async.pushRetryDelayInMills") public int eventMeshTcpMsgRetryAsyncDelayInMills = 500; + @ConfigFiled(field = "retry.pushRetryQueueSize") public int eventMeshTcpMsgRetryQueueSize = 10000; + @ConfigFiled(field = "tcp.RebalanceIntervalInMills") public Integer eventMeshTcpRebalanceIntervalInMills = 30 * 1000; + @ConfigFiled(field = "admin.http.port") public int eventMeshServerAdminPort = 10106; - + @ConfigFiled(field = "tcp.sendBack.enabled") public boolean eventMeshTcpSendBackEnabled = Boolean.TRUE; + @ConfigFiled(field = "") public int eventMeshTcpSendBackMaxTimes = 3; + @ConfigFiled(field = "tcp.pushFailIsolateTimeInMills") public int eventMeshTcpPushFailIsolateTimeInMills = 30 * 1000; + @ConfigFiled(field = "gracefulShutdown.sleepIntervalInMills") public int gracefulShutdownSleepIntervalInMills = 1000; + @ConfigFiled(field = "rebalanceRedirect.sleepIntervalInM") public int sleepIntervalInRebalanceRedirectMills = 200; + @ConfigFiled(field = "maxEventSize") public int eventMeshEventSize = 1000; + @ConfigFiled(field = "maxEventBatchSize") public int eventMeshEventBatchSize = 10; - private TrafficShapingConfig gtc = new TrafficShapingConfig(0, 10_000, 1_000, 2000); - private TrafficShapingConfig ctc = new TrafficShapingConfig(0, 2_000, 1_000, 10_000); - - public EventMeshTCPConfiguration(ConfigurationWrapper configurationWrapper) { - super(configurationWrapper); - } - - @Override - public void init() { - super.init(); - eventMeshTcpServerPort = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_TCP_PORT, eventMeshTcpServerPort); - - eventMeshTcpIdleReadSeconds = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_READER_IDLE_SECONDS, - eventMeshTcpIdleReadSeconds); - - eventMeshTcpIdleWriteSeconds = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_WRITER_IDLE_SECONDS, - eventMeshTcpIdleWriteSeconds); - - eventMeshTcpIdleAllSeconds = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_ALL_IDLE_SECONDS, - eventMeshTcpIdleAllSeconds); - - eventMeshTcpMsgReqnumPerSecond = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECONDS, - eventMeshTcpMsgReqnumPerSecond); - - eventMeshTcpClientMaxNum = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_CLIENT_MAX_NUM, - eventMeshTcpClientMaxNum); - - eventMeshTcpGlobalScheduler = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_GLOBAL_SCHEDULER, - eventMeshTcpGlobalScheduler); - - eventMeshTcpTaskHandleExecutorPoolSize = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_TCP_TASK_HANDLE_POOL_SIZE, eventMeshTcpTaskHandleExecutorPoolSize); - - eventMeshTcpMsgDownStreamExecutorPoolSize = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_TCP_MSG_DOWNSTREAM_POOL_SIZE, eventMeshTcpMsgDownStreamExecutorPoolSize); - - eventMeshTcpSessionExpiredInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_SESSION_EXPIRED_TIME, eventMeshTcpSessionExpiredInMills); - - eventMeshTcpSessionUpstreamBufferSize = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_SESSION_UPSTREAM_BUFFER_SIZE, eventMeshTcpSessionUpstreamBufferSize); - - //========================================eventMesh retry config=============================================// - eventMeshTcpMsgAsyncRetryTimes = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_RETRY_ASYNC_PUSH_RETRY_TIMES, eventMeshTcpMsgAsyncRetryTimes); - - eventMeshTcpMsgSyncRetryTimes = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_RETRY_SYNC_PUSH_RETRY_TIMES, eventMeshTcpMsgSyncRetryTimes); - - eventMeshTcpMsgRetryAsyncDelayInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_RETRY_ASYNC_PUSH_RETRY_DELAY, eventMeshTcpMsgRetryAsyncDelayInMills); - - eventMeshTcpMsgRetrySyncDelayInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_RETRY_SYNC_PUSH_RETRY_DELAY, eventMeshTcpMsgRetrySyncDelayInMills); - - eventMeshTcpMsgRetryQueueSize = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_RETRY_PUSH_RETRY_QUEUE_SIZE, eventMeshTcpMsgRetryQueueSize); - - eventMeshTcpRebalanceIntervalInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_TCP_REBALANCE_INTERVAL, eventMeshTcpRebalanceIntervalInMills); - - eventMeshServerAdminPort = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_ADMIN_HTTP_PORT, eventMeshServerAdminPort); - - eventMeshTcpSendBackEnabled = configurationWrapper.getBoolProp( - ConfKeys.KEYS_EVENTMESH_TCP_SEND_BACK_ENABLED, eventMeshTcpSendBackEnabled); - - eventMeshTcpPushFailIsolateTimeInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_PUSH_FAIL_ISOLATE_TIME, eventMeshTcpPushFailIsolateTimeInMills); - - gracefulShutdownSleepIntervalInMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_GRACEFUL_SHUTDOWN_SLEEP_TIME, gracefulShutdownSleepIntervalInMills); - - sleepIntervalInRebalanceRedirectMills = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_REBALANCE_REDIRECT_SLEEP_TIME, sleepIntervalInRebalanceRedirectMills); - - eventMeshEventSize = configurationWrapper.getIntProp(ConfKeys.KEYS_EVENTMESH_SERVER_EVENTSIZE, eventMeshEventSize); - - eventMeshEventBatchSize = configurationWrapper.getIntProp( - ConfKeys.KEYS_EVENTMESH_SERVER_EVENT_BATCHSIZE, eventMeshEventBatchSize); - } - - public TrafficShapingConfig getGtc() { - return gtc; - } - - public TrafficShapingConfig getCtc() { - return ctc; - } - - static class ConfKeys { - - public static final String KEYS_EVENTMESH_SERVER_TCP_PORT = "eventMesh.server.tcp.port"; - public static final String KEYS_EVENTMESH_SERVER_READER_IDLE_SECONDS = "eventMesh.server.tcp.readerIdleSeconds"; - public static final String KEYS_EVENTMESH_SERVER_WRITER_IDLE_SECONDS = "eventMesh.server.tcp.writerIdleSeconds"; - public static final String KEYS_EVENTMESH_SERVER_ALL_IDLE_SECONDS = "eventMesh.server.tcp.allIdleSeconds"; - public static final String KEYS_EVENTMESH_SERVER_CLIENT_MAX_NUM = "eventMesh.server.tcp.clientMaxNum"; - public static final String KEYS_EVENTMESH_SERVER_MSG_REQ_NUM_PER_SECONDS = "eventMesh.server.tcp.msgReqnumPerSecond"; - public static final String KEYS_EVENTMESH_SERVER_TCP_REBALANCE_INTERVAL = "eventMesh.server.tcp.RebalanceIntervalInMills"; - public static final String KEYS_EVENTMESH_SERVER_GLOBAL_SCHEDULER = "eventMesh.server.global.scheduler"; - public static final String KEYS_EVENTMESH_SERVER_TCP_TASK_HANDLE_POOL_SIZE = "eventMesh.server.tcp.taskHandleExecutorPoolSize"; - public static final String KEYS_EVENTMESH_SERVER_TCP_MSG_DOWNSTREAM_POOL_SIZE = "eventMesh.server.tcp.msgDownStreamExecutorPoolSize"; - public static final String KEYS_EVENTMESH_SERVER_SESSION_EXPIRED_TIME = "eventMesh.server.session.expiredInMills"; - public static final String KEYS_EVENTMESH_SERVER_SESSION_UPSTREAM_BUFFER_SIZE = "eventMesh.server.session.upstreamBufferSize"; - public static final String KEYS_EVENTMESH_SERVER_SESSION_DOWNSTREAM_UNACK_SIZE = "eventMesh.server.session.downstreamUnackSize"; - public static final String KEYS_EVENTMESH_SERVER_RETRY_ASYNC_PUSH_RETRY_TIMES = "eventMesh.server.retry.async.pushRetryTimes"; - public static final String KEYS_EVENTMESH_SERVER_RETRY_SYNC_PUSH_RETRY_TIMES = "eventMesh.server.retry.sync.pushRetryTimes"; - public static final String KEYS_EVENTMESH_SERVER_RETRY_ASYNC_PUSH_RETRY_DELAY = "eventMesh.server.retry.async.pushRetryDelayInMills"; - public static final String KEYS_EVENTMESH_SERVER_RETRY_SYNC_PUSH_RETRY_DELAY = "eventMesh.server.retry.sync.pushRetryDelayInMills"; - public static final String KEYS_EVENTMESH_SERVER_RETRY_PUSH_RETRY_QUEUE_SIZE = "eventMesh.server.retry.pushRetryQueueSize"; - public static final String KEYS_EVENTMESH_SERVER_ADMIN_HTTP_PORT = "eventMesh.server.admin.http.port"; - public static final String KEYS_EVENTMESH_TCP_SEND_BACK_ENABLED = "eventMesh.server.tcp.sendBack.enabled"; - public static final String KEYS_EVENTMESH_SERVER_PUSH_FAIL_ISOLATE_TIME = "eventMesh.server.tcp.pushFailIsolateTimeInMills"; - public static final String KEYS_EVENTMESH_SERVER_GRACEFUL_SHUTDOWN_SLEEP_TIME = "eventMesh.server.gracefulShutdown.sleepIntervalInMills"; - public static final String KEYS_EVENTMESH_SERVER_REBALANCE_REDIRECT_SLEEP_TIME = "eventMesh.server.rebalanceRedirect.sleepIntervalInM"; - public static final String KEYS_EVENTMESH_SERVER_EVENTSIZE = "eventMesh.server.maxEventSize"; - public static final String KEYS_EVENTMESH_SERVER_EVENT_BATCHSIZE = "eventMesh.server.maxEventBatchSize"; - } + private final TrafficShapingConfig gtc = new TrafficShapingConfig(0, 10_000, 1_000, 2000); + private final TrafficShapingConfig ctc = new TrafficShapingConfig(0, 2_000, 1_000, 10_000); + @Data + @NoArgsConstructor + @AllArgsConstructor public static class TrafficShapingConfig { long writeLimit = 0; long readLimit = 1000; long checkInterval = 1000; long maxTime = 5000; - - public TrafficShapingConfig(long writeLimit, long readLimit, long checkInterval, long maxTime) { - this.writeLimit = writeLimit; - this.readLimit = readLimit; - this.checkInterval = checkInterval; - this.maxTime = maxTime; - } - - public TrafficShapingConfig() { - - } - - public long getWriteLimit() { - return writeLimit; - } - - public void setWriteLimit(long writeLimit) { - this.writeLimit = writeLimit; - } - - public long getReadLimit() { - return readLimit; - } - - public void setReadLimit(long readLimit) { - this.readLimit = readLimit; - } - - public long getCheckInterval() { - return checkInterval; - } - - public void setCheckInterval(long checkInterval) { - this.checkInterval = checkInterval; - } - - public long getMaxTime() { - return maxTime; - } - - public void setMaxTime(long maxTime) { - this.maxTime = maxTime; - } - - @Override - public String toString() { - return "TrafficShapingConfig{" - + - "writeLimit=" + writeLimit - + - ", readLimit=" + readLimit - + - ", checkInterval=" + checkInterval - + - ", maxTime=" + maxTime - + - '}'; - } } - } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/ConsumerManager.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/ConsumerManager.java index d777158222..b5eb7a2b55 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/ConsumerManager.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/ConsumerManager.java @@ -208,7 +208,7 @@ public synchronized void restartEventMeshConsumer(final String consumerGroup) th } private void startClientCheck() { - final int clientTimeout = eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshSessionExpiredInMills; + final int clientTimeout = eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshSessionExpiredInMills(); if (clientTimeout > 0) { scheduledExecutorService.scheduleAtFixedRate(() -> { if (LOGGER.isDebugEnabled()) { diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/EventMeshConsumer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/EventMeshConsumer.java index 69ca65465b..dc08f86f30 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/EventMeshConsumer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/consumer/EventMeshConsumer.java @@ -145,7 +145,6 @@ public synchronized void init() throws Exception { keyValue.put(EVENT_MESH_IDC, eventMeshGrpcConfiguration.getEventMeshIDC()); keyValue.put(INSTANCE_NAME, EventMeshUtil.buildMeshClientID(consumerGroup, eventMeshGrpcConfiguration.getEventMeshCluster())); - persistentMqConsumer.init(keyValue); persistentMqConsumer.registerEventListener(createEventListener(SubscriptionMode.CLUSTERING)); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/push/WebhookPushRequest.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/push/WebhookPushRequest.java index 1341eb27c1..458bb0eca5 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/push/WebhookPushRequest.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/push/WebhookPushRequest.java @@ -108,7 +108,7 @@ public void tryPushRequest() { builder.addHeader(ProtocolKey.VERSION, ProtocolVersion.V1.getVersion()); builder.addHeader(ProtocolKey.EventMeshInstanceKey.EVENTMESHCLUSTER, eventMeshGrpcConfiguration.getEventMeshCluster()); - builder.addHeader(ProtocolKey.EventMeshInstanceKey.EVENTMESHIP, eventMeshGrpcConfiguration.eventMeshIp); + builder.addHeader(ProtocolKey.EventMeshInstanceKey.EVENTMESHIP, eventMeshGrpcConfiguration.getEventMeshIp()); builder.addHeader(ProtocolKey.EventMeshInstanceKey.EVENTMESHENV, eventMeshGrpcConfiguration.getEventMeshEnv()); builder.addHeader(ProtocolKey.EventMeshInstanceKey.EVENTMESHIDC, eventMeshGrpcConfiguration.getEventMeshIDC()); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/retry/GrpcRetryer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/retry/GrpcRetryer.java index f65bfc63d2..0022e24ffb 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/retry/GrpcRetryer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/retry/GrpcRetryer.java @@ -49,7 +49,7 @@ public GrpcRetryer(EventMeshGrpcServer eventMeshGrpcServer) { private Thread dispatcher; public void pushRetry(DelayRetryable delayRetryable) { - if (failed.size() >= grpcConfiguration.eventMeshServerRetryBlockQueueSize) { + if (failed.size() >= grpcConfiguration.getEventMeshServerRetryBlockQueueSize()) { retryLogger.error("[RETRY-QUEUE] is full!"); return; } @@ -57,22 +57,21 @@ public void pushRetry(DelayRetryable delayRetryable) { } public void init() { - pool = new ThreadPoolExecutor(grpcConfiguration.eventMeshServerRetryThreadNum, - grpcConfiguration.eventMeshServerRetryThreadNum, - 60000, - TimeUnit.MILLISECONDS, - new ArrayBlockingQueue<>(grpcConfiguration.eventMeshServerRetryBlockQueueSize), - new ThreadFactory() { - private AtomicInteger count = new AtomicInteger(); - - @Override - public Thread newThread(Runnable r) { - Thread thread = new Thread(r, "grpc-retry-" + count.incrementAndGet()); - thread.setPriority(Thread.NORM_PRIORITY); - thread.setDaemon(true); - return thread; - } - }, new ThreadPoolExecutor.AbortPolicy()); + pool = new ThreadPoolExecutor( + grpcConfiguration.getEventMeshServerRetryThreadNum(), + grpcConfiguration.getEventMeshServerRetryThreadNum(), 60000, TimeUnit.MILLISECONDS, + new ArrayBlockingQueue<>(grpcConfiguration.getEventMeshServerRetryBlockQueueSize()), + new ThreadFactory() { + private AtomicInteger count = new AtomicInteger(); + + @Override + public Thread newThread(Runnable r) { + Thread thread = new Thread(r, "grpc-retry-" + count.incrementAndGet()); + thread.setPriority(Thread.NORM_PRIORITY); + thread.setDaemon(true); + return thread; + } + }, new ThreadPoolExecutor.AbortPolicy()); dispatcher = new Thread(() -> { try { diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ConsumerService.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ConsumerService.java index ec77420b06..b3108a7c69 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ConsumerService.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ConsumerService.java @@ -55,7 +55,7 @@ public ConsumerService(EventMeshGrpcServer eventMeshGrpcServer, public void subscribe(Subscription request, StreamObserver responseObserver) { log.info("cmd={}|{}|client2eventMesh|from={}|to={}", "subscribe", EventMeshConstants.PROTOCOL_GRPC, - request.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + request.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(); EventEmitter emitter = new EventEmitter<>(responseObserver); @@ -80,14 +80,14 @@ public void onNext(Subscription subscription) { if (!subscription.getSubscriptionItemsList().isEmpty()) { log.info("cmd={}|{}|client2eventMesh|from={}|to={}", "subscribeStream", EventMeshConstants.PROTOCOL_GRPC, - subscription.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + subscription.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(); handleSubscriptionStream(subscription, emitter); } else { log.info("cmd={}|{}|client2eventMesh|from={}|to={}", "reply-to-server", EventMeshConstants.PROTOCOL_GRPC, - subscription.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + subscription.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); handleSubscribeReply(subscription, emitter); } @@ -148,7 +148,7 @@ private SimpleMessage buildSimpleMessage(Subscription subscription) { public void unsubscribe(Subscription request, StreamObserver responseObserver) { log.info("cmd={}|{}|client2eventMesh|from={}|to={}", "unsubscribe", EventMeshConstants.PROTOCOL_GRPC, - request.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + request.getHeader().getIp(), eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(); EventEmitter emitter = new EventEmitter<>(responseObserver); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/HeartbeatService.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/HeartbeatService.java index 962cf401bd..c302d01d2e 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/HeartbeatService.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/HeartbeatService.java @@ -49,7 +49,7 @@ public HeartbeatService(final EventMeshGrpcServer eventMeshGrpcServer, public void heartbeat(Heartbeat request, StreamObserver responseObserver) { LOGGER.info("cmd={}|{}|client2eventMesh|from={}|to={}", "heartbeat", EventMeshConstants.PROTOCOL_GRPC, request.getHeader().getIp(), - eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); EventEmitter emitter = new EventEmitter<>(responseObserver); threadPoolExecutor.submit(() -> { diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ProducerService.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ProducerService.java index 0254bdb8f9..c7759a743d 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ProducerService.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/grpc/service/ProducerService.java @@ -54,7 +54,7 @@ public ProducerService(EventMeshGrpcServer eventMeshGrpcServer, public void publish(SimpleMessage request, StreamObserver responseObserver) { cmdLogger.info("cmd={}|{}|client2eventMesh|from={}|to={}", "AsyncPublish", EventMeshConstants.PROTOCOL_GRPC, request.getHeader().getIp(), - eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(); EventEmitter emitter = new EventEmitter<>(responseObserver); @@ -73,7 +73,7 @@ public void publish(SimpleMessage request, StreamObserver responseObse public void requestReply(SimpleMessage request, StreamObserver responseObserver) { cmdLogger.info("cmd={}|{}|client2eventMesh|from={}|to={}", "RequestReply", EventMeshConstants.PROTOCOL_GRPC, request.getHeader().getIp(), - eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(); EventEmitter emitter = new EventEmitter<>(responseObserver); @@ -92,7 +92,7 @@ public void requestReply(SimpleMessage request, StreamObserver re public void batchPublish(BatchMessage request, StreamObserver responseObserver) { cmdLogger.info("cmd={}|{}|client2eventMesh|from={}|to={}", "BatchPublish", EventMeshConstants.PROTOCOL_GRPC, request.getHeader().getIp(), - eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp); + eventMeshGrpcServer.getEventMeshGrpcConfiguration().getEventMeshIp()); eventMeshGrpcServer.getMetricsMonitor().recordReceiveMsgFromClient(request.getMessageItemCount()); EventEmitter emitter = new EventEmitter<>(responseObserver); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageProcessor.java index 418395bbb6..45a5a1c763 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageProcessor.java @@ -112,14 +112,14 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext String producerGroup = ""; int eventSize = eventList.size(); - if (eventSize > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventBatchSize) { + if (eventSize > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventBatchSize()) { batchMessageLogger.error("Event batch size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventBatchSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventBatchSize()); responseEventMeshCommand = asyncContext.getRequest().createHttpCommandResponse( sendMessageBatchResponseHeader, SendMessageBatchResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR.getRetCode(), - "Event batch size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventBatchSize)); + "Event batch size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventBatchSize())); asyncContext.onComplete(responseEventMeshCommand); return; } @@ -140,14 +140,14 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext } String content = event.getData() == null ? "" : new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { batchMessageLogger.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); responseEventMeshCommand = asyncContext.getRequest().createHttpCommandResponse( - sendMessageBatchResponseHeader, - SendMessageBatchResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_HEADER_ERR.getRetCode(), - "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize)); + sendMessageBatchResponseHeader, + SendMessageBatchResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_HEADER_ERR.getRetCode(), + "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize())); asyncContext.onComplete(responseEventMeshCommand); return; } @@ -284,7 +284,7 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext long delta = eventSize; eventMeshHTTPServer.getMetrics().getSummaryMetrics().recordSendBatchMsg(delta); - if (eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshServerBatchMsgBatchEnabled) { + if (eventMeshHTTPServer.getEventMeshHttpConfiguration().isEventMeshServerBatchMsgBatchEnabled()) { for (List eventlist : topicBatchMessageMappings.values()) { // TODO: Implementation in API. Consider whether to put it in the plug-in. CloudEvent event = null; diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageV2Processor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageV2Processor.java index 46514b9eb7..cdcb9ebce7 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageV2Processor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/BatchSendMessageV2Processor.java @@ -159,14 +159,14 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext } String content = new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { batchMessageLogger.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); responseEventMeshCommand = asyncContext.getRequest().createHttpCommandResponse( sendMessageBatchV2ResponseHeader, SendMessageBatchV2ResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR.getRetCode(), - "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize)); + "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize())); asyncContext.onComplete(responseEventMeshCommand); return; } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/LocalSubscribeEventProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/LocalSubscribeEventProcessor.java index eb854072e6..88db9db219 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/LocalSubscribeEventProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/LocalSubscribeEventProcessor.java @@ -52,7 +52,7 @@ @EventMeshTrace @Slf4j public class LocalSubscribeEventProcessor extends AbstractEventProcessor { - + public LocalSubscribeEventProcessor(final EventMeshHTTPServer eventMeshHTTPServer) { super(eventMeshHTTPServer); } @@ -135,12 +135,12 @@ public void handler(final HandlerService.HandlerSpecific handlerSpecific, final // validate URL try { - if (!IPUtils.isValidDomainOrIp(url, eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshIpv4BlackList, - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshIpv6BlackList)) { + if (!IPUtils.isValidDomainOrIp(url, eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshIpv4BlackList(), + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshIpv6BlackList())) { if (log.isErrorEnabled()) { log.error("subscriber url {} is not valid", url); } - + handlerSpecific.sendErrorResponse(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR, responseHeaderMap, responseBodyMap, null); return; diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteSubscribeEventProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteSubscribeEventProcessor.java index 503ac87dd5..a3019a32e6 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteSubscribeEventProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteSubscribeEventProcessor.java @@ -150,8 +150,8 @@ public void handler(HandlerService.HandlerSpecific handlerSpecific, HttpRequest // validate URL try { - if (!IPUtils.isValidDomainOrIp(url, eventMeshHttpConfiguration.eventMeshIpv4BlackList, - eventMeshHttpConfiguration.eventMeshIpv6BlackList)) { + if (!IPUtils.isValidDomainOrIp(url, eventMeshHttpConfiguration.getEventMeshIpv4BlackList(), + eventMeshHttpConfiguration.getEventMeshIpv6BlackList())) { httpLogger.error("subscriber url {} is not valid", url); handlerSpecific.sendErrorResponse(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR, responseHeaderMap, responseBodyMap, null); @@ -179,7 +179,7 @@ public void handler(HandlerService.HandlerSpecific handlerSpecific, HttpRequest try { // local subscription url String localUrl = "http://" + localAddress + ":" - + eventMeshHttpConfiguration.httpServerPort + + eventMeshHttpConfiguration.getHttpServerPort() + RequestURI.PUBLISH_BRIDGE.getRequestURI(); Map remoteBodyMap = new HashMap<>(); remoteBodyMap.put(EventMeshConstants.URL, localUrl); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteUnSubscribeEventProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteUnSubscribeEventProcessor.java index c6b5b2ad4b..b5edf8ab9a 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteUnSubscribeEventProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/RemoteUnSubscribeEventProcessor.java @@ -93,7 +93,7 @@ public void handler(HandlerService.HandlerSpecific handlerSpecific, HttpRequest Map sysHeaderMap = requestWrapper.getSysHeaderMap(); Map responseBodyMap = new HashMap<>(); - + //validate header if (validateSysHeader(sysHeaderMap)) { @@ -130,11 +130,11 @@ public void handler(HandlerService.HandlerSpecific handlerSpecific, HttpRequest String sysId = eventMeshHttpConfiguration.getSysID(); String meshGroup = String.join("-", env, idc, cluster, sysId); - + // local unSubscription url String unsubscribeUrl = "http://" + localAddress + ":" - + eventMeshHttpConfiguration.httpServerPort + + eventMeshHttpConfiguration.getHttpServerPort() + RequestURI.PUBLISH_BRIDGE.getRequestURI(); Map remoteBodyMap = new HashMap<>(); @@ -197,6 +197,6 @@ public String[] paths() { return new String[] {RequestURI.UNSUBSCRIBE_REMOTE.getRequestURI()}; } - + } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/ReplyMessageProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/ReplyMessageProcessor.java index 118cc65b6c..f74e707893 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/ReplyMessageProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/ReplyMessageProcessor.java @@ -154,14 +154,14 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext } String content = event.getData() == null ? "" : new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { httpLogger.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); responseEventMeshCommand = asyncContext.getRequest().createHttpCommandResponse( replyMessageResponseHeader, ReplyMessageResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR.getRetCode(), - "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize)); + "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize())); asyncContext.onComplete(responseEventMeshCommand); return; } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncEventProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncEventProcessor.java index d202531637..9dcd973a58 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncEventProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncEventProcessor.java @@ -198,10 +198,10 @@ public void handler(final HandlerService.HandlerSpecific handlerSpecific, final } final String content = new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (Objects.requireNonNull(content).length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (Objects.requireNonNull(content).length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { if (log.isErrorEnabled()) { log.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); } handlerSpecific.sendErrorResponse(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_SIZE_ERR, responseHeaderMap, responseBodyMap, EventMeshUtil.getCloudEventExtensionMap(SpecVersion.V1.toString(), event)); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncMessageProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncMessageProcessor.java index 86d12dd860..fc73c4baac 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncMessageProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncMessageProcessor.java @@ -228,14 +228,14 @@ public void processRequest(ChannelHandlerContext ctx, AsyncContext } String content = event.getData() == null ? "" : new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (content.length() > eventMeshHttpConfiguration.eventMeshEventSize) { + if (content.length() > eventMeshHttpConfiguration.getEventMeshEventSize()) { httpLogger.error("Event size exceeds the limit: {}", - eventMeshHttpConfiguration.eventMeshEventSize); + eventMeshHttpConfiguration.getEventMeshEventSize()); responseEventMeshCommand = request.createHttpCommandResponse( sendMessageResponseHeader, SendMessageResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_SIZE_ERR.getRetCode(), - "Event size exceeds the limit: " + eventMeshHttpConfiguration.eventMeshEventSize)); + "Event size exceeds the limit: " + eventMeshHttpConfiguration.getEventMeshEventSize())); asyncContext.onComplete(responseEventMeshCommand); Span excepSpan = TraceUtils.prepareServerSpan(EventMeshUtil.getCloudEventExtensionMap(protocolVersin, event), diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncRemoteEventProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncRemoteEventProcessor.java index 7425bd9e45..0e270f364e 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncRemoteEventProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendAsyncRemoteEventProcessor.java @@ -243,10 +243,10 @@ public void handler(final HandlerService.HandlerSpecific handlerSpecific, final } final String content = event.getData() == null ? "" : new String(event.getData().toBytes(), StandardCharsets.UTF_8); - if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { if (log.isErrorEnabled()) { log.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); } handlerSpecific.sendErrorResponse(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_SIZE_ERR, responseHeaderMap, responseBodyMap, EventMeshUtil.getCloudEventExtensionMap(SpecVersion.V1.toString(), event)); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendSyncMessageProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendSyncMessageProcessor.java index 6b5deb4c72..92b116f62f 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendSyncMessageProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SendSyncMessageProcessor.java @@ -193,16 +193,16 @@ public void processRequest(final ChannelHandlerContext ctx, final AsyncContext eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize) { + if (content.length() > eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()) { if (log.isErrorEnabled()) { log.error("Event size exceeds the limit: {}", - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize); + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize()); } responseEventMeshCommand = asyncContext.getRequest().createHttpCommandResponse( sendMessageResponseHeader, SendMessageResponseBody.buildBody(EventMeshRetCode.EVENTMESH_PROTOCOL_BODY_ERR.getRetCode(), - "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshEventSize)); + "Event size exceeds the limit: " + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshEventSize())); asyncContext.onComplete(responseEventMeshCommand); return; } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SubscribeProcessor.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SubscribeProcessor.java index 3f6c24387c..97303a9280 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SubscribeProcessor.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/processor/SubscribeProcessor.java @@ -145,8 +145,8 @@ public void processRequest(final ChannelHandlerContext ctx, final AsyncContext requestBodyMap) { return requestBodyMap.get(EventMeshConstants.URL) == null || requestBodyMap.get(EventMeshConstants.MANAGE_TOPIC) == null || requestBodyMap.get(EventMeshConstants.CONSUMER_GROUP) == null; - + } /** @@ -274,5 +274,5 @@ public static String post(CloseableHttpClient client, String uri, return client.execute(httpPost, responseHandler); } - + } diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/retry/HttpRetryer.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/retry/HttpRetryer.java index 6448db3fe4..b7faae46a9 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/retry/HttpRetryer.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/http/retry/HttpRetryer.java @@ -48,7 +48,7 @@ public HttpRetryer(EventMeshHTTPServer eventMeshHTTPServer) { private Thread dispatcher; public void pushRetry(DelayRetryable delayRetryable) { - if (failed.size() >= eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshServerRetryBlockQSize) { + if (failed.size() >= eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshServerRetryBlockQSize()) { retryLogger.error("[RETRY-QUEUE] is full!"); return; } @@ -56,11 +56,11 @@ public void pushRetry(DelayRetryable delayRetryable) { } public void init() { - pool = new ThreadPoolExecutor(eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshServerRetryThreadNum, - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshServerRetryThreadNum, + pool = new ThreadPoolExecutor(eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshServerRetryThreadNum(), + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshServerRetryThreadNum(), 60000, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<>( - eventMeshHTTPServer.getEventMeshHttpConfiguration().eventMeshServerRetryBlockQSize), + eventMeshHTTPServer.getEventMeshHttpConfiguration().getEventMeshServerRetryBlockQSize()), new ThreadFactory() { private AtomicInteger count = new AtomicInteger(); diff --git a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/tcp/client/group/ClientGroupWrapper.java b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/tcp/client/group/ClientGroupWrapper.java index 4e91a3d2da..894759f711 100644 --- a/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/tcp/client/group/ClientGroupWrapper.java +++ b/eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/core/protocol/tcp/client/group/ClientGroupWrapper.java @@ -71,7 +71,7 @@ @Slf4j public class ClientGroupWrapper { - + private final String sysId; private String group; @@ -130,7 +130,6 @@ public ClientGroupWrapper(String sysId, String group, eventMeshTCPServer.getEventMeshTCPConfiguration().getEventMeshConnectorPluginType()); this.mqProducerWrapper = new MQProducerWrapper( eventMeshTCPServer.getEventMeshTCPConfiguration().getEventMeshConnectorPluginType()); - } public ConcurrentHashMap> getTopic2sessionInGroupMapping() { diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/controller/ClientManageControllerTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/controller/ClientManageControllerTest.java index 092369abbf..4984fa9c0c 100644 --- a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/controller/ClientManageControllerTest.java +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/controller/ClientManageControllerTest.java @@ -21,11 +21,11 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyInt; -import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; import org.apache.eventmesh.admin.rocketmq.controller.AdminController; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.metrics.api.model.HttpSummaryMetrics; import org.apache.eventmesh.metrics.api.model.TcpSummaryMetrics; import org.apache.eventmesh.runtime.boot.EventMeshGrpcServer; @@ -50,19 +50,20 @@ public class ClientManageControllerTest { @Test - public void testStart() { - EventMeshTCPServer eventMeshTCPServer = mock(EventMeshTCPServer.class); - EventMeshHTTPServer eventMeshHTTPServer = mock(EventMeshHTTPServer.class); - EventMeshGrpcServer eventMeshGrpcServer = mock(EventMeshGrpcServer.class); - Registry registry = mock(Registry.class); + public void testStart() throws Exception { AdminController adminController = mock(AdminController.class); - EventMeshTCPConfiguration tcpConfiguration = mock(EventMeshTCPConfiguration.class); - doNothing().when(tcpConfiguration).init(); + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + EventMeshTCPConfiguration tcpConfiguration = configService.buildConfigInstance(EventMeshTCPConfiguration.class); + + EventMeshTCPServer eventMeshTCPServer = mock(EventMeshTCPServer.class); when(eventMeshTCPServer.getEventMeshTCPConfiguration()).thenReturn(tcpConfiguration); HttpSummaryMetrics httpSummaryMetrics = mock(HttpSummaryMetrics.class); HTTPMetricsServer metrics = mock(HTTPMetricsServer.class); + EventMeshHTTPServer eventMeshHTTPServer = mock(EventMeshHTTPServer.class); when(eventMeshHTTPServer.getMetrics()).thenReturn(metrics); when(eventMeshHTTPServer.getMetrics().getSummaryMetrics()).thenReturn(httpSummaryMetrics); @@ -76,11 +77,13 @@ public void testStart() { WebHookConfigOperation webHookConfigOperation = mock(WebHookConfigOperation.class); when(adminWebHookConfigOperationManage.getWebHookConfigOperation()).thenReturn(webHookConfigOperation); + EventMeshGrpcServer eventMeshGrpcServer = mock(EventMeshGrpcServer.class); + Registry registry = mock(Registry.class); ClientManageController controller = new ClientManageController(eventMeshTCPServer, eventMeshHTTPServer, eventMeshGrpcServer, registry); controller.setAdminWebHookConfigOperationManage(adminWebHookConfigOperationManage); - when(eventMeshTCPServer.getEventMeshTCPConfiguration().getEventMeshConnectorPluginType()).thenReturn("standalone"); + eventMeshTCPServer.getEventMeshTCPConfiguration().setEventMeshConnectorPluginType("standalone"); try (MockedStatic dummyStatic = Mockito.mockStatic(HttpServer.class)) { HttpServer server = mock(HttpServer.class); diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/handler/QueryRecommendEventMeshHandlerTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/handler/QueryRecommendEventMeshHandlerTest.java index 1a7c3a2732..0e6b5e5a8c 100644 --- a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/handler/QueryRecommendEventMeshHandlerTest.java +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/admin/handler/QueryRecommendEventMeshHandlerTest.java @@ -21,13 +21,13 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyString; -import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.doThrow; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.mockConstruction; import static org.mockito.Mockito.mockStatic; import static org.mockito.Mockito.when; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.runtime.admin.controller.HttpHandlerManager; import org.apache.eventmesh.runtime.boot.EventMeshTCPServer; import org.apache.eventmesh.runtime.configuration.EventMeshTCPConfiguration; @@ -54,8 +54,7 @@ public class QueryRecommendEventMeshHandlerTest { public void testHandle() throws Exception { // mock eventMeshTCPServer EventMeshTCPServer eventMeshTCPServer = mock(EventMeshTCPServer.class); - EventMeshTCPConfiguration tcpConfiguration = mock(EventMeshTCPConfiguration.class); - doNothing().when(tcpConfiguration).init(); + EventMeshTCPConfiguration tcpConfiguration = new EventMeshTCPConfiguration(); when(eventMeshTCPServer.getEventMeshTCPConfiguration()).thenReturn(tcpConfiguration); URI uri = mock(URI.class); @@ -70,7 +69,6 @@ public void testHandle() throws Exception { // case 1: normal case tcpConfiguration.setEventMeshServerRegistryEnable(true); - outputStream.write("result".getBytes(StandardCharsets.UTF_8)); when(httpExchange.getResponseBody()).thenReturn(outputStream); try (MockedConstruction ignored = mockConstruction(EventMeshRecommendImpl.class, (mock, context) -> when(mock.calculateRecommendEventMesh(anyString(), anyString())).thenReturn(returnValue))) { @@ -81,7 +79,6 @@ public void testHandle() throws Exception { // case 2: params illegal outputStream = new ByteArrayOutputStream(); - outputStream.write("params illegal!".getBytes(StandardCharsets.UTF_8)); when(httpExchange.getResponseBody()).thenReturn(outputStream); try (MockedStatic dummyStatic = mockStatic(StringUtils.class)) { dummyStatic.when(() -> StringUtils.isBlank(any())).thenReturn(true); diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/boot/EventMeshServerTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/boot/EventMeshServerTest.java new file mode 100644 index 0000000000..84aedc1fd0 --- /dev/null +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/boot/EventMeshServerTest.java @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.runtime.boot; + +import org.apache.eventmesh.common.config.CommonConfiguration; +import org.apache.eventmesh.common.config.ConfigService; +import org.apache.eventmesh.runtime.configuration.EventMeshTCPConfiguration; +import org.apache.eventmesh.runtime.constants.EventMeshConstants; + +import java.io.File; +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +public class EventMeshServerTest { + + /** + * True Environment variables need to be set during startup + */ + @Test + public void testGetConfigWhenStartup() throws Exception { + + testGetConfigWhenStartup(Boolean.FALSE); + } + + private void testGetConfigWhenStartup(Boolean hasEnv) throws Exception { + String eventMeshConfFile = "configuration.properties"; + + if (hasEnv) { + ConfigService.getInstance() + .setConfigPath(EventMeshConstants.EVENTMESH_CONF_HOME + File.separator) + .setRootConfig(eventMeshConfFile); + } else { + eventMeshConfFile = "classPath://" + eventMeshConfFile; + ConfigService.getInstance().setRootConfig(eventMeshConfFile); + } + + ConfigService configService = ConfigService.getInstance(); + EventMeshTCPConfiguration eventMeshTCPConfiguration = configService.buildConfigInstance(EventMeshTCPConfiguration.class); + + assertCommonConfig(eventMeshTCPConfiguration); + assertTCPConfig(eventMeshTCPConfiguration); + } + + private void assertTCPConfig(EventMeshTCPConfiguration config) { + Assert.assertEquals(config.eventMeshTcpServerPort, 816); + Assert.assertEquals(config.eventMeshTcpIdleAllSeconds, 1816); + Assert.assertEquals(config.eventMeshTcpIdleWriteSeconds, 2816); + Assert.assertEquals(config.eventMeshTcpIdleReadSeconds, 3816); + Assert.assertEquals(config.eventMeshTcpMsgReqnumPerSecond, Integer.valueOf(4816)); + Assert.assertEquals(config.eventMeshTcpClientMaxNum, 5816); + Assert.assertEquals(config.eventMeshTcpGlobalScheduler, 6816); + Assert.assertEquals(config.eventMeshTcpTaskHandleExecutorPoolSize, 7816); + Assert.assertEquals(config.eventMeshTcpMsgDownStreamExecutorPoolSize, 8816); + Assert.assertEquals(config.eventMeshTcpSessionExpiredInMills, 1816); + Assert.assertEquals(config.eventMeshTcpSessionUpstreamBufferSize, 11816); + Assert.assertEquals(config.eventMeshTcpMsgAsyncRetryTimes, 12816); + Assert.assertEquals(config.eventMeshTcpMsgSyncRetryTimes, 13816); + Assert.assertEquals(config.eventMeshTcpMsgRetrySyncDelayInMills, 14816); + Assert.assertEquals(config.eventMeshTcpMsgRetryAsyncDelayInMills, 15816); + Assert.assertEquals(config.eventMeshTcpMsgRetryQueueSize, 16816); + Assert.assertEquals(config.eventMeshTcpRebalanceIntervalInMills, Integer.valueOf(17816)); + Assert.assertEquals(config.eventMeshServerAdminPort, 18816); + Assert.assertEquals(config.eventMeshTcpSendBackEnabled, Boolean.TRUE); + Assert.assertEquals(config.eventMeshTcpSendBackMaxTimes, 3); + Assert.assertEquals(config.eventMeshTcpPushFailIsolateTimeInMills, 21816); + Assert.assertEquals(config.gracefulShutdownSleepIntervalInMills, 22816); + Assert.assertEquals(config.sleepIntervalInRebalanceRedirectMills, 23816); + Assert.assertEquals(config.eventMeshEventSize, 22816); + Assert.assertEquals(config.eventMeshEventBatchSize, 23816); + } + + private void assertCommonConfig(CommonConfiguration config) { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + Assert.assertEquals("idc-succeed!!!", config.getEventMeshIDC()); + Assert.assertEquals("cluster-succeed!!!", config.getEventMeshCluster()); + Assert.assertEquals("name-succeed!!!", config.getEventMeshName()); + Assert.assertEquals("816", config.getSysID()); + Assert.assertEquals("connector-succeed!!!", config.getEventMeshConnectorPluginType()); + Assert.assertEquals("security-succeed!!!", config.getEventMeshSecurityPluginType()); + Assert.assertEquals("registry-succeed!!!", config.getEventMeshRegistryPluginType()); + Assert.assertEquals("trace-succeed!!!", config.getEventMeshTracePluginType()); + Assert.assertEquals("hostIp-succeed!!!", config.getEventMeshServerIp()); + + List list = new ArrayList<>(); + list.add("metrics-succeed1!!!"); + list.add("metrics-succeed2!!!"); + list.add("metrics-succeed3!!!"); + Assert.assertEquals(list, config.getEventMeshMetricsPluginType()); + + Assert.assertTrue(config.isEventMeshServerSecurityEnable()); + Assert.assertTrue(config.isEventMeshServerRegistryEnable()); + Assert.assertTrue(config.isEventMeshServerTraceEnable()); + + Assert.assertEquals("eventmesh.idc-succeed!!!", config.getEventMeshWebhookOrigin()); + } +} \ No newline at end of file diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/client/common/Server.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/client/common/Server.java index 3cc0899318..8c69fd522f 100644 --- a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/client/common/Server.java +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/client/common/Server.java @@ -17,9 +17,7 @@ package org.apache.eventmesh.runtime.client.common; -import org.apache.eventmesh.common.config.ConfigurationWrapper; import org.apache.eventmesh.runtime.boot.EventMeshServer; -import org.apache.eventmesh.runtime.constants.EventMeshConstants; public class Server { @@ -33,10 +31,8 @@ public class Server { } public void startAccessServer() throws Exception { - ConfigurationWrapper configurationWrapper = - new ConfigurationWrapper(EventMeshConstants.EVENTMESH_CONF_HOME, - EventMeshConstants.EVENTMESH_CONF_FILE, false); - new EventMeshServer(configurationWrapper).start(); + eventMeshServer = new EventMeshServer(); + eventMeshServer.start(); } public void shutdownAccessServer() throws Exception { diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfigurationTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfigurationTest.java new file mode 100644 index 0000000000..5f107ab3bc --- /dev/null +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshGrpcConfigurationTest.java @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.runtime.configuration; + +import org.apache.eventmesh.common.config.CommonConfiguration; +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +public class EventMeshGrpcConfigurationTest { + + @Test + public void testGetConfigForEventMeshGrpcConfiguration() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + + EventMeshGrpcConfiguration config = configService.buildConfigInstance(EventMeshGrpcConfiguration.class); + + assertCommonConfig(config); + assertGrpcConfig(config); + } + + private void assertGrpcConfig(EventMeshGrpcConfiguration config) { + Assert.assertEquals(config.getGrpcServerPort(), 816); + Assert.assertEquals(config.getEventMeshSessionExpiredInMills(), 1816); + Assert.assertEquals(config.isEventMeshServerBatchMsgBatchEnabled(), Boolean.FALSE); + Assert.assertEquals(config.getEventMeshServerBatchMsgThreadNum(), 2816); + Assert.assertEquals(config.getEventMeshServerSendMsgThreadNum(), 3816); + Assert.assertEquals(config.getEventMeshServerPushMsgThreadNum(), 4816); + Assert.assertEquals(config.getEventMeshServerReplyMsgThreadNum(), 5816); + Assert.assertEquals(config.getEventMeshServerSubscribeMsgThreadNum(), 6816); + Assert.assertEquals(config.getEventMeshServerRegistryThreadNum(), 7816); + Assert.assertEquals(config.getEventMeshServerAdminThreadNum(), 8816); + Assert.assertEquals(config.getEventMeshServerRetryThreadNum(), 9816); + Assert.assertEquals(config.getEventMeshServerPullRegistryInterval(), 11816); + Assert.assertEquals(config.getEventMeshServerAsyncAccumulationThreshold(), 12816); + Assert.assertEquals(config.getEventMeshServerRetryBlockQueueSize(), 13816); + Assert.assertEquals(config.getEventMeshServerBatchBlockQueueSize(), 14816); + Assert.assertEquals(config.getEventMeshServerSendMsgBlockQueueSize(), 15816); + Assert.assertEquals(config.getEventMeshServerPushMsgBlockQueueSize(), 16816); + Assert.assertEquals(config.getEventMeshServerSubscribeMsgBlockQueueSize(), 17816); + Assert.assertEquals(config.getEventMeshServerBusyCheckInterval(), 18816); + Assert.assertEquals(config.isEventMeshServerConsumerEnabled(), Boolean.TRUE); + Assert.assertEquals(config.isEventMeshServerUseTls(), Boolean.TRUE); + Assert.assertEquals(config.getEventMeshBatchMsgRequestNumPerSecond(), 21816); + Assert.assertEquals(config.getEventMeshMsgReqNumPerSecond(), 19816); + } + + private void assertCommonConfig(CommonConfiguration config) { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + Assert.assertEquals("idc-succeed!!!", config.getEventMeshIDC()); + Assert.assertEquals("cluster-succeed!!!", config.getEventMeshCluster()); + Assert.assertEquals("name-succeed!!!", config.getEventMeshName()); + Assert.assertEquals("816", config.getSysID()); + Assert.assertEquals("connector-succeed!!!", config.getEventMeshConnectorPluginType()); + Assert.assertEquals("security-succeed!!!", config.getEventMeshSecurityPluginType()); + Assert.assertEquals("registry-succeed!!!", config.getEventMeshRegistryPluginType()); + Assert.assertEquals("trace-succeed!!!", config.getEventMeshTracePluginType()); + Assert.assertEquals("hostIp-succeed!!!", config.getEventMeshServerIp()); + + List list = new ArrayList<>(); + list.add("metrics-succeed1!!!"); + list.add("metrics-succeed2!!!"); + list.add("metrics-succeed3!!!"); + Assert.assertEquals(list, config.getEventMeshMetricsPluginType()); + + Assert.assertTrue(config.isEventMeshServerSecurityEnable()); + Assert.assertTrue(config.isEventMeshServerRegistryEnable()); + Assert.assertTrue(config.isEventMeshServerTraceEnable()); + + Assert.assertEquals("eventmesh.idc-succeed!!!", config.getEventMeshWebhookOrigin()); + } +} \ No newline at end of file diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfigurationTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfigurationTest.java new file mode 100644 index 0000000000..72c0b5e14f --- /dev/null +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshHTTPConfigurationTest.java @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.runtime.configuration; + +import org.apache.eventmesh.common.config.CommonConfiguration; +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +import inet.ipaddr.AddressStringException; +import inet.ipaddr.IPAddress; +import inet.ipaddr.IPAddressString; + +public class EventMeshHTTPConfigurationTest { + + @Test + public void testGetEventMeshHTTPConfiguration() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + + EventMeshHTTPConfiguration config = configService.buildConfigInstance(EventMeshHTTPConfiguration.class); + + assertCommonConfig(config); + assertHTTPConfig(config); + } + + private void assertHTTPConfig(EventMeshHTTPConfiguration config) throws AddressStringException { + Assert.assertEquals(config.getHttpServerPort(), 1816); + Assert.assertEquals(config.isEventMeshServerBatchMsgBatchEnabled(), Boolean.FALSE); + Assert.assertEquals(config.getEventMeshServerBatchMsgThreadNum(), 2816); + Assert.assertEquals(config.getEventMeshServerSendMsgThreadNum(), 3816); + Assert.assertEquals(config.getEventMeshServerPushMsgThreadNum(), 4816); + Assert.assertEquals(config.getEventMeshServerReplyMsgThreadNum(), 5816); + Assert.assertEquals(config.getEventMeshServerClientManageThreadNum(), 6816); + Assert.assertEquals(config.getEventMeshServerRegistryThreadNum(), 7816); + Assert.assertEquals(config.getEventMeshServerAdminThreadNum(), 8816); + + Assert.assertEquals(config.getEventMeshServerRetryThreadNum(), 9816); + Assert.assertEquals(config.getEventMeshServerPullRegistryInterval(), 11816); + Assert.assertEquals(config.getEventMeshServerAsyncAccumulationThreshold(), 12816); + Assert.assertEquals(config.getEventMeshServerRetryBlockQSize(), 13816); + Assert.assertEquals(config.getEventMeshServerBatchBlockQSize(), 14816); + Assert.assertEquals(config.getEventMeshServerSendMsgBlockQSize(), 15816); + Assert.assertEquals(config.getEventMeshServerPushMsgBlockQSize(), 16816); + Assert.assertEquals(config.getEventMeshServerClientManageBlockQSize(), 17816); + Assert.assertEquals(config.getEventMeshServerBusyCheckInterval(), 18816); + Assert.assertEquals(config.isEventMeshServerConsumerEnabled(), Boolean.TRUE); + Assert.assertEquals(config.isEventMeshServerUseTls(), Boolean.TRUE); + Assert.assertEquals(config.getEventMeshHttpMsgReqNumPerSecond(), 19816); + Assert.assertEquals(config.getEventMeshBatchMsgRequestNumPerSecond(), 21816); + Assert.assertEquals(config.getEventMeshEventSize(), 22816); + Assert.assertEquals(config.getEventMeshEventBatchSize(), 23816); + + List list4 = new ArrayList<>(); + list4.add(new IPAddressString("127.0.0.1").toAddress()); + list4.add(new IPAddressString("127.0.0.2").toAddress()); + Assert.assertEquals(config.getEventMeshIpv4BlackList(), list4); + List list6 = new ArrayList<>(); + list6.add(new IPAddressString("0:0:0:0:0:0:7f00:01").toAddress()); + list6.add(new IPAddressString("0:0:0:0:0:0:7f00:02").toAddress()); + Assert.assertEquals(config.getEventMeshIpv6BlackList(), list6); + } + + private void assertCommonConfig(CommonConfiguration config) { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + Assert.assertEquals("idc-succeed!!!", config.getEventMeshIDC()); + Assert.assertEquals("cluster-succeed!!!", config.getEventMeshCluster()); + Assert.assertEquals("name-succeed!!!", config.getEventMeshName()); + Assert.assertEquals("816", config.getSysID()); + Assert.assertEquals("connector-succeed!!!", config.getEventMeshConnectorPluginType()); + Assert.assertEquals("security-succeed!!!", config.getEventMeshSecurityPluginType()); + Assert.assertEquals("registry-succeed!!!", config.getEventMeshRegistryPluginType()); + Assert.assertEquals("trace-succeed!!!", config.getEventMeshTracePluginType()); + Assert.assertEquals("hostIp-succeed!!!", config.getEventMeshServerIp()); + + List list = new ArrayList<>(); + list.add("metrics-succeed1!!!"); + list.add("metrics-succeed2!!!"); + list.add("metrics-succeed3!!!"); + Assert.assertEquals(list, config.getEventMeshMetricsPluginType()); + + Assert.assertTrue(config.isEventMeshServerSecurityEnable()); + Assert.assertTrue(config.isEventMeshServerRegistryEnable()); + Assert.assertTrue(config.isEventMeshServerTraceEnable()); + + Assert.assertEquals("eventmesh.idc-succeed!!!", config.getEventMeshWebhookOrigin()); + } +} \ No newline at end of file diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfigurationTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfigurationTest.java new file mode 100644 index 0000000000..a26617842c --- /dev/null +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/configuration/EventMeshTCPConfigurationTest.java @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.runtime.configuration; + +import org.apache.eventmesh.common.config.CommonConfiguration; +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +public class EventMeshTCPConfigurationTest { + + @Test + public void testGetEventMeshTCPConfiguration() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://configuration.properties"); + + EventMeshTCPConfiguration config = configService.buildConfigInstance(EventMeshTCPConfiguration.class); + + assertCommonConfig(config); + assertTCPConfig(config); + } + + private void assertTCPConfig(EventMeshTCPConfiguration config) { + Assert.assertEquals(config.eventMeshTcpServerPort, 816); + Assert.assertEquals(config.eventMeshTcpIdleAllSeconds, 1816); + Assert.assertEquals(config.eventMeshTcpIdleWriteSeconds, 2816); + Assert.assertEquals(config.eventMeshTcpIdleReadSeconds, 3816); + Assert.assertEquals(config.eventMeshTcpMsgReqnumPerSecond, Integer.valueOf(4816)); + Assert.assertEquals(config.eventMeshTcpClientMaxNum, 5816); + Assert.assertEquals(config.eventMeshTcpGlobalScheduler, 6816); + Assert.assertEquals(config.eventMeshTcpTaskHandleExecutorPoolSize, 7816); + Assert.assertEquals(config.eventMeshTcpMsgDownStreamExecutorPoolSize, 8816); + Assert.assertEquals(config.eventMeshTcpSessionExpiredInMills, 1816); + Assert.assertEquals(config.eventMeshTcpSessionUpstreamBufferSize, 11816); + Assert.assertEquals(config.eventMeshTcpMsgAsyncRetryTimes, 12816); + Assert.assertEquals(config.eventMeshTcpMsgSyncRetryTimes, 13816); + Assert.assertEquals(config.eventMeshTcpMsgRetrySyncDelayInMills, 14816); + Assert.assertEquals(config.eventMeshTcpMsgRetryAsyncDelayInMills, 15816); + Assert.assertEquals(config.eventMeshTcpMsgRetryQueueSize, 16816); + Assert.assertEquals(config.eventMeshTcpRebalanceIntervalInMills, Integer.valueOf(17816)); + Assert.assertEquals(config.eventMeshServerAdminPort, 18816); + Assert.assertEquals(config.eventMeshTcpSendBackEnabled, Boolean.TRUE); + Assert.assertEquals(config.eventMeshTcpSendBackMaxTimes, 3); + Assert.assertEquals(config.eventMeshTcpPushFailIsolateTimeInMills, 21816); + Assert.assertEquals(config.gracefulShutdownSleepIntervalInMills, 22816); + Assert.assertEquals(config.sleepIntervalInRebalanceRedirectMills, 23816); + Assert.assertEquals(config.eventMeshEventSize, 22816); + Assert.assertEquals(config.eventMeshEventBatchSize, 23816); + } + + private void assertCommonConfig(CommonConfiguration config) { + Assert.assertEquals("env-succeed!!!", config.getEventMeshEnv()); + Assert.assertEquals("idc-succeed!!!", config.getEventMeshIDC()); + Assert.assertEquals("cluster-succeed!!!", config.getEventMeshCluster()); + Assert.assertEquals("name-succeed!!!", config.getEventMeshName()); + Assert.assertEquals("816", config.getSysID()); + Assert.assertEquals("connector-succeed!!!", config.getEventMeshConnectorPluginType()); + Assert.assertEquals("security-succeed!!!", config.getEventMeshSecurityPluginType()); + Assert.assertEquals("registry-succeed!!!", config.getEventMeshRegistryPluginType()); + Assert.assertEquals("trace-succeed!!!", config.getEventMeshTracePluginType()); + Assert.assertEquals("hostIp-succeed!!!", config.getEventMeshServerIp()); + + List list = new ArrayList<>(); + list.add("metrics-succeed1!!!"); + list.add("metrics-succeed2!!!"); + list.add("metrics-succeed3!!!"); + Assert.assertEquals(list, config.getEventMeshMetricsPluginType()); + + Assert.assertTrue(config.isEventMeshServerSecurityEnable()); + Assert.assertTrue(config.isEventMeshServerRegistryEnable()); + Assert.assertTrue(config.isEventMeshServerTraceEnable()); + + Assert.assertEquals("eventmesh.idc-succeed!!!", config.getEventMeshWebhookOrigin()); + } +} \ No newline at end of file diff --git a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/util/HttpTinyClientTest.java b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/util/HttpTinyClientTest.java index 09e66657d3..bc09994d1f 100644 --- a/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/util/HttpTinyClientTest.java +++ b/eventmesh-runtime/src/test/java/org/apache/eventmesh/runtime/util/HttpTinyClientTest.java @@ -31,6 +31,7 @@ import java.util.List; import org.junit.Assert; +import org.junit.Ignore; import org.junit.Test; import org.mockito.MockedStatic; import org.mockito.Mockito; diff --git a/eventmesh-runtime/src/test/resources/configuration.properties b/eventmesh-runtime/src/test/resources/configuration.properties new file mode 100644 index 0000000000..190fd2fc5d --- /dev/null +++ b/eventmesh-runtime/src/test/resources/configuration.properties @@ -0,0 +1,106 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# CommonConfiguration config +eventMesh.server.env=env-succeed!!! +eventMesh.server.idc=idc-succeed!!! +eventMesh.sysid=816 +eventMesh.server.cluster=cluster-succeed!!! +eventMesh.server.name=name-succeed!!! +eventMesh.server.hostIp=hostIp-succeed!!! +eventMesh.connector.plugin.type=connector-succeed!!! +eventMesh.security.plugin.type=security-succeed!!! +eventMesh.registry.plugin.type=registry-succeed!!! +eventMesh.trace.plugin=trace-succeed!!! +eventMesh.server.registry.registerIntervalInMills=816 +eventMesh.server.registry.fetchRegistryAddrIntervalInMills=1816 +eventMesh.metrics.plugin=metrics-succeed1!!!,metrics-succeed2!!!,metrics-succeed3!!! +eventMesh.registry.plugin.server-addr=server-addr-succeed1!!! + +eventMesh.server.security.enabled=true +eventMesh.server.registry.enabled=true +eventMesh.server.trace.enabled=true + +eventMesh.server.provide.protocols=TCP,HTTP,GRPC +eventMesh.registry.plugin.username=username-succeed!!! +eventMesh.registry.plugin.password=password-succeed!!! + +# EventMeshHTTPConfiguration config +eventMesh.server.http.port=1816 +eventMesh.server.batchmsg.batch.enabled=false +eventMesh.server.batchmsg.threads.num=2816 +eventMesh.server.sendmsg.threads.num=3816 +eventMesh.server.pushmsg.threads.num=4816 +eventMesh.server.replymsg.threads.num=5816 +eventMesh.server.clientmanage.threads.num=6816 +eventMesh.server.registry.threads.num=7816 +eventMesh.server.admin.threads.num=8816 +eventMesh.server.retry.threads.num=9816 +eventMesh.server.pull.registry.interval=11816 +eventMesh.server.async.accumulation.threshold=12816 +eventMesh.server.retry.blockQ.size=13816 +eventMesh.server.batchmsg.blockQ.size=14816 +eventMesh.server.sendmsg.blockQ.size=15816 +eventMesh.server.pushmsg.blockQ.size=16816 +eventMesh.server.clientM.blockQ.size=17816 +eventMesh.server.busy.check.interval=18816 +eventMesh.server.consumer.enabled=true +eventMesh.server.useTls.enabled=true +eventMesh.server.http.msgReqnumPerSecond=19816 +eventMesh.server.batchmsg.reqNumPerSecond=21816 +eventMesh.server.maxEventSize=22816 +eventMesh.server.maxEventBatchSize=23816 +eventMesh.server.blacklist.ipv4=127.0.0.1,127.0.0.2 +eventMesh.server.blacklist.ipv6=0:0:0:0:0:0:7f00:01,0:0:0:0:0:0:7f00:02 + +# EventMeshGrpcConfiguration config +eventMesh.server.grpc.port=816 +eventMesh.server.session.expiredInMills=1816 + + +# EventMeshTCPConfiguration config +eventMesh.server.tcp.port=816 +eventMesh.server.tcp.allIdleSeconds=1816 +eventMesh.server.tcp.writerIdleSeconds=2816 +eventMesh.server.tcp.readerIdleSeconds=3816 +eventMesh.server.tcp.msgReqnumPerSecond=4816 +eventMesh.server.tcp.clientMaxNum=5816 +eventMesh.server.tcp.enabled=true +eventMesh.server.global.scheduler=6816 +eventMesh.server.tcp.taskHandleExecutorPoolSize=7816 +eventMesh.server.tcp.msgDownStreamExecutorPoolSize=8816 +eventMesh.server.session.upstreamBufferSize=11816 +eventMesh.server.retry.async.pushRetryTimes=12816 +eventMesh.server.retry.sync.pushRetryTimes=13816 +eventMesh.server.retry.sync.pushRetryDelayInMills=14816 +eventMesh.server.retry.async.pushRetryDelayInMills=15816 +eventMesh.server.retry.pushRetryQueueSize=16816 +eventMesh.server.tcp.RebalanceIntervalInMills=17816 +eventMesh.server.admin.http.port=18816 +eventMesh.server.tcp.sendBack.enabled=true +eventMesh.server.tcp.pushFailIsolateTimeInMills=21816 +eventMesh.server.gracefulShutdown.sleepIntervalInMills=22816 +eventMesh.server.rebalanceRedirect.sleepIntervalInM=23816 + + + + + + + + + diff --git a/eventmesh-sdk-java/src/main/java/org/apache/eventmesh/client/tcp/impl/eventmeshmessage/EventMeshMessageTCPSubClient.java b/eventmesh-sdk-java/src/main/java/org/apache/eventmesh/client/tcp/impl/eventmeshmessage/EventMeshMessageTCPSubClient.java index f0257050f5..93e4b92ae3 100644 --- a/eventmesh-sdk-java/src/main/java/org/apache/eventmesh/client/tcp/impl/eventmeshmessage/EventMeshMessageTCPSubClient.java +++ b/eventmesh-sdk-java/src/main/java/org/apache/eventmesh/client/tcp/impl/eventmeshmessage/EventMeshMessageTCPSubClient.java @@ -88,7 +88,7 @@ public void reconnect() throws EventMeshException { @Override public void subscribe(String topic, SubscriptionMode subscriptionMode, SubscriptionType subscriptionType) - throws EventMeshException { + throws EventMeshException { try { subscriptionItems.add(new SubscriptionItem(topic, subscriptionMode, subscriptionType)); Package request = MessageUtils.subscribe(topic, subscriptionMode, subscriptionType); @@ -146,7 +146,7 @@ public EventMeshMessage getProtocolMessage(Package tcpPackage) { public void callback(EventMeshMessage eventMeshMessage, ChannelHandlerContext ctx) { if (callback != null) { callback.handle(eventMeshMessage).ifPresent( - responseMessage -> ctx.writeAndFlush(MessageUtils.buildPackage(responseMessage, Command.RESPONSE_TO_SERVER)) + responseMessage -> ctx.writeAndFlush(MessageUtils.buildPackage(responseMessage, Command.RESPONSE_TO_SERVER)) ); } } diff --git a/eventmesh-security-plugin/eventmesh-security-api/src/main/java/org/apache/eventmesh/api/common/ConfigurationWrapper.java b/eventmesh-security-plugin/eventmesh-security-api/src/main/java/org/apache/eventmesh/api/common/ConfigurationWrapper.java deleted file mode 100644 index 8f07b05512..0000000000 --- a/eventmesh-security-plugin/eventmesh-security-api/src/main/java/org/apache/eventmesh/api/common/ConfigurationWrapper.java +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.api.common; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; -import java.io.IOException; -import java.net.URL; -import java.util.Properties; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class ConfigurationWrapper { - - private static final Logger LOGGER = LoggerFactory.getLogger(ConfigurationWrapper.class); - - private static final String EVENTMESH_CONFIG_HOME = System.getProperty("confPath", System.getenv("confPath")); - - public static Properties getConfig(String configFile) throws IOException { - String configFilePath; - - // get from classpath - URL resource = Thread.currentThread().getContextClassLoader().getResource(configFile); - if (resource != null && new File(resource.getPath()).exists()) { - configFilePath = resource.getPath(); - } else { - // get from config home - configFilePath = EVENTMESH_CONFIG_HOME + File.separator + configFile; - } - - if (LOGGER.isInfoEnabled()) { - LOGGER.info("loading auth config: {}", configFilePath); - } - - Properties properties = new Properties(); - try (BufferedReader br = new BufferedReader(new FileReader(configFilePath))) { - properties.load(br); - } - - return properties; - } -} diff --git a/eventmesh-security-plugin/eventmesh-security-api/src/test/java/org/apache/eventmesh/api/common/ConfigurationWrapperTest.java b/eventmesh-security-plugin/eventmesh-security-api/src/test/java/org/apache/eventmesh/api/common/ConfigurationWrapperTest.java deleted file mode 100644 index d517c912fb..0000000000 --- a/eventmesh-security-plugin/eventmesh-security-api/src/test/java/org/apache/eventmesh/api/common/ConfigurationWrapperTest.java +++ /dev/null @@ -1,53 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eventmesh.api.common; - -import java.net.URL; -import java.util.Properties; - -import org.junit.Assert; -import org.junit.Test; - -public class ConfigurationWrapperTest { - - @Test - public void testGetDefaultConfig() { - try { - URL resource = Thread.currentThread().getContextClassLoader().getResource("testpath"); - String directoryPath = resource.getPath(); - System.setProperty("confPath", directoryPath); - Properties p = ConfigurationWrapper.getConfig("test1.properties"); - String v = (String) p.get("a"); - Assert.assertEquals(v, "2"); - } catch (Exception e) { - Assert.fail(e.getMessage()); - } - - } - - @Test - public void testGetSpecifiedConfig() { - try { - Properties p = ConfigurationWrapper.getConfig("test.properties"); - String v = (String) p.get("a"); - Assert.assertEquals(v, "1"); - } catch (Exception e) { - Assert.fail(e.getMessage()); - } - } -} diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/build.gradle b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/build.gradle index 044b2e6b41..9a35c3b1b1 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/build.gradle +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/build.gradle @@ -18,6 +18,9 @@ dependencies { implementation project(":eventmesh-security-plugin:eventmesh-security-api") implementation "org.apache.commons:commons-lang3" + implementation project(":eventmesh-common") + compileOnly 'org.projectlombok:lombok:1.18.22' + annotationProcessor 'org.projectlombok:lombok:1.18.22' testImplementation project(":eventmesh-security-plugin:eventmesh-security-api") testImplementation "org.apache.commons:commons-lang3" diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigs.java b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigs.java index acc7379897..d9441c3a9a 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigs.java +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigs.java @@ -17,45 +17,18 @@ package org.apache.eventmesh.auth.http.basic.config; -import org.apache.eventmesh.api.common.ConfigurationWrapper; -import org.apache.eventmesh.api.exception.AuthException; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; -import java.io.IOException; -import java.util.Properties; +import lombok.Data; +@Data +@Config(prefix = "auth", path = "classPath://auth-http-basic.properties") public class AuthConfigs { - private static final String AUTH_CONFIG_FILE_NAME = "auth-http-basic.properties"; - - private static final String AUTH_CONFIG_KEY_USERNAME = "auth.username"; - - private static final String AUTH_CONFIG_KEY_PASSWORD = "auth.password"; - + @ConfigFiled(field = "username") private String username; - private String password; - - private static AuthConfigs instance; - - public static synchronized AuthConfigs getConfigs() throws AuthException { - try { - if (instance == null) { - Properties props = ConfigurationWrapper.getConfig(AUTH_CONFIG_FILE_NAME); - instance = new AuthConfigs(); - instance.username = props.getProperty(AUTH_CONFIG_KEY_USERNAME); - instance.password = props.getProperty(AUTH_CONFIG_KEY_PASSWORD); - } - return instance; - } catch (IOException e) { - throw new AuthException("getConfigs error", e); - } - } - - public String getUsername() { - return username; - } - - public String getPassword() { - return password; - } + @ConfigFiled(field = "password") + public String password; } diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicService.java b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicService.java index 42dad2c5cc..71075fd4cd 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicService.java +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/main/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicService.java @@ -20,6 +20,7 @@ import org.apache.eventmesh.api.auth.AuthService; import org.apache.eventmesh.api.exception.AuthException; import org.apache.eventmesh.auth.http.basic.config.AuthConfigs; +import org.apache.eventmesh.common.config.Config; import org.apache.commons.lang3.Validate; @@ -28,13 +29,17 @@ import java.util.HashMap; import java.util.Map; +@Config(field = "authConfigs") public class AuthHttpBasicService implements AuthService { + /** + * Unified configuration class corresponding to auth-http-basic.properties + */ private AuthConfigs authConfigs; @Override public void init() throws AuthException { - authConfigs = AuthConfigs.getConfigs(); + } @Override @@ -49,16 +54,17 @@ public void shutdown() throws AuthException { @Override public Map getAuthParams() throws AuthException { - if (authConfigs == null) { - init(); - } - - Validate.notNull(authConfigs); - String token = Base64.getEncoder().encodeToString((authConfigs.getUsername() + authConfigs.getPassword()) - .getBytes(StandardCharsets.UTF_8)); + String password = authConfigs.getPassword(); + String username = authConfigs.getUsername(); + String token = Base64.getEncoder() + .encodeToString((username + password).getBytes(StandardCharsets.UTF_8)); Map authParams = new HashMap<>(2); authParams.put("Authorization", "Basic " + token); return authParams; } + + public AuthConfigs getClientConfiguration() { + return this.authConfigs; + } } diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigsTest.java b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigsTest.java index 275530328d..7bfee74258 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigsTest.java +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/config/AuthConfigsTest.java @@ -17,17 +17,26 @@ package org.apache.eventmesh.auth.http.basic.config; +import org.apache.eventmesh.api.auth.AuthService; +import org.apache.eventmesh.auth.http.basic.impl.AuthHttpBasicService; +import org.apache.eventmesh.spi.EventMeshExtensionFactory; + import org.junit.Assert; import org.junit.Test; public class AuthConfigsTest { @Test - public void testGetConfigs() { - AuthConfigs configs = AuthConfigs.getConfigs(); - String password = configs.getPassword(); - String username = configs.getUsername(); - Assert.assertEquals(password, "password"); - Assert.assertEquals(username, "usera"); + public void getConfigWhenAuthHttpBasicServiceInit() { + AuthHttpBasicService authService = (AuthHttpBasicService) EventMeshExtensionFactory.getExtension( + AuthService.class, "auth-http-basic"); + + AuthConfigs config = authService.getClientConfiguration(); + assertConfig(config); + } + + private void assertConfig(AuthConfigs config) { + Assert.assertEquals(config.getUsername(), "username-success!!!"); + Assert.assertEquals(config.getPassword(), "password-success!!!"); } -} +} \ No newline at end of file diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicServiceTest.java b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicServiceTest.java index 0216252d16..f3b389089e 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicServiceTest.java +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/java/org/apache/eventmesh/auth/http/basic/impl/AuthHttpBasicServiceTest.java @@ -18,6 +18,9 @@ package org.apache.eventmesh.auth.http.basic.impl; +import org.apache.eventmesh.api.auth.AuthService; +import org.apache.eventmesh.spi.EventMeshExtensionFactory; + import java.util.Map; import org.junit.Assert; @@ -30,7 +33,8 @@ public class AuthHttpBasicServiceTest { @BeforeClass public static void beforeClass() { - service = new AuthHttpBasicService(); + service = (AuthHttpBasicService) EventMeshExtensionFactory.getExtension( + AuthService.class, "auth-http-basic"); } @Test diff --git a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/resources/auth-http-basic.properties b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/resources/auth-http-basic.properties index 367e5a4467..0e46ee68cb 100644 --- a/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/resources/auth-http-basic.properties +++ b/eventmesh-security-plugin/eventmesh-security-auth-http-basic/src/test/resources/auth-http-basic.properties @@ -14,5 +14,5 @@ # See the License for the specific language governing permissions and # limitations under the License. # -auth.username = usera -auth.password = password \ No newline at end of file +auth.username = username-success!!! +auth.password = password-success!!! \ No newline at end of file diff --git a/eventmesh-spi/src/main/java/org/apache/eventmesh/spi/EventMeshExtensionFactory.java b/eventmesh-spi/src/main/java/org/apache/eventmesh/spi/EventMeshExtensionFactory.java index 54d239619c..6de216b40d 100644 --- a/eventmesh-spi/src/main/java/org/apache/eventmesh/spi/EventMeshExtensionFactory.java +++ b/eventmesh-spi/src/main/java/org/apache/eventmesh/spi/EventMeshExtensionFactory.java @@ -17,12 +17,14 @@ package org.apache.eventmesh.spi; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.spi.loader.ExtensionClassLoader; import org.apache.eventmesh.spi.loader.JarExtensionClassLoader; import org.apache.eventmesh.spi.loader.MetaInfExtensionClassLoader; import org.apache.commons.lang3.StringUtils; +import java.io.IOException; import java.lang.reflect.InvocationTargetException; import java.util.ArrayList; import java.util.List; @@ -85,12 +87,18 @@ private static T getSingletonExtension(Class extensionType, String extens if (extensionInstanceClass == null) { return null; } - T extensionInstance = extensionInstanceClass.getDeclaredConstructor().newInstance(); + T extensionInstance = extensionInstanceClass.getDeclaredConstructor().newInstance(); + ConfigService.getInstance().populateConfigForObject(extensionInstance); + logger.info("initialize extension instance success, extensionType: {}, extensionInstanceName: {}", - extensionType, extensionInstanceName); + extensionType, extensionInstanceName); return extensionInstance; } catch (NoSuchMethodException | InstantiationException | IllegalAccessException | InvocationTargetException e) { throw new ExtensionException("Extension initialize error", e); + } catch (NoSuchFieldException | IOException e) { + logger.error("initialize extension instance config failed, extensionType: {}, extensionInstanceName: {}", + extensionType, extensionInstanceName, e); + throw new ExtensionException("Extension initialize error", e); } }); } @@ -102,11 +110,17 @@ private static T getPrototypeExtension(Class extensionType, String extens return null; } T extensionInstance = extensionInstanceClass.getDeclaredConstructor().newInstance(); + ConfigService.getInstance().populateConfigForObject(extensionInstance); + logger.info("initialize extension instance success, extensionType: {}, extensionName: {}", - extensionType, extensionInstanceName); + extensionType, extensionInstanceName); return extensionInstance; } catch (NoSuchMethodException | InstantiationException | IllegalAccessException | InvocationTargetException e) { throw new ExtensionException("Extension initialize error", e); + } catch (NoSuchFieldException | IOException e) { + logger.error("initialize extension instance config failed, extensionType: {}, extensionInstanceName: {}", + extensionType, extensionInstanceName, e); + throw new ExtensionException("Extension initialize error", e); } } diff --git a/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/common/EventMeshTraceConstants.java b/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/common/EventMeshTraceConstants.java index d1e2db093d..41c1f34192 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/common/EventMeshTraceConstants.java +++ b/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/common/EventMeshTraceConstants.java @@ -29,9 +29,4 @@ public class EventMeshTraceConstants { public static final String TRACE_DOWNSTREAM_EVENTMESH_CLIENT_SPAN = "downstream-eventmesh-client-span"; public static final String TRACE_EVENTMESH_SDK_SERVER_SPAN = "eventmesh-sdk-server-span"; - - public static final String TRACE_EVENTMESH_MAX_EXPORT_SIZE = "eventmesh.trace.max.export.size"; - public static final String TRACE_EVENTMESH_MAX_QUEUE_SIZE = "eventmesh.trace.max.queue.size"; - public static final String TRACE_EVENTMESH_EXPORT_TIMEOUT = "eventmesh.trace.export.timeout"; - public static final String TRACE_EVENTMESH_EXPORT_INTERVAL = "eventmesh.trace.export.interval"; } diff --git a/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/config/ExporterConfiguration.java b/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/config/ExporterConfiguration.java index ef07c2619e..b59eed4431 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/config/ExporterConfiguration.java +++ b/eventmesh-trace-plugin/eventmesh-trace-api/src/main/java/org/apache/eventmesh/trace/api/config/ExporterConfiguration.java @@ -17,101 +17,27 @@ package org.apache.eventmesh.trace.api.config; -import org.apache.eventmesh.common.utils.PropertiesUtils; -import org.apache.eventmesh.trace.api.common.EventMeshTraceConstants; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; -import org.apache.commons.lang3.StringUtils; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.net.URL; -import java.util.Properties; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; +import lombok.Data; /** * to load the properties form exporter.properties */ -@Slf4j -@UtilityClass +@Data +@Config(prefix = "eventmesh.trace", path = "classPath://exporter.properties") public class ExporterConfiguration { - private static final String CONFIG_FILE = "exporter.properties"; - private static final Properties properties = new Properties(); - + @ConfigFiled(field = "max.export.size") private int eventMeshTraceMaxExportSize = 512; - private int eventMeshTraceMaxQueueSize = 2048; - private int eventMeshTraceExportTimeout = 30; - private int eventMeshTraceExportInterval = 5; - - static { - loadProperties(); - initializeConfig(); - } - - public static int getEventMeshTraceMaxExportSize() { - return eventMeshTraceMaxExportSize; - } - public static int getEventMeshTraceMaxQueueSize() { - return eventMeshTraceMaxQueueSize; - } - - public static int getEventMeshTraceExportTimeout() { - return eventMeshTraceExportTimeout; - } - - public static int getEventMeshTraceExportInterval() { - return eventMeshTraceExportInterval; - } - - private void initializeConfig() { - String eventMeshTraceMaxExportSizeStr = properties.getProperty(EventMeshTraceConstants.TRACE_EVENTMESH_MAX_EXPORT_SIZE); - if (StringUtils.isNotEmpty(eventMeshTraceMaxExportSizeStr)) { - eventMeshTraceMaxExportSize = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshTraceMaxExportSizeStr)); - } - - String eventMeshTraceMaxQueueSizeStr = properties.getProperty(EventMeshTraceConstants.TRACE_EVENTMESH_MAX_QUEUE_SIZE); - if (StringUtils.isNotEmpty(eventMeshTraceMaxQueueSizeStr)) { - eventMeshTraceMaxQueueSize = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshTraceMaxQueueSizeStr)); - } - - String eventMeshTraceExportTimeoutStr = properties.getProperty(EventMeshTraceConstants.TRACE_EVENTMESH_EXPORT_TIMEOUT); - if (StringUtils.isNotEmpty(eventMeshTraceExportTimeoutStr)) { - eventMeshTraceExportTimeout = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshTraceExportTimeoutStr)); - } + @ConfigFiled(field = "max.queue.size") + private int eventMeshTraceMaxQueueSize = 2048; - String eventMeshTraceExportIntervalStr = properties.getProperty(EventMeshTraceConstants.TRACE_EVENTMESH_EXPORT_INTERVAL); - if (StringUtils.isNotEmpty(eventMeshTraceExportIntervalStr)) { - eventMeshTraceExportInterval = - Integer.parseInt(StringUtils.deleteWhitespace(eventMeshTraceExportIntervalStr)); - } - } + @ConfigFiled(field = "export.timeout") + private int eventMeshTraceExportTimeout = 30; - private void loadProperties() { - URL resource = ExporterConfiguration.class.getClassLoader().getResource(CONFIG_FILE); - if (resource != null) { - try (InputStream inputStream = resource.openStream()) { - if (inputStream.available() > 0) { - properties.load(new BufferedReader(new InputStreamReader(inputStream))); - } - } catch (IOException e) { - throw new RuntimeException("Load exporter.properties file from classpath error"); - } - } - // get from config home - try { - String configPath = System.getProperty("confPath", System.getenv("confPath")) + File.separator + CONFIG_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } catch (IOException e) { - throw new IllegalArgumentException("Cannot load exporter.properties file from conf"); - } - } + @ConfigFiled(field = "export.interval") + private int eventMeshTraceExportInterval = 5; } diff --git a/eventmesh-trace-plugin/eventmesh-trace-api/src/test/java/org/apache/eventmesh/trace/api/config/ExporterConfigurationTest.java b/eventmesh-trace-plugin/eventmesh-trace-api/src/test/java/org/apache/eventmesh/trace/api/config/ExporterConfigurationTest.java new file mode 100644 index 0000000000..a0900cf05a --- /dev/null +++ b/eventmesh-trace-plugin/eventmesh-trace-api/src/test/java/org/apache/eventmesh/trace/api/config/ExporterConfigurationTest.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.trace.api.config; + +import org.apache.eventmesh.common.config.ConfigService; + +import org.junit.Assert; +import org.junit.Test; + +public class ExporterConfigurationTest { + + @Test + public void testGetExporterConfiguration() { + ConfigService configService = ConfigService.getInstance(); + ExporterConfiguration config = configService.buildConfigInstance(ExporterConfiguration.class); + + Assert.assertEquals(816, config.getEventMeshTraceMaxExportSize()); + Assert.assertEquals(1816, config.getEventMeshTraceMaxQueueSize()); + Assert.assertEquals(2816, config.getEventMeshTraceExportTimeout()); + Assert.assertEquals(3816, config.getEventMeshTraceExportInterval()); + } +} \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-api/src/test/resources/exporter.properties b/eventmesh-trace-plugin/eventmesh-trace-api/src/test/resources/exporter.properties new file mode 100644 index 0000000000..a951d48918 --- /dev/null +++ b/eventmesh-trace-plugin/eventmesh-trace-api/src/test/resources/exporter.properties @@ -0,0 +1,25 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +#set the maximum batch size to use +eventmesh.trace.max.export.size=816 +#set the queue size. This must be >= the export batch size +eventmesh.trace.max.queue.size=1816 +#set the max amount of time an export can run before getting(TimeUnit=SECONDS) +eventmesh.trace.export.timeout=2816 +#set time between two different exports(TimeUnit=SECONDS) +eventmesh.trace.export.interval=3816 \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/JaegerTraceService.java b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/JaegerTraceService.java index 78bb747583..bd0ad8c527 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/JaegerTraceService.java +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/JaegerTraceService.java @@ -19,6 +19,7 @@ import static io.opentelemetry.api.common.AttributeKey.stringKey; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.trace.api.EventMeshTraceService; import org.apache.eventmesh.trace.api.config.ExporterConfiguration; import org.apache.eventmesh.trace.api.exception.TraceException; @@ -49,6 +50,8 @@ import io.opentelemetry.sdk.trace.SpanProcessor; import io.opentelemetry.sdk.trace.export.BatchSpanProcessor; +@Config(field = "jaegerConfiguration") +@Config(field = "exporterConfiguration") public class JaegerTraceService implements EventMeshTraceService { private String eventMeshJaegerIp; @@ -73,16 +76,26 @@ public class JaegerTraceService implements EventMeshTraceService { private TextMapPropagator textMapPropagator; + /** + * Unified configuration class corresponding to jaeger.properties + */ + private JaegerConfiguration jaegerConfiguration; + + /** + * Unified configuration class corresponding to exporter.properties + */ + private ExporterConfiguration exporterConfiguration; + @Override public void init() throws TraceException { // jaeger's config - eventMeshJaegerIp = JaegerConfiguration.getEventMeshJaegerIp(); - eventMeshJaegerPort = JaegerConfiguration.getEventMeshJaegerPort(); + eventMeshJaegerIp = jaegerConfiguration.getEventMeshJaegerIp(); + eventMeshJaegerPort = jaegerConfiguration.getEventMeshJaegerPort(); // exporter's config - eventMeshTraceExportInterval = ExporterConfiguration.getEventMeshTraceExportInterval(); - eventMeshTraceExportTimeout = ExporterConfiguration.getEventMeshTraceExportTimeout(); - eventMeshTraceMaxExportSize = ExporterConfiguration.getEventMeshTraceMaxExportSize(); - eventMeshTraceMaxQueueSize = ExporterConfiguration.getEventMeshTraceMaxQueueSize(); + eventMeshTraceExportInterval = exporterConfiguration.getEventMeshTraceExportInterval(); + eventMeshTraceExportTimeout = exporterConfiguration.getEventMeshTraceExportTimeout(); + eventMeshTraceMaxExportSize = exporterConfiguration.getEventMeshTraceMaxExportSize(); + eventMeshTraceMaxQueueSize = exporterConfiguration.getEventMeshTraceMaxQueueSize(); String httpEndpoint = String.format("http://%s:%s", eventMeshJaegerIp, eventMeshJaegerPort); JaegerGrpcSpanExporter jaegerExporter = JaegerGrpcSpanExporter.builder() @@ -165,4 +178,12 @@ public Span createSpan(String spanName, SpanKind spanKind, Context context, bool public void shutdown() throws TraceException { sdkTracerProvider.close(); } + + public JaegerConfiguration getClientConfiguration() { + return this.jaegerConfiguration; + } + + public ExporterConfiguration getExporterConfiguration() { + return this.exporterConfiguration; + } } \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/common/JaegerConstants.java b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/common/JaegerConstants.java index 775cd89939..622b637744 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/common/JaegerConstants.java +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/common/JaegerConstants.java @@ -20,8 +20,4 @@ public class JaegerConstants { public static final String SERVICE_NAME = "eventmesh_trace"; - - public static final String KEY_JAEGER_IP = "eventmesh.trace.jaeger.ip"; - - public static final String KEY_JAEGER_PORT = "eventmesh.trace.jaeger.port"; } \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfiguration.java b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfiguration.java index f08feac869..1e60ca665f 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfiguration.java +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/main/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfiguration.java @@ -17,81 +17,18 @@ package org.apache.eventmesh.trace.jaeger.config; -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; -import org.apache.eventmesh.trace.jaeger.common.JaegerConstants; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; -import org.apache.commons.lang3.StringUtils; +import lombok.Data; -import java.io.BufferedReader; -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.net.URL; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import com.google.common.base.Preconditions; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; - -@Slf4j -@UtilityClass +@Data +@Config(prefix = "eventmesh.trace.jaeger", path = "classPath://jaeger.properties") public class JaegerConfiguration { - private static final String CONFIG_FILE = "jaeger.properties"; - - private static final Properties PROPERTIES = new Properties(); - + @ConfigFiled(field = "ip", notEmpty = true) private String eventMeshJaegerIp = "localhost"; + @ConfigFiled(field = "port", notEmpty = true) private int eventMeshJaegerPort = 14250; - - static { - loadProperties(); - initializeConfig(); - } - - private void loadProperties() { - URL resource = JaegerConfiguration.class.getClassLoader().getResource(CONFIG_FILE); - if (resource != null) { - try (InputStream inputStream = resource.openStream(); - BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8))) { - if (inputStream.available() > 0) { - PROPERTIES.load(reader); - } - } catch (IOException e) { - throw new RuntimeException("Load jaeger.properties file from classpath error", e); - } - } - // get from config home - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONFIG_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(PROPERTIES, configPath); - } catch (IOException e) { - throw new IllegalArgumentException("Cannot load jaeger.properties file from conf", e); - } - } - - private void initializeConfig() { - String jaegerIp = PROPERTIES.getProperty(JaegerConstants.KEY_JAEGER_IP); - Preconditions.checkState(StringUtils.isNotEmpty(jaegerIp), - String.format("%s error", JaegerConstants.KEY_JAEGER_IP)); - eventMeshJaegerIp = StringUtils.deleteWhitespace(jaegerIp); - - String jaegerPort = PROPERTIES.getProperty(JaegerConstants.KEY_JAEGER_PORT); - if (StringUtils.isNotEmpty(jaegerPort)) { - eventMeshJaegerPort = Integer.parseInt(StringUtils.deleteWhitespace(jaegerPort)); - } - } - - public static String getEventMeshJaegerIp() { - return eventMeshJaegerIp; - } - - public static int getEventMeshJaegerPort() { - return eventMeshJaegerPort; - } } \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/JaegerTraceServiceTest.java b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/JaegerTraceServiceTest.java index 19e1702bcd..b3b4756c46 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/JaegerTraceServiceTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/JaegerTraceServiceTest.java @@ -19,6 +19,8 @@ import static org.junit.Assert.assertThrows; +import org.apache.eventmesh.trace.api.TracePluginFactory; + import java.lang.reflect.Field; import org.junit.Assert; @@ -31,7 +33,8 @@ public class JaegerTraceServiceTest { @Test public void testInit() { - JaegerTraceService jaegerTraceService = new JaegerTraceService(); + JaegerTraceService jaegerTraceService = + (JaegerTraceService) TracePluginFactory.getEventMeshTraceService("jaeger"); jaegerTraceService.init(); Assert.assertNotNull(jaegerTraceService.sdkTracerProvider); @@ -47,7 +50,8 @@ public void testInit() { @Test public void testShutdown() throws NoSuchFieldException, IllegalAccessException { SdkTracerProvider mockSdkTracerProvider = Mockito.mock(SdkTracerProvider.class); - JaegerTraceService jaegerTraceService = new JaegerTraceService(); + JaegerTraceService jaegerTraceService = + (JaegerTraceService) TracePluginFactory.getEventMeshTraceService("jaeger"); jaegerTraceService.init(); Field sdkTracerProviderField = JaegerTraceService.class.getDeclaredField("sdkTracerProvider"); sdkTracerProviderField.setAccessible(true); diff --git a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfigurationTest.java b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfigurationTest.java index 7f1245fe0a..03e62c89c1 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfigurationTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/java/org/apache/eventmesh/trace/jaeger/config/JaegerConfigurationTest.java @@ -17,6 +17,10 @@ package org.apache.eventmesh.trace.jaeger.config; +import org.apache.eventmesh.trace.api.TracePluginFactory; +import org.apache.eventmesh.trace.api.config.ExporterConfiguration; +import org.apache.eventmesh.trace.jaeger.JaegerTraceService; + import org.junit.Assert; import org.junit.Test; @@ -24,7 +28,24 @@ public class JaegerConfigurationTest { @Test public void testGetConfiguration() { - Assert.assertEquals("localhost", JaegerConfiguration.getEventMeshJaegerIp()); - Assert.assertEquals(14250, JaegerConfiguration.getEventMeshJaegerPort()); + JaegerTraceService jaegerTrace = + (JaegerTraceService) TracePluginFactory.getEventMeshTraceService("jaeger"); + + JaegerConfiguration config = jaegerTrace.getClientConfiguration(); + assertClientConfig(config); + ExporterConfiguration exporterConfig = jaegerTrace.getExporterConfiguration(); + assertBaseConfig(exporterConfig); + } + + private void assertClientConfig(JaegerConfiguration config) { + Assert.assertEquals("localhost", config.getEventMeshJaegerIp()); + Assert.assertEquals(14250, config.getEventMeshJaegerPort()); + } + + private void assertBaseConfig(ExporterConfiguration config) { + Assert.assertEquals(816, config.getEventMeshTraceMaxExportSize()); + Assert.assertEquals(1816, config.getEventMeshTraceMaxQueueSize()); + Assert.assertEquals(2816, config.getEventMeshTraceExportTimeout()); + Assert.assertEquals(3816, config.getEventMeshTraceExportInterval()); } } \ No newline at end of file diff --git a/eventmesh-security-plugin/eventmesh-security-api/src/test/resources/testpath/test1.properties b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/resources/exporter.properties similarity index 84% rename from eventmesh-security-plugin/eventmesh-security-api/src/test/resources/testpath/test1.properties rename to eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/resources/exporter.properties index 9dc12ee523..14f6980df0 100644 --- a/eventmesh-security-plugin/eventmesh-security-api/src/test/resources/testpath/test1.properties +++ b/eventmesh-trace-plugin/eventmesh-trace-jaeger/src/test/resources/exporter.properties @@ -14,4 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. # -a=2 \ No newline at end of file +eventmesh.trace.max.export.size=816 +eventmesh.trace.max.queue.size=1816 +eventmesh.trace.export.timeout=2816 +eventmesh.trace.export.interval=3816 \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/build.gradle b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/build.gradle index 63e892e554..3b7409252b 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/build.gradle +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/build.gradle @@ -22,6 +22,9 @@ dependencies { implementation project(":eventmesh-common") implementation 'org.slf4j:slf4j-api' + compileOnly 'org.projectlombok:lombok:1.18.22' + annotationProcessor 'org.projectlombok:lombok:1.18.22' + implementation 'io.opentelemetry:opentelemetry-api' implementation 'io.opentelemetry:opentelemetry-semconv' implementation 'io.opentelemetry:opentelemetry-sdk' diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceService.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceService.java index f0e0217dc8..bde6490334 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceService.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceService.java @@ -17,6 +17,7 @@ package org.apache.eventmesh.trace.pinpoint; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.trace.api.EventMeshTraceService; import org.apache.eventmesh.trace.api.config.ExporterConfiguration; import org.apache.eventmesh.trace.api.exception.TraceException; @@ -48,6 +49,8 @@ /** * https://github.com/pinpoint-apm/pinpoint */ +@Config(field = "pinpointConfiguration") +@Config(field = "exporterConfiguration") public class PinpointTraceService implements EventMeshTraceService { private SdkTracerProvider sdkTracerProvider; @@ -58,20 +61,29 @@ public class PinpointTraceService implements EventMeshTraceService { protected Thread shutdownHook; + /** + * Unified configuration class corresponding to pinpoint.properties + */ + private PinpointConfiguration pinpointConfiguration; + + /** + * Unified configuration class corresponding to exporter.properties + */ + private ExporterConfiguration exporterConfiguration; @Override public void init() throws TraceException { - long eventMeshTraceExportInterval = ExporterConfiguration.getEventMeshTraceExportInterval(); - long eventMeshTraceExportTimeout = ExporterConfiguration.getEventMeshTraceExportTimeout(); - int eventMeshTraceMaxExportSize = ExporterConfiguration.getEventMeshTraceMaxExportSize(); - int eventMeshTraceMaxQueueSize = ExporterConfiguration.getEventMeshTraceMaxQueueSize(); + long eventMeshTraceExportInterval = exporterConfiguration.getEventMeshTraceExportInterval(); + long eventMeshTraceExportTimeout = exporterConfiguration.getEventMeshTraceExportTimeout(); + int eventMeshTraceMaxExportSize = exporterConfiguration.getEventMeshTraceMaxExportSize(); + int eventMeshTraceMaxQueueSize = exporterConfiguration.getEventMeshTraceMaxQueueSize(); SpanProcessor spanProcessor = BatchSpanProcessor.builder( new PinpointSpanExporter( - PinpointConfiguration.getAgentId(), - PinpointConfiguration.getAgentName(), - PinpointConfiguration.getApplicationName(), - PinpointConfiguration.getGrpcTransportConfig())) + pinpointConfiguration.getAgentId(), + pinpointConfiguration.getAgentName(), + pinpointConfiguration.getApplicationName(), + pinpointConfiguration.getGrpcTransportConfig())) .setScheduleDelay(eventMeshTraceExportInterval, TimeUnit.SECONDS) .setExporterTimeout(eventMeshTraceExportTimeout, TimeUnit.SECONDS) .setMaxExportBatchSize(eventMeshTraceMaxExportSize) @@ -148,4 +160,12 @@ public Span createSpan(String spanName, SpanKind spanKind, Context context, bool public void shutdown() throws TraceException { sdkTracerProvider.close(); } + + public PinpointConfiguration getClientConfiguration() { + return this.pinpointConfiguration; + } + + public ExporterConfiguration getExporterConfiguration() { + return this.exporterConfiguration; + } } diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/common/PinpointConstants.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/common/PinpointConstants.java index 8ccf5affc0..9ebf68fd70 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/common/PinpointConstants.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/common/PinpointConstants.java @@ -17,22 +17,10 @@ package org.apache.eventmesh.trace.pinpoint.common; -import org.apache.eventmesh.common.Constants; - public class PinpointConstants { public static final String SERVICE_NAME = "eventmesh_trace"; - public static final String PROPERTY_KEY_PREFIX = "eventmesh.trace.pinpoint"; - - public static final String AGENT_ID_KEY = PROPERTY_KEY_PREFIX + Constants.DOT + "agentId"; - - public static final String AGENT_NAME_KEY = PROPERTY_KEY_PREFIX + Constants.DOT + "agentName"; - - public static final String APPLICATION_NAME = "applicationName"; - - public static final String APPLICATION_NAME_KEY = PROPERTY_KEY_PREFIX + Constants.DOT + APPLICATION_NAME; - public static final String REQ_IP = "req0ip"; public static final String UNKNOWN_REQ_IP = "unknown"; diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfiguration.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfiguration.java index 14d2254b9a..31596eb3b0 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfiguration.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/main/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfiguration.java @@ -17,17 +17,10 @@ package org.apache.eventmesh.trace.pinpoint.config; -import static org.apache.eventmesh.trace.pinpoint.common.PinpointConstants.AGENT_ID_KEY; -import static org.apache.eventmesh.trace.pinpoint.common.PinpointConstants.AGENT_NAME_KEY; -import static org.apache.eventmesh.trace.pinpoint.common.PinpointConstants.APPLICATION_NAME; -import static org.apache.eventmesh.trace.pinpoint.common.PinpointConstants.APPLICATION_NAME_KEY; -import static org.apache.eventmesh.trace.pinpoint.common.PinpointConstants.PROPERTY_KEY_PREFIX; - -import static java.util.Objects.requireNonNull; - import org.apache.eventmesh.common.Constants; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; import org.apache.eventmesh.common.exception.JsonException; -import org.apache.eventmesh.common.utils.PropertiesUtils; import org.apache.eventmesh.common.utils.RandomStringUtils; import org.apache.commons.lang3.StringUtils; @@ -46,56 +39,34 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.navercorp.pinpoint.profiler.context.grpc.config.GrpcTransportConfig; -public final class PinpointConfiguration { - - private static final String CONFIG_FILE = "pinpoint.properties"; - - private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper() - .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); - - private static final Properties properties = new Properties(); - - private static String agentId; - private static String agentName; - private static String applicationName; - private static GrpcTransportConfig grpcTransportConfig; +import lombok.Data; - static { - loadProperties(); - initializeConfig(); - } +@Data +@Config(prefix = "eventmesh.trace.pinpoint", path = "classPath://pinpoint.properties") +public final class PinpointConfiguration { - public static String getAgentId() { - return agentId; - } + @ConfigFiled(field = "agentId", reload = true) + private String agentId; - public static String getAgentName() { - return agentName; - } + @ConfigFiled(field = "agentName", reload = true) + private String agentName; - public static String getApplicationName() { - return applicationName; - } + @ConfigFiled(field = "applicationName", findEnv = true, notNull = true) + private String applicationName; - public static GrpcTransportConfig getGrpcTransportConfig() { - return grpcTransportConfig; - } + @ConfigFiled(field = "", reload = true) + private Properties grpcTransportProperties; - private static void initializeConfig() { - applicationName = properties.getProperty(APPLICATION_NAME_KEY); - if (StringUtils.isBlank(applicationName)) { - applicationName = Optional.ofNullable(System.getProperty(APPLICATION_NAME)) - .orElseGet(() -> System.getenv(APPLICATION_NAME)); - } + private GrpcTransportConfig grpcTransportConfig; - requireNonNull(applicationName, String.format("%s can not be null", APPLICATION_NAME_KEY)); + private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper() + .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); - agentName = properties.getProperty(AGENT_NAME_KEY); + public void reload() { if (StringUtils.isBlank(agentName)) { agentName = applicationName; } - agentId = properties.getProperty(AGENT_ID_KEY); if (StringUtils.isBlank(agentId)) { // refer to: com.navercorp.pinpoint.common.util.IdValidateUtils#validateId agentId = StringUtils.substring(agentName, 0, 15) @@ -103,32 +74,8 @@ private static void initializeConfig() { + RandomStringUtils.generateNum(8); } - Properties temporary = PropertiesUtils.getPropertiesByPrefix(properties, PROPERTY_KEY_PREFIX); - // Map to Pinpoint property configuration. - grpcTransportConfig = convertValue(temporary, GrpcTransportConfig.class); - } - - private static void loadProperties() { - URL resource = PinpointConfiguration.class.getClassLoader().getResource(CONFIG_FILE); - if (resource != null) { - try (InputStream inputStream = resource.openStream(); - BufferedReader reader = new BufferedReader( - new InputStreamReader(inputStream, StandardCharsets.UTF_8))) { - if (inputStream.available() > 0) { - properties.load(reader); - } - } catch (IOException e) { - throw new RuntimeException(String.format("Load %s file from classpath error", CONFIG_FILE)); - } - } - // get from config home - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONFIG_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath, StandardCharsets.UTF_8); - } catch (IOException e) { - throw new IllegalArgumentException(String.format("Can not load %s file from conf", CONFIG_FILE)); - } + grpcTransportConfig = convertValue(grpcTransportProperties, GrpcTransportConfig.class); } public static T convertValue(Object fromValue, Class toValueType) { diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceServiceTest.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceServiceTest.java index c14ee5986a..b49d7ca588 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceServiceTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/PinpointTraceServiceTest.java @@ -19,6 +19,8 @@ import static org.junit.Assert.assertThrows; +import org.apache.eventmesh.trace.api.TracePluginFactory; + import java.lang.reflect.Field; import org.junit.Assert; @@ -31,7 +33,8 @@ public class PinpointTraceServiceTest { @Test public void testInit() { - PinpointTraceService pinpointTraceService = new PinpointTraceService(); + PinpointTraceService pinpointTraceService = + (PinpointTraceService) TracePluginFactory.getEventMeshTraceService("pinpoint"); pinpointTraceService.init(); IllegalArgumentException illegalArgumentException = @@ -43,7 +46,9 @@ public void testInit() { @Test public void testShutdown() throws Exception { SdkTracerProvider mockSdkTracerProvider = Mockito.mock(SdkTracerProvider.class); - PinpointTraceService pinpointTraceService = new PinpointTraceService(); + + PinpointTraceService pinpointTraceService = + (PinpointTraceService) TracePluginFactory.getEventMeshTraceService("pinpoint"); pinpointTraceService.init(); Field sdkTracerProviderField = PinpointTraceService.class.getDeclaredField("sdkTracerProvider"); sdkTracerProviderField.setAccessible(true); diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfigurationTest.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfigurationTest.java index 6388be27c6..ca0cf070e6 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfigurationTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/config/PinpointConfigurationTest.java @@ -17,10 +17,10 @@ package org.apache.eventmesh.trace.pinpoint.config; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getAgentId; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getAgentName; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getApplicationName; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getGrpcTransportConfig; + +import org.apache.eventmesh.trace.api.TracePluginFactory; +import org.apache.eventmesh.trace.api.config.ExporterConfiguration; +import org.apache.eventmesh.trace.pinpoint.PinpointTraceService; import org.junit.Assert; import org.junit.Test; @@ -30,11 +30,22 @@ public class PinpointConfigurationTest { @Test - public void testCase() { - Assert.assertEquals("eventmesh", getApplicationName()); - Assert.assertEquals("eventmesh", getAgentName()); - Assert.assertEquals("eventmesh-01", getAgentId()); - GrpcTransportConfig grpcTransportConfig = getGrpcTransportConfig(); + public void getConfigWhenPinpointTraceInit() { + PinpointTraceService pinpointTrace = + (PinpointTraceService) TracePluginFactory.getEventMeshTraceService("pinpoint"); + + PinpointConfiguration config = pinpointTrace.getClientConfiguration(); + assertClientConfig(config); + ExporterConfiguration exporterConfig = pinpointTrace.getExporterConfiguration(); + assertBaseConfig(exporterConfig); + } + + private void assertClientConfig(PinpointConfiguration config) { + Assert.assertEquals("eventmesh", config.getApplicationName()); + Assert.assertEquals("eventmesh", config.getAgentName()); + Assert.assertEquals("eventmesh-01", config.getAgentId()); + + GrpcTransportConfig grpcTransportConfig = config.getGrpcTransportConfig(); Assert.assertNotNull(grpcTransportConfig); Assert.assertEquals("127.0.0.1", grpcTransportConfig.getAgentCollectorIp()); Assert.assertEquals(9991, grpcTransportConfig.getAgentCollectorPort()); @@ -44,4 +55,11 @@ public void testCase() { Assert.assertEquals(123, grpcTransportConfig.getSpanClientOption().getLimitCount()); Assert.assertEquals(6700, grpcTransportConfig.getSpanClientOption().getLimitTime()); } + + private void assertBaseConfig(ExporterConfiguration config) { + Assert.assertEquals(816, config.getEventMeshTraceMaxExportSize()); + Assert.assertEquals(1816, config.getEventMeshTraceMaxQueueSize()); + Assert.assertEquals(2816, config.getEventMeshTraceExportTimeout()); + Assert.assertEquals(3816, config.getEventMeshTraceExportInterval()); + } } \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/exporter/PinpointSpanExporterTest.java b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/exporter/PinpointSpanExporterTest.java index 028049fdea..3284f9d413 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/exporter/PinpointSpanExporterTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/java/org/apache/eventmesh/trace/pinpoint/exporter/PinpointSpanExporterTest.java @@ -17,12 +17,11 @@ package org.apache.eventmesh.trace.pinpoint.exporter; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getAgentId; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getAgentName; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getApplicationName; -import static org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration.getGrpcTransportConfig; import org.apache.eventmesh.common.utils.RandomStringUtils; +import org.apache.eventmesh.trace.api.TracePluginFactory; +import org.apache.eventmesh.trace.pinpoint.PinpointTraceService; +import org.apache.eventmesh.trace.pinpoint.config.PinpointConfiguration; import java.util.ArrayList; import java.util.Collection; @@ -52,11 +51,16 @@ public class PinpointSpanExporterTest { @Before public void setup() { + PinpointTraceService pinpointTrace = + (PinpointTraceService) TracePluginFactory.getEventMeshTraceService("pinpoint"); + + PinpointConfiguration config = pinpointTrace.getClientConfiguration(); + this.exporter = new PinpointSpanExporter( - getAgentId(), - getAgentName(), - getApplicationName(), - getGrpcTransportConfig() + config.getAgentId(), + config.getAgentName(), + config.getApplicationName(), + config.getGrpcTransportConfig() ); } diff --git a/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/resources/exporter.properties b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/resources/exporter.properties new file mode 100644 index 0000000000..14f6980df0 --- /dev/null +++ b/eventmesh-trace-plugin/eventmesh-trace-pinpoint/src/test/resources/exporter.properties @@ -0,0 +1,20 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +eventmesh.trace.max.export.size=816 +eventmesh.trace.max.queue.size=1816 +eventmesh.trace.export.timeout=2816 +eventmesh.trace.export.interval=3816 \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceService.java b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceService.java index fcac705842..6eda020fc3 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceService.java +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceService.java @@ -19,6 +19,7 @@ import static io.opentelemetry.api.common.AttributeKey.stringKey; +import org.apache.eventmesh.common.config.Config; import org.apache.eventmesh.trace.api.EventMeshTraceService; import org.apache.eventmesh.trace.api.config.ExporterConfiguration; import org.apache.eventmesh.trace.api.exception.TraceException; @@ -51,8 +52,10 @@ /** - * + * ZipkinTraceService */ +@Config(field = "zipkinConfiguration") +@Config(field = "exporterConfiguration") public class ZipkinTraceService implements EventMeshTraceService { private String eventMeshZipkinIP; private int eventMeshZipkinPort; @@ -69,16 +72,27 @@ public class ZipkinTraceService implements EventMeshTraceService { private Tracer tracer; private TextMapPropagator textMapPropagator; + /** + * Unified configuration class corresponding to zipkin.properties + */ + private ZipkinConfiguration zipkinConfiguration; + + /** + * Unified configuration class corresponding to exporter.properties + */ + private ExporterConfiguration exporterConfiguration; + @Override public void init() { //zipkin's config - eventMeshZipkinIP = ZipkinConfiguration.getEventMeshZipkinIP(); - eventMeshZipkinPort = ZipkinConfiguration.getEventMeshZipkinPort(); + eventMeshZipkinIP = zipkinConfiguration.getEventMeshZipkinIP(); + eventMeshZipkinPort = zipkinConfiguration.getEventMeshZipkinPort(); + //exporter's config - eventMeshTraceExportInterval = ExporterConfiguration.getEventMeshTraceExportInterval(); - eventMeshTraceExportTimeout = ExporterConfiguration.getEventMeshTraceExportTimeout(); - eventMeshTraceMaxExportSize = ExporterConfiguration.getEventMeshTraceMaxExportSize(); - eventMeshTraceMaxQueueSize = ExporterConfiguration.getEventMeshTraceMaxQueueSize(); + eventMeshTraceExportInterval = exporterConfiguration.getEventMeshTraceExportInterval(); + eventMeshTraceExportTimeout = exporterConfiguration.getEventMeshTraceExportTimeout(); + eventMeshTraceMaxExportSize = exporterConfiguration.getEventMeshTraceMaxExportSize(); + eventMeshTraceMaxQueueSize = exporterConfiguration.getEventMeshTraceMaxQueueSize(); String httpUrl = String.format("http://%s:%s", eventMeshZipkinIP, eventMeshZipkinPort); ZipkinSpanExporter zipkinExporter = @@ -168,4 +182,12 @@ public void shutdown() { //todo: turn the value of useTrace in AbstractHTTPServer into false } + + public ZipkinConfiguration getClientConfiguration() { + return this.zipkinConfiguration; + } + + public ExporterConfiguration getExporterConfiguration() { + return this.exporterConfiguration; + } } diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/common/ZipkinConstants.java b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/common/ZipkinConstants.java index aa3ad33f1b..cd1888bfd2 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/common/ZipkinConstants.java +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/common/ZipkinConstants.java @@ -20,9 +20,7 @@ public class ZipkinConstants { // Name of the service(using the instrumentationName) public static final String SERVICE_NAME = "eventmesh_trace"; + // Zipkin API Endpoints for uploading spans public static final String ENDPOINT_V2_SPANS = "/api/v2/spans"; - - public static final String KEY_ZIPKIN_IP = "eventmesh.trace.zipkin.ip"; - public static final String KEY_ZIPKIN_PORT = "eventmesh.trace.zipkin.port"; } diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfiguration.java b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfiguration.java index 40d30bb7d7..fcc2ac0dec 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfiguration.java +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/main/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfiguration.java @@ -17,82 +17,22 @@ package org.apache.eventmesh.trace.zipkin.config; -import org.apache.eventmesh.common.Constants; -import org.apache.eventmesh.common.utils.PropertiesUtils; -import org.apache.eventmesh.trace.zipkin.common.ZipkinConstants; +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; -import org.apache.commons.lang3.StringUtils; - -import java.io.BufferedReader; -import java.io.File; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.net.URL; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import com.google.common.base.Preconditions; - -import lombok.experimental.UtilityClass; -import lombok.extern.slf4j.Slf4j; +import lombok.Data; /** * to load the properties form zipkin.properties */ -@Slf4j -@UtilityClass -public class ZipkinConfiguration { - private static final String CONFIG_FILE = "zipkin.properties"; - private static final Properties properties = new Properties(); +@Data +@Config(prefix = "eventmesh.trace.zipkin", path = "classPath://zipkin.properties") +public class ZipkinConfiguration { + @ConfigFiled(field = "ip", notNull = true) private String eventMeshZipkinIP = "localhost"; - private int eventMeshZipkinPort = 9411; - static { - loadProperties(); - initializeConfig(); - } - - public static String getEventMeshZipkinIP() { - return eventMeshZipkinIP; - } - - public static int getEventMeshZipkinPort() { - return eventMeshZipkinPort; - } - - private void initializeConfig() { - String eventMeshZipkinIPStr = properties.getProperty(ZipkinConstants.KEY_ZIPKIN_IP); - Preconditions.checkState(StringUtils.isNotEmpty(eventMeshZipkinIPStr), - String.format("%s error", ZipkinConstants.KEY_ZIPKIN_IP)); - eventMeshZipkinIP = StringUtils.deleteWhitespace(eventMeshZipkinIPStr); - - String eventMeshZipkinPortStr = properties.getProperty(ZipkinConstants.KEY_ZIPKIN_PORT); - if (StringUtils.isNotEmpty(eventMeshZipkinPortStr)) { - eventMeshZipkinPort = Integer.parseInt(StringUtils.deleteWhitespace(eventMeshZipkinPortStr)); - } - } - - private void loadProperties() { - URL resource = ZipkinConfiguration.class.getClassLoader().getResource(CONFIG_FILE); - if (resource != null) { - try (InputStream inputStream = resource.openStream(); - BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8))) { - if (inputStream.available() > 0) { - properties.load(reader); - } - } catch (IOException e) { - throw new RuntimeException("Load zipkin.properties file from classpath error", e); - } - } - // get from config home - try { - String configPath = Constants.EVENTMESH_CONF_HOME + File.separator + CONFIG_FILE; - PropertiesUtils.loadPropertiesWhenFileExist(properties, configPath); - } catch (IOException e) { - throw new IllegalArgumentException("Cannot load zipkin.properties file from conf", e); - } - } + @ConfigFiled(field = "port") + private int eventMeshZipkinPort = 9411; } diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceServiceTest.java b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceServiceTest.java index ccb102e5c3..1aca19cb3c 100644 --- a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceServiceTest.java +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/ZipkinTraceServiceTest.java @@ -19,6 +19,8 @@ import static org.junit.Assert.assertThrows; +import org.apache.eventmesh.trace.api.TracePluginFactory; + import java.lang.reflect.Field; import org.junit.Assert; @@ -31,7 +33,8 @@ public class ZipkinTraceServiceTest { @Test public void testInit() { - ZipkinTraceService zipkinTraceService = new ZipkinTraceService(); + ZipkinTraceService zipkinTraceService = + (ZipkinTraceService) TracePluginFactory.getEventMeshTraceService("zipkin"); zipkinTraceService.init(); Assert.assertNotNull(zipkinTraceService.sdkTracerProvider); @@ -46,7 +49,9 @@ public void testInit() { @Test public void testShutdown() throws Exception { SdkTracerProvider mockSdkTracerProvider = Mockito.mock(SdkTracerProvider.class); - ZipkinTraceService zipkinTraceService = new ZipkinTraceService(); + + ZipkinTraceService zipkinTraceService = + (ZipkinTraceService) TracePluginFactory.getEventMeshTraceService("zipkin"); zipkinTraceService.init(); Field sdkTracerProviderField = ZipkinTraceService.class.getDeclaredField("sdkTracerProvider"); sdkTracerProviderField.setAccessible(true); diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfigurationTest.java b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfigurationTest.java new file mode 100644 index 0000000000..51adfe249a --- /dev/null +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/java/org/apache/eventmesh/trace/zipkin/config/ZipkinConfigurationTest.java @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.trace.zipkin.config; + +import org.apache.eventmesh.trace.api.TracePluginFactory; +import org.apache.eventmesh.trace.api.config.ExporterConfiguration; +import org.apache.eventmesh.trace.zipkin.ZipkinTraceService; + +import org.junit.Assert; +import org.junit.Test; + +public class ZipkinConfigurationTest { + + @Test + public void getConfigWhenZipkinTraceInit() { + ZipkinTraceService zipkinTraceService = + (ZipkinTraceService) TracePluginFactory.getEventMeshTraceService("zipkin"); + + ZipkinConfiguration config = zipkinTraceService.getClientConfiguration(); + assertConfig(config); + ExporterConfiguration exporterConfig = zipkinTraceService.getExporterConfiguration(); + assertBaseConfig(exporterConfig); + } + + private void assertConfig(ZipkinConfiguration config) { + Assert.assertEquals("127.0.0.1", config.getEventMeshZipkinIP()); + Assert.assertEquals(816, config.getEventMeshZipkinPort()); + } + + private void assertBaseConfig(ExporterConfiguration config) { + Assert.assertEquals(816, config.getEventMeshTraceMaxExportSize()); + Assert.assertEquals(1816, config.getEventMeshTraceMaxQueueSize()); + Assert.assertEquals(2816, config.getEventMeshTraceExportTimeout()); + Assert.assertEquals(3816, config.getEventMeshTraceExportInterval()); + } +} \ No newline at end of file diff --git a/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/exporter.properties b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/exporter.properties new file mode 100644 index 0000000000..14f6980df0 --- /dev/null +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/exporter.properties @@ -0,0 +1,20 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +eventmesh.trace.max.export.size=816 +eventmesh.trace.max.queue.size=1816 +eventmesh.trace.export.timeout=2816 +eventmesh.trace.export.interval=3816 \ No newline at end of file diff --git a/eventmesh-security-plugin/eventmesh-security-api/src/test/resources/test.properties b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/zipkin.properties similarity index 88% rename from eventmesh-security-plugin/eventmesh-security-api/src/test/resources/test.properties rename to eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/zipkin.properties index ce1c5a1eba..e62cf9b02c 100644 --- a/eventmesh-security-plugin/eventmesh-security-api/src/test/resources/test.properties +++ b/eventmesh-trace-plugin/eventmesh-trace-zipkin/src/test/resources/zipkin.properties @@ -14,4 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. # -a=1 \ No newline at end of file + +#zipkin's working ip and port +eventmesh.trace.zipkin.ip=127.0.0.1 +eventmesh.trace.zipkin.port=816 \ No newline at end of file diff --git a/eventmesh-webhook/eventmesh-webhook-admin/build.gradle b/eventmesh-webhook/eventmesh-webhook-admin/build.gradle index ce7104d52a..26b948fd18 100644 --- a/eventmesh-webhook/eventmesh-webhook-admin/build.gradle +++ b/eventmesh-webhook/eventmesh-webhook-admin/build.gradle @@ -1,4 +1,4 @@ -/* + /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. @@ -26,6 +26,12 @@ dependencies { implementation "com.fasterxml.jackson.core:jackson-core" implementation "com.fasterxml.jackson.core:jackson-annotations" + compileOnly 'org.projectlombok:lombok:1.18.22' + annotationProcessor 'org.projectlombok:lombok:1.18.22' + + testCompileOnly 'org.projectlombok:lombok:1.18.22' + testAnnotationProcessor 'org.projectlombok:lombok:1.18.22' + testImplementation project(":eventmesh-webhook:eventmesh-webhook-api") diff --git a/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManage.java b/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManage.java index 5148d19d40..a7ded53719 100644 --- a/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManage.java +++ b/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManage.java @@ -17,9 +17,12 @@ package org.apache.eventmesh.webhook.admin; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.OPERATION_MODE_FILE; +import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.OPERATION_MODE_NACOS; + +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.webhook.api.WebHookConfigOperation; -import org.apache.eventmesh.webhook.api.WebHookOperationConstant; +import org.apache.eventmesh.webhook.config.AdminConfiguration; import java.lang.reflect.Constructor; import java.lang.reflect.InvocationTargetException; @@ -36,42 +39,38 @@ public class AdminWebHookConfigOperationManage { private static final Map> map = new HashMap<>(); static { - map.put("file", FileWebHookConfigOperation.class); - map.put("nacos", NacosWebHookConfigOperation.class); + map.put(OPERATION_MODE_FILE, FileWebHookConfigOperation.class); + map.put(OPERATION_MODE_NACOS, NacosWebHookConfigOperation.class); } public Logger logger = LoggerFactory.getLogger(this.getClass()); - private ConfigurationWrapper configurationWrapper; + private AdminConfiguration adminConfiguration; private WebHookConfigOperation webHookConfigOperation; - public void setConfigurationWrapper(ConfigurationWrapper configurationWrapper) { - this.configurationWrapper = configurationWrapper; - } - public WebHookConfigOperation getWebHookConfigOperation() { return webHookConfigOperation; } public void init() throws Exception { - - if (!configurationWrapper.getBoolProp(WebHookOperationConstant.ADMIN_START_CONFIG_NAME, false)) { + adminConfiguration = ConfigService.getInstance().buildConfigInstance(AdminConfiguration.class); + if (!adminConfiguration.isAdminStart()) { return; } - String operationMode = configurationWrapper.getProp(WebHookOperationConstant.OPERATION_MODE_CONFIG_NAME); - + String operationMode = adminConfiguration.getOperationMode(); if (!map.containsKey(operationMode)) { throw new IllegalStateException("operationMode is not supported."); } - Constructor constructor = map.get(operationMode).getDeclaredConstructor(Properties.class); + Constructor constructor = + map.get(operationMode).getDeclaredConstructor(Properties.class); constructor.setAccessible(true); + Properties operationProperties = adminConfiguration.getOperationProperties(); try { - Properties properties = configurationWrapper.getPropertiesByConfig("eventMesh.webHook." + operationMode + "Mode", true); - logger.info("operationMode is {} properties is {} ", operationMode, properties); - this.webHookConfigOperation = constructor.newInstance(properties); + logger.info("operationMode is {} properties is {} ", operationMode, operationProperties); + this.webHookConfigOperation = constructor.newInstance(operationProperties); } catch (InstantiationException | IllegalAccessException | InvocationTargetException e) { logger.error("can't find WebHookConfigOperation implementation"); throw new Exception("can't find WebHookConfigOperation implementation", e); diff --git a/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/config/AdminConfiguration.java b/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/config/AdminConfiguration.java new file mode 100644 index 0000000000..1afecc78c7 --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-admin/src/main/java/org/apache/eventmesh/webhook/config/AdminConfiguration.java @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.webhook.config; + +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; + +import java.util.Properties; + +import lombok.Data; + +@Data +@Config(prefix = "eventMesh.webHook") +public class AdminConfiguration { + + @ConfigFiled(field = "admin.start") + private boolean adminStart = false; + + @ConfigFiled(field = "operationMode") + private String operationMode; + + @ConfigFiled(field = "", reload = true) + private Properties operationProperties; + + public void reload() { + processOperationProperties(); + } + + public void processOperationProperties() { + String prefix = operationMode + "Mode"; + this.operationProperties = (Properties) operationProperties.get(prefix); + } +} + diff --git a/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigurationWrapperTest.java b/eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManageTest.java similarity index 51% rename from eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigurationWrapperTest.java rename to eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManageTest.java index 56ce9191ce..ea26b2a600 100644 --- a/eventmesh-common/src/test/java/org/apache/eventmesh/common/config/ConfigurationWrapperTest.java +++ b/eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/admin/AdminWebHookConfigOperationManageTest.java @@ -15,34 +15,24 @@ * limitations under the License. */ -package org.apache.eventmesh.common.config; +package org.apache.eventmesh.webhook.admin; -import java.io.File; +import org.apache.eventmesh.common.config.ConfigService; import org.junit.Assert; -import org.junit.Before; import org.junit.Test; -public class ConfigurationWrapperTest { - - private ConfigurationWrapper wraper; - - @Before - public void before() { - String file = ConfigurationWrapperTest.class.getResource("/configuration.properties").getFile(); - File f = new File(file); - wraper = new ConfigurationWrapper(f.getParent(), f.getName(), false); - } +public class AdminWebHookConfigOperationManageTest { @Test - public void testGetProp() { - Assert.assertEquals("value1", wraper.getProp("eventMesh.server.env")); - Assert.assertEquals("value2", wraper.getProp("eventMesh.server.idc")); - } + public void init() throws Exception { + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://eventmesh.properties"); - @Test(expected = NullPointerException.class) - public void construct() { - ConfigurationWrapper newWrapper = new ConfigurationWrapper(null, "eventmesh.properties", false); - } + AdminWebHookConfigOperationManage adminWebHookConfigOperationManage = new AdminWebHookConfigOperationManage(); + adminWebHookConfigOperationManage.init(); -} + Assert.assertTrue( + adminWebHookConfigOperationManage.getWebHookConfigOperation() instanceof FileWebHookConfigOperation); + } +} \ No newline at end of file diff --git a/eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/config/AdminConfigurationTest.java b/eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/config/AdminConfigurationTest.java new file mode 100644 index 0000000000..34e88c96fd --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-admin/src/test/java/org/apache/eventmesh/webhook/config/AdminConfigurationTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.webhook.config; + +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.Properties; + +import org.junit.Assert; +import org.junit.Test; + +public class AdminConfigurationTest { + + @Test + public void testGetAdminConfiguration() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://eventmesh.properties"); + + AdminConfiguration config = configService.buildConfigInstance(AdminConfiguration.class); + + assertAdminConfiguration(config); + } + + private void assertAdminConfiguration(AdminConfiguration config) { + Assert.assertTrue(config.isAdminStart()); + Assert.assertEquals("file", config.getOperationMode()); + + Properties properties = new Properties(); + properties.put("filePath", "."); + Assert.assertEquals(properties, config.getOperationProperties()); + } +} \ No newline at end of file diff --git a/eventmesh-webhook/eventmesh-webhook-admin/src/test/resources/eventmesh.properties b/eventmesh-webhook/eventmesh-webhook-admin/src/test/resources/eventmesh.properties new file mode 100644 index 0000000000..4429d16f40 --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-admin/src/test/resources/eventmesh.properties @@ -0,0 +1,28 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# webhook +# Start webhook admin service +eventMesh.webHook.admin.start=true +# Webhook event configuration storage mode. Currently, only file and Nacos are supported +eventMesh.webHook.operationMode=file +# The file storage path of the file storage mode. If #{eventmeshhome} is written, it is in the eventmesh root directory +eventMesh.webHook.fileMode.filePath=. +# Nacos storage mode, and the configuration naming rule is eventmesh webHook. nacosMode. {nacos native configuration key} please see the specific configuration [nacos github api](https://github.com/alibaba/nacos/blob/develop/api/src/main/java/com/alibaba/nacos/api/SystemPropertyKeyConst.java) +## Address of Nacos +eventMesh.webHook.nacosMode.serverAddr=127.0.0.1:8848 +# Webhook eventcloud sending mode. And eventmesh connector. plugin. The type configuration is the same +eventMesh.webHook.producer.connector=standalone diff --git a/eventmesh-webhook/eventmesh-webhook-api/src/main/java/org/apache/eventmesh/webhook/api/WebHookOperationConstant.java b/eventmesh-webhook/eventmesh-webhook-api/src/main/java/org/apache/eventmesh/webhook/api/WebHookOperationConstant.java index de79244e08..d4386ded28 100644 --- a/eventmesh-webhook/eventmesh-webhook-api/src/main/java/org/apache/eventmesh/webhook/api/WebHookOperationConstant.java +++ b/eventmesh-webhook/eventmesh-webhook-api/src/main/java/org/apache/eventmesh/webhook/api/WebHookOperationConstant.java @@ -37,9 +37,9 @@ public class WebHookOperationConstant { public static final long TIMEOUT_MS = 3 * 1000L; - public static final String ADMIN_START_CONFIG_NAME = "eventMesh.webHook.admin.start"; + public static final String OPERATION_MODE_FILE = "file"; - public static final String OPERATION_MODE_CONFIG_NAME = "eventMesh.webHook.operationMode"; + public static final String OPERATION_MODE_NACOS = "nacos"; public static final String EVENTMESH_HOME = "#{eventMeshHome}"; diff --git a/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/WebHookController.java b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/WebHookController.java index c1da97232f..c2fe780521 100644 --- a/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/WebHookController.java +++ b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/WebHookController.java @@ -20,29 +20,26 @@ import org.apache.eventmesh.api.SendCallback; import org.apache.eventmesh.api.SendResult; import org.apache.eventmesh.api.exception.OnExceptionContext; -import org.apache.eventmesh.common.config.ConfigurationWrapper; +import org.apache.eventmesh.common.config.ConfigService; import org.apache.eventmesh.common.protocol.ProtocolTransportObject; import org.apache.eventmesh.common.protocol.http.WebhookProtocolTransportObject; import org.apache.eventmesh.protocol.api.ProtocolAdaptor; import org.apache.eventmesh.protocol.api.ProtocolPluginFactory; import org.apache.eventmesh.webhook.api.WebHookConfig; +import org.apache.eventmesh.webhook.receive.config.ReceiveConfiguration; import org.apache.eventmesh.webhook.receive.protocol.ProtocolManage; import org.apache.eventmesh.webhook.receive.storage.HookConfigOperationManage; import java.util.Map; import java.util.Objects; +import java.util.Properties; import java.util.UUID; import org.slf4j.Logger; import org.slf4j.LoggerFactory; - -import lombok.Setter; - public class WebHookController { - private static final String WEBHOOK_PRODUCER_CONNECTOR_PROP = "eventMesh.webHook.producer.connector"; - private static final String PROTOCOL_ADAPTOR = "webhook"; private static final String CONTENT_TYPE = "content-type"; @@ -55,7 +52,9 @@ public class WebHookController { * protocol pool */ private final ProtocolManage protocolManage = new ProtocolManage(); + public Logger logger = LoggerFactory.getLogger(this.getClass()); + /** * config pool */ @@ -65,13 +64,14 @@ public class WebHookController { private ProtocolAdaptor protocolAdaptor; - @Setter - private ConfigurationWrapper configurationWrapper; + private ReceiveConfiguration receiveConfiguration; public void init() throws Exception { - this.webHookMQProducer = new WebHookMQProducer(configurationWrapper.getProperties(), - configurationWrapper.getProp(WEBHOOK_PRODUCER_CONNECTOR_PROP)); - this.hookConfigOperationManage = new HookConfigOperationManage(configurationWrapper); + receiveConfiguration = ConfigService.getInstance().buildConfigInstance(ReceiveConfiguration.class); + Properties rootConfig = ConfigService.getInstance().getRootConfig(); + + this.webHookMQProducer = new WebHookMQProducer(rootConfig, receiveConfiguration.getConnectorPluginType()); + this.hookConfigOperationManage = new HookConfigOperationManage(receiveConfiguration); this.protocolAdaptor = ProtocolPluginFactory.getProtocolAdaptor(PROTOCOL_ADAPTOR); } diff --git a/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfiguration.java b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfiguration.java new file mode 100644 index 0000000000..deb84b5b25 --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfiguration.java @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.webhook.receive.config; + +import org.apache.eventmesh.common.config.Config; +import org.apache.eventmesh.common.config.ConfigFiled; + +import java.util.Properties; + +import lombok.Data; + +@Data +@Config(prefix = "eventMesh.webHook") +public class ReceiveConfiguration { + @ConfigFiled(field = "producer.connector") + private String connectorPluginType; + + @ConfigFiled(field = "operationMode") + private String operationMode; + + @ConfigFiled(field = "fileMode.filePath") + private String filePath; + + @ConfigFiled(field = "", reload = true) + private Properties operationProperties; + + public void reload() { + processOperationProperties(); + } + + public void processOperationProperties() { + String prefix = operationMode + "Mode"; + this.operationProperties = (Properties) operationProperties.get(prefix); + } +} diff --git a/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/storage/HookConfigOperationManage.java b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/storage/HookConfigOperationManage.java index ecae94f57d..32d78885a9 100644 --- a/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/storage/HookConfigOperationManage.java +++ b/eventmesh-webhook/eventmesh-webhook-receive/src/main/java/org/apache/eventmesh/webhook/receive/storage/HookConfigOperationManage.java @@ -19,14 +19,15 @@ import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.DATA_ID_EXTENSION; import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.GROUP_PREFIX; +import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.OPERATION_MODE_FILE; +import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.OPERATION_MODE_NACOS; import static org.apache.eventmesh.webhook.api.WebHookOperationConstant.TIMEOUT_MS; -import org.apache.eventmesh.common.config.ConfigurationWrapper; import org.apache.eventmesh.common.utils.JsonUtils; import org.apache.eventmesh.webhook.api.WebHookConfig; import org.apache.eventmesh.webhook.api.WebHookConfigOperation; -import org.apache.eventmesh.webhook.api.WebHookOperationConstant; import org.apache.eventmesh.webhook.api.utils.StringUtils; +import org.apache.eventmesh.webhook.receive.config.ReceiveConfiguration; import java.io.FileNotFoundException; import java.util.List; @@ -43,15 +44,6 @@ import com.alibaba.nacos.api.exception.NacosException; public class HookConfigOperationManage implements WebHookConfigOperation { - - private static final String FILE_OPERATION_MODE = "file"; - - private static final String WEBHOOK_FILEMODE_FILEPATH_PROP = "eventMesh.webHook.fileMode.filePath"; - - private static final String NACOS_OPERATION_MODE = "nacos"; - - private static final String WEBHOOK_NACOSMODE = "eventMesh.webHook.nacosMode"; - /** * webhook config pool -> key is CallbackPath */ @@ -66,16 +58,16 @@ public HookConfigOperationManage() { /** * Initialize according to operationMode * - * @param configurationWrapper + * @param receiveConfiguration receiveConfiguration */ - public HookConfigOperationManage(ConfigurationWrapper configurationWrapper) throws FileNotFoundException, NacosException { + public HookConfigOperationManage(ReceiveConfiguration receiveConfiguration) throws FileNotFoundException, NacosException { - this.operationMode = configurationWrapper.getProp(WebHookOperationConstant.OPERATION_MODE_CONFIG_NAME); + this.operationMode = receiveConfiguration.getOperationMode(); - if (FILE_OPERATION_MODE.equals(operationMode)) { - new WebhookFileListener(configurationWrapper.getProp(WEBHOOK_FILEMODE_FILEPATH_PROP), cacheWebHookConfig); - } else if (NACOS_OPERATION_MODE.equals(operationMode)) { - nacosModeInit(configurationWrapper.getPropertiesByConfig(WEBHOOK_NACOSMODE, true)); + if (OPERATION_MODE_FILE.equals(operationMode)) { + new WebhookFileListener(receiveConfiguration.getFilePath(), cacheWebHookConfig); + } else if (OPERATION_MODE_NACOS.equals(operationMode)) { + nacosModeInit(receiveConfiguration.getOperationProperties()); } } @@ -85,9 +77,9 @@ private void nacosModeInit(Properties config) throws NacosException { @Override public WebHookConfig queryWebHookConfigById(WebHookConfig webHookConfig) { - if (FILE_OPERATION_MODE.equals(operationMode)) { + if (OPERATION_MODE_FILE.equals(operationMode)) { return cacheWebHookConfig.get(StringUtils.getFileName(webHookConfig.getCallbackPath())); - } else if (NACOS_OPERATION_MODE.equals(operationMode)) { + } else if (OPERATION_MODE_NACOS.equals(operationMode)) { try { String content = nacosConfigService.getConfig(webHookConfig.getManufacturerEventName() + DATA_ID_EXTENSION, GROUP_PREFIX + webHookConfig.getManufacturerName(), TIMEOUT_MS); diff --git a/eventmesh-webhook/eventmesh-webhook-receive/src/test/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfigurationTest.java b/eventmesh-webhook/eventmesh-webhook-receive/src/test/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfigurationTest.java new file mode 100644 index 0000000000..d5b2c45aa8 --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-receive/src/test/java/org/apache/eventmesh/webhook/receive/config/ReceiveConfigurationTest.java @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.eventmesh.webhook.receive.config; + +import org.apache.eventmesh.common.config.ConfigService; + +import java.util.Properties; + +import org.junit.Assert; +import org.junit.Test; + +public class ReceiveConfigurationTest { + + @Test + public void testGetReceiveConfiguration() throws Exception { + + ConfigService configService = ConfigService.getInstance(); + configService.setRootConfig("classPath://eventmesh.properties"); + + Properties rootConfig = ConfigService.getInstance().getRootConfig(); + Assert.assertEquals("DEFAULT", rootConfig.get("eventMesh.server.idc")); + + ReceiveConfiguration config = configService.buildConfigInstance(ReceiveConfiguration.class); + assertReceiveConfiguration(config); + } + + private void assertReceiveConfiguration(ReceiveConfiguration config) { + Assert.assertEquals("nacos", config.getOperationMode()); + + Properties properties = new Properties(); + properties.put("serverAddr", "127.0.0.1:8848"); + Assert.assertEquals(properties, config.getOperationProperties()); + Assert.assertEquals("standalone", config.getConnectorPluginType()); + Assert.assertEquals(".", config.getFilePath()); + } +} \ No newline at end of file diff --git a/eventmesh-webhook/eventmesh-webhook-receive/src/test/resources/eventmesh.properties b/eventmesh-webhook/eventmesh-webhook-receive/src/test/resources/eventmesh.properties new file mode 100644 index 0000000000..9e2151dd6b --- /dev/null +++ b/eventmesh-webhook/eventmesh-webhook-receive/src/test/resources/eventmesh.properties @@ -0,0 +1,30 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +eventMesh.server.idc=DEFAULT + +# webhook +# Start webhook admin service +eventMesh.webHook.admin.start=true +# Webhook event configuration storage mode. Currently, only file and Nacos are supported +eventMesh.webHook.operationMode=nacos +# The file storage path of the file storage mode. If #{eventmeshhome} is written, it is in the eventmesh root directory +eventMesh.webHook.fileMode.filePath=. +# Nacos storage mode, and the configuration naming rule is eventmesh webHook. nacosMode. {nacos native configuration key} please see the specific configuration [nacos github api](https://github.com/alibaba/nacos/blob/develop/api/src/main/java/com/alibaba/nacos/api/SystemPropertyKeyConst.java) +## Address of Nacos +eventMesh.webHook.nacosMode.serverAddr=127.0.0.1:8848 +# Webhook eventcloud sending mode. And eventmesh connector. plugin. The type configuration is the same +eventMesh.webHook.producer.connector=standalone