[Bug] unable to start event mesh with storage plugin eventMesh.storage.plugin.type=kafka #4444
Description
Search before asking
- I had searched in the issues and found no similar issues.
Environment
Mac
EventMesh version
master
What happened
#storage plugin
eventMesh.storage.plugin.type=kafka
got exception
09-15 00:31:00,359 DEBUG [main] ResourceLeakDetector(ResourceLeakDetector.java:130) - -Dio.grpc.netty.shaded.io.netty.leakDetection.targetRecords: 4
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:155) - -Dio.netty.allocator.numHeapArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:156) - -Dio.netty.allocator.numDirectArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:158) - -Dio.netty.allocator.pageSize: 8192
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:163) - -Dio.netty.allocator.maxOrder: 11
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:167) - -Dio.netty.allocator.chunkSize: 16777216
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:168) - -Dio.netty.allocator.smallCacheSize: 256
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:169) - -Dio.netty.allocator.normalCacheSize: 64
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:170) - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:171) - -Dio.netty.allocator.cacheTrimInterval: 8192
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:172) - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:173) - -Dio.netty.allocator.useCacheForAllThreads: true
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:174) - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,396 INFO [eventMesh-tcp-server] EventMeshTCPServer(EventMeshTCPServer.java:189) - EventMeshTCPServer[port=10000] started.....
2023-09-15 00:31:00,407 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:79) - -Dio.netty.processId: 51938 (auto-detected)
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:135) - -Djava.net.preferIPv4Stack: false
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:136) - -Djava.net.preferIPv6Addresses: false
2023-09-15 00:31:00,411 DEBUG [grpc-nio-boss-ELG-1-1] NetUtilInitializations(NetUtilInitializations.java:129) - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1%lo0)
2023-09-15 00:31:00,412 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:187) - Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128
2023-09-15 00:31:00,414 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:101) - -Dio.netty.machineId: 4c:32:75:ff:fe:92:a4:71 (auto-detected)
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,452 INFO [main] EventMeshGrpcServer(EventMeshGrpcServer.java:150) - ---------------EventMeshGRPCServer running-------------------
2023-09-15 00:31:00,478 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,479 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 ERROR [main] MQAdminWrapper(MQAdminWrapper.java:40) - can't load the meshMQAdmin plugin, please check.
2023-09-15 00:31:00,485 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]
2023-09-15 00:31:00,486 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]...
2023-09-15 00:31:00,480 ERROR [main] EventMeshStartup(EventMeshStartup.java:57) - EventMesh start fail.
java.lang.RuntimeException: doesn't load the meshMQAdmin plugin, please check.
at org.apache.eventmesh.runtime.core.plugin.MQAdminWrapper.(MQAdminWrapper.java:41) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.handler.TopicHandler.(TopicHandler.java:59) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.initClientHandler(ClientManageController.java:129) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.start(ClientManageController.java:95) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshServer.start(EventMeshServer.java:169) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshStartup.main(EventMeshStartup.java:40) [eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
2023-09-15 00:31:00,489 pool-1-thread-1 DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG OutputStream closed
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false, all resources released: true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Appender console stopped with status true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Log4j2 ConfigurationScheduler shutting down threads in java.util.concurrent.ScheduledThreadPoolExecutor@69532cf3[Running, pool size = 1, active threads = 0, queued tasks = 1, completed tasks = 0]
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Stopped XmlConfiguration[location=/Users/tuan/Desktop/dbs/apache-eventmesh-1.9.0/conf/log4j2.xml] OK
2023-09-15 00:31:00,491 pool-1-thread-1 DEBUG Stopped LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa] with status true
Tuans-MacBook-Pro:apache-eventmesh-1.9.0 tuan$
How to reproduce
java -Deventmesh.log.home=/Users/tuan/Desktop/apache-eventmesh-1.9.0 -DconfPath=${EVENTMESH_HOME}/conf -classpath ${EVENTMESH_HOME}/conf:${EVENTMESH_HOME}/apps/:${EVENTMESH_HOME}/lib/ org.apache.eventmesh.runtime.boot.EventMeshStartup
Debug logs
#storage plugin
eventMesh.storage.plugin.type=kafka
got exception
09-15 00:31:00,359 DEBUG [main] ResourceLeakDetector(ResourceLeakDetector.java:130) - -Dio.grpc.netty.shaded.io.netty.leakDetection.targetRecords: 4
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:155) - -Dio.netty.allocator.numHeapArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:156) - -Dio.netty.allocator.numDirectArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:158) - -Dio.netty.allocator.pageSize: 8192
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:163) - -Dio.netty.allocator.maxOrder: 11
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:167) - -Dio.netty.allocator.chunkSize: 16777216
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:168) - -Dio.netty.allocator.smallCacheSize: 256
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:169) - -Dio.netty.allocator.normalCacheSize: 64
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:170) - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:171) - -Dio.netty.allocator.cacheTrimInterval: 8192
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:172) - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:173) - -Dio.netty.allocator.useCacheForAllThreads: true
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:174) - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,396 INFO [eventMesh-tcp-server] EventMeshTCPServer(EventMeshTCPServer.java:189) - EventMeshTCPServer[port=10000] started.....
2023-09-15 00:31:00,407 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:79) - -Dio.netty.processId: 51938 (auto-detected)
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:135) - -Djava.net.preferIPv4Stack: false
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:136) - -Djava.net.preferIPv6Addresses: false
2023-09-15 00:31:00,411 DEBUG [grpc-nio-boss-ELG-1-1] NetUtilInitializations(NetUtilInitializations.java:129) - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1%lo0)
2023-09-15 00:31:00,412 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:187) - Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128
2023-09-15 00:31:00,414 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:101) - -Dio.netty.machineId: 4c:32:75:ff:fe:92:a4:71 (auto-detected)
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,452 INFO [main] EventMeshGrpcServer(EventMeshGrpcServer.java:150) - ---------------EventMeshGRPCServer running-------------------
2023-09-15 00:31:00,478 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,479 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 ERROR [main] MQAdminWrapper(MQAdminWrapper.java:40) - can't load the meshMQAdmin plugin, please check.
2023-09-15 00:31:00,485 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]
2023-09-15 00:31:00,486 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]...
2023-09-15 00:31:00,480 ERROR [main] EventMeshStartup(EventMeshStartup.java:57) - EventMesh start fail.
java.lang.RuntimeException: doesn't load the meshMQAdmin plugin, please check.
at org.apache.eventmesh.runtime.core.plugin.MQAdminWrapper.(MQAdminWrapper.java:41) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.handler.TopicHandler.(TopicHandler.java:59) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.initClientHandler(ClientManageController.java:129) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.start(ClientManageController.java:95) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshServer.start(EventMeshServer.java:169) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshStartup.main(EventMeshStartup.java:40) [eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
2023-09-15 00:31:00,489 pool-1-thread-1 DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG OutputStream closed
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false, all resources released: true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Appender console stopped with status true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Log4j2 ConfigurationScheduler shutting down threads in java.util.concurrent.ScheduledThreadPoolExecutor@69532cf3[Running, pool size = 1, active threads = 0, queued tasks = 1, completed tasks = 0]
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Stopped XmlConfiguration[location=/Users/tuan/Desktop/dbs/apache-eventmesh-1.9.0/conf/log4j2.xml] OK
2023-09-15 00:31:00,491 pool-1-thread-1 DEBUG Stopped LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa] with status true
Tuans-MacBook-Pro:apache-eventmesh-1.9.0 tuan$
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct *