Skip to content

[Bug] unable to start event mesh with storage plugin eventMesh.storage.plugin.type=kafka #4444

Closed
@nguyenminhtuanfit

Description

Search before asking

  • I had searched in the issues and found no similar issues.

Environment

Mac

EventMesh version

master

What happened

#storage plugin
eventMesh.storage.plugin.type=kafka
got exception

09-15 00:31:00,359 DEBUG [main] ResourceLeakDetector(ResourceLeakDetector.java:130) - -Dio.grpc.netty.shaded.io.netty.leakDetection.targetRecords: 4
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:155) - -Dio.netty.allocator.numHeapArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:156) - -Dio.netty.allocator.numDirectArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:158) - -Dio.netty.allocator.pageSize: 8192
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:163) - -Dio.netty.allocator.maxOrder: 11
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:167) - -Dio.netty.allocator.chunkSize: 16777216
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:168) - -Dio.netty.allocator.smallCacheSize: 256
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:169) - -Dio.netty.allocator.normalCacheSize: 64
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:170) - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:171) - -Dio.netty.allocator.cacheTrimInterval: 8192
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:172) - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:173) - -Dio.netty.allocator.useCacheForAllThreads: true
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:174) - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,396 INFO [eventMesh-tcp-server] EventMeshTCPServer(EventMeshTCPServer.java:189) - EventMeshTCPServer[port=10000] started.....
2023-09-15 00:31:00,407 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:79) - -Dio.netty.processId: 51938 (auto-detected)
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:135) - -Djava.net.preferIPv4Stack: false
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:136) - -Djava.net.preferIPv6Addresses: false
2023-09-15 00:31:00,411 DEBUG [grpc-nio-boss-ELG-1-1] NetUtilInitializations(NetUtilInitializations.java:129) - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1%lo0)
2023-09-15 00:31:00,412 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:187) - Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128
2023-09-15 00:31:00,414 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:101) - -Dio.netty.machineId: 4c:32:75:ff:fe:92:a4:71 (auto-detected)
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,452 INFO [main] EventMeshGrpcServer(EventMeshGrpcServer.java:150) - ---------------EventMeshGRPCServer running-------------------
2023-09-15 00:31:00,478 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,479 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 ERROR [main] MQAdminWrapper(MQAdminWrapper.java:40) - can't load the meshMQAdmin plugin, please check.
2023-09-15 00:31:00,485 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]
2023-09-15 00:31:00,486 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]...
2023-09-15 00:31:00,480 ERROR [main] EventMeshStartup(EventMeshStartup.java:57) - EventMesh start fail.
java.lang.RuntimeException: doesn't load the meshMQAdmin plugin, please check.
at org.apache.eventmesh.runtime.core.plugin.MQAdminWrapper.(MQAdminWrapper.java:41) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.handler.TopicHandler.(TopicHandler.java:59) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.initClientHandler(ClientManageController.java:129) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.start(ClientManageController.java:95) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshServer.start(EventMeshServer.java:169) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshStartup.main(EventMeshStartup.java:40) [eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
2023-09-15 00:31:00,489 pool-1-thread-1 DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG OutputStream closed
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false, all resources released: true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Appender console stopped with status true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Log4j2 ConfigurationScheduler shutting down threads in java.util.concurrent.ScheduledThreadPoolExecutor@69532cf3[Running, pool size = 1, active threads = 0, queued tasks = 1, completed tasks = 0]
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Stopped XmlConfiguration[location=/Users/tuan/Desktop/dbs/apache-eventmesh-1.9.0/conf/log4j2.xml] OK
2023-09-15 00:31:00,491 pool-1-thread-1 DEBUG Stopped LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa] with status true
Tuans-MacBook-Pro:apache-eventmesh-1.9.0 tuan$

How to reproduce

java -Deventmesh.log.home=/Users/tuan/Desktop/apache-eventmesh-1.9.0 -DconfPath=${EVENTMESH_HOME}/conf -classpath ${EVENTMESH_HOME}/conf:${EVENTMESH_HOME}/apps/:${EVENTMESH_HOME}/lib/ org.apache.eventmesh.runtime.boot.EventMeshStartup

Debug logs

#storage plugin
eventMesh.storage.plugin.type=kafka
got exception

09-15 00:31:00,359 DEBUG [main] ResourceLeakDetector(ResourceLeakDetector.java:130) - -Dio.grpc.netty.shaded.io.netty.leakDetection.targetRecords: 4
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:155) - -Dio.netty.allocator.numHeapArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:156) - -Dio.netty.allocator.numDirectArenas: 16
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:158) - -Dio.netty.allocator.pageSize: 8192
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:163) - -Dio.netty.allocator.maxOrder: 11
2023-09-15 00:31:00,360 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:167) - -Dio.netty.allocator.chunkSize: 16777216
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:168) - -Dio.netty.allocator.smallCacheSize: 256
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:169) - -Dio.netty.allocator.normalCacheSize: 64
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:170) - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:171) - -Dio.netty.allocator.cacheTrimInterval: 8192
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:172) - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:173) - -Dio.netty.allocator.useCacheForAllThreads: true
2023-09-15 00:31:00,361 DEBUG [main] PooledByteBufAllocator(PooledByteBufAllocator.java:174) - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,377 DEBUG [eventMesh-tcp-server] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,396 INFO [eventMesh-tcp-server] EventMeshTCPServer(EventMeshTCPServer.java:189) - EventMeshTCPServer[port=10000] started.....
2023-09-15 00:31:00,407 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:79) - -Dio.netty.processId: 51938 (auto-detected)
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:135) - -Djava.net.preferIPv4Stack: false
2023-09-15 00:31:00,409 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:136) - -Djava.net.preferIPv6Addresses: false
2023-09-15 00:31:00,411 DEBUG [grpc-nio-boss-ELG-1-1] NetUtilInitializations(NetUtilInitializations.java:129) - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1%lo0)
2023-09-15 00:31:00,412 DEBUG [grpc-nio-boss-ELG-1-1] NetUtil(NetUtil.java:187) - Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128
2023-09-15 00:31:00,414 DEBUG [grpc-nio-boss-ELG-1-1] DefaultChannelId(DefaultChannelId.java:101) - -Dio.netty.machineId: 4c:32:75:ff:fe:92:a4:71 (auto-detected)
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:87) - -Dio.netty.allocator.type: pooled
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:96) - -Dio.netty.threadLocalDirectBufferSize: 0
2023-09-15 00:31:00,439 DEBUG [grpc-nio-boss-ELG-1-1] ByteBufUtil(ByteBufUtil.java:99) - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2023-09-15 00:31:00,452 INFO [main] EventMeshGrpcServer(EventMeshGrpcServer.java:150) - ---------------EventMeshGRPCServer running-------------------
2023-09-15 00:31:00,478 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,479 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.rocketmq.admin.RocketMQAdminAdaptor
2023-09-15 00:31:00,480 INFO [main] JarExtensionClassLoader(JarExtensionClassLoader.java:132) - load extension class success, extensionType: interface org.apache.eventmesh.api.admin.Admin, extensionClass: class org.apache.eventmesh.storage.standalone.admin.StandaloneAdminAdaptor
2023-09-15 00:31:00,480 ERROR [main] MQAdminWrapper(MQAdminWrapper.java:40) - can't load the meshMQAdmin plugin, please check.
2023-09-15 00:31:00,485 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]
2023-09-15 00:31:00,486 pool-1-thread-1 DEBUG Stopping LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa]...
2023-09-15 00:31:00,480 ERROR [main] EventMeshStartup(EventMeshStartup.java:57) - EventMesh start fail.
java.lang.RuntimeException: doesn't load the meshMQAdmin plugin, please check.
at org.apache.eventmesh.runtime.core.plugin.MQAdminWrapper.(MQAdminWrapper.java:41) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.handler.TopicHandler.(TopicHandler.java:59) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.initClientHandler(ClientManageController.java:129) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.admin.controller.ClientManageController.start(ClientManageController.java:95) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshServer.start(EventMeshServer.java:169) ~[eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
at org.apache.eventmesh.runtime.boot.EventMeshStartup.main(EventMeshStartup.java:40) [eventmesh-runtime-1.9.0-release.jar:1.9.0-release]
2023-09-15 00:31:00,489 pool-1-thread-1 DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG OutputStream closed
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false, all resources released: true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Appender console stopped with status true
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Log4j2 ConfigurationScheduler shutting down threads in java.util.concurrent.ScheduledThreadPoolExecutor@69532cf3[Running, pool size = 1, active threads = 0, queued tasks = 1, completed tasks = 0]
2023-09-15 00:31:00,490 pool-1-thread-1 DEBUG Stopped XmlConfiguration[location=/Users/tuan/Desktop/dbs/apache-eventmesh-1.9.0/conf/log4j2.xml] OK
2023-09-15 00:31:00,491 pool-1-thread-1 DEBUG Stopped LoggerContext[name=1b6d3586, org.apache.logging.log4j.core.LoggerContext@428640fa] with status true
Tuans-MacBook-Pro:apache-eventmesh-1.9.0 tuan$

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Assignees

No one assigned

    Labels

    StalebugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions