[BUG] JsonEOFException : Unexpected end-of-input | AzureOpenAiStreamingChatModel #1848
Comments
|
Hey, I think I filled up the same issue a couple of days ago :-) |
|
Thanks @jdubois -- I tried searching the discussion/bug history and couldn't find anything related. I hate creating duplicates. I will add a note to your defect report and link it to mine. |
|
Oh it's not really a duplicate as it's with another SDK, and I believe it gives them more context. And that's my last important issue for GitHub Models support, so I'm super happy to have more people pushing for this :-D |
|
Fixed by #1872 |
|
I just upgraded to the 0.36.0 build and I am still getting the same errors. It should be noted that I have been able to get all my other dev.langchain4j groupIds to 0.35.0/0.36.0, with the exception of: Which I need to keep at 0.31.0 to keep the errors in the original post at bay. |
|
I see a similar issue on 0.36.0 when using |
|
I was not setting the property:
at all, so whatever the default is. I also noticed that it is deprecated. With that said, when I do set the value to errors. Back to |
|
@cato-dll could you please provide the latest logs you have with 0.36.0? |
|
Here you go: |
|
@cato-dll thank you! I guess we have to wait untill Azure/azure-sdk-for-java#42037 or Azure/azure-sdk-for-java#41164 is fixed. Perharps adding a 👍 to that issues will help to speed it up :) |
|
Hey guys, today is Dec 24, I also got a similar error on 0.36.2 or 1.0.0-alpha1. |
|
For 0.36.2 I'm not surprised, but for 1.0.0-alpha1 I need to check as it uses https://github.com/Azure/azure-sdk-for-java/releases/tag/azure-ai-openai_1.0.0-beta.13 which in the release notes should fix Azure/azure-sdk-for-java#41164 (which should be your issue). |
jdubois
added a commit
to jdubois/jdubois-langchain4j-demo
that referenced
this issue
Dec 24, 2024
|
I tried to reproduce it at jdubois/jdubois-langchain4j-demo@aeecf3e but I don't have the bug anymore... I also tried with This is using the latest 1.0.0-alpha1 Can you provide a test where I can reproduce this easily? Maybe by modifying my current test code? |
|
I just bumped my dependencies up to:
and I no longer receive the:
exceptions. So it would appear, at least for my use case, the issue has been resolved. |
|
So it looks solved, only @gumutianqi you say you have a "similar" issue, do you have more exact details? Otherwise I think we can close this one. |
|
Yep, I will pick the exception stack at here. <properties>
<java.version>21</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-undertow</artifactId>
<version>3.3.5</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>3.3.5</version>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-azure-open-ai</artifactId>
<version>1.0.0-alpha1</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-ai-openai</artifactId>
<version>1.0.0-beta.13</version>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
<version>3.6.11</version>
</dependency>
</dependencies2024-12-25 23:11:04.625 [adec2282bdda70c7413ffcbe76bac1a0] [XNIO-1 task-2] INFO c.a.c.h.n.implementation.NettyUtility : {"az.sdk.message":"The following Netty versions were found on the classpath and have a mismatch with the versions used by azure-core-http-netty. If your application runs without issue this message can be ignored, otherwise please align the Netty versions used in your application. For more information, see https://aka.ms/azsdk/java/dependency/troubleshoot.","azure-netty-version":"4.1.110.Final","azure-netty-native-version":"2.0.65.Final","classpath-netty-version-io.netty:netty-common":"4.1.114.Final","classpath-netty-version-io.netty:netty-handler":"4.1.114.Final","classpath-netty-version-io.netty:netty-handler-proxy":"4.1.114.Final","classpath-netty-version-io.netty:netty-buffer":"4.1.114.Final","classpath-netty-version-io.netty:netty-codec":"4.1.114.Final","classpath-netty-version-io.netty:netty-codec-http":"4.1.114.Final","classpath-netty-version-io.netty:netty-codec-http2":"4.1.114.Final","classpath-netty-version-io.netty:netty-transport-native-unix-common":"4.1.114.Final","classpath-netty-version-io.netty:netty-transport-native-epoll":"4.1.100.Final","classpath-netty-version-io.netty:netty-transport-native-kqueue":"4.1.114.Final","classpath-native-netty-version-io.netty:netty-tcnative-boringssl-static":"2.0.66.Final"}
2024-12-25 23:11:04.782 [adec2282bdda70c7413ffcbe76bac1a0] [XNIO-1 task-2] INFO c.a.a.o.i.O.getChatCompletions : {"az.sdk.message":"HTTP request","method":"POST","url":"https://xxx.openai.azure.com//openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview","tryCount":1,"Date":"Wed, 25 Dec 2024 15:11:04 GMT","Content-Type":"application/json","x-ms-client-request-id":"08ac1f75-002e-4260-be4a-546a0bb6f639","Accept":"application/json","User-Agent":"langchain4j-azure-openai","redactedHeaders":"api-key","content-length":290,"body":"{\"messages\":[{\"content\":\"you are a smart develpment. **Always talk to me in Chinese.**\",\"role\":\"system\"},{\"role\":\"user\",\"content\":\"Please summary the movie 《Silicon valley》at 100.\"}],\"max_tokens\":10,\"temperature\":0.7,\"stop\":[\"STOP\"],\"stream\":true,\"model\":\"gpt-4o-laigu\"}"}
2024-12-25 23:11:06.692 [] [reactor-http-nio-1] INFO c.a.a.o.i.O.getChatCompletions : {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://xxx.openai.azure.com//openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview","durationMs":1909,"Date":"Wed, 25 Dec 2024 15:11:06 GMT","Transfer-Encoding":"chunked","Content-Type":"text/event-stream; charset=utf-8","x-ms-client-request-id":"08ac14jf-002e-4260-be4a-546a0bb6f639","redactedHeaders":"x-request-id,x-ms-region,x-envoy-upstream-service-time,apim-request-id,x-ratelimit-remaining-tokens,x-ratelimit-remaining-requests,Strict-Transport-Security,azureml-model-session,x-accel-buffering,x-content-type-options,x-ms-rai-invoked"}
2024-12-25 23:11:06.770 [] [loomBoundedElastic-3] ERROR c.a.c.i.MethodHandleReflectiveInvoker : Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"十二"},"finish_reason":null,"index":0"; line: 1, column: 13]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"十二"},"finish_reason":null,"index":0"; line: 1, column: 411]
2024-12-25 23:11:06.770 [] [loomBoundedElastic-3] ERROR c.a.c.i.s.DefaultJsonSerializer : com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"十二"},"finish_reason":null,"index":0"; line: 1, column: 13]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"十二"},"finish_reason":null,"index":0"; line: 1, column: 411] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
If I try to update LangChain4J beyond 0.31.0 I get a Jackson JsonEOFException, when making an API call to my Azure OpenAI endpoint. The LLM streamed response does work despite the exception -- but I can't have this filling up my logs :).
NOTE: the results are streamed.
Log and Stack trace
2024-09-27T08:38:30.634-04:00 ERROR 26320 --- [XXXXX] [oundedElastic-1] c.a.c.i.s.DefaultJsonSerializer : com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input in VALUE_STRING at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":" are"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727440708,"id":"chatcmpl-AC4Wqu"; line: 1, column: 346]To Reproduce
The exception is reproducible every time (Streaming instance) when using:
but does not create an error (Non-streaming) when using:
Expected behavior
No exceptions.
Please complete the following information:
The text was updated successfully, but these errors were encountered: