[BUG] Azure AI Inference API throwing "Unexpected end-of-input: expected close marker for Object" when using ChatCompletionsAsyncClient #42037
Open
Description
Describe the bug
I'm using azure-ai-inference_1.0.0-beta.1.
When I use ChatCompletionsAsyncClient, I got correct answers but there are several log messages that indicates there are errors going underneath.
Here are some of the logs from the sample that I share below:
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":" Java"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727"; line: 1, column: 1]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":" Java"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727"; line: 1, column: 319]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":" Java"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727"; line: 1, column: 1]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":" Java"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727"; line: 1, column: 319]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: Unexpected end-of-input: was expecting closing '"' for name at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},""; line: 1, column: 180]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input: was expecting closing '"' for name at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},""; line: 1, column: 180]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"; line: 1, column: 39]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"; line: 1, column: 40]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"choices":[{"content_filter_results":{"; line: 1, column: 39]) at [Source: (byte[])"{"choices":[{"content_filter_results":{"; line: 1, column: 40]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: Unexpected end-of-input in field name at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"less"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727270634,"i"; line: 1, column: 327]
2024-09-25 15:23:55 [boundedElastic-1] com.azure.core.util.logging.ClientLogger.performLogging()
ERROR: com.azure.json.implementation.jackson.core.io.JsonEOFException: Unexpected end-of-input in field name at [Source: (byte[])"{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":"less"},"finish_reason":null,"index":0,"logprobs":null}],"created":1727270634,"i"; line: 1, column: 327]
Exception or Stack Trace
There is no stack trace, only the logs shared above.
To Reproduce
To reproduce this issue, execute the following code: https://github.com/jdubois/langchain4j-examples/blob/f6932500e8e61abbef15d02e6de8af7e2cfdcb29/github-models-examples/src/main/java/GitHubModelsStreamingChatModelExamples.java#L14
If you want to test the whole example, it's in this branch https://github.com/jdubois/langchain4j-examples/tree/azure-sdk-for-java-issue-2 that I created for this issue.
Code Snippet
The code is at https://github.com/jdubois/langchain4j-examples/blob/f6932500e8e61abbef15d02e6de8af7e2cfdcb29/github-models-examples/src/main/java/GitHubModelsStreamingChatModelExamples.java#L14
Expected behavior
There should not errors in the logs.
Setup (please complete the following information):
- OS: Mac OS
- IDE: IntelliJ
- Library/Libraries: azure-ai-inference_1.0.0-beta.1
- Java version: 23
- App Server/Environment: none
- Frameworks: none
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- Bug Description Added
- Repro Steps Added
- Setup information Added
Activity