[Microsoft.Extensions.AI.OpenAI] Add logging support to HandleToolCallsAsync

When using `OpenAIRealtimeExtensions.HandleToolCallsAsync` there is no way to capture the error information from failed tool calls. The errors are reported to the model but then swallowed without any additional logging:
https://github.com/dotnet/extensions/blob/5161cb90e1db3c3b6192ce40a3406dabc53db35a/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIRealtimeExtensions.cs#L107-L145

The equivalent logging code for the non realtime case is in place:
[extensions/src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs at 5161cb90e1db3c3b6192ce40a3406dabc53db35a · dotnet/extensions](https://github.com/dotnet/extensions/blob/5161cb90e1db3c3b6192ce40a3406dabc53db35a/src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs#L637).

	private static async Task<ConversationItem?> GetFunctionCallOutputAsync(
	this ConversationItemStreamingFinishedUpdate update,
	IReadOnlyList<AIFunction> tools,
	bool? detailedErrors = false,
	JsonSerializerOptions? jsonSerializerOptions = null,
	CancellationToken cancellationToken = default)
	{
	if (!string.IsNullOrEmpty(update.FunctionName)
	&& tools.FirstOrDefault(t => t.Metadata.Name == update.FunctionName) is AIFunction aiFunction)
	{
	var jsonOptions = jsonSerializerOptions ?? AIJsonUtilities.DefaultOptions;

	var functionCallContent = FunctionCallContent.CreateFromParsedArguments(
	update.FunctionCallArguments, update.FunctionCallId, update.FunctionName,
	argumentParser: json => JsonSerializer.Deserialize(json,
	(JsonTypeInfo<IDictionary<string, object>>)jsonOptions.GetTypeInfo(typeof(IDictionary<string, object>)))!);

	try
	{
	var result = await aiFunction.InvokeAsync(functionCallContent.Arguments, cancellationToken).ConfigureAwait(false);
	var resultJson = JsonSerializer.Serialize(result, jsonOptions.GetTypeInfo(typeof(object)));
	return ConversationItem.CreateFunctionCallOutput(update.FunctionCallId, resultJson);
	}
	catch (JsonException)
	{
	return ConversationItem.CreateFunctionCallOutput(update.FunctionCallId, "Invalid JSON");
	}
	catch (Exception e) when (!cancellationToken.IsCancellationRequested)
	{
	var message = "Error calling tool";

	if (detailedErrors == true)
	{
	message += $": {e.Message}";
	}

	return ConversationItem.CreateFunctionCallOutput(update.FunctionCallId, message);
	}
	}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Microsoft.Extensions.AI.OpenAI] Add logging support to HandleToolCallsAsync #5715

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development