Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: cleaner completion traces (RUN-160) (#545)
Runtime component: voiceflow/general-runtime#1053 Instead of having 3 different top level completion traces, we should be protective of the "trace.type" namespace - makes for easier documentation. The pattern we want to encourage for users is to build specialized handlers around each type of trace, and there will be a centralized switch/if else where this happens. The less we have here, the better: https://github.com/voiceflow/libs/blob/6cb7a7b7667cc18f7e49abc78afe9e34c187086d/packages/base-types/src/trace/index.ts#L111-L128 There is now a `trace.payload.status` property that denotes if it is `start` `continue` or `end`. In the future we can attach more specific metadata to the `start` state. I believe users DO need an explicit start or stop message, because otherwise it would be on them to check the previous and next trace after the completion events to know to when to perform interface events. Imagine a webchat, there's likely an explicit function to create a chat bubble, then it gets streamed in text, and there is another function to end the bubble. In the future for Audio, this is extremely obvious on web interfaces: https://developer.mozilla.org/en-US/docs/Web/API/HTMLAudioElement Giving the users tokens mid-stream is kind of useless and just redundant data. For example, OpenAI/Anthropic's API has token metrics only on the last chunk: https://platform.openai.com/docs/api-reference/chat/object#chat/object-usage https://docs.anthropic.com/en/api/messages-streaming Finally, we will no longer be giving tokens are part of completion events. Even for regular text/speak steps we don't really give tokens are part of them. Instead we should have a unified type of trace for all token or billing usage in the future, including for TTS, AI Response, AI Set, etc. Today it's a debug trace, which you will still get with `completion_events=true`. Example client code **before**: ``` // completion events flag case "completion-start": process.stdout.write(trace.payload.completion ?? ""); break; case "completion-continue": process.stdout.write(trace.payload.completion ?? ""); break; case "completion-end": process.stdout.write(trace.payload.completion ?? "" + "\n"); break; ``` example client code after: ``` case "completion": process.stdout.write(trace.payload.content ?? ""); if (trace.payload.state === "end") { process.stdout.write("\n"); } break; ``` Co-authored-by: Tyler <tylerhan97@gmail.com>
- Loading branch information