Skip to content

Commit

Permalink
[docs][serve] Fix subsection title hierarchy (ray-project#41746)
Browse files Browse the repository at this point in the history
Subsections are appearing in the Serve Examples page as separate docs vs sections of the Streaming doc.

Signed-off-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
  • Loading branch information
angelinalg authored Dec 8, 2023
1 parent 7001982 commit aa86ef6
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions doc/source/serve/tutorials/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ This tutorial serves the [DialoGPT](https://huggingface.co/microsoft/DialoGPT-sm
pip install transformers
```

# Create a Streaming Deployment
## Create a Streaming Deployment

Open a new Python file called `textbot.py`. First, add the imports and the [Serve logger](serve-logging).

Expand Down Expand Up @@ -73,7 +73,7 @@ Run the model with `serve run textbot:app`, and query it from another terminal w

You should see the output printed token by token.

# Stream inputs and outputs using WebSockets
## Stream inputs and outputs using WebSockets

WebSockets let you stream input into the application and stream output back to the client. Use WebSockets to create a chatbot that stores a conversation with a user.

Expand Down Expand Up @@ -134,7 +134,7 @@ Run the model with `serve run chatbot:app`. Query it using the `websockets` pack

You should see the outputs printed token by token.

# Batch requests and stream the output for each
## Batch requests and stream the output for each

Improve model utilization and request latency by batching requests together when running the model.

Expand Down

0 comments on commit aa86ef6

Please sign in to comment.