Skip to content

Server exits abnormally on API call after long period of inactivity #1991

Closed
@QuantiusBenignus

Description

Have been using whisper.cpp server on my local machine in lieu of the "main" program, to serve Blurt and BlahST because of the performance advantage (~90 x real time, as described here ).

I load the server on machine startup and keep it running, but many times now, after a long period of inactivity (say an hour), when I send an API request, the server will blow up and exit abnormally. ps call shows it as zombie i.e. defunct and the logs say:

image
... ending with the curl API call being unsuccessful: "curl: (52) Empty reply from server"
(Timeline is bottom to top, I do not understand why it is repeating the request receipt and inference steps, a successful call doesn't look like that)

Anecdotally, "unspecified launch failure" is most often a segfault but could it be thread synchronization issue in this case?
Or is it indeed some sort of out-of-bounds memory access on memory that has been released on a timeout?
GPU is RTX3060, 12GB

I can probably test a server run without cuBLAS support to see if the issue persists.

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions