I think this would be quite useful.

I've pulled your branch and it has some problems, it saves the params in the settings.json as strings which makes the server bug out if you don't manually edit them.

@clementine-of-whitewind Interesting, I haven't noticed that. Which params?

for example if I change the temperature the ooba server bugs out because it didn't expect a string when doing some comparison operation. ive ran ooba with llama-7b. manually removing the parantheses in the settings.json for temperature and other parameters restored functionality.

Okay. It seems to work with strings for me as well, but storing them that way was certainly bug which should be now fixed in this PR branch.

Hey, @im-not-tom
I'll pull that to my fork if you don't mind.

@SillyLossy I don't mind at all.

It is excellent, probably the simplest way to get LLaMa+GPTQ working.

Look's like the API usage with streaming is different: https://github.com/oobabooga/text-generation-webui/blob/main/api-example-stream.py
It would be great if you implement stream support.

FYI: the other way around is now possible, ie making text-generation-webui pretend to be kobold while leaving TavernAI unchanged

oobabooga/text-generation-webui#342

In my experience, this PR worked a bit better than the KoboldAI API implementation on the text-generation-webgui. For example, with --auto-devices --gpu-memory 8 --cpu-memory 45 --no-stream --extensions api on text-generation-webgui side set as parameters using the KoboldAI API extension, the model oftentimes generates strange non-related output about Python (not errors, but text about Python), which in my experience, was not the issue with this PR. Too bad it now conflicts with the mainstream main branch

In my experience, this PR worked a bit better than the KoboldAI API implementation on the text-generation-webgui. For example, with --auto-devices --gpu-memory 8 --cpu-memory 45 --no-stream --extensions api on text-generation-webgui side set as parameters using the KoboldAI API extension, the model oftentimes generates strange non-related output about Python (not errors, but text about Python), which in my experience, was not the issue with this PR. Too bad it now conflicts with the mainstream main branch

That being said, singleline mode does not seem to work either with the KoboldAI API on the text-generation-webgui, or this may be an issue on TavernAI's side. I have multigen, free name mode, and singleline mode enabled. The behavior I am getting is TavernAI not ceasing the text generation even after the line has been generated, which results in lines being generated that are completely irrelevant to the dialogue, which of course results in waste of time.