Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(experimental): Add Configurable System Prompts Feature #744

Merged
merged 11 commits into from
Dec 16, 2024

Conversation

thecodacus
Copy link
Collaborator

Add Configurable System Prompts Feature

Overview

This PR introduces a dynamic system prompt configuration feature that allows users to select different LLM system prompts based on their needs. This enhancement provides flexibility in how the AI assistant behaves and processes requests, with initial testing showing promising results across models including GPT-4 and Claude 2.5 Coder 32k.

Key Changes

1. Prompt Library Implementation

  • Added new PromptLibrary class to manage different system prompts
  • Implemented two initial prompts:
    • Default: The battle-tested standard system prompt
    • Optimized: An experimental version designed for lower token usage
  • Created extensible architecture for future prompt additions

2. Settings Integration

  • Added prompt selection UI in the Features tab
  • Implemented prompt persistence using cookies
  • Added new promptId state management in settings store

3. Backend Integration

  • Modified chat API to support dynamic prompt selection
  • Updated stream text functionality to incorporate selected prompts
  • Maintained backward compatibility with existing chat functionality

4. File Structure Updates

  • Moved prompts to a dedicated directory under lib/common/prompts/
  • Created new optimized.ts for the experimental prompt version
  • Reorganized prompt-related utilities for better maintainability

Technical Details

Prompt Library Structure

The PromptLibrary class provides:

  • Centralized prompt management
  • Type-safe prompt registration and retrieval
  • Configurable prompt options (working directory, HTML elements, etc.)
  • Easy extensibility for adding new prompts

Settings Integration

  • New promptStore atom in settings store
  • Persistent storage using cookies
  • UI integration in Features tab with dropdown selection
  • Real-time prompt switching without page reload

Testing Results

  • Tested with GPT-4 and Claude 2.5 Coder 32k
  • Optimized prompt showed improved performance in token usage
  • Maintained response quality across different models
  • Verified prompt persistence across sessions

Migration Impact

  • No breaking changes to existing functionality
  • Default prompt remains unchanged for existing users
  • No database migrations required

Future Improvements

  • Add more specialized prompts for different use cases
  • Implement prompt customization interface
  • Add prompt performance metrics
  • Support for model-specific prompt optimizations

How to Test

  1. Navigate to Settings > Features
  2. Select different prompts from the dropdown
  3. Test chat functionality with each prompt
  4. Verify prompt persistence after page reload
  5. Test with different LLM models if available

Preview

GPT4o:

Optimized.Prompt.gpt4o.mp4

qwen 2.5 Coder 32B

Optimized.Prompt.Qwen.mp4

@thecodacus thecodacus changed the title System prompt variations feat(experimental): Add Configurable System Prompts Feature Dec 15, 2024
@thecodacus thecodacus requested review from dustinwloring1988 and wonderwhy-er and removed request for dustinwloring1988 December 15, 2024 15:31
@dustinwloring1988
Copy link
Collaborator

@thecodacus My day just got better, cant wait to use this.

@dustinwloring1988
Copy link
Collaborator

works on windows.

@thecodacus
Copy link
Collaborator Author

we need code template import after this. I would like to add starter templates which will super charge the project outputs

@dustinwloring1988
Copy link
Collaborator

that would take this to the next level.

@wonderwhy-er
Copy link
Collaborator

But can't we already use them with github import?
Like here is bolt.new starters
https://boltstarter.com/

It has link like this
https://bolt.new/github.com/donvito/bolt-admin-dashboard

In reality its this repo
https://github.com/donvito/bolt-admin-dashboard

I just go to bolt.diy and import git
It took a bit of time but I got this
image

Bolt.new works faster for this for some reason
But it does work with bolt.diy

@thecodacus
Copy link
Collaborator Author

thecodacus commented Dec 15, 2024

But can't we already use them with github import?

yes but thats manual selection. I am talking about

user: i want a todo app
llm: selects vite starter templates and start the project

another scenario

user: i want to design an admin dashboard with schadcn ui and typescript 
llm: selects schadcn starter templates and start the project

basically based on user's qurey it should select the best starter template and start.

currently lots of users are saying they are not getting good results compared to bolt.new this is the reason.
for any general query bolt/new is first pulling a suitable starter template, then starting the work

@thecodacus
Copy link
Collaborator Author

Bolt.new works faster for this for some reason

cuz they are not cloning it using browser resources 😅

@wonderwhy-er
Copy link
Collaborator

Bolt.new works faster for this for some reason

cuz they are not cloning it using browser resources 😅

Even if they were pulling on server side and then streaming it to client it would be the same no?
May be only part I can imagine is being isomorphic git not being good.

Another is some form of caching they may be do.

@wonderwhy-er
Copy link
Collaborator

But can't we already use them with github import?

yes but thats manual selection. I am talking about

user: i want a todo app
llm: selects vite starter templates and start the project

another scenario

user: i want to design an admin dashboard with schadcn ui and typescript 
llm: selects schadcn starter templates and start the project

basically based on user's qurey it should select the best starter template and start.

currently lots of users are saying they are not getting good results compared to bolt.new this is the reason. for any general query bolt/new is first pulling a suitable starter template, then starting the work

I guess you mean dozen of templates for various setups than? That are very barebone? Like next.js starter, vue starter and so on?

@thecodacus
Copy link
Collaborator Author

thecodacus commented Dec 15, 2024

I guess you mean dozen of templates for various setups than? That are very barebone? Like next.js starter, vue starter and so on?

Yes exactly. it should have most of the skeleton ready. so that the llm does not have to write them. and the results will instantly become pretty nice

I was building for this.. https://github.com/thecodacus/vite-react-ts-template
notice the .bolt folder

@thecodacus
Copy link
Collaborator Author

May be only part I can imagine is being isomorphic git not being good.

yes isomorphic-git is trying to do an actual git operation in browser, and using a simulated file system plus it also needs to get loaded and initialized, where as they are using github api it to pull all the files directly and them pushing it.. both has its pros and cons.

@thecodacus thecodacus merged commit de2cb43 into stackblitz-labs:main Dec 16, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants