Skip to content

Commit

Permalink
Merge pull request HamaWhiteGG#48 from HamaWhiteGG/dev
Browse files Browse the repository at this point in the history 
Implements i18n(pt-BR) for prompts in SQLDatabaseChain
  • Loading branch information
@HamaWhiteGG
HamaWhiteGG authored Jul 20, 2023
2 parents 09a0886 + 16d17ea commit afdbfbc
Show file tree
Hide file tree
Showing 6 changed files with 281 additions and 131 deletions.
19 changes: 16 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,20 @@ println(output);
Certainly! I am an AI language model developed by OpenAI called GPT-3. I have been trained on a vast amount of text data from the internet, which allows me to generate human-like responses to a wide range of queries and engage in conversations. My purpose is to assist and provide information to the best of my abilities. Is there anything specific you would like to know about me?
```
## 4. Run Test Cases from Source
## 4. i18n for SQLDatabaseChain
If you want to choose other language instead english, just set environment variable on your host. If you not set, then **en-US** will be default
```shell
export USE_LANGUAGE=pt_BR
```
#### 4.1 Available Languages
| Language | Value |
|--------------------|-------|
| English(default) | en_US |
| Portuguese(Brazil) | pt_BR |
## 5. Run Test Cases from Source
```shell
git clone https://github.com/HamaWhiteGG/langchain-java.git
Expand All @@ -342,12 +355,12 @@ This project uses Spotless to format the code. If you make any modifications, pl
mvn spotless:apply
```
## 5. Support
## 6. Support
Don’t hesitate to ask!
[Open an issue](https://github.com/HamaWhiteGG/langchain-java/issues) if you find a bug in Flink.
## 6. Fork and Contribute
## 7. Fork and Contribute
This is an active open-source project. We are always open to people who want to use the system or contribute to it. Please note that pull requests should be merged into the **dev** branch.
Contact [me](baisongxx@gmail.com) if you are looking for implementation tasks that fit your skills.
Expand Down
138 changes: 10 additions & 128 deletions langchain-core/src/main/java/com/hw/langchain/chains/sql/database/prompt/Prompt.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

import com.hw.langchain.output.parsers.list.CommaSeparatedListOutputParser;
import com.hw.langchain.prompts.prompt.PromptTemplate;
import com.hw.langchain.utils.ResourceBundleUtils;

import java.util.List;
import java.util.Map;
Expand All @@ -30,164 +31,45 @@
*/
public class Prompt {

private static String PROMPT_SUFFIX = """
Only use the following tables:
{table_info}
private static String PROMPT_SUFFIX = ResourceBundleUtils.getString("prompt.suffix");

Question: {input}
""";

private static String _DEFAULT_TEMPLATE =
"""
Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. Unless the user specifies in his question a specific number of examples he wishes to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.
Never query for all the columns from a specific table, only ask for a the few relevant columns given the question.
Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _DEFAULT_TEMPLATE = ResourceBundleUtils.getString("prompt.default.template");

public static PromptTemplate PROMPT = new PromptTemplate(List.of("input", "table_info", "dialect", "top_k"),
_DEFAULT_TEMPLATE + PROMPT_SUFFIX);

private static String _DECIDER_TEMPLATE =
"""
Given the below input question and list of potential tables, output a comma separated list of the table names that may be necessary to answer this question.
Question: {query}
Table Names: {table_names}
Relevant Table Names:
""";
private static String _DECIDER_TEMPLATE = ResourceBundleUtils.getString("prompt.decider.template");

public static PromptTemplate DECIDER_PROMPT = new PromptTemplate(_DECIDER_TEMPLATE,
List.of("query", "table_names"),
new CommaSeparatedListOutputParser());

private static String _mysql_prompt =
"""
You are a MySQL expert. Given an input question, first create a syntactically correct MySQL query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per MySQL. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in backticks (`) to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use CURDATE() function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _mysql_prompt = ResourceBundleUtils.getString("prompt.database.mysql");

public static PromptTemplate MYSQL_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _mysql_prompt + PROMPT_SUFFIX);

private static String _h2_prompt =
"""
You are a H2 expert. Given an input question, first create a syntactically correct H2 query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per H2. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in backticks (`) to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use CURDATE() function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _h2_prompt = ResourceBundleUtils.getString("prompt.database.h2");;

public static PromptTemplate H2_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _h2_prompt + PROMPT_SUFFIX);

private static String _mariadb_prompt =
"""
You are a MariaDB expert. Given an input question, first create a syntactically correct MariaDB query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per MariaDB. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in backticks (`) to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use CURDATE() function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _mariadb_prompt = ResourceBundleUtils.getString("prompt.database.mariadb");;

public static PromptTemplate MARIADB_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _mariadb_prompt + PROMPT_SUFFIX);

private static String _oracle_prompt =
"""
You are an Oracle SQL expert. Given an input question, first create a syntactically correct Oracle SQL query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the FETCH FIRST n ROWS ONLY clause as per Oracle SQL. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use TRUNC(SYSDATE) function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _oracle_prompt = ResourceBundleUtils.getString("prompt.database.oracle");

public static PromptTemplate ORACLE_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _oracle_prompt + PROMPT_SUFFIX);

private static String _postgres_prompt =
"""
You are a PostgreSQL expert. Given an input question, first create a syntactically correct PostgreSQL query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per PostgreSQL. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use CURRENT_DATE function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _postgres_prompt = ResourceBundleUtils.getString("prompt.database.postgres");

public static PromptTemplate POSTGRES_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _postgres_prompt + PROMPT_SUFFIX);

private static String _sqlite_prompt =
"""
You are a SQLite expert. Given an input question, first create a syntactically correct SQLite query to run, then look at the results of the query and return the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for at most {top_k} results using the LIMIT clause as per SQLite. You can order the results to return the most informative data in the database.
Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.
Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
Pay attention to use date('now') function to get the current date, if the question involves "today".
Use the following format:
Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
""";
private static String _sqlite_prompt = ResourceBundleUtils.getString("prompt.database.sqlite");

public static PromptTemplate SQLITE_PROMPT =
new PromptTemplate(List.of("input", "table_info", "top_k"), _sqlite_prompt + PROMPT_SUFFIX);
Expand Down
59 changes: 59 additions & 0 deletions langchain-core/src/main/java/com/hw/langchain/utils/ResourceBundleUtils.java
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.hw.langchain.utils;

import org.apache.commons.lang3.LocaleUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import lombok.experimental.UtilityClass;

import java.util.Locale;
import java.util.ResourceBundle;

/**
* @author Borges
*/
@UtilityClass
public class ResourceBundleUtils {

private static final Logger LOG = LoggerFactory.getLogger(ResourceBundleUtils.class);

private static final String BUNDLE_NAME = "i18n.messages";
private static final String USE_LANGUAGE_ENV = "USE_LANGUAGE";

public static String getString(String key) {
String language = System.getenv(USE_LANGUAGE_ENV);

Locale locale = new Locale("en", "US");

if (language != null) {
try {
locale = LocaleUtils.toLocale(language);
LOG.info("Using locale from {}", language);
} catch (Exception ex) {
LOG.warn("Can't load locale for language {}, setting default en_US", language);
}
}

ResourceBundle bundle = ResourceBundle.getBundle(BUNDLE_NAME, locale);
return bundle.getString(key);
}

}
Loading

0 comments on commit afdbfbc

Please sign in to comment.