[DH-5733] Support schemas column to add a db connection (#466)

* [DH-5733] Support schemas column to add a db connection

* DBs without schema should store None

* Add ids in sync-schemas endpoint

* Support multi-schemas for refresh endpoint

* Add schema not support error exception

* Add documentation for multi-schemas

* Fix sync schema method

* Sync-schemas endpoint let adding ids from different db connection

* Fix refresh endpoint

* Fix table-description storage

* Fix schema_name filter in table-description repository

* DH-5735/add support for multiple schemas for agents

* DH-5766/adding the validation to raise exception for queries without schema in multiple schema setting

* DH-5765/add support multiple schema for finetuning

---------

Co-authored-by: mohammadrezapourreza <m1378.prz@gmail.com>

DH5682/fixing the wrong exception handling with tools (#451)

[DH-5517] Create a CustomError to send a description (#416)

Start Kula changes, and new endpoints and remove the old ones (#275)

* Start Kula changes, and new endpoints and remove the old ones

* Dh 5073/agents generating sql queries (#276)

* DH-5073/text-to-SQL agents

* DH-5073/removing the intermediate steps

* DH-5073/removing the intermediate steps

* DH-5073/update tests

* Create services and improve models

* DH-5080/refactor_engine_with_new_resources (#277)

* DH-5080/refactor_engine_with_new_resources

* DH-5080/update the tests

* DH-5080/update the testst

* DH-5080/updating the evaluatord

* Dh 5080/sql generations services (#278)

* DH-5080/sql_generations service

* DH-5080/reformat with black

* DH-5080/removing generator from env example

* DH-5082/nl_generations implementation (#279)

* DH-5080/removing db connection validation (#280)

* Create Prompt service (#281)

* DH-5086/endpoints for nl and sql generation (#282)

* DH-5085 Re-name golden-records as golden-sqls (#283)

* [DH-5068] Add metadata and created_at fields (#284)

* [DH-5087] Add Prompts GET endpoints (#285)

* DH-5089/fixing_the_new_agent (#286)

* DH-5089/fixing_the_new_agent

* DH-5089/the get endpoints

* DH-5089/refactor with black

* DH-5089/renaming golden records

* DH-5090/execute sql generation endpoint (#287)

* DH-5090/execute sql generation endpoint

* DH-5090/do not initiate the repo

* DH-5099/save sql_generation on initial then update (#288)

* [DH-5088] Add endpoints to update metadata (#289)

* DH-5103/nl-generation bug (#290)

* [DH-5100] Script to migrate old collections (#291)

* Replace PATCH endpoint for PUT (#292)

* Fix migrate script (#293)

* solving the issue with PUT sql_generations (#294)

* solving the issue with PUT sql_generations

* reformat

* DH-5104/update the fine-tuning metadata (#295)

* DH-5104/fixing the issue of finetuning (#297)

* Fix datatime responses (#298)

* Fix metadata in table-descriptions (#299)

* Dh 5104/metadata issue for endpoints (#300)

* DH-5104/fixing the golden sqls endpoint

* DH-5401/reformat

* Replace nl_answer by text (#301)

* DH-5110/fixing evalutor issue (#302)

* DH-5110/fixing evalutor issue

* DH-5110/fixing the 500 error for sql execution

* Improve resouces requests (#303)

* DH-5109/csv export endpoint (#304)

* DH-5094/fix the ARRAY issue (#309)

* [DH-5113] Return datetimes with explicit UTC (#311)

* Fix migration script (#306)

* Dh-5094/reverting_the_fixes (#310)

* Dh-5094/reverting_the_fixes

* Update the states of the scanner

* change the states of the scanner

* DH-5094/add exception handling

* DH-5094/add exception handling

* DH-5094/exception handling

* DH-3135/Nulltype handling plus script for scanner (#312)

* DH-5137/removing the markdown from queries (#313)

* DH-5147/rename the csv endpoint (#314)

* DH-5146/add get finetunings to engine (#316)

* DH-5149/add delete endpoint and make basellm and alias optional (#317)

* DH-5149/add delete endpoint and make basellm and alias optional

* DH-5149/update states

* DH-5158/update the finetuning statuses (#318)

* DH-5158/update the finetuning statuses

* finxing the openai states

* Add value

* DH-5153 Add golden sqls in batches (#319)

* DH-5167/update the finetuning statuses (#320)

* DH-5167/update the finetuning statuses

* DH-5167/refromat with black

* Dh 5168/raise exception for not availlable finetuning (#321)

* DH-5168/better exception handling for SQL generation

* DH-5168/reformat with black

* DH-5166/the parser to extract SQL queries (#322)

* [DH-5173] Fix table-descriptions PUT endpoint response (#323)

* DH-5177/fixing the empty sql queries (#325)

* DH-5175/fix_the_status_update_on_engine (#324)

* DH-5181/fixing the prompts of fientuning agent (#326)

* DH-5184/fix (#327)

* [DH-5187] Fix table-descriptions filter by table_name field (#328)

* DH-5184/fix the query parameters of finetuning (#329)

* DH-5184/fix the query parameters of finetuning

* DH-5184/reformat

* DH-5204/sql generation error (#330)

* DH-5205/chromadb script fix (#331)

* [DH-5212] Fix UTC timezone (#332)

* Fix datatime fields (#333)

* DH-5225/fix the sql injection (#336)

* Fix datetime timezone format for the responses (#337)

* DH-5232/changing the error message for no sql (#338)

* Rename migration file (#339)

* Dh 5228/docs for new endpoints (#340)

* DH-5228/add finetuning module

* DH-5228/add finetuning endpoints

* DH-5225/adding the docs for prompts

* DH-5228/add nl generations docs

* DH-5228/adding the docs for sql generations

* DH-5228/add sql generations resource

* [DH-5238] Fix migration script (#341)

* DH-5225/persist nl generations and return id (#343)

* [DH-5117] Sync-schema returns a Table descriptions list (#342)

* [DH-5239] Fix datetime format (#344)

* DH-5251/avoid question rephrasing (#346)

* DH-5253/fix the finetuning get endpoint (#347)

---------

Co-authored-by: Mohammadreza Pourreza <71866535+MohammadrezaPourreza@users.noreply.github.com>

Dh 4983 (#244)

* [DH-4983] Document response endpoint in Readme

* Document in read the docs the CSV download feature

DH-4907 Rename llm_credentials.api_key as llm_api_key for body params…

… and responses (#229)

DH-4784 Store all responses and code refactoring (#195)

* DH-4784 Store all responses and refactoring

* Calculate confidence score when a response is created

* Add GET endpoints to retrieve one resource for responses, table-descriptions and questions

DH-4703 Set table-desription status (#183)

* DH-4703 Set table-desription status

* DH-4703 Added script to update table_descriptions rows

DH-4608 Refactor Dataherald Engine API and resources (#151)

* DH-4608 Replace db_alias by db_connection_id

* Improve database-connections endpoints

* Improve table-descriptions endpoints

* Improve queries endpoints

* Add filters for table descriptions endpoint

* Fix nl-query-responses endpoints and sql-query-executions endpoint

* Use DatabaseConnectionRepository

* Fix nl-query-responses POST endpoint

* Fix tests

* Add scrip to update version

* Update documentation and readme file

* DH-4598 ported changes in other branch for docs into the refactor branch

* remove reference to contributing from modules

---------

Co-authored-by: Amir A. Zohrenejad <aazo11@gmail.com>