• oxdb-lite is toy database, the initial iteration of oxdb completely written in python loosely implementing oxdb Architecture design refer oxdb-compressed desing to understand the System Architecture concepts of oxdb
• oxdb-engine core is build groundup in rust using oxdb-compressed desing which is more optimized production ready design

• 1 about
• 2 installation
• 3 docs
• 4 oxdb access interfaces
• 5 oxdb-lite shell
• 6 oxdb-lite core
• 7 oxdb-lite client-server
• 8 perfomance profiling
• 9 lib implementation
• 10 directory tree

• oxdb-compressed desing core design which is losely implemented here
• oxdb-engine main production level implementation of the oxdb design
• ox-onnx onnx runtime and model interface manager

• oxdb-lite is an open-source, AI-native vector embedding database core written in python tailored for efficient storage and retrieval of vector embeddings. It is also designed to support the construction of Retrieval-Augmented Generation (RAG) systems, making it an ideal solution for managing knowledge databases in AI assistant applications.

• oxdb-lite is custamized to run in navtive machine with minimal memory and uses onnx models in ox-onnx runtime for generating vectore embaddings from data

• oxdb-lite is build on top of oxdoc core oxdb-lite data management and documents handelling storage retrieval serialization presistance are all done using oxdoc

always build from source for latest and bug free version

• refer ox-onnx if there is any installation dependency issues

pip install git+https://github.com/ox-ai/oxdb-lite.git

not recomented as yet not published the latest stable relese in pypi

pip install oxdb-lite

• refere py-notebooks and test.log.ipynb and core.log for understanding the underlying usage
• oxdb_lite
  • core.log
  • shell.log
  • server.log
  • client.log
  • ai.embed
  • oxdoc
    • oxdld
    • oxdsd
    • oxdmem
    • doc
      • oxdbin

• oxdb-lite shell mode
• oxdb-lite core mode
• oxdb-lite client-server mode

• initiate a shell session in terminall for quick access

                                                                                                             
┌──(lokesh㉿kali)-[~/Documents/0-lab/ox-ai/oxdb-lite]
└─$ oxdblite.shell 
oxdb> search "data"
oxdb : 
{'data': ['data'],
 'embeddings': [],
 'entries': 1,
 'idx': ['1'],
 'index': [{'doc': 'log-doc',
            'hid': '3a6eb0790f39ac87c94f3856b2dd2c5d110e6811602261a9a923d3bb23adc8b7'}],
 'sim_score': [0.9999999932464888]}
oxdb> ^C
Exiting shell...
Initiating Clean Up
Clean Up Compelete
                          

to start oxdb-lite shell run below cmd refere shell.log.md for further detials

• cmd to initiate terminal session which can intract directly to oxdb-lite which means its a thick client that runs the oxdb-line engine with in
• seprate client server mode thin client will be made avilable

oxdblite.shell

• on terminal access : to send db query with out starting a session

oxdblite.shell 'search("data")'

• if path not correctly assigned due too sudo or admin access use below cmd

python -m oxdblite.shell

• to directly work with oxdb-lite
• direct access gives lot of low level api access for inspecting the db
• refere db.log

from oxdb_lite.core import Oxdb

# init disk persisted vectore database
db=Oxdb("noteDB")

# create a doc which holds all reverlent data together
log = db.get_doc("note-doc")

# can push data by single line cmd
log.push("data-1")
# can add uid or uuids and additional metadata
log.push(data="need to implement pdfsearch db",uid="uuid",metadata={"note-type" :"project-note","org":"ox-ai"})
# support different data types
log.push(datax ={"datas":["project-queue","priority is db" ]})

# can retrive data by metadata filters or string search
log.pull(uid="projects")

# can also apply metadata filers, search string, and all other query methods methods
log.search("data",2)

# # here comes the main plot do vector search in different methods
# by (Optional[str], optional): The search method. Defaults to "dp".
#                 - "dp": Dot Product (default)
#                 - "ed": Euclidean Distance
#                 - "cs": Cosine Similarity
log.search(query="project plan",topn=2,by="ed",where={"org":"ox-ai"},where_data={"search_string":"db"})

# output
{'entries': 2,
 'data': ['need to implement pdfsearch db',
          'need to implement pdfsearch db with ui'],
 'sim_score': [1.343401897402224, 1.3484805614337063]}

• in clien server mode need to start the server first with command

#default apikey = "oxdb_lite-prime"
oxdblite.server --apikey "your-api-key" --host --port 8008

• you can use python client binding high level interface code which is same as core db access, refere test.client and client.log
• java script api coming soon u can directly acces using api
• for more detials refer server.log.md
• if path not correctly assigned due to sudo or admin access use below cmd

#default apikey = "oxdb_lite-prime"
python -m oxdb_lite.server --apikey "hi0x" --host --port 8008

# Set the environment variable
export OXDB_API_KEY="oxdb-apikey-001"
# Access the environment variable
echo $OXDB_API_KEY

import os
# Set the environment variable
os.environ["OXDB_API_KEY"] = "oxdb-lite-101"
# Access the environment variable
api_key = os.getenv("OXDB_API_KEY")

will update soon

oxdb
.
├── __init__.py
├── ai
│   ├── __init__.py
│   └── embed.py
├── client
│   ├── __init__.py
│   └── log.py
├── core
│   ├── __init__.py
│   ├── log.py
│   └── types.py
├── oxdoc_lite
│   ├── __init__.py
│   ├── db
│   │   ├── __init__.py
│   │   ├── cache.py
│   │   ├── freeindex.py
│   │   ├── ld.py
│   │   ├── mem.py
│   │   └── sd.py
│   ├── doc
│   │   └── markdown.py
│   ├── dp.py
│   ├── oxdbin.py
│   └── utils.py
├── server
│   ├── __init__.py
│   ├── __main__.py
│   └── log.py
├── settings.py
├── shell
│   ├── __init__.py
│   ├── __main__.py
│   ├── cli.py
│   └── log.py
└── utils
    ├── __init__.py
    └── dp.py

10 directories, 29 files