This project is part of the @thi.ng/umbrella monorepo.
@thi.ng/rstream based triple store & reactive query engine with declarative query specs related to Datalog / SPARQL. Inserted triples / facts are broadcast to multiple indexing streams and any query subscriptions attached to them. This enables push-based, auto-updating query results, which are changing each time upstream transformations & filters have been triggered.
Triples are 3-tuples of
[subject, predicate, object]
. Unlike with traditional
RDF
triple stores, any JS data types can be used as subject, predicate or
object (though support for such must be explicitly enabled & this
feature is currently WIP).
- Dynamic & declarative dataflow graph construction via high-level data specs and/or functions
- Entirely based on stream abstractions provided by @thi.ng/rstream
- All data transformations done using dynamically composed tranducers
- Query optimizations
- Extensive re-use of existing sub-query results (via subscriptions)
- Interim result de-duplication / dataflow gates
- Push-based, auto-updating query results
ALPHA - bleeding edge / work-in-progress
This project is currently still in early development and intended as a continuation of the Clojure based thi.ng/fabric, this time built on the streaming primitives provided by @thi.ng/rstream.
yarn add @thi.ng/rstream-query
Package sizes (gzipped, pre-treeshake): ESM: 2.66 KB / CJS: 2.75 KB / UMD: 2.62 KB
- @thi.ng/api
- @thi.ng/associative
- @thi.ng/checks
- @thi.ng/equiv
- @thi.ng/errors
- @thi.ng/math
- @thi.ng/rstream
- @thi.ng/rstream-dot
- @thi.ng/transducers
- tslib
Several demos in this repo's /examples directory are using this package.
A selection:
Screenshot | Description | Live demo | Source |
---|---|---|---|
Triple store query results & sortable table | Demo | Source |
import { TripleStore, asTriples } from "@thi.ng/rstream-query";
import { trace } from "@thi.ng/rstream";
// create store with initial set of triples / facts
const store = new TripleStore([
["london", "type", "city"],
["london", "part-of", "uk"],
["portland", "type", "city"],
["portland", "partOf", "oregon"],
["portland", "partOf", "usa"],
["oregon", "type", "state"],
["usa", "type", "country"],
["uk", "type", "country"],
]);
// alternatively, convert an object into a sequence of triples
const store = new TripleStore(asTriples({
london: {
type: "city",
partOf: "uk"
},
portland: {
type: "city",
partOf: ["oregon", "usa"]
},
oregon: { type: "state" },
uk: { type: "country" },
usa: { type: "country" },
});
// compile the below query spec into a dataflow graph
// pattern items prefixed w/ "?" are query variables
// this query matches the following relationships
// using all currently known triples in the store
// when matching triples are added or removed, the query
// result updates automatically...
// currently only "where" and bounded "path" sub-queries are possible
// in the near future, more query types will be supported
// (e.g. optional relationships, pre/post filters etc.)
store.addQueryFromSpec({
q: [
{
// all "where" subqueries are joined (logical AND)
where: [
// match any subject of type "city"
["?city", "type", "city"],
// match each ?city var's "part-of" relationships (if any)
["?city", "partOf", "?country"],
// matched ?country var must have type = "country"
["?country", "type", "country"]
]
}
],
// `bind` is an (optional) query post-processor and
// allows injection of new variables into the result set
// here we create a new var "answer" whose values are derived from
// the other two query vars
bind: {
answer: (res) => `${res.city} is located in ${res.country}`
},
// another post-processing step, only keeps "answer" var in results
select: ["answer"]
})
.subscribe(trace("results"))
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' } }
// helper fn to insert new city relationship to the store
const addCity = (name, country) =>
store.into([
[name, "type", "city"],
[name, "partOf", country],
[country, "type", "country"],
]);
addCity("berlin", "germany");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' } }
addCity("paris", "france");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' },
// { answer: 'paris is located in france' } }
After setting up the above query and its internal transformations, the generated dataflow topology then looks as follows:
- The blue nodes are
TripleStore
-internal index stream sources, emitting changes when new triples are added - The left set of red nodes are the sub-queries of the above
where
clause, responsible for joining the individual (S)ubject, (P)redicate and (O)bject sub-queries. - The results of these are then further joined (right red node) & transformed to produce the final solution set and post-process it
Btw. The diagram has been generated using
@thi.ng/rstream-dot
and can be recreated by calling store.toDot()
(for the above example)
The source code for the above example is here
(Many) more features forthcoming...
Karsten Schmidt
© 2018 - 2020 Karsten Schmidt // Apache Software License 2.0