Skip to content

Jenesis Data Store: a dynamic, cross platform, high performance, ORM data-mapper. Designed to assist in rapid development and data mining

License

Notifications You must be signed in to change notification settings

SubiyaCryolite/jds

Repository files navigation

Build Status Maven Central Javadocs License Size

Jenesis Data Store

Jenesis Data Store (JDS) was created to help developers persist data to a strongly-typed portable JSON format.

JDS has four goals:

  • To allow for the rapid development of complex Java/Kotlin systems with stringent data definition / quality requirements
  • To provide a flexible and reliable framework to persist and retrieve data against.
  • To provide a robust, strongly-typed Field Dictionary.
  • To leverage JSON as a datastore over EAV based paradigms.

The library eliminates the need to modify schemas once a class has been altered.

It also eliminates all concerns regarding "breaking changes" in regards to fields and their addition and/or removal.

Put simply, JDS is useful for any developer that requires a flexible data store running on top of a Relational databases.

JDS is licensed under the 3-Clause BSD License

Design

The concept behind JDS is quite simple. Extend a base Entity class, define strongly-typed Fields and then map them against implementations of the Property interface.

JDS was designed to avoid reflection and its potential performance pitfalls. As such mapping functions which are overridden, and invoked at least once at runtime, are used to enforce/validate typing as instead of annotations. This is discussed below in section 1.1.4 "Binding properties".

Features

  • Transparent persistence
  • Supports the persistence of NULL values for boxed types (e.g Integer, Double)
  • Full support for generics and inheritance
  • Easily integrates with new or existing databases
  • Portable format which can be serialised to JSON allows for the flexibility of EAV without its drawbacks
  • Supports MySQL, T-SQL, PostgreSQL, Oracle 11G, MariaDB and SQLite
  • Supports a robust Field Dictonary allowing for metadata such as Tags and Alternate Coding to be applied to Fields and Entities.

Maven Central

You can search on The Central Repository with GroupId and ArtifactId Maven Search for Maven Search

Maven

<dependency>
    <groupId>io.github.subiyacryolite</groupId>
    <artifactId>jds</artifactId>
    <version>20.4</version>
</dependency>

Gradle

compile 'io.github.subiyacryolite:jds:20.4'

Dependencies

The library depends on Java 1.8. Both 64 and 32 bit variants should suffice. Both the Development Kit and Runtime can be downloaded from here.

Supported Databases

The API currently supports the following Relational Databases, each of which has their own dependencies, versions and licensing requirements. Please consult the official sites for specifics.

Database Version Tested Against Official Site JDBC Driver Tested Against
PostgreSQL 9.5 Official Site org.postgresql
MySQL 5.7.14 Official Site com.mysql.cj.jdbc.Driver
MariaDb 10.2.12 Official Site org.mariadb.jdbc.Driver
Microsoft SQL Server 2008 R2 Official Site com.microsoft.sqlserver
SQLite 3.16.1 Official Site org.sqlite.JDBC
Oracle 11g Release 2 Official Site oracle.jdbc.driver.OracleDriver

1 How it works

1.1 Creating Classes

Classes that use JDS need to extend Entity.

import io.github.subiyacryolite.jds.Entity;

public class Address : Entity

However, if you plan on using interfaces they must extend IEntity. Concrete classes can then extend Entity

import io.github.subiyacryolite.jds.Entity;
import io.github.subiyacryolite.jds.IEntity;

public interface IAddress : IEntity

public class Address : IAddress

1.1.1 Annotating Classes

Every class/interface which extends Entity/IEntity must have its own unique Entity ID as well as an Entity Name. This is done by annotating the class, or its (parent) interface.

@EntityAnnotation(id = 1, name = "address", description = "An entity representing address information")
class Address : Entity()

Entity IDs MUST be unique in your application, any value of type long is valid. Entity Names do not enforce unique constraints but its best to use a unique name regardless. These values can be referenced to mine data.

1.1.2 Defining Fields

Fields are big part of the JDS framework. Each Field MUST have a unique Field Id. Field Names do not enforce unique constraints but its best to use a unique name regardless. These values can be referenced to mine data. Every Field that you define can be one of the following types.

JDS Field Type Java Type Description
DateTimeCollection Collection<LocalDateTime> Collection of type LocalDateTime
DoubleCollection Collection<Double> Collection of type Double
EntityCollection Collection<Class<? extends Entity>> Collection of type Entity
FloatCollection Collection<Float> Collection of type Float
IntCollection Collection<Integer> Collection of type Integer
ShortCollection Collection<Short> Collection of type Short
LongCollection Collection<Long> Collection of type Long
StringCollection Collection<String> Collection of type String
UuidCollection Collection<UUID> Collection of type UUID
Blob byte[] or InputStream Blob values
Boolean boolean / Boolean Boolean values
Entity Class<? extends Entity> Object of type Entity
DateTime LocalDateTime DateTime instances based on the host machines local timezone
Date LocalDate Local date instances
Double double / Double Numeric double values
Duration Duration Object of type Duration
EnumCollection Collection<Enum> Collection of type Enum
Enum Enum Object of type Enum
Float float / Float Numeric float values
Int int / Integer Numeric integer values
Short short / Short Numeric SHORT values
Long long / Long Numeric long values
MonthDay MonthDay Object of type MonthDay
Period Period Object of type Period
String String String values with no max limit
Time LocalTime Local time instances
YearMonth YearMonth Object of type YearMonth
ZonedDateTime ZonedDateTime Zoned DateTime instances
Uuid UUID UUID instances

We recommend defining your Fields as static constants

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String)
    val PlotNumber = Field(2, "plot_number", FieldType.Int)
    val Area = Field(3, "area_name", FieldType.String)
    val ProvinceOrState = Field(4, "province_name", FieldType.String)
    val City = Field(5, "city_name", FieldType.String)
    val Country = Field(7, "country_name", FieldType.String)
    val PrimaryAddress = Field(8, "primary_address", FieldType.Boolean)
    val Timestamp = Field(9, "timestamp", FieldType.DateTime)
}

Furthermore, you can add descriptions, of up to 256 characters, to each field

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String, "The street name of the address")
    val PlotNumber = Field(2, "plot_number", FieldType.Int, "The street name of the address")
    val Area = Field(3, "area_name", FieldType.String, "The name of the area / neighbourhood")
    //...
}

JDS also supports Tags which can be applied to each Field and Entity definitions. Tags can be defined as a set of strings, there is no limit on how many tags a field can have. This can be useful for categorising certain kinds of information

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String, description = "The street name of the address", tags = setOf("AddressInfo", "ClientInfo", "IdentifiableInfo"))
    //...
}

1.1.3 Defining Enums

JDS Enums are an extension of Fields. Usually these values would be represented by Check Boxes, Radio Buttons or Combo Boxes on the front-end.

First we'd define a standard JDS Field of type Enum.

import io.github.subiyacryolite.jds.Field
import io.github.subiyacryolite.jds.enums.FieldType

public class Fields
{
    val Direction = Field(10, "direction", FieldType.Enum)
}

Then, we can define our actual enum in the following manner.

enum class Direction {
    North, West, South, East
}

Lastly, we create an instance of the JDS Field Enum type.

import io.github.subiyacryolite.jds.FieldEnum

object Enums {
    val Directions = FieldEnum(Direction::class.java, Fields.Direction, *Direction.values())
}

Behind the scenes these Enums will be stored as either:

  • an Integer (FieldType.Enum);
  • a String (FieldType.EnumString);
  • an Integer Array (FieldType.EnumCollection); or
  • a String Collection (FieldType.EnumStringCollection)

1.1.4 Binding Properties

Depending on the type of Field, JDS will require that you set you objects properties to one of the following IValue container types.

JDS Field Type Container Java Mapping Call Kotlin Mapping Call
DateTimeCollection MutableCollection<LocalDateTime> mapDateTimes map
DoubleCollection MutableCollection<Double> mapDoubles map
EntityCollection MutableCollection<Class<? extends Entity>> map map
FloatCollection MutableCollection<Float> mapFloats map
IntCollection MutableCollection<Integer> mapInts map
LongCollection MutableCollection<Long> mapLongs map
StringCollection MutableCollection<String> mapStrings map
Boolean IValue<Boolean> mapBoolean map
Blob IValue<ByteArray> map map
Entity Class<? extends Entity> map map
Date IValue<LocalDate> mapDate map
DateTime IValue<LocalDateTime> mapDateTime map
Double IValue<Double> mapNumeric map
Duration IValue<Duration> mapDuration map
Enum IValue<Enum> mapEnum map
EnumCollection Collection<Enum> mapEnums map
Float IValue<Float> mapNumeric map
Int IValue<Integer> mapNumeric map
Long IValue<Long> mapNumeric map
MonthDay IValue<MonthDay> mapMonthDay map
Period IValue<Period> mapPeriod map
String IValue<String> mapString map
Time IValue<LocalTime> mapTime map
YearMonth IValue<YearMonth> mapYearMonth map
ZonedDateTime IValue<ZonedDateTime> mapZonedDateTime map
Uuid IValue<ZonedDateTime> mapZonedDateTime map

To simplify the mapping Process Jds has the following helper classes defined:

  • Generic containers (Entities and Enums)
    • ObjectValue
  • Non null containers
    • BlobValue
    • BooleanValue
    • DoubleValue
    • DurationValue
    • EnumValue
    • FloatValue
    • IntegerValue
    • LocalDateValue
    • LocalDateTimeValue
    • LocalTimeValue
    • LongValue
    • MonthDayValue
    • PeriodValue
    • ShortValue
    • StringValue
    • UuidValue
    • YearMonthValue
    • ZonedDateTimeValue
  • Nullable containers
    • NullableBlobValue
    • NullableBooleanValue
    • NullableDoubleValue
    • NullableDurationValue
    • NullableEnumValue
    • NullableFloatValue
    • NullableIntegerValue
    • NullableLocalDateValue
    • NullableLocalDateTimeValue
    • NullableLocalTimeValue
    • NullableLongValue
    • NullableMonthDayValue
    • NullablePeriodValue
    • NullableShortValue
    • NullableStringValue
    • NullableUuidValue
    • NullableYearMonthValue
    • NullableZonedDateTimeValue

Note: JDS assumes that all collection types will not contain null entries.

Note: Collection types can be of any valid type e.g. ArrayList, LinkedList, HashSet etc

After your class and its properties have been defined you must map the property to its corresponding Field using the map() method. I recommend doing this in your primary constructor.

The example below shows a class definition with valid properties and bindings. With this your class can be persisted.

Note that the example below has a 3rd parameter to the map method, this is the Property Name

The Property Name is used by the JDS Field Dictionary to know which property a particular Field is mapped to within an Entity.

This is necessary as one Field definition can be mapped to a different property amongst different Entities.

For example a Field called "FirstName" could be mapped to a property called "firstName" in one Entity and a property called "givenName" in another.

package io.github.subiyacryolite.jds.tests.entities

import io.github.subiyacryolite.jds.Entity
import io.github.subiyacryolite.jds.annotations.EntityAnnotation
import io.github.subiyacryolite.jds.beans.property.NullableBooleanValue
import io.github.subiyacryolite.jds.beans.property.NullableShortValue
import io.github.subiyacryolite.jds.tests.constants.Fields
import java.time.LocalDateTime

data class Address(
        private val _streetName: IValue<String> = StringValue(),
        private val _plotNumber: IValue<Short?> = NullableShortValue(),
        private val _area: IValue<String> = StringValue(),
        private val _city: IValue<String> = StringValue(),
        private val _provinceOrState: IValue<String> = StringValue(),
        private val _country: IValue<String> = StringValue(),
        private val _primaryAddress: IValue<Boolean?> = NullableBooleanValue(),
        private val _timestamp: IValue<LocalDateTime> = LocalDateTimeValue()
) : Entity(), IAddress {

    override fun bind() {
        super.bind()
        map(Fields.StreetName, _streetName, "streetName")
        map(Fields.PlotNumber, _plotNumber, "plotNumber")
        map(Fields.ResidentialArea, _area, "area")
        map(Fields.City, _city, "city")
        map(Fields.ProvinceOrState, _provinceOrState, "provinceOrState")
        map(Fields.Country, _country, "country")
        map(Fields.PrimaryAddress, _primaryAddress, "primaryAddress")
        map(Fields.TimeStamp, _timestamp, "timestamp")
    }

    var primaryAddress: Boolean?
        get() = _primaryAddress.get()
        set(value) = _primaryAddress.set(value)

    var streetName: String
        get() = _streetName.get()
        set(value) = _streetName.set(value)

    var plotNumber: Short?
        get() = _plotNumber.get()
        set(value) = _plotNumber.set(value)

    var area: String
        get() = _area.get()
        set(value) = _area.set(value)

    var city: String
        get() = _city.get()
        set(value) = _city.set(value)

    var provinceOrState: String
        get() = _provinceOrState.get()
        set(value) = _provinceOrState.set(value)

    var country: String
        get() = _country.get()
        set(value) = _country.set(value)

    var timeOfEntry: LocalDateTime
        get() = _timestamp.get()
        set(timeOfEntry) = _timestamp.set(timeOfEntry)
}

1.1.5 Binding Objects and Object Arrays

JDS can also persist embedded objects and object arrays.

All that's required is a valid Entity or IEntity subclass to be mapped to a Field of type Entity or EntityCollection .

import io.github.subiyacryolite.jds.Field
import io.github.subiyacryolite.jds.enums.FieldType

object Fields
{
    val Addresses = Field(23, "addresses", FieldType.EntityCollection, "A collection of addresses")
}
import io.github.subiyacryolite.jds.FieldEntity

object Entities {
    val Addresses: FieldEntity<Address> = FieldEntity(Address::class.java, Fields.Addresses)
}
import io.github.subiyacryolite.jds.Entity
import io.github.subiyacryolite.jds.annotations.EntityAnnotation
import io.github.subiyacryolite.jds.tests.constants.Entities

@EntityAnnotation(id = 2, name = "address_book")
data class AddressBook(
        val addresses: MutableCollection<IAddress> = ArrayList()
) : Entity() {

    override fun bind() {
        super.bind()
        map(Entities.Addresses, addresses, "addresses")
    }
}

1.2 CRUD Operations

1.2.1 Initialising the database

In order to use JDS you will need an instance of DbContext. Your instance of DbContext will have to extend one of the following classes:

  • MariaDbContext;
  • MySqlContext;
  • OracleContext;
  • SqLiteDbContext; or
  • TransactionalSqlContext

After this you must override the dataSource property.

Please note that your project must have the correct JDBC driver in its class path. The drivers that were used during development are listed under Supported Databases above.

These samples use HikariCP to provide connection pooling for enhanced performance.

Postgres example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.PostGreSqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class PostGreSqlContextImplementation : PostGreSqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.pg.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:postgresql://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

MySQL Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.MySqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class MySqlContextImplementation : MySqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.mysql.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:mysql://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

MariaDb Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.MariaDbContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class MariaDbContextImplementation : MariaDbContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.mariadb.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:mariadb://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"

        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Microsoft SQL Server Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.TransactionalSqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class TransactionalSqlContextImplementation : TransactionalSqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.tsql.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.jdbcUrl = "jdbc:sqlserver://${properties["dbUrl"]}\\${properties["dbInstance"]};databaseName=${properties["dbName"]}"
        hikariConfig.dataSourceProperties = properties //additional props
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Oracle Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.OracleContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class OracleContextImplementation : OracleContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.ora.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:oracle:thin:@${properties["dbUrl"]}:${properties["dbPort"]}:${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Sqlite Example

import io.github.subiyacryolite.jds.context.SqLiteDbContext
import org.sqlite.SQLiteConfig
import org.sqlite.SQLiteDataSource
import java.io.File
import javax.sql.DataSource

class SqLiteDbContextImplementation : SqLiteDbContext() {

    private val sqLiteDataSource: DataSource

    init {
        val path = File(System.getProperty("user.home") + File.separator + ".jdstest" + File.separator + "jds.db")
        if (!path.exists())
            if (!path.parentFile.exists())
                path.parentFile.mkdirs()
        val sqLiteConfig = SQLiteConfig()
        sqLiteConfig.enforceForeignKeys(true) //You must enable foreign keys in SQLite
        sqLiteDataSource = SQLiteDataSource(sqLiteConfig)
        sqLiteDataSource.url = "jdbc:sqlite:${path.absolutePath}"
    }

    override val dataSource: DataSource
        get () {
            Class.forName("org.sqlite.JDBC")
            return sqLiteDataSource
        }
}

With this you should have a valid data source allowing you to access your database. JDS will automatically setup its tables and procedures at runtime.

Furthermore, you can use the getConnection() method OR connection property from this dataSource property in order to return a standard java.sql.Connection in your application.

1.2.2 Initialising JDS

Once you have initialised your database you can go ahead and initialise all your JDS classes. You can achieve this by mapping ALL your JDS classes in the following manner.

fun initialiseJdsClasses(dbContext: DbContext)
{
    dbContext.map(Address::class.java);
    dbContext.map(AddressBook::class.java);
}

You only have to do this once at start-up. Without this you will not be able to persist or load data.

1.2.3 Creating objects

Once you have defined your class you can initialise them. A dynamic id is created for every Entity by default (using javas UUID class). This value is used to uniquely identify an object and it data in the database. You can set your own values if you wish.

    val primaryAddress = Address()
    primaryAddress.overview.id = "primaryAddress" //explicit id defined, JDS assigns a value by default on instantiation
    primaryAddress.area = "Norte Broad"
    primaryAddress.city = "Livingstone"
    primaryAddress.country = "Zambia"
    primaryAddress.plotNumber = null
    primaryAddress.provinceOrState = "Southern"
    primaryAddress.streetName = "East Street"
    primaryAddress.timeOfEntry = LocalTime.now()
    primaryAddress.primaryAddress = PrimaryAddress.YES

1.2.4 Saving objects (Portable Format)

...

1.2.5 Loading objects (Portable Format)

...

Development

I highly recommend the use of the IntelliJ IDE for development.

Contributing to Jenesis Data Store

If you would like to contribute code you can do so through Github by forking the repository and sending a pull request targeting the current development branch.

When submitting code, please make every effort to follow existing conventions and style in order to keep the code as readable as possible.

Bugs and Feedback

For bugs, questions and discussions please use the Github Issues.

Special Thanks

To all our users and contributors!