LianjaDev

Lianja 6.3 introduced a new Key/Value Store (KVS) to support data streaming and IoT applications.

A key-value database is a type of nonrelational database that uses a simple key-value method to store data. A key-value database stores data as a collection of key-value pairs in which a key serves as a unique identifier. Both keys and values can be anything, ranging from simple objects to complex compound objects.

KVS is implemented as a collection of functions built into Lianja.

It is a high performance object store for storing keys with values that may be objects that are automatically JSON encoded and decoded.

The KVS itself is a B+ tree index which contains keys and values inside the KVS file.

It is a 64-bit file store so it can contain a huge number of indexed Key/Value pairs. Deleted space is reused.

It handles automatic locking supporting concurrent high speed CRUD operations.

KVS can be used for data streaming applications as its very high performance and with the kvs_removeLast() function can be used to implement a stack of data, together with kvs_removeFirst() which can be used to implement a queue of data.

KVS is used in Lianja to handle forthcoming offline database support. This will be explained later in another article.

My intension is to enhance KVS later with sharding/partitioning to facilitate high performance clustered indexing.

Let's look at the built-in functions that are used with KVS.

Code:

// create a new KVS with keys of nKeylen length
kvsid = kvs_create( cFilename[.kvs], nKeylen [, bEnableLogging] )

// open an existing KVS
kvsid = kvs_open( cFilename[.kvs] )

// close the KVS
kvs_close( kvsid )

// Add/update key/value for the specified key. 
// If the value is an object or array it is automatically JSON encoded.
ok = kvs_set( kvsid, cKey | nKey, objectName | arrayName | cExpr)
 
// Add key/value for the specified key. 
// If the value is an object or array it is automatically JSON encoded.
ok = kvs_add( kvsid, cKey | nKey, objectName | arrayName | cExpr)
 
// Update key/value for the specified key. 
// If the value is an object or array it is automatically JSON encoded.
ok = kvs_update( kvsid, cKey | nKey, objectName | arrayName | cExpr)
 
// fetch key/value for the specified key. 
// If the value is an object then cMembername specifies a member (property) to return.
value = kvs_get( kvsid, cKey | nKey [,cDefault [,cMembername] ] )

// remove the specified key. 
kvs_remove( kvsid, cKey | nKey )

// fetch all keys/values for the specified key. 
// CKeyPattern may end in * to match wildcard keys. returns an dynarray of arrays.
object = kvs_getAll( kvsid [, cKey | cKeyPattern ] )

// fetch all keys for the specified key. CKeyPattern may end in * to match wildcard keys.
array = kvs_keys( ksvid, [, cKey | cKeyPattern ] )

// fetch all values for the specified key. CKeyPattern may end in * to match wildcard keys.
array = kvs_values( ksvid, [, cKey | cKeyPattern ] )

// fetch a range of key/value pairs (offset starts at 1). 
// returns an array of two elements, key and value
array = kvs_getRange( ksvid, nOffset1, nCount )

// returns count of number of key/value pairs in the store
numeric = kvs_count( kvsid )

// remove all key/value pairs from the store
kvs_clear( kvsid )

// Use for queue. returns an array of two elements, key and value
array = kvs_removeFirst( kvsid )  

// Use for stack. returns an array of two elements, key and value
array = kvs_removeLast( kvsid )

// position on first key/value pair returns an array of two elements, key and value
array = kvs_rewind( kvsid )

// position on next key/value pair returns an array of two elements, key and value
array = kvs_next( kvsid )

// returns true if key contained in the store
logical = kvs_contains( kvsid,  cKey | nKey )

// list binary transaction log associated with the store to the console 
// (useful during development for debugging)
kvs_listLog( kvsid )

// list the key/values in the store to the console 
// (useful during development for debugging)
kvs_list( kvsid )

// rebuild the KVS file from the binary transaction log
kvs_rebuild( cFilename )

// returns min key
kvs_min( kvsid )

// returns max key
kvs_max( kvsid )