Developing a Simple Database
This is an introduction to using the Simple Database API to develop an application on a MckoiDDB network installation. If you have not yet installed and setup MckoiDDB, you should follow the Quick Start Guide to make a MckoiDDB installation.
What is the Simple Database API?
Once MckoiDDB has been installed on a network it is able to perform basic data management and query tasks across a cluster of machines. Unfortunately this is not useful unless there is also a way for developers to model data structures appropriate for their needs. The Simple Database API is a data model that does just this - allowing developers to create and manipulate structures such as tables with indexes, maps (associative arrays), and objects that may be changed arbitrarily in a strictly consistent and parallel way using transactions and snapshot isolation.
In this guide we will create a Simple Database instance on our MckoiDDB installation and demonstrate some Java applications that store and retrieve information using structures provided by the Simple Database data model.
Setting up a Simple Database
Before we start writing any Java code I will show how to create a Simple Database instance on a root server in the network. To create the new instance we will use the 'console' script to access the administration console. The console can also be accessed by running the following command;
java -cp ..\..\lib\mckoiddb.jar com.mckoi.runtime.AdminConsole -netpassword [the network password]
From the administration console prompt enter the following command;
add path com.mckoi.sdb.SimpleDatabase testdb to 127.0.0.1:3500
The above example assumes you have set up a single machine installation on your local machine as described in the Quick Start Guide. If you are running MckoiDDB over a network, 127.0.0.1:3500 would instead be the IP address of the machine you have set as the root server. After running this command you will have a Simple Database instance path named 'testdb'.
If you'd like to check the current paths you have set up on your network you can use the 'show paths' command from the console prompt which should now include the following report;
MckoiDDB> show paths Root server: 127.0.0.1:3500 +Name: testdb Consensus Class: com.mckoi.sdb.SimpleDatabase Status: 114 bytes (in 0 tables 0 files)
Connecting to the Simple Database
The following shows the basic Java code needed to connect to our Simple Database instance;
import com.mckoi.data.*;
import com.mckoi.network.*;
import com.mckoi.sdb.*;
import java.io.IOException;
import java.util.Properties;
class MyConnectApp {
public static void main(String[] args) {
// This is the server we need to connect to get onto the network.
Properties prop = new Properties();
prop.setProperty("manager_address", "127.0.0.1:3500");
prop.setProperty("network_password", [your network password]);
// Connect to the network,
MckoiDDBClient client;
try {
client = MckoiDDBClientUtils.connectTCP(prop);
}
catch (IOException e) {
e.printStackTrace();
return;
}
// Create an SDBSession object on our named path.
SDBSession session = new SDBSession(client, "testdb");
// ... Now we can interact with the database
}
}
If you are connecting to a MckoiDDB installation over a network, the manager_address IP address in the above example should be changed to the location of the Manager server on your network. If this code fails to connect then you should check that the network.conf file of all the machine nodes in the network includes the IP address of the client machine in the connect_whitelist property.
Transactions and Concurrency Control
Before we move on to some more example code, a quick overview of the transaction system in MckoiDDB should be covered.
In MckoiDDB a transaction is a fully isolated snapshot 'view' of the database. To make a permanent change to the database, you must first create a transaction (SDBTransaction), then make your modification, and then commit the transaction. MckoiDDB is strongly consistent meaning it is impossible for a transaction to see changes made by other concurrent transactions. Any changes committed are visible to future transactions started after the commit. Also, MckoiDDB provides optimistic concurrency control which provides a guarantee that read and writes can not lock.
When a transaction tries to commit a conflicting update (for example, when a transaction tries to delete a row that was concurrently removed by another transaction), the commit function of the second transaction raises a com.mckoi.network.CommitFaultException. The first transaction to commit will succeed provided it does not also make conflicting changes in other structures. The transaction is rolled back when a commit fault happens.
The Simple Database API offers two types of primary data structures; Files and Tables, each which support their own concurrency control models.
For files, concurrency conflicts will happen at commit when;
- A file with the same name is created, modified or deleted by concurrent transactions.
- The contents of a file are modified by concurrent transactions.
For tables, a commit fault happens when;
- A table with the same name is created or deleted by concurrent transactions.
- The structure of a table is altered by concurrent transactions (for example, a column is deleted and added).
- The same row is updated or deleted by concurrent transactions.
Note that Simple Database tables do not have constraint definitions that could cause a conflict to happen when rows are inserted into a table. Row inserts alone can not cause a commit fault.
File Structures in a Simple Database
As mentioned in the previous section the Simple Database API provides a File like data structure. A File is a binary object that can grow and shrink to any size and data can be written to and read from any position. The Simple Database file object works a lot like java.io.RandomAccessFile but with one important difference - data can be efficiently shifted inside the file by any amount. The 'shift' function makes it possible to support other structured data objects such as sorted list/sets and maps over the top of the file data structure simply by wrapping the File object.
The following example creates a com.mckoi.sdb.SDBFile and inserts some data into the file;
// Create a transaction
SDBTransaction transaction = session.createTransaction();
// Make sure the file 'BinaryFile' is created and fetch it
boolean created = transaction.createFile("BinaryFile");
SDBFile binary_file = transaction.getFile("BinaryFile");
// Move to start of the file and insert 3 integers, {1, 2, 3}
binary_file.position(0);
binary_file.putInt(1);
binary_file.putInt(2);
binary_file.putInt(3);
// Commit the change,
try {
transaction.commit();
}
catch (CommitFaultException e) {
// This will only happen if another transaction has
// concurrently created, deleted or modified the file.
System.out.println("Oops, we failed to commit.");
}
The next example wraps an SDBFile with a com.mckoi.data.StringData object. StringData allows free manipulation of a character string stored in the database;
SDBTransaction transaction = session.createTransaction();
boolean created = transaction.createFile("StringFile");
SDBFile string_file = transaction.getFile("StringFile");
// Wrap the file with a StringData object and add "Hello World"
// to the end and insert "Goodbye World " at the start.
StringData string_ob = new StringData(string_file);
string_ob.append("Hello World");
string_ob.insert(0, "Goodbye World ");
// Print the complete string,
System.out.println(string_ob.toString());
try {
transaction.commit();
}
catch (CommitFaultException e) {
System.out.println("Oops, we failed to commit.");
}
SDBFile can also support sorted lists and maps. The following example uses the com.mckoi.data.PropertySet class to wrap a file and provide a String->String associative array.
boolean created = transaction.createFile("MyPropertySet");
SDBFile property_file = transaction.getFile("MyPropertySet");
// Wrap the file with the PropertySet class and make some
// associations.
PropertySet properties = new PropertySet(property_file);
properties.setProperty("Hello", "world");
properties.setProperty("Foo", "bar");
properties.setProperty("Austin", "Powers");
// Output a property,
System.out.println("Austin = " + properties.getProperty("Austin"));
As you can see, these File objects are very expressive and can be used to implement all sorts of other more complex persistent data structures. Internally in MckoiDDB these file-like data structures are used as the primitive building blocks of all other persistent data structures. Do not be afraid to create as many Files as you need as small or large as necessary. They are designed to be efficient and light weight.
As mentioned in the Transactions section, one drawback to using Simple Database files is the simple concurrency control model they have. If the same file is modified concurrently a commit fault will occur on the second transaction when it commits. Simple Database tables are a more appropriate structure for handling data structures with frequent changes in a distributed environment.
Table Structures in a Simple Database
The Simple Database API includes an implementation of a table structure called com.mckoi.sdb.SDBTable. SDBTable can store string values in the cell elements of the table, supports indexes on columns (SDBIndex), and provides a cursor interface for traversing rows and indexes (RowCursor). Simple Database tables have a more complex commit model than Simple Database files allowing concurrent inserts, updates and deletes to the same table object provided the changes don't conflict.
In the following example we create a new SDBTable called 'BooksWithRobots' and set up the columns and indexes of the table and populates it with some data.
// Create an SDBSession object on our named path.
SDBSession session = new SDBSession(client, "testdb");
// Create a new transaction,
SDBTransaction transaction = session.createTransaction();
// Create the table 'BooksWithRobots'
boolean created = transaction.createTable("BooksWithRobots");
// If we created a new table then we populate it with data
if (created) {
SDBTable robot_books = transaction.getTable("BooksWithRobots");
// Add the columns and indexes of the table,
robot_books.addColumn("name");
robot_books.addColumn("author");
robot_books.addColumn("year");
robot_books.addIndex("author");
robot_books.addIndex("year");
// Insert 4 rows,
robot_books.insert();
robot_books.setValue("name", "The Hitchhiker's Guide to the Galaxy");
robot_books.setValue("author", "Douglas Adams");
robot_books.setValue("year", "1979");
robot_books.complete();
robot_books.insert();
robot_books.setValue("name", "I, Robot");
robot_books.setValue("author", "Isaac Asimov");
robot_books.setValue("year", "1950");
robot_books.complete();
robot_books.insert();
robot_books.setValue("name", "2001 : A Space Odyssey");
robot_books.setValue("author", "Authur C. Clarke");
robot_books.setValue("year", "1968");
robot_books.complete();
robot_books.insert();
robot_books.setValue("name", "The Rest of the Robots");
robot_books.setValue("author", "Isaac Asimov");
robot_books.setValue("year", "1964");
robot_books.complete();
}
// Commit the changes,
try {
transaction.commit();
}
catch (CommitFaultException e) {
// In this example, this could only happen if a
// concurrent transaction also created a table called
// 'BooksWithRobots'
System.out.println("Oops, we failed to commit.");
}
The following example queries the author index and prints a list of all the books in the table from the author "Isaac Asimov".
// Create a new transaction,
SDBTransaction transaction = session.createTransaction();
// Fetch the 'BooksWithRobots' table,
SDBTable robot_books = transaction.getTable("BooksWithRobots");
// Fetch the author index,
SDBIndex index = robot_books.getIndex("author");
// Find the sub-index of all entries that are 'Isaac Asimov' from the
// author index.
index = index.sub("Isaac Asimov", true, "Isaac Asimov", true);
// Print the name and year of the books,
for (SDBRow row : index) {
System.out.print(row.getValue("name"));
System.out.print(" (");
System.out.print(row.getValue("year"));
System.out.println(")");
}
Where to go for more information
Check out Simple Database API for the JavaDocs of these APIs. You can also follow the documentation link at the top of any page for more technical information about MckoiDDB.