I’ve been tasked with explaining the concepts (and origin) of the MV database (more commonly referred to as: Pick, Universe, Unidata and Reality), to a bunch of developers who are under the impression that there is only one database: SQL Server. They believe that the database world is flat! And, alarmingly, they are part of the vast majority of software developers, who have never gone beyond relational.

History of MV

It all started back in 1965. Yes! Way before Mr Codd “invented” the database in 1970 and IBM “invented” the database retrieval language, SEQUEL – later to become SQL. Two programmers (Dick Pick & Don Nelson), employed by TRW, were working on a US Army project and developed something called Generalized Information Retrieval Language System (GIRLS), which was up and running by 1968. See wikipedia for more info.

English

The first commercial port of the MV database, was Microdata’s Reality, which incorporated GIRLS under the name English. The core strength of English (as the name might suggest) was it’s ability to retrieve data using a human readable language. An example better illustrates than an explanation:

LIST ORDER.HEAD BY CUSTNO WITH DATE = "22/03/2009" DATE BREAK-ON CUSTNO ORDNO TOTAL ORDVAL

The keywords in this command are:

• BY – which tells us that the following field (CUSTNO) is what we need to sort by.
• WITH – indicates the selection criteria (DATE = “22/03/2009″)
• BREAK-ON – tells us what we need to sub-total on (CUSTNO)
• TOTAL – tells us what needs to be totalled (ORDVAL)

The rest is fairly obvious: ORDER.HEAD is the file/table and CUSTNO, DATE, ORDNO & ORDVAL are the fields/columns.

The other major strength of English is the Dictionary – which is used to define the fields/columns in the table. The dictionary, not only gives names to the columns (e.g. CUSTNO, DATE, ORDNO & ORDVAL), it also allows manipulation of the data. For instance, you probably want to see the DATE in human-readable form (rather than internal format: i.e. number of days since 01/01/1968), and you probably want to see the order value with 2DP rather than as an integer. It goes way beyond this, however. In addition to string manipulation; data formatting and database joins, all of these operations could be combined, in the wonderful F-Correlative.

Reverse Polish

The F-Correlative uses a notation called Reverse Polish, which allows operations to be performed on the top one or two items in the stack. Again, an example is the easiest way to explain:

8,3,2,+,-

Will first put “8″ onto the stack, followed by “3″, then “2″. So, “2″ is at the top.

We then reach our first operation (“+”), which will sum the top two values (“2″ and “3″) and remove them from the stack, and put the result (“5″) on the top.

The next item we reach is another operation (“-”) which will subtract “5″ from “8″.

Many implementations of Reverse Polish are limited to mathematical operations, but the F-Correlative provides conditional operators and access to any of the other functions available in the Dictionary – e.g. string manipulation, formatting, and database joins. Over time, yet more powerful functionality has been incorporated into the Dictionary (e.g. subroutine calls),

Multi-Values

The multi-value is just one part of a database structure called the Dynamic Array. The dynamic array provides the MV database with it’s flexibility, efficiency and power. In simple terms, the dynamic array is a record consisting of delimited (rather than fixed-length) fields. A well-known example of this is a CSV file. The dynamic array, however, doesn’t use the comma as a delimiter – it uses something called a Field Mark (also called Attribute Mark). It then goes one step further, by adding an additional dimension to the array (multi-values), which are delimited by the Value Mark.

The purpose of the multi-value is to allow repeating data to be easily accomodated within the database. This is quite useful, because, in “real life” a lot of “stuff” occurs more than once, and a database that accomodates this, requires less code. If we compare a CSV record with the MV equivalent (using a family tree as an example), here’s how they differ:

CSV:

ID,Parent Name,Child 1,Child 2,Child n,,,,,
1,Steve,Sarah,Charlie,,,,,,

MV (where “^” indicates a Field Mark and “]” is a Value Mark):

ID^Parent Name^Children
1^Steve^Sarah]Charlie

In the CSV record, we have an unknown number of columns, whereas in the MV record it’s fixed. And if we want to view this data, we enter the command “LIST FAMILY.TREE PARENT CHILDREN” which gives us this:

ID Parent Children
1  Steve  Sarah
          Charlie

I concede that comparing any type of database to a CSV file is a little one-sided. And, yes, we could achieve the same result with a “real” RDMS, but at the expense of having two tables. I would also concede that the MV concept is open to abuse. I’ve seen many examples of multi-values being used when a “normailised” approach would have been much better. And multi-values being used to collect up a bunch of non-repeating fields. This is where programming standards are required – a topic for another day, I think!

You have to admit that Apple have the perfect set up – being in total control of both hardware and software. It’s not surprising that they deliver such a great product. Next comes Microsoft, who at least have complete control of the software. But, in aiming for hardware ubiquity, have to accomodate the vagaries of many different hardware manufacturers.

The Linux world, however, is a far more complicated place. In many respects, the approach to hardware portability is even more ambitious than Microsoft. Linux distributions frequently offer ports for Sparc/Risc processors, in addition to Intel/AMD x86 and 64-bit CPUs. But that’s not the complicated bit! It’s how these distributions are put together that makes life so difficult. A Linux distribution is a compilation of a whole bunch of different components, all developed by different individuals, with different goals.

The Linux Kernel is just a small (but obviously core) part of what you get when installing a distribution. In addition, there are a number of libraries, utilities and compilers provided by the GNU project; the X-windows GUI framework; and a choice of desktop managers (including KDE & Gnome). And, when one of these components goes off track, it can be a frustrating time for both distributors and users.

For the past three years I’ve been running Kubuntu on my desktop – successfully moving through the releases (Edgy, Feisty & Gutsy) and then coming to a complete halt. For me, the current release (Hardy) has been a non-starter. Some major hacks to X-windows have left ATI Radeon graphics cards (virtually) unsupported. The distributor (Canonical) are not really at fault – they need to put together the pieces of each “distribution jigsaw puzzle” in a way that allows them to keep moving forward. And this is one of the really great things about open source: the pace of change & progress. I would still not hesitate to recommend the Ubuntu/Kubuntu distributions. But, for now, I sit in a cul-de-sac, hoping that some time soon, a hack to X-windows will set me on my way again.

The IT department consists of about 40: web developers, back-office system developers and support staff. In the 14 years I’ve been there, it has always been a struggle to get system documentation up to date and in line with reality.

Wow! How the wiki has changed that. We now have approaching 900 pages of documentation and contributions from 100% of the team. The reason, I believe it has been so successful is that developers who would normally adhere to Klingon Programming Standards, actually seem to like using the wiki.

The other major benefit of the wiki, is it’s version history. Now, when our software is modified, the associated wiki is updated and a comment is added referencing back to the project. A simple click (on the diff button) shows what changes were made for each project.

Anyone involved in software development should look at implementing a wiki.