Data models.

First Principles.

  1. Programs need persistent storage.
  2. Each program is free to create their own storage methods and rules.
  3. Users want to exchange data between programs.

Data-Freedoms

  1. Programs should make the user's data available to any process that requests it. This data is free and the application has no rights over how it can be used.
  2. Programs can use proprietary, binary or custom backends provided no data essential to the re-creation of the original is omitted from the free user data.
 

Problems with DCOP, CORBA, RPC etc.

Exchanging data is not just exchanging a document or sharing a file.
Users need more than simply passing data as a file between applications. This is useful but requires both programs to be locked into the one shared format.
Exchanging data is more than asking a procedure to provide a fixed data result.
Users need more than a fixed result. Calling a specific function in a specific CORBA or RPC compatible program will only give the result designed by the original developer. If the user wants data that is not specifically supported by the RPC calls, the data is inaccessible.

QOF tries to provide a generic interface to generic data without requiring the original program. It does this without preventing the original program from importing the data at a later date by always maintaining the data at the level of an object. Objects can be translated between programs but the original remains and can be updated by data in the translated objects.


Program- or backend- driven models.

QOF can provide data exchange in either situation but the question is central to how you design your program.

Program-centric design.

Is your database a backend to the program that can use other backends?
Do you want your program to work with different databases?

program-centric designs

The problem here is that each backend is incompatible with ProgramB. The data is locked.

Database-centric design.

Is your program a front end to a database that can use another front-end?
do you want your database to work with different programs?

database-centric designs

The problem here is that each program is limited to the shared backend. If the user does not have this particular database (or any database) server installed, the data is locked.


Data-centric design

Neither approach considers the data - each concentrates on the formatting of the data, the file format. Whether it is binary, text, SQL or anything else is irrelevant. What matters in a data-centric world is that the data is available on-demand to whichever process can use it.

Reconciling the two models.

GnuCash works with different databases. The database is a backend and it can be replaced. This is how GnuCash can work with SQLlite, how the postgres backend exists and how it also uses XML. One program, multiple backends.

QOF fits naturally into this scheme because it can provide a mechanism for adding backends, such as QSF, to an existing program. This is what I've done with pilot-link. I've added QOF to libpisock and in doing so added a backend as well. QOF has a second backend that is in development - a GUI data mining interface called DWI - and that will become available to any application using QOF as a library.

Alternatively, if you take the database-imperative to it's limits, you'll be creating a data set that can be exported to another database and use any number of front-ends but which cannot be used without a compatible database.

As QOF has no pre-determined front-end, it can work here too and by doing so, it automatically adds more backends. In effect, it gives you the best of both. You get an engine that can work with multiple front ends and multiple backends, with the same data being mapped or translated between them.

free design

Each backend is generic and part of QOF. Therefore any program that can exchange data with QOF can use whichever backend is suitable. Any other QOF compatible program can read the data, even if it doesn't use that backend normally. This increases platform independence, data accessibility and program flexibility.


Example

In example below, backendA1 is incompatible with programB which has it's own incompatible backend, B1 that programA cannot use.

However, all programs linked to QOF can share any or all of the three (in this example) QOF backends and adding a backend to QOF adds a backend to all QOF programs. By mapping ProgramA objects to ProgramB objects within QOF, the data in any of the Q backends also becomes available to both programs.

Work is under way to port the newer QOF code to other QOF applications that use internal QOF code like GnoTime and I aim to have all QOF programs using QOF only as an external library eventually.

There's nothing wrong with having a backend of your own, it's how GnuCash works, as long as there are sufficient QOF objects to import/export all the essential data.

example design

In this way, data from Backend, A1, can be loaded by ProgramA and saved, exported or streamed via backendQ1 to ProgramB, which is then free to save the data to it's own Backend, B1.

Equally, ProgramB can provide data via backendQ2 that ProgramA can understand.