The illusion of DataBase Abstraction Layers or Classes

Many people think that DB abstraction is cool. I get emails about this topic all the time. People suggesting we add a DB abstraction layer to b2evolution, or offering to do so themselves.

So why haven't we added one yet? Well, simply because DB abstraction doesn't work! :!:

DB abstraction layers will mostly hide the specific interface semantics between the application and the database. Thus you don't have to worry about the specific system call to connect to each particular DB. OK nice. So what? That's only the easiest part in porting an app from a DBMS to another... (BTW, we use a class (/evocore/db.class) in b2evolution for this too. You just have to slighlty alter this class if you want to connect to another DBMS than MySQL).

More sophisticated layers will also translate between different datatypes from one DBMS to another. But since datatypes tend to standardize among all popular DBMSes this is getting less of an issue as time passes by...

Finally, we get to the point where it hurts: the SQL syntax! There are vitually no two DBMSes that share the exact same SQL syntax, except for the most basic SELECT, INSERT, UPDATE and DELETE statements! >:-[ And it doesn't seem to be standardizing really... some still won't support even the minimalistic SQL 92 standard! >:XX

Just check it out with your 2 favorite DBMSes:

  • How do you perform a LEFT OUTER JOIN?
  • How do you concatenate columns into a single result string?
  • How do you limit results in the WHERE clause to a regular expression?
  • How do you handle the fact that when you INSERT and the primary key already exists, you want to UPDATE instead? (MySQL's REPLACE INTO syntax...)?

These are just a few common examples, but they're probably already enough to show you why you DB abstraction layer won't magically translate from you first favorite DBMS to your second... You'll have to rewrite many queries too... and sometimes you won't find equivalent functionnality (subqueries?) and you'll have to write extra application code...

Not to mention triggers and stored procedures where you wouldn't even dare to dream about some kind of compatibility.

The most advanced DB abstraction I have ever worked with was ODBC. Yes that thing actually did SQL syntax translation! But it depended on specific drivers to implement translation against a standardized syntax defined by Microsoft.

So today when I see some open source library pretending to perform DB abstraction and I can't find anything closely related to SQL syntax translation, I won't even consider it... It's useless to me. The real difficulty with handling multiple DBs is rewriting the SQL queries, triggers and stored procedures... it's not connecting to yet another fancy DBMS and SELECT 'hello'... |-|

There's more than the code...

Oh well... I think it's been too long since I last read some great wisdom like the one on Joel on Software.

I read this really insightful peace today about all the important things beyond just the actual software code.

Here's a funny quote:

Human emotions can be really, really superficial. In particular people ridiculously overvalue aesthetics and beauty when evaluating products. It's one of the reasons iPods, and, for that matter, Keanu Reeves, are so successful.

...but the whole article is definitely a must read!

Of course, this so much applies to b2evolution as well... :-/

Sendo X Flashlight

There's this freeware app, called Torch, that allows you to buse your Series 60 phone as a torch/flashlight by blanking the screen and maintaining the backlight on until you exit the app.

Pretty interesting (you know when you drop your garage keys at night... :P), except the screen backlight isn't even remotely as bright as the the camera Flash LED on the Sendo X. That's what I would like: an app allowing me to turn on/off the camera Flash on demand!

The incredible thing is that apparently noone ever bothered to release something like this... although it's very likely to be just a simple API call... Well... I guess not enough Series 60 developers own a Sendo X... :'(

MySQL Data Integrity Enforcement Caveats

Okay, I desperately lack time to write full articles posts lately, so I'm going to make this quick! :P

Background: Suppose you have a table named Songs and a table named Genres. Genres only contains Genre IDs and Genre Names. Songs contains all sorts of data, but at some point there is one field (let's call it Song_Genre_ID) that points to the Genre ID. Get the picture? I'm sorry, I don't have the time for an actual picture of this. That "pointer" is called a Foreign Key. The Genre_ID, being the Primary key.

Data Integrity Enforcement means that MySQL is going to prevent you from putting any crappy value into the Foreign Key (FK) that would not exist in the refered Primary Key (PK). It will also prevent you from deleting a Genre if there are songs pointing to it. The bottom line is: if you're serious about your DB, you can't do without!

Now how would you enforce the integrity? Basically with something like this (but you may want to check the MySQL manual for details, right? :P):

REFERENCES Genres (Genre_ID)

Now, here's my point: from my experience I have found that a hell lot of things can go wrong when you try to add that constraint, and it can be a nightmare to find out what's going wrong! As a matter of fact, you'll find out MySQL's error messages aren't very helpful >:XX...

So here's a checklist to follow in order to find out what's been going wrong:

  • If the error message is even less helpful than you could possibly imagine, check your version of MySQL. Versions 4.1+ definitely have better messages than previous versions.
  • Check that both of your tables are of type InnoDB. Don't even try it otherwise.
  • Check that the PK you are refering to is actually defined as the Primary Key for its table.
  • On versions < 4.1, check that the the FK is being properly INDEXed before creating the foreign key.
  • Check the data types of the FK and the PK. They should be exactly the same. And I really mean *exactly*! Check the "UNSIGNED" attribute also. If it's signed on one side and unsigned on the other, it won't work! (I recommend using INT UNSIGNED for most of your FKs and PKs).
  • Check that any existing values in the PK column have matches in the FK column at the time you try to create the constraint.
  • Check that the DEFAULT value for your FK also has a match in the FK column at the time you try to create the constraint.

Okay I hope I didn't forget anything. Of course, if you find another reason for the constraint creation to fail, I'd love to hear about it! ;)

Good luck! :>>

MySQL features by version breakdown

I could not find anything like this on the net, so I thought I'd make my own chart and share it... ;)

MySQL version 3.23.58
4.0.22 4.1.7 5.0
Release date 11-sept-03 27-oct-04 23-oct-04 not stable yet
Row level locking yes yes yes yes
Transactions yes yes yes yes
Foreign keys yes yes yes yes
UNION no yes yes yes
Subqueries (derived tables) no no yes yes
Multiple table DELETEs no yes yes yes
Unicode UTF-8 support yes yes yes yes
Column level charset support no no yes yes
Timezone handling no no yes yes
Full text indexing (MyISAM only) no UTF-8 no UTF-8 yes yes
Stored procedures no no no yes
Rudimentary triggers no no no yes
Updatatable views no no no yes
Server side cursors no no no yes
Master/slave one way replication yes yes yes yes
Replication over SSL no no yes yes
Clustering (not available on Windows) no no yes yes

Note: I have not included MySQL 3.23 without InnoDB because that version can do vitually nothing interesting. It's a pure joke. Unfortunately, it is the kind of joke hosting providers like to try on you when they claim they offer you a "DBMS"... yeah right... call it an electronic rollodex, no more :>>

I'm a little bit too lazy for styling this right now, sorry.
If you notice an error, please be so nice and report it. Thanks ;)