A brief history of MySQLdb

It was recently pointed out to me what a confusing mess the source repositories were for MySQLdb, so I spent a good bit of time cleaning them up. To understand how things got into the state they were in, and where we are going, a brief history lesson  required.

I first started working on MySQLdb in 1998. At the time, the best option for source control was CVS, and I had a local CVS repository for the project, and occasionally put up some tarballs. In 2001, I got the OK to make it an open source project, and development moved to SourceForge. Whatever you might think of SourceForge nowadays, it was a safe place to host your code, distribute files, and you got a couple nice extras like a bug tracker and forums. And SourceForge had CVS, of course, but nothing else at the time. Subversion existed in early form but was not yet available at SourceForge.

SourceForge was a bit slow to pick up on Subversion. Even though Subversion 1.0 was released in 2004, it looks like they didn't make it available until 2006. I do remember that both MySQL and Python both used SourceForge for CVS repositories, and both left SourceForge not too long before it became available, because they just couldn't stand to wait any longer. In any case, once Subversion because available, I switched to using it, because it sucked a lot less than CVS. But one side effect of this was the copying of CVS's branches and tags. I could rename branches and re-tag things, but the old CVS branches and tags were still in there for good.

In 2008, sometime after 1.2.2,  I was ready to make a clean start on MySQLdb, and re-engineer most of it. By this time, Mercurial was available and I decided to port just the SVN trunk over to a new Hg repository.  I really had hoped to stop using the SVN repository altogether, but it was still being used for ZMySQLDA, and it also turned out a 1.2.3 bugfix release was needed.

Work has been sporadic on MySQLdb-2.0 over the last years. Part of it not having a lot of time, and part of it was developer burnout. But there is stuff that needs to be done. In particular, Python 3 compatibility is needed now more than ever. I thought I could do this in 1.2.4, but I would have to sacrifice compatibility for Python < 2.7. So MySQLdb-1.2.4 will be a bugfix release and fully Python 2.7 compatible (and should be Python 2.8 compatible), and very soon thereafter there will be a 1.3.0 which will require Python 2.7 or newer and be compatible with Python 3.

One parallel development over the life of MySQLdb has been the life of MySQL itself. First they were sold to Sun, which I was not very happy about, but life went on. Then Sun was bought by Oracle, which I'm not happy about at all. Still, since then, you can still get MySQL for free, so it hasn't been huge impact on me. The Oracle acquisition led to two new MySQL forks: MariaDB, led by original MySQL creator Michael "Monty" Widenius; and Drizzle, led by MySQL guru Brian Aker. MariaDB is a straight-up fork of MySQL-5.1 and is supposed to be a binary replacement for MySQL, so the current MySQLdb should work with it just fine. Drizzle, on the other hand, has been heavily refactored and converted from C to C++, and has a different API. There is a Drizzle Python project (which borrows heavily from MySQLdb in places) already.

Anyway... My plan is to be able to support MySQL, MariaDB, and probably Drizzle, which is going to require having different driver backends. Not so much for MySQL and MariaDB, but definitely for Drizzle. One of the other drawbacks of MySQLdb-1.x is, although there's some support for using the embedded server library, it's hard to have both the embedded server and the regular client library in the same installation. Additionally, building the C module on Windows has historically been painful, and there have been requests for a pure Python connector. I've also considered doing a ctypes-based driver; at the time, ctypes was not ready for prime time, but it is in the Python core for quite a while now.

With all these various back-end drivers possible, and support no longer being strictly limited to MySQL, I decided it would be best to rename MySQLdb-2.0, and for better or worse, the name I came up with was... Moist. Why, oh why "moist", you ask? Well, MySQL has the dolphin, MariaDB has the seal, and Drizzle has.. .the drizzle... all things that are wet, or perhaps moist. I thought about "Soggy" but there was already a project by that name on GitHub.

So: The SVN repository on SourceForge lives on, but only for ZMySQLDA: It's been migrated to a git repository (MySQLdb1) that is synced between SourceForge and GitHub, and this is where MySQLdb-1.x will live. The Hg repository on SourceForge that was for MySQLdb-2.0 has now been cloned to become moist on GitHub. The Hg repository will probably be disappearing in the near future, once I'm convinced everything is OK with the moist repository. I've also added some READMEs to all the repositories with a map of where all the code is buried.

Any questions?

MySQLdb book and other news

Did you know there is a book specifically addressing how to use MySQLdb? MySQL for Python was published by Packt Publishing in September 2010. If you're worried about two years being an eternity on the internet, don't: There hasn't been a new release of MySQLdb since the book was published.

Speaking of new releases, I built a Windows installer of MySQLdb-1.2.3 for Python-2.7 and using the MySQL Connector/C 6.0. This should be able to connect to any modern version of MySQL (4.0 through 5.5 and newer), and as I understand it, it should also work with MariaDB, but this is not yet tested.

Additionally, there is a 1.2.4 release coming out in the near future which will be a bug-fix release only. It will support Python-2.4 through 2.7 (and should be compatible with Python-2.8 when it arrives). There is some work towards Python-3 compatibility (specifically, 3.2.3), but unfortunately, I can't easily support Python < 2.7 and Python >= 3 at the same time. So after 1.2.4 is out, there will be a 1.3.0 release shortly thereafter which will be for Python-2.7 and 3.2 or newer (and probably earlier).

Longer term: I've also been working on a MySQLdb-2.0 version which is almost completely rewritten from scratch on the Python side, and modularized (and some code removed) on the C side. It's going to have pluggable drivers, with the intention of having the option of using libmysqlclient (the standard C library), libmysqld (the embedded server library), some kind of pure Python driver, and eventually a driver for Drizzle. MySQLdb-2.0 is actually going to be renamed Moist, and no, I don't yet have the code in github yet, but it will be what is the MySQLdb2 repository on SourceForge.

Is the project leaving SourceForge? It remains to be seen. Every time I get fed up and I'm sure I'm about to move, they have some big update which makes things better. The current location is pretty well-known, and the project has been there since 2001. I do think I will be mirroring a git repository between SF and github at the very least; all the cool kids are on github these days.