Automatic builds, packaging

Specifying all compiler options manually, as we did in a previous section, gets more complicated as the number of files grows. When a plugin consists of more than a couple of files, an appropriate Makefile becomes almost a requirement. And it is absolutely unavoidable if we want to distribute our great plugin, as we cannot expect our users to copy and paste complex command lines from a README file. We want the process of configuring and building a plugin to be as simple as possible. But first we need to decide whether a plugin should be built from inside the MySQL source tree or standalone.

UDFs and standalone plugins

UDFs and certain plugin types (for example, full-text parser plugins, some Daemon plugins, or newer plugin types added after MySQL 5.1) do not require MySQL sources for building; the API for them is complete and self-sufficient. These plugins can be distributed and built independently from MySQL. Writing a Makefile or configure.ac for such a plugin does not differ from writing them for any other project—we only need to set the installation path correctly. When using automake and libtool, a simple Makefile.am can look like this:

plugindir= $(libdir)/mysql/plugin
plugin_LTLIBRARIES= my_plugin.la
my_plugin_la_SOURCES= my_plugin.c
my_plugin_la_LDFLAGS= -module -rpath $(plugindir)
my_plugin_la_CFLAGS= -DMYSQL_DYNAMIC_PLUGIN

This file sets the installation directory to be mysql/plugin/ inside the library path, which is usually /usr/lib. However, strictly speaking, the user has to use the same library path that his MySQL installation uses. It specifies the build target to be my_plugin.la—it is a libtool control file, a text file with information about my_plugin.so. The latter will be built automatically. It tells the libtool that we are building a library for dynamic loading with dlopen() (the -module option does that) and where it will be installed. The last line adds ‑DMYSQL_DYNAMIC_PLUGIN to the compiler command line. There is no need to specify ‑fPIC, ‑shared, or ‑bundle; libtool will use them automatically, depending on the platform we are building on. It knows a large number of operating systems, compilers, linkers, and their corresponding command-line switches for building dynamically loaded modules.

In addition to Makefile.am, a complete project will need a configure.ac file, AUTHORS, NEWS, ChangeLog, and README files. The last four files are required by automake, but they can be empty. The configure.ac file is used by autoconf to generate a configure script, which, in turn, will generate Makefile. A minimal configure.ac could be as simple as:

AC_INIT(my_plugin, 0.1)
AM_INIT_AUTOMAKE
AC_PROG_LIBTOOL
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

It sets the name and version of our software package, initializes automake and libtool, and specifies that the result of the configure script should be a Makefile.

Plugins that are built from the MySQL source tree

If we need to have access to the MySQL source tree for our plugin, we can at least do it with style. Plugins that are built from the MySQL source tree can be integrated seamlessly into the MySQL build system. Additionally, we will get support for Microsoft Windows builds and the ability to link the plugin statically into the server, so that it becomes a part of the mysqld binary. Unlike standalone plugins, we will only need three auxiliary files here.

On UNIX-like systems, MySQL 5.1 is built using autotools and make. A plug.in file will be the source file for autoconf, and Makefile.am for automake. To build MySQL on Windows one needs CMake, and thus our plugin should come with a CMakeLists.txt file. All of these three files can use the full power of autotools or CMake, if necessary, but for a minimal working plugin they only need to contain a few simple lines.

plug.in

The plug.in file describes the plugin to the MySQL configure script. A plugin is detected automatically by autoconf as long as its plug.in file can be found in a directory located in the plugin/ or storage/ subdirectory in the MySQL source tree (in other words, it should be either plugin/*/plug.in or storage/*/plug.in). A plug.in file can use all autoconf and m4 macros as usual. Additionally, MySQL defines a few macros specifically for using in the plug.in files. They all are documented in the config/ac-macros/plugin.m4 file in the MySQL source tree. The most important of them are described as follows:

• MYSQL_PLUGIN([name],[long name], [description], [group,group...])

  This is usually the first line in any plug.in file. This macro is mandatory. It declares a new plugin. The name will be used in the configure options such as ‑‑with‑plugin‑foo and ‑‑without‑plugin‑foo. The long name and the description will be printed in the ./configure ‑‑help output. "Groups" are preset configuration names that one can specify in the ‑‑with‑plugin=group option. Any group name can be used, but max, max‑no‑ndb, and default are commonly used. Most plugins add themselves to the max and max‑no‑ndb groups.

• MYSQL_PLUGIN_STATIC([name],[libmyplugin.a])

  This macro declares that a plugin name supports static builds, that is, it can be built as a static library and linked statically into the server binary. It specifies the name of this static library, which can be later referred to in Makefile.am as @plugin_myplugin_static_target@. It will be expanded to libmyplugin.a if a static build is selected, otherwise it will be empty.

• MYSQL_PLUGIN_DYNAMIC([name],[myplugin.la])

  Similarly, this macro declares that a plugin can be built as a shared library and loaded into the server dynamically. It introduces a Makefile.am substitution @plugin_myplugin_dynamic_target@, which is myplugin.la if this shared library needs to be built, and empty otherwise.

• MYSQL_PLUGIN_ACTIONS([name],[ ACTION-IF-SELECTED ])

  The ACTION‑IF‑SELECTED code will be executed only if this plugin is selected by configure either for static or dynamic builds. Here we can check for system headers, libraries, and functions that are used by the plugin. Normal AC_ macros can be used here freely.

An example of a plug.in file can look like

MYSQL_PLUGIN(my_plugin,[My Plugin Example],
[An example of My Plugin], [max,max-no-ndb])
MYSQL_PLUGIN_STATIC(my_plugin,[libmy_plugin.a])
MYSQL_PLUGIN_DYNAMIC(my_plugin,[my_plugin.la])

With such a file in place, say in plugin/my_plugin/plug.in, all we need to do is to run autoreconf ‑f to recreate the configure script. After that, there is no distinction between our plugin and official MySQL plugins:

$ ./configure --help
`configure' configures this package to adapt to many kinds of systems.
...
--with-plugins=PLUGIN[[[,PLUGIN..]]]
Plugins to include in mysqld. (default is: none)
Must be a configuration name or a comma separated
list of plugins.
Available configurations are: none max max-no-ndb
all.
Available plugins are: partition daemon_example
ftexample archive blackhole csv example federated
heap ibmdb2i innobase innodb_plugin myisam
myisammrg my_plugin ndbcluster.
...
=== My Plugin Example ===
Plugin Name: my_plugin
Description: An example of My Plugin
Supports build: static and dynamic
Configurations: max, max-no-ndb

A new plugin is mentioned in the "available plugins" list and described in detail at the end of the configure ‑‑help output.

Makefile.am

As in the case of standalone plugins, we need a Makefile.am file. It will be converted by automake and the configure script to a Makefile, and it defines how the plugin, static or shared library, should be built. This file is more complex than for standalone plugins because it needs to cover both static and dynamic builds. Of course, when a plugin supports only one way of linking, only static or only dynamic, Makefile.am gets much simpler. Let's analyze it line by line:

pkgplugindir = $(pkglibdir)/plugin
INCLUDES = -I$(top_srcdir)/include -I$(top_builddir)/include \
-I$(top_srcdir)/sql
pkgplugin_LTLIBRARIES = @plugin_my_plugin_shared_target@
my_plugin_la_LDFLAGS = -module -rpath $(pkgplugindir)
my_plugin_la_CXXFLAGS= -DMYSQL_DYNAMIC_PLUGIN
my_plugin_la_SOURCES = my_plugin.c
noinst_LIBRARIES = @plugin_my_plugin_static_target@
libmy_plugin_a_SOURCES= my_plugin.c
EXTRA_LTLIBRARIES = my_plugin.la
EXTRA_LIBRARIES = libmy_plugin.a
EXTRA_DIST = CMakeLists.txt plug.in

The file starts with defining pkgplugindir—a place where a plugin will be installed.

Then we set the search path for the #include directives; it needs to contain at least the include/ directory where most of the headers are, and often the sql/ directory too, especially when we need to access internal server structures.

Now we can specify automake build rules for the targets. A shared target is a libtool library (LTLIBRARIES) that should be installed in pkgplugindir (because we used the pkgplugin_ prefix). And we specify the source files, compiler and linker flags that are needed to build the my_plugin.la library. If a user decides not to build a dynamic version of our plugin, @plugin_my_plugin_shared_target@ will be empty and no libtool library will be built.

Similarly, we specify rules for the static target. It is a library (LIBRARIES), and it should not be installed (noinst_). Indeed, as it will be linked into the server statically, becoming a part of the server binary, there is no need to install it separately. In this case, we do not need any special compiler or linker flags, we only specify the sources.

Because a user may decide to build either a static or a shared library, the name of the build target is not known before the configure script is run. However, automake needs to know all possible targets in advance, and we list them in the EXTRA_ variables.

We end the file by listing all remaining files that are part of the plugin source distribution, but are not mentioned in other rules. The automake needs to know about them, otherwise the make dist command will work incorrectly.

CMakeLists.txt

In MySQL 5.1, of all plugin types, only Storage Engine plugins can be integrated into the MySQL build system on Windows. One does this by providing an appropriate CMakeLists.txt file. All of the CMake power is available there, but a minimal CMakeLists.txt file is as simple as this:

INCLUDE("${PROJECT_SOURCE_DIR}/storage/mysql_storage_engine.cmake")
SET(my_plugin_SOURCES my_plugin.c)
MYSQL_STORAGE_ENGINE(my_plugin)

We only specify the name of the storage engine, my_plugin, and the source files, and include the file that does all of the heavy job.