OpenWrt Buildroot

Usage and documentation by Felix Fietkau and Waldemar Brodkorb, based on uClibc Buildroot documentation by Thomas Petazzoni. Contributions from Karsten Kruse, Ned Ludd, Martin Herren. OpenWrt Kernel Module Creation Howto by Markus Becker.

Last modification : $Id$

About OpenWrt Buildroot

OpenWrt Buildroot is a set of Makefiles and patches that allows to easily generate both a cross-compilation toolchain and a root filesystem for your Wireless Router. The cross-compilation toolchain uses uClibc (http://www.uclibc.org/), a tiny C standard library.

A compilation toolchain is the set of tools that allows to compile code for your system. It consists of a compiler (in our case, gcc), binary utils like assembler and linker (in our case, binutils) and a C standard library (for example GNU Libc, uClibc or dietlibc). The system installed on your development station certainly already has a compilation toolchain that you can use to compile application that runs on your system. If you're using a PC, your compilation toolchain runs on an x86 processor and generates code for a x86 processor. Under most Linux systems, the compilation toolchain uses the GNU libc as C standard library. This compilation toolchain is called the "host compilation toolchain", and more generally, the machine on which it is running, and on which you're working is called the "host system". The compilation toolchain is provided by your distribution, and OpenWrt Buildroot has nothing to do with it.

As said above, the compilation toolchain that comes with your system runs and generates code for the processor of your host system. As your embedded system has a different processor, you need a cross-compilation toolchain: it's a compilation toolchain that runs on your host system but that generates code for your target system (and target processor). For example, if your host system uses x86 and your target system uses MIPS, the regular compilation toolchain of your host runs on x86 and generates code for x86, while the cross-compilation toolchain runs on x86 and generates code for MIPS.

You might wonder why such a tool is needed when you can compile gcc, binutils, uClibc and all the tools by hand. Of course, doing so is possible. But dealing with all configure options, with all problems of every gcc or binutils version is very time-consuming and uninteresting. OpenWrt Buildroot automates this process through the use of Makefiles, and has a collection of patches for each gcc and binutils version to make them work on the MIPS architecture of most Wireless Routers.

Obtaining OpenWrt Buildroot

OpenWrt Buildroot is available via CVS - Concurrent Version System. For any kind of OpenWrt development you should get the latest version from cvs via:

 $ cvs -d:pserver:anonymous@openwrt.org:/openwrt co openwrt

If you only like to create your own custom firmware images and pakages we strongely suggest to use the CVS branch of the stable version (whiterussian): 

 $ cvs -d:pserver:anonymous@openwrt.org:/openwrt co -rwhiterussian openwrt

Using OpenWrt Buildroot

OpenWrt Buildroot has a nice configuration tool similar to the one you can find in the Linux Kernel (http://www.kernel.org/) or in Busybox (http://www.busybox.org/). Note that you can run everything as a normal user. There is no need to be root to configure and use the Buildroot. The first step is to run the configuration assistant:

 $ make menuconfig

For each entry of the configuration tool, you can find associated help that describes the purpose of the entry.

Once everything is configured, the configuration tool has generated a .config file that contains the description of your configuration. It will be used by the Makefiles to do what's needed.

Let's go:

 $ make

This command will download, configure and compile all the selected tools, and finally generate target firmware images and additional packages (depending on your selections in make menuconfig. All the target files can be found in the bin/ subdirectory. You can compile firmware images containing two different filesystem types:

jffs2 contains a writable root filesystem, which will expand to the size of your flash image. Note: if you use the generic firmware image, you need to pick the right image for your flash size, because of different eraseblock sizes.

squashfs contains a read-only root filesystem using a modified squashfs filesystem with LZMA compression. When booting it, you can create a writable second filesystem, which will contain your modifications to the root filesystem, including the packages you install.

Customizing the target filesystem

There are two ways to customize the resulting target filesystem:

Customizing the Busybox configuration

Busybox is very configurable, and you may want to customize it. Its configuration is completely integrated into the main menuconfig system. You can find it under "OpenWrt Package Selection" => "Busybox Configuration"

Customizing the uClibc configuration

Just like BusyBox, uClibc offers a lot of configuration options. They allow to select various functionalities, depending on your needs and limitations.

The easiest way to modify the configuration of uClibc is to follow these steps :

  1. Make a first compilation of buildroot without trying to customize uClibc.
  2. Go into the directory toolchain_build_ARCH/uClibc/ and run make menuconfig. The nice configuration assistant, similar to the one used in the Linux Kernel appears. Make your configuration as appropriate.
  3. Copy the .config file to toolchain/uClibc/uClibc.config or toolchain/uClibc/uClibc.config-locale. The former is used if you haven't selected locale support in the Buildroot configuration, and the latter is used if you have selected locale support.
  4. Run the compilation again.

Otherwise, you can simply change toolchain/uClibc/uClibc.config or toolchain/uClibc/uClibc.config-locale without running the configuration assistant.

How OpenWrt Buildroot works

As said above, OpenWrt is basically a set of Makefiles that download, configure and compiles software with the correct options. It also includes some patches for various software, mainly the ones involved in the cross-compilation tool chain (gcc, binutils and uClibc).

There is basically one Makefile per software, and they are named Makefile. Makefiles are split into three sections:

Each directory contains at least 2 files :

The main Makefile do the job through the following steps (once the configuration is done):

  1. Create the download directory (dl/ by default). This is where the tarballs will be downloaded. It is interesting to know that the tarballs are in this directory because it may be useful to save them somewhere to avoid further downloads.
  2. Create the build directory (build_ARCH/ by default, where ARCH is your architecture). This is where all user-space tools while be compiled.
  3. Create the toolchain build directory (toolchain_build_ARCH/ by default, where ARCH is your architecture). This is where the cross compilation toolchain will be compiled.
  4. Setup the staging directory (staging_dir_ARCH/ by default). This is where the cross-compilation toolchain will be installed. If you want to use the same cross-compilation toolchain for other purposes, such as compiling third-party applications, you can add staging_dir_ARCH/bin to your PATH, and then use arch-linux-gcc to compile your application. In order to setup this staging directory, it first removes it, and then it creates various subdirectories and symlinks inside it.
  5. Create the target directory (build_ARCH/root/ by default) and the target filesystem skeleton. This directory will contain the final root filesystem. To set it up, it first deletes it, then it copies the skeleton available in target/default/target_skeleton and then removes useless CVS/ directories.
  6. Call the prepare, compile and install targets for the subdirectories toolchain, package and target

Using the uClibc toolchain

You may want to compile your own programs or other software that are not packaged in OpenWrt. In order to do this, you can use the toolchain that was generated by the Buildroot.

The toolchain generated by the Buildroot by default is located in staging_dir_ARCH. The simplest way to use it is to add staging_dir_ARCH/bin/ to your PATH environment variable, and then to use arch-linux-gcc, arch-linux-objdump, arch-linux-ld, etc.

For example, you may add the following to your .bashrc (considering you're building for the MIPS architecture and that Buildroot is located in ~/openwrt/) :

export PATH=$PATH:~/openwrt/staging_dir_mipsel/bin/

Then you can simply do :

mipsel-linux-uclibc-gcc -o foo foo.c

Important : do not try to move the toolchain to an other directory, it won't work. There are some hard-coded paths in the gcc configuration. If the default toolchain directory doesn't suit your needs, please refer to the Using the uClibc toolchain outside of buildroot section.

Using the uClibc toolchain outside of buildroot

By default, the cross-compilation toolchain is generated inside staging_dir_ARCH/. But sometimes, it may be useful to install it somewhere else, so that it can be used to compile other programs or by other users. Moving the staging_dir_ARCH/ directory elsewhere is not possible, because they are some hardcoded paths in the toolchain configuration.

If you want to use the generated toolchain for other purposes, you can configure Buildroot to generate it elsewhere using the option of the configuration tool : Build options -> Toolchain and header file location, which defaults to staging_dir_ARCH/.

Location of downloaded packages

It might be useful to know that the various tarballs that are downloaded by the Makefiles are all stored in the DL_DIR which by default is the dl directory. It's useful for example if you want to keep a complete version of Buildroot which is known to be working with the associated tarballs. This will allow you to regenerate the toolchain and the target filesystem with exactly the same versions.

Extending OpenWrt with more software

This section will only consider the case in which you want to add user-space software.

Package directory

First of all, create a directory under the package directory for your software, for example foo.

Config.in file

Then, create a file named Config.in. This file will contain the portion of options description related to our foo software that will be used and displayed in the configuration tool. It should basically contain :

config BR2_PACKAGE_FOO
        tristate "foo - some nice tool"
        default m if CONFIG_DEVEL
        help
	     This is a comment that explains what foo is.

If you depend on other software or library inside the Buildroot, it is important that you automatically select these packages in your Config.in. Example if foo depends on bar library: 

config BR2_PACKAGE_FOO
        tristate "foo - some nice tool"
        default m if CONFIG_DEVEL
	select BR2_PACKAGE_LIBBAR
        help
        This is a comment that explains what foo is.

Of course, you can add other options to configure particular things in your software.

Config.in in the package directory

To add your package to the configuration tool, you need to add the following line to package/Config.in, please add it to a section, which fits the purpose of foo: 

comment "Networking"
source "package/foo/Config.in"

Makefile in the package directory

To add your package to the build process, you need to edit the Makefile in the package/ directory. Locate the lines that look like the following:

package-$(BR2_PACKAGE_FOO) += foo

As you can see, this short line simply adds the target foo to the list of targets handled by OpenWrt Buildroot.

In addition to the default dependencies, you make your package depend on another package (e.g. a library) by adding a line: 

foo-compile: bar-compile

The ipkg control file

Additionally, you need to create a control file which contains information about your package, readable by the ipkg package utility. It should be created as file: package/foo/ipkg/foo.control

The file looks like this

     1  Package: foo
     2  Priority: optional
     3  Section: net
     4  Maintainer: Foo Software <foo@foosoftware.com>
     5  Source: http://foosoftware.com
     6  Depends: libbar
     7  Description: Package Description

You can skip the usual Version: and Architecture fields, as they will be generated by the make-ipkg-dir.sh script called from your Makefile. The Depends field is important, so that ipkg will automatically fetch all dependend software on your target system.

The real Makefile

Finally, here's the hardest part. Create a file named Makefile. It will contain the Makefile rules that are in charge of downloading, configuring, compiling and installing the software. Below is an example that we will comment afterwards.

     1  # $Id$
     2	
     3  include $(TOPDIR)/rules.mk
     4
     5  PKG_NAME:=foo
     6  PKG_VERSION:=1.0
     7  PKG_RELEASE:=1
     8  PKG_MD5SUM:=4584f226523776a3cdd2fb6f8212ba8d
     9 
    10  PKG_SOURCE_URL:=http://www.foosoftware.org/downloads
    11  PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
    12 	PKG_CAT:=zcat
    13	
    14	PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)
    15	PKG_INSTALL_DIR:=$(PKG_BUILD_DIR)/ipkg-install
    16
    17	include $(TOPDIR)/package/rules.mk
    18
    19	$(eval $(call PKG_template,FOO,foo,$(PKG_VERSION)-$(PKG_RELEASE),$(ARCH)))
    20
    21  $(PKG_BUILD_DIR)/.configured: $(PKG_BUILD_DIR)/.prepared
    22          (cd $(PKG_BUILD_DIR); \
    23                  $(TARGET_CONFIGURE_OPTS) \
    24                  CFLAGS="$(TARGET_CFLAGS)" \
    25                  ./configure \
    26                  --target=$(GNU_TARGET_NAME) \
    27                  --host=$(GNU_TARGET_NAME) \
    28                  --build=$(GNU_HOST_NAME) \
    29                  --prefix=/usr \
    30                  --sysconfdir=/etc \
    31 			--with-bar="$(STAGING_DIR)/usr" \
    32          );
    33          touch $@
    34
    35  $(PKG_BUILD_DIR)/.built:
    36      	rm -rf $(PKG_INSTALL_DIR)
    37		mkdir -p $(PKG_INSTALL_DIR)
    38		$(MAKE) -C $(PKG_BUILD_DIR) \
    39 	          $(TARGET_CONFIGURE_OPTS) \
    40            install_prefix="$(PKG_INSTALL_DIR)" \
    41 	          all install
    42		touch $@
    43 
    44  $(IPKG_FOO):
    46		install -d -m0755 $(IDIR_FOO)/usr/sbin
    47    	cp -fpR $(PKG_INSTALL_DIR)/usr/sbin/foo $(IDIR_FOO)/usr/sbin
    49		$(RSTRIP) $(IDIR_FOO)
    50		$(IPKG_BUILD) $(IDIR_FOO) $(PACKAGE_DIR)
    51	
    52	mostlyclean:
    53  	make -C $(PKG_BUILD_DIR) clean
    54    	rm $(PKG_BUILD_DIR)/.built

First of all, this Makefile example works for a single binary software. For other software such as libraries or more complex stuff with multiple binaries, it should be adapted. Look at the other Makefile files in the package/ directory.

At lines 5-15, a couple of useful variables are defined:

In Line 3 and 17 we include common variables and routines to simplify the process of ipkg creation. It includes routines to download, verify and extract the software package archives.

Line 19 contains the magic line which automatically creates the ipkg for us.

Lines 21-33 defines a target and associated rules that configures the software. It depends on the previous target (the hidden .prepared file) so that we are sure the software has been uncompressed. In order to configure it, it basically runs the well-known ./configurescript. As we may be doing cross-compilation, target, host and build arguments are given. The prefix is also set to /usr, not because the software will be installed in /usr on your host system, but in the target filesystem. Finally it creates a .configured file to mark the software as configured.

Lines 35-42 defines a target and a rule that compiles the software. This target will create the binary file in the compilation directory, and depends on the software being already configured (hence the reference to the .configured file). Afterwards it installs the resulting binary into the PKG_INSTALL_DIR. It basically runs make install inside the source directory.

Lines 44-50 defines a target and associated rules that create the ipkg package, which can optionally be embedded into the resulting firmware image. It manually installs all files you want to integrate in your resulting ipkg. RSTRIP will recursevily strip all binaries and libraries. Finally IPKG_BUILD is called to create the package.

Conclusion

As you can see, adding a software to buildroot is simply a matter of writing a Makefile using an already existing example and to modify it according to the compilation process of the software.

If you package software that might be useful for other persons, don't forget to send a patch to OpenWrt developers! Use the mail address: openwrt-devel@openwrt.org

Resources

To learn more about OpenWrt you can visit this website: http://openwrt.org/

OpenWrt Kernel Module Creation Howto

About OpenWrt Kernel Module Compilation

You are planning to compile a kernel module? This howto will explain what you have to do, to have your kernel module installable as an ipkg.

Enable the kernel options

Enable the kernel options you want by modifying build_mipsel/linux/.config. We are assuming, that you already had your kernel compiled once here. You can do the modification by hand or by 

$ cd build_mipsel/linux
$ make menuconfig
And copy it, so your changes are not getting lost, when doing a 'make dirclean'. Here we assume that you are compiling for Broadcom chipset based devices: 
 $ cp .config ../../../target/linux/linux-2.4/config/brcm

Create a buildroot option

Create a buildroot option by modifying/inserting into target/linux/Config.in, e.g. 

config BR2_PACKAGE_KMOD_USB_KEYBOARD
        tristate "Support for USB keyboards"
        default m
        depends BR2_PACKAGE_KMOD_USB_CONTROLLER

Define the binary files for the kernel module

Define the binary files for the kernel module by modifying/inserting into target/linux/linux-2.4/Makefile, e.g. 

$(eval $(call KMOD_template,USB_KEYBOARD,usb-kbd,\
	$(MODULES_DIR)/kernel/drivers/input/input.o \
	$(MODULES_DIR)/kernel/drivers/input/keybdev.o \
	$(MODULES_DIR)/kernel/drivers/usb/usbkbd.o \
,CONFIG_USB_KEYB,kmod-usb-core,60,input keybdev usbkbd))
Where CONFIG_USB_KEYB is the kernel option, USB_KEYBOARD is the last part of BR2_PACKAGE_KMOD_USB_KEYBOARD and usb-kbd is part of the filename of the created ipkg.

Specify the ipkg control file

Create e.g. target/linux/control/kmod-usb-kbd.control with content similar to this: 

Package: kmod-usb-kbd
Priority: optional
Section: sys
Maintainer: Markus Becker <mab@comnets.uni-bremen.de>
Source: buildroot internal
Description: Kernel Support for USB Keyboards

Compile the kernel module

Enable the kernel module with 

$ make menuconfig
in TOPDIR and selecting it.
Compile with 
$ make dirclean && make