From d38e13b12b10060e210d7c5ace366cc82eaf9da9 Mon Sep 17 00:00:00 2001 From: nbd Date: Mon, 16 Oct 2006 04:21:44 +0000 Subject: add modified version of mbm's 'introduction to buildroot-ng' to the documentation git-svn-id: svn://svn.openwrt.org/openwrt/trunk@5151 3c298f89-4303-0410-b956-a3cf2f4a3e73 --- docs/Makefile | 2 +- docs/build.tex | 320 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ docs/openwrt.tex | 7 +- 3 files changed, 325 insertions(+), 4 deletions(-) create mode 100644 docs/build.tex diff --git a/docs/Makefile b/docs/Makefile index 543dbebad..f311a8fd9 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,4 +1,4 @@ -openwrt.pdf: Makefile openwrt.tex config.tex network.tex network-scripts.tex network-scripts.tex wireless.tex +openwrt.pdf: Makefile openwrt.tex config.tex network.tex network-scripts.tex network-scripts.tex wireless.tex build.tex $(MAKE) cleanup pdflatex openwrt.tex pdflatex openwrt.tex diff --git a/docs/build.tex b/docs/build.tex new file mode 100644 index 000000000..1484dc6f8 --- /dev/null +++ b/docs/build.tex @@ -0,0 +1,320 @@ +One of the biggest challenges to getting started with embedded devices is that you +just can't install a copy of Linux and expect to be able to compile a firmware. +Even if you did remember to install a compiler and every development tool offered, +you still wouldn't have the basic set of tools needed to produce a firmware image. +The embedded device represents an entirely new hardware platform, which is +incompatible with the hardware on your development machine, so in a process called +cross compiling you need to produce a new compiler capable of generating code for +your embedded platform, and then use it to compile a basic Linux distribution to +run on your device. + +The process of creating a cross compiler can be tricky, it's not something that's +regularly attempted and so the there's a certain amount of mystery and black magic +associated with it. In many cases when you're dealing with embedded devices you'll +be provided with a binary copy of a compiler and basic libraries rather than +instructions for creating your own -- it's a time saving step but at the same time +often means you'll be using a rather dated set. Likewise, it's also common to be +provided with a patched copy of the Linux kernel from the board or chip vendor, +but this is also dated and it can be difficult to spot exactly what has been +changed to make the kernel run on the embedded platform. + +\subsection{Building an image} + +OpenWrt takes a different approach to building a firmware, downloading, patching +and compiling everything from scratch, including the cross compiler. Or to put it +in simpler terms, OpenWrt doesn't contain any executables or even sources, it's an +automated system for downloading the sources, patching them to work with the given +platform and compiling them correctly for the platform. What this means is that +just by changing the template, you can change any step in the process. + + +As an example, if a new kernel is released, a simple change to one of the Makefiles +will download the latest kernel, patch it to run on the embedded platform and produce +a new firmware image -- there's no work to be done trying to track down an unmodified +copy of the existing kernel to see what changes had been made, the patches are +already provided and the process ends up almost completely transparent. This doesn't +just apply to the kernel, but to anything included with OpenWrt -- It's this one +simple understated concept which is what allows OpenWrt to stay on the bleeding edge +with the latest compilers, latest kernels and latest applications. + +So let's take a look at OpenWrt and see how this all works + + +\subsubsection{Download openwrt} + +This article refers to the "Kamikaze" branch of OpenWrt, which can be downloaded via +subversion using the following command: + +\begin{Verbatim} +svn co https://svn.openwrt.org/openwrt/trunk kamikaze +\end{Verbatim} + +Additionally, there's a trac interface on \href{https://dev.openwrt.org/}{https://dev.openwrt.org/} +which can be used to monitor svn commits and browse the sources. + + +\subsubsection{The directory structure} + +There are four key directories in the base: + +\begin{itemize} + \item tools + \item toolchain + \item package + \item target +\end{itemize} + +\texttt{tools} and \texttt{toolchain} refer to common tools which will be +used to build the firmware image and the compiler and c library. +The result of this is three new directories, \texttt{tool\_build}, which is a temporary +directory for building the target independent tools, \texttt{toolchain\_build\_\textit{}} +which is used for building the toolchain for a specific architecture, and +\texttt{staging\_dir\_\textit{}} where the resulting toolchain is installed. +You won't need to do anything with the toolchain directory unless you intend to +add a new version of one of the components above. + +\texttt{package} is for exactly that -- packages. In an OpenWrt firmware, almost everything +is an \texttt{.ipk}, a software package which can be added to the firmware to provide new +features or removed to save space. + +\texttt{target} refers to the embedded platform, this contains items which are specific to +a specific embedded platform. Of particular interest here is the "\texttt{target/linux}" +directory which is broken down by platform and contains the kernel config and patches +to the kernel for a particular platform. There's also the "\texttt{target/image}" directory +which describes how to package a firmware for a specific platform. + +Both the target and package steps will use the directory "\texttt{build\_\textit{}}" +as a temporary directory for compiling. Additionally, anything downloaded by the toolchain, +target or package steps will be placed in the "\texttt{dl}" directory. + + +\subsubsection{Building OpenWrt} + +While the OpenWrt build environment was intended mostly for developers, it also has to be +simple enough that an inexperienced end user can easily build his or her own customized firmware. + +Running the command "\texttt{make menuconfig}" will bring up OpenWrt's configuration menu +screen, through this menu you can select which platform you're targeting, which versions of +the toolchain you want to use to build and what packages you want to install into the +firmware image. Similar to the linux kernel config, almost every option has three choices, +\texttt{y/m/n} which are represented as follows: + +\begin{itemize} + \item{\texttt{<*>} (pressing y)} \\ + This will be included in the firmware image + \item{\texttt{} (pressing m)} \\ + This will be compiled but not included (for later install) + \item{\texttt{< >} (pressing n)} \\ + This will not be compiled +\end{itemize} + +After you've finished with the menu configuration, exit and when prompted, save your +configuration changes. To begin compiling the firmware, type "\texttt{make}". By default +OpenWrt will only display a high level overview of the compile process and not each individual +command. + +\subsubsection{Example:} + +\begin{Verbatim} +make[2] toolchain/install +make[3] -C toolchain install +make[2] target/compile +make[3] -C target compile +make[4] -C target/utils prepare + +[...] +\end{Verbatim} + +This makes it easier to monitor which step it's actually compiling and reduces the amount +of noise caused by the compile output. To see the full output, run the command +"\texttt{make V=99}". + +During the build process, buildroot will download all sources to the "\texttt{dl}" +directory and will start patching and compiling them in the "\texttt{build\_\textit{}}" +directory. When finished, the resulting firmware will be in the "\texttt{bin}" directory +and packages will be in the "\texttt{bin/packages}" directory. + + +\subsection{Creating packages} + + +One of the things that we've attempted to do with OpenWrt's template system is make it +incredibly easy to port software to OpenWrt. If you look at a typical package directory +in OpenWrt you'll find two things: + +\begin{itemize} + \item \texttt{package/\textit{}/Makefile} + \item \texttt{package/\textit{}/patches} +\end{itemize} + +The patches directory is optional and typically contains bug fixes or optimizations to +reduce the size of the executable. The package makefile is the important item, provides +the steps actually needed to download and compile the package. + +Looking at one of the package makefiles, you'd hardly recognize it as a makefile. +Through what can only be described as blatant disregard and abuse of the traditional +make format, the makefile has been transformed into an object oriented template which +simplifies the entire ordeal. + +Here for example, is \texttt{package/bridge/Makefile}: + +\begin{Verbatim}[frame=single,numbers=left] +include $(TOPDIR)/rules.mk + +PKG_NAME:=bridge +PKG_VERSION:=1.0.6 +PKG_RELEASE:=1 + +PKG_BUILD_DIR:=$(BUILD_DIR)/bridge-utils-$(PKG_VERSION) +PKG_SOURCE:=bridge-utils-$(PKG_VERSION).tar.gz +PKG_SOURCE_URL:=@SF/bridge +PKG_MD5SUM:=9b7dc52656f5cbec846a7ba3299f73bd +PKG_CAT:=zcat + +include $(INCLUDE_DIR)/package.mk + +define Package/bridge + SECTION:=base + CATEGORY:=Network + DEFAULT:=y + TITLE:=Ethernet bridging configuration utility + URL:=http://bridge.sourceforge.net/ +endef + +define Package/bridge/description +Ethernet bridging configuration utility + Manage ethernet bridging; a way to connect networks together + to form a larger network. +endef + +define Build/Configure + $(call Build/Configure/Default, \ + --with-linux-headers=$(LINUX_DIR)) +endef + +define Package/bridge/install + install -m0755 -d $(1)/usr/sbin + install -m0755 $(PKG_BUILD_DIR)/brctl/brctl \ + $(1)/usr/sbin/ +endef + +$(eval $(call BuildPackage,bridge)) +\end{Verbatim} + + +As you can see, there's not much work to be done; everything is hidden in other makefiles +and abstracted to the point where you only need to specify a few variables. + +\begin{itemize} + \item \texttt{PKG\_NAME} \\ + The name of the package, as seen via menuconfig and ipkg + \item \texttt{PKG\_VERSION} \\ + The upstream version number that we're downloading + \item \texttt{PKG\_RELEASE} \\ + The version of this package Makefile + \item \texttt{PKG\_BUILD\_DIR} \\ + Where to compile the package + \item \texttt{PKG\_SOURCE} \\ + The filename of the original sources + \item \texttt{PKG\_SOURCE\_URL} \\ + Where to download the sources from + \item \texttt{PKG\_MD5SUM} \\ + A checksum to validate the download + \item \texttt{PKG\_CAT} \\ + How to decompress the sources (zcat, bzcat, unzip) +\end{itemize} + +The \texttt{PKG\_*} variables define where to download the package from; +\texttt{@SF} is a special keyword for downloading packages from sourceforge. +The md5sum is used to verify the package was downloaded correctly and +\texttt{PKG\_BUILD\_DIR} defines where to find the package after the sources are +uncompressed into \texttt{\$(BUILD\_DIR)}. + +At the bottom of the file is where the real magic happens, "BuildPackage" is a macro +setup by the earlier include statements. BuildPackage only takes one argument directly -- +the name of the package to be built, in this case "\texttt{bridge}". All other information +is taken from the define blocks. This is a way of providing a level of verbosity, it's +inherently clear what the contents of the \texttt{description} template in +\texttt{Package/bridge} is, which wouldn't be the case if we passed this information +directly as the Nth argument to \texttt{BuildPackage}. + +\texttt{BuildPackage} uses the following defines: + +\textbf{\texttt{Package/\textit{}}:} \\ + \texttt{\textit{}} matches the argument passed to buildroot, this describes + the package the menuconfig and ipkg entries. Within \texttt{Package/\textit{}} + you can define the following variables: + + \begin{itemize} + \item \texttt{SECTION} \\ + The type of package (currently unused) + \item \texttt{CATEGORY} \\ + Which menu it appears in menuconfig + \item \texttt{TITLE} \\ + A short description of the package + \item \texttt{URL} \\ + Where to find the original software + \item \texttt{MAINTAINER} (optional) \\ + Who to contact concerning the package + \item \texttt{DEPENDS} (optional) \\ + Which packages must be built/installed before this package + \end{itemize} + +\textbf{\texttt{Package/\textit{}/conffiles} (optional):} \\ + A list of config files installed by this package, one file per line. + +\textbf{\texttt{Build/Prepare} (optional):} \\ + A set of commands to unpack and patch the sources. You may safely leave this + undefined. + +\textbf{\texttt{Build/Configure} (optional):} \\ + You can leave this undefined if the source doesn't use configure or has a + normal config script, otherwise you can put your own commands here or use + "\texttt{\$(call Build/Configure/Default,\textit{})}" as above to + pass in additional arguments for a standard configure script. + +\textbf{\texttt{Build/Compile} (optional):} \\ + How to compile the source; in most cases you should leave this undefined. + +\textbf{\texttt{Package/\textit{}/install}:} \\ + A set of commands to copy files out of the compiled source and into the ipkg + which is represented by the \texttt{\$(1)} directory. + +The reason that some of the defines are prefixed by "\texttt{Package/\textit{}}" +and others are simply "\texttt{Build}" is because of the possibility of generating +multiple packages from a single source. OpenWrt works under the assumption of one +source per package makefile, but you can split that source into as many packages as +desired. Since you only need to compile the sources once, there's one global set of +"\texttt{Build}" defines, but you can add as many "Package/" defines as you want +by adding extra calls to \texttt{BuildPackage} -- see the dropbear package for an example. + +After you've created your \texttt{package/\textit{}/Makefile}, the new package +will automatically show in the menu the next time you run "make menuconfig" and if selected +will be built automatically the next time "\texttt{make}" is run. + +\subsubsection{Troubleshooting} + +If you find your package doesn't show up in menuconfig, try the following command to +see if you get the correct description: + +\begin{Verbatim} + TOPDIR=$PWD make -C package/ DUMP=1 V=99 +\end{Verbatim} + +If you're just having trouble getting your package to compile, there's a few +shortcuts you can take. Instead of waiting for make to get to your package, you can +run one of the following: + +\begin{itemize} + \item \texttt{make package/\textit{}-clean V=99} + \item \texttt{make package/\textit{}-install V=99} +\end{itemize} + +Another nice trick is that if the source directory under \texttt{build\_\textit{}} +is newer than the package directory, it won't clobber it by unpacking the sources again. +If you were working on a patch you could simply edit the sources under the +\texttt{build\_\textit{}/\textit{}} directory and run the install command above, +when satisfied, copy the patched sources elsewhere and diff them with the unpatched +sources. A warning though - if you go modify anything under \texttt{package/\textit{}} +it will remove the old sources and unpack a fresh copy. + diff --git a/docs/openwrt.tex b/docs/openwrt.tex index f1269e296..e63eac3f2 100644 --- a/docs/openwrt.tex +++ b/docs/openwrt.tex @@ -5,6 +5,9 @@ \usepackage{ae,aecompl,aeguill} \usepackage{fancyvrb} +\setlength{\parindent}{0pt} +\setlength{\parskip}\medskipamount + \begin{document} \tableofcontents @@ -28,9 +31,7 @@ \input{network-scripts} \chapter{Development issues} \section{The build system} - \subsection{Building an image} - \subsection{Integrating packages} - \subsection{Creating packages} + \input{build} \section{Extra tools} \subsection{Image Builder} \subsection{SDK} -- cgit v1.2.3