Using Devtools on Arch Linux

The package Devtools was originally made for Trusted Users to properly create packages for the official repositories. However, it can be used by ordinary users as well to build AUR packages, or even modified official packages.

Refer to this guide for understanding and using the AUR in general, including obtaining the PKGBUILD. This doc only shows the steps specific to Devtools, if it is the method you choose to compile a package.

Devtools maintains a separate clean Arch installation, located in /var/lib/archbuild/<TARGET>/root, which only contains package groups base and base-devel. If this clean installation doesn’t exist, it automatically creates it. If it does exist, it automatically updates any packages in it. When Devtools is used to build a package, it starts with a copy of this clean installation, installs required packages into the copy only, copies the source code into it, performs the compiling and packaging in it, and only copies out the resulting package, in identical form from what is found in the official repositories.

There are advantages to Devtools over running makepkg directly. One advantage is that base-devel and other packages necessary to compile, but not run, the package you’re making never wind up in your main system. That’s less packages to have to periodically upgrade, and have concerns about. Although primarily a benefit for Arch package maintainers, this process easily exposes when a PKGBUILD is incorrect, such as if a dependency is missed from being listed that the maintainer happens to have already installed in their main system. You can also use a machine that is faster at building packages, and copy the resulting package to a slower machine that will run it, without polluting the building machine’s installation.

The main disadvantage is that the clean root is always there, taking about 800MB, and usually a single copy is there taking more space. Note, if /var/lib/archbuild/ uses Btrfs, the copy of the clean root starts off being a Btrfs snapshot, so those files do not take double the space. The clean root is always kept there to avoid re-installing it every time a package is being made.

Compiling Using Devtools

Install Devtools:

# pacman -S devtools

To build a package, Devtools includes archbuild, but you don’t run this directly. It also includes symlinks of {extra, gnome-unstable, kde-unstable, staging, testing}-x86_64-build. The symlink is being used to run it will be inspected by archbuild, to determine which target you want it to use. It can be ran to use these unstable/staging/testing repositories, which may have newer versions than have been released to the official repositories. To use the official repositories for non-AUR packages, in the directory with the PKGBUILD, for example the directory made by git clone, run the following:

$ extra-x86_64-build

Note: The rest of this guide will simply refer to extra-x86_64-build.

After it finishes running, the following will be the results:

• /var/lib/archbuild/extra-x86_64/root – A clean chroot, which is an up to date installation with only package groups base and base-devel.
• /var/lib/archbuild/extra-x86_64/<USERNAME> – This will contain a build chroot. This is a copy of the clean chroot with any dependencies required to build or run the package being built, as well as its source code, compilation results, and package.
• The directory you’re in will contain the package and build log files, as well as any downloaded source code.

At the end, you may notice “Checking PKGBUILD“, and “Checking <PKGNAME>-<PKGVER>-<PKGREL>-<ARCH>.pkg.tar.xz“. Any lines after these are output from namcap, which automatically looks for problems like malformed PKGBUILD files, dependencies included that the package doesn’t appear to use, dependencies not included that the package appears to use, and more. False positives are often generated by namcap, but is a great tool for giving things to investigate. If your package works correctly, it’s not a good idea to alert the maintainer to namcap output, unless you’ve looked into it and verified a change should be made.

You can use pacman to install the package, which will install any dependencies required to run the package as long as they’re in official repositories or a local repository.

Either use a Local Repository as explained here, or install the file directly:

# pacman -U <PKGNAME>-<PKGVER>-<PKGREL>-<ARCH>.pkg.tar.xz

If you were to run extra-x86_64-build again, right now, or anytime later on with this or another package, it will update the clean chroot if needed, delete the build chroot and make it a fresh copy of the clean chroot, and perform the same process. If your directory still has the source code downloaded from the last time, it will use it. If the package is a developmental AUR package, it will pull new changes rather than re-clone.

Internally, extra-x86_64-build runs makechrootpkg, which internally calls makepkg. The options for extra-x86_64-build include the following:

• -c : Clean the chroots, by removing and recreating the entire /var/lib/archbuild/extra-x86_64/ directory, including its clean chroot and all build chroot directories. This is rarely needed, only if the clean chroot gets corrupted, or if Devtools is upgraded in a way that breaks backwards compatibility.
• -r <dir> : Use a different directory than /var/lib/archbuild/extra-x86_64/ to contain the chroots.

Any arguments to extra-x86_64-build after -- are passed to makechrootpkg, when it internally uses it. Several arguments are always automatically passed from extra-x86_64-build to makechrootpkg. These automatic arguments are -r <value given to extra-x86_64-build -r option if given, /var/lib/archbuild/extra-x86_64 otherwise> -c -n. They tell makechrootpkg to remove the build chroot and make it a fresh copy of the clean chroot, and to run namcap on the package if it successfully builds. A commonly used option that can be passed to makechrootpkg is -l <copy name>. This is the directory name to give the build chroot, instead of <USERNAME>, which is useful for maintaining multiple copies or compiling multiple packages at the same time.

Any arguments to makechrootpkg after -- are passed to makepkg, when it internally uses it to build the package. The first time makepkg is run by makechrootpkg, it is done with its own unchangeable options, to download source files, if needed, and perform integrity checks; thus nothing can be forwarded on this run. It runs makepkg a second time to build the package, and always automatically passes makepkg arguments of --syncdeps --noconfirm --log --holdver --skipinteg which tells makepkg to, within the build chroot, automatically install missing dependencies required for building and using the package, not to ask for confirmation during pacman, log the build process to text files in addition to stdout, don’t update source code if in a version control system and don’t perform source file verification checks.

You can chain these together by using the following form:

$ extra-x86_64-build <DEVTOOLS-OPTIONS> -- <MAKECHROOTPKG-OPTIONS> -- <MAKEPKG-OPTIONS>

Note that /var/lib/archbuild can be treated as if it were a temporary directory. If you have multiple Vultr hard drives, it is worthwhile to mount a RAID0 (stripe) filesystem here. If you have a lot of RAM, you can also mount a RAM backed file-system like tmpfs. After a package is built, it’s copied out into the directory you ran extra-x86_64-build from and if you wanted to, at this point you could delete /var/lib/archbuild. The next run would be slower, because it would need to make a new clean root. Alternatively, you could delete /var/lib/archbuild/<USERNAME> to reclaim extra space from the build chroot before it is automatically deleted by the next run of Devtools. So, even if you had a RAID0 filesystem mounted here fail, the most you would lose would be a compilation in process.

Devtools Configuration Files

There are a few specifics to note with Devtools configuration files. They are located in /usr/share/devtools/, such as makepkg-x86_64.conf and pacman-extra.conf:

• For /etc files like makepkg.conf and pacman.conf, you can safely edit them in place, and when the package is upgraded, it won’t overwrite your changes. Rather it will save the new configuration files (if they changed from the previous version) ending with .pacnew. However, Devtools configuration files are in /usr/share/ which is not intended to be user edited, so when Devtools is upgraded, it will completely overwrite your changes to these files without alerting you. A change to this behavior has been proposed and rejected, because this helps ensure packages are sent to the official repositories all with the same compilation settings.
• The value for MAKEFLAGS, PACKAGER, and {SRC,SRCPKG,PKG,LOG}DEST are taken from /etc/makepkg.conf rather than /usr/share/devtools/makepkg-x86_64.conf.

Local Repository

If you are building packages that have dependencies on other packages you’ve built, you need to use a local repository, so that when pacman runs within the build chroot, it finds the dependencies.

To setup a local repository, refer to this guide’s “Local Repository” section.

Create a custom target:

# ln -s archbuild /usr/bin/custom-x86_64-build
# cp /usr/share/devtools/pacman-{extra,custom}.conf

Edit /usr/share/devtools/pacman-custom.conf, and add the following at the end:

[archLocalRepo]
SigLevel = Optional TrustAll
Server = file:///archLocalRepo

Edit /etc/pacman.conf, and add the following. This forces the directory to be bind mounted in the chroot:

CacheDir    = /var/cache/pacman/pkg/ /archLocalRepo/

Now, instead of using extra-x86_64-build use this:

$ custom-x86_64-build

If you always want to use the custom target, you can delete the /var/lib/archbuild/extra-x86_64-build/ directory if it exists, as the chroots will now be in /var/lib/archbuild/custom-x86_64-build/.

Package Faster

Note enabling threaded packaging involves editing the /usr/share/devtools configuration files, which isn’t officially supported, so you’ll need to perform this change each time Devtools is upgraded.

Devtools combines an entire package into an archive format. By default, it makes a .tar.xz using a single thread for the xz compression.

On multi CPU systems, you can allow xz to use multiple threads by editing /usr/share/devtools/makepkg-x86_64.conf, and change the following line:

COMPRESSXZ=(xz -c -z -)

To allow as many threads as you have virtual cores:

COMPRESSXZ=(xz -c -z - --threads=0)

To allow using multiple virtual cores, but not all of them, so as to reduce impact to overall system performance, add a specific number:

COMPRESSXZ=(xz -c -z - --threads=21)

Specifying more threads than the number of virtual cores you have will decrease performance.

If you don’t mind the package file being (potentially much) larger, disable compression by editing /usr/share/devtools/makepkg-x86_64.conf, and change the following line:

PKGEXT='.pkg.tar.xz'

Change it to look like the following:

PKGEXT='.pkg.tar'