silas.net.br
home | tech | misc | code | bookmarks (broken) | contact | README
This file documents my efforts to make pkgsrc bulkbuild work.
Two very important resources are the bulkbuild chapter of the pkgsrc guide and the bulkbuild HOWTO in the pkgsrc "inner" documentation. Also, take a look at the pbulk wiki HOWTO.
In this guide modern pbulk is made instead of classic bulkbuild, as shown in the bulkbuild chapter, although the terminology "bulkbuild" is used.
I setup a machine only for bulkbuilds, because I didn't want to setup a chroot environment in one of the work machines (the bulkbuild chapter recommends to either set a special machine or a chroot'ed environment, because bulkbuilds usually mess things up, invalidating the machine for anything that is not building packages).
In this machine, I downloaded pkgsrc and untared it in the default place: /usr/pkgsrc:
$ cd /tmp $ ftp ftp://ftp.netbsd.org/pub/pkgsrc/pkgsrc-2009Q2/pkgsrc.tar.gz $ su - # cd /tmp # tar zxf pkgsrc.tar.gz -C /usr
The guide says, yet, that it is necessary to bootstrap pkgsrc, even if NetBSD is being used. So:
# cd /usr/pkgsrc/bootstrap # ./bootstrap --prefix /usr/pbulk --varbase /usr/pbulk/var
Important
If you are trying to run pbulk in a non-NetBSD environment, you need to make a bootstrap kit as told in bulkbuild HOWTO. So, after the previous bootstrap command, execute the following commands:
# ./cleanup
# ./bootstrap --mk-fragment /tmp/mk.conf \
--gzip-binary-kit bootstrap_kit.tar.gz
If you cannot find the tarball, where is it? See if it is in /.
(For more information on bootstrapping pkgsrc, check the pkgsrc-bootstrap page).
Then, install the pbulk tools:
# cd /usr/pkgsrc/pkgtools/pbulk # /usr/pbulk/bin/bmake install
All configuration is made by a single file, located in ${prefix}/etc/pbulk.conf, in our case, /usr/pbulk/etc/pbulk.conf.
Important changes to this file are:
Set master_mode=no if you don't want to use multiple machines for building the packages.
Set bootstrapkit= if you are using NetBSD (according to the bulkbuild HOWTO).
Set limited_list=/limited_list to the path to the file that you want to specify what packages you want to build. For example:
limited_list=/home/user/pbulk-limited_list
And, in the file, put the new-line separated list of packages you want to build:
audio/bladeenc audio/cdparanoia chat/irssi devel/scmgit editors/vim lang/sbcl mail/mutt misc/screen security/sudo www/elinks www/firefox3
pkgsrc now has user-destdir enabled by default and you should tell in pbulk.conf what is the unprivileged user you want it to use to build pacakges in the unprivileged_user variable. Create this user manually (if it doesn't exist yet) and make sure destination directory of packages and pkgsrc both have their owner set to this user.
If you are using NetBSD, set make=/usr/bin/make.
If you don't want to publish your packets automatically, set mail=: and rsync=:. I forgot to do that and it produced a lot of non-clear problems.
The environment and configuration are set up.
Note
Before executing the following steps, I had problems making bulkbuild finding bmake in the right directory, although I set the make variable correctly in pbulk.conf. To make it work, I needed to:
# export PATH=/usr/pkg/bin:/usr/pkg/sbin:${PATH}
Let's build the packages:
# cd /usr/pbulk/bin # ./bulkbuild
It will scan for the packages and start to build them. It will deliver the built packages where the PACKAGES variable (in /etc/mk.conf) points to, defaults to /usr/pkgsrc/packages.
Most errors can arrive in this step. Check the Troubleshooting section.
Since the bulkbuild is complete, if you change your limited_list file and run ./builbuild script again, it will build all the packages in the limited_list, including those that are already built. To build only the packages you added, follow the following steps:
Editor the limited_list file normally.
Run the scan fase of the pbulk:
# cd /usr/pbulk/libexec/pbulk # ./scan
Run the build fase:
# ./build
The version of pkg_admin you have installed in the base system is probably too old. If you bootstrapped pkgsrc, it compiled a newer version of pkg_admin for you. You have to tell pkgsrc that you want to use the bootstrapped version:
# export PKG_TOOLS_BIN=/usr/pbulk/sbin
More information about bulk builds environment variables in pkgsrc/doc/mk/bulk/environment.
Postfix is probably not working. Try /etc/rc.d/postfix forcestart. Before doing that, check whether or not you need it (read bulkbuild HOWTO again). If not, make rsync=: and mail=: in pbulk.conf.
Does the variable PACKAGES in mk.conf points to the same place where the variable packages in pbulk.conf points to?
They should point to the same place.
Check report.html. Does it tell that you that pkgsrc needs X11? Did you forgot to install the X11 sets in your NetBSD installation?
You may get an error like:
===> Installing for tnftp-20070806 ERROR: tnftp-20070806 is already installed - perhaps an older version? ERROR: If so, you may use either of: ERROR: - "pkg_delete tnftp-20070806" and "/usr/pbulk/bin/bmake reinstall" ERROR: to upgrade properly ERROR: - "/usr/pbulk/bin/bmake update" to rebuild the package and all ERROR: of its dependencies ERROR: - "/usr/pbulk/bin/bmake replace" to replace only the package without ERROR: re-linking dependencies, risking various problems. *** Error code 1
And /usr/pbulk is your bootstrapped pkgsrc, you are probably wondering why it is calling /usr/pbulk/bin/bmake and not /usr/pkg/bin/bmake. Check if you set make variable correctly in pbulk.conf according to your architecture and the bulkbuild HOWTO.
Besides links on the text, there are other references worth to have a look:
If you get an error like:
Unresolvable dependencies found, exiting: databases/py-sqlite3
databases/py-sqlite3 exists. What is the problem, then? On the scan/resolve phase of a bulkbuild (pbulk), it is very likely to be a version mismatch problem.
In this case I had to add this line, to avoid version 2.4 of Python being considered:
PYTHON_VERSIONS_ACCEPTED=27 26 25
Joerg Sonnenberger here helped me with this problem.