Upgrade FreeBSD ports with portsnap, pkg_version, and portupgrade
Upgrade FreeBSD ports with portsnap, pkg_version, and portupgrade
Before reading this tutorial, I recommend you to check out Using the Ports Collection from the FreeBSD Handbook, which may have some more up-to-date information that is missing here.
Then you may come back here for some tips and troubleshooting.
This tutorial is about upgrading the FreeBSD ports collection. I started to write it many years ago, and have added things during the years, but left old things as they were. Thus, the further down you read, bigger the chances are that you find obsolete information, so please check the date when each paragraph was written. Check See also external links for up-to-date information.
If you come from a Linux desktop environment (i.e. Ubuntu), you will probably find the FreeBSD ports upgrade procedure quite painful in comparision to Update Manager.
In the case of FreeBSD, the upgrade procedure is a bit tricky, and we have to stick to the command-line.
The upgrade of the ports collection is done in two steps:
1. Update your ports tree. This basically means downloading the newest available Makefile for each port. Each port's Makefile indicates which version is newest one available (which may or may not be the one already installed). This step is the quick one.
2. Update your ports. This means downloading and installing either precompiled packages (faster, preferrable), or source code to be compiled (slower, but sometimes necessary, see comments below). This step may take a long time. Or a very, very, very long time. Read on.
All-in-one ports upgrade, using portsnap, pkg_version, and portupgrade(top) ((2012-04-17, tested on FreeBSD 9.0)
Step 1.
Update your ports tree using portsnap.
First time you use portsnap, you must run:
portsnap fetch
portsnap extract
To update the ports tree (always, first time included):
portsnap fetch
portsnap update
Step 2.
Upgrade your ports collection using pkg_version and portupgrade.
A script to do it all-in-one (yes, as root): download
#!/bin/sh# Update your ports tree using <code>portsnap</code>.<br>
# First time you use portsnap, you must run:
portsnap fetch
portsnap extract
# To update the ports tree (always, first time included):portsnap fetch
portsnap update
# Upgrade the ports using only precompiled packages, never source code to compile (faster).
# The BATCH environment variable avoids the annoying blue popup screens, and the port is installed with the default configuration.BATCH=yes portupgrade -arPP
# Save a list of the ports which still need to be upgraded (normally ports which only exist as source code, but also precompiled packaged which may have failed to upgrade correctly).
pkg_version -q -l '<' > /tmp/upgrade-me.txt
# Upgrade the ports using precompiled packages if possible, otherwise source code.BATCH=yes portupgrade -rP `cat /tmp/upgrade-me.txt`# Save a new, updated list.pkg_version -q -l '<' > /tmp/upgrade-me.txt
# Force (re)install of ports that still haven't been upgraded correctly.
BATCH=yes portupgrade -f `cat /tmp/upgrade-me.txt`
2009-09-18:
First of all, upgrading your ports collection will take a lot of time.
A FreeBSD system using a desktop environment like GNOME and/or KDE have several hundred ports installed, which easily can take about a week to upgrade.
At the time of this writing, (2012-04-17) there are (AFAK) three different tools to upgrade the FreeBSD ports collection, portupgrade, portmanager, and portmaster, the choice is yours. I mention cvsup in this tutorial, but nowadays csup is the default tool to synchronize your /usr/ports/ directory with a CVSup server. (You can also synchronize /usr/ports/ using portsnap or sysinstall, but I will not mention these tools here.)
To minimize both headache and time for the upgrade procedure, there are a few tips:
Synchronize /usr/ports/ from scratch, from a reliable CVSup server. It will take a little bit longer than just updating an existing /usr/ports/, but it will guarantee you to have a fresh copy of the ports tree.
These are the steps:
Delete you existing ports tree and get a fresh tree:
rm -rf /usr/ports
# Change cvsup.FreeBSD.org for a server near you to have a faster update.
csup -L 2 -h cvsup.FreeBSD.org /usr/share/examples/cvsup/ports-supfile
Read /usr/ports/UPDATING!
This is probably your biggest time saver, especially if you haven't upgraded you ports collection recently.
The are ports that portupgrade isn't able to upgrade correctly. There are detailed instructions in /usr/ports/UPDATING how to upgrade these ports manually before upgrading other ports. If you do not take this step, the complete upgrade will most probably fail. Other ports, depending on ports which need to be manually upgraded, will try to build (which may be really time consuming) but will fail due to the port dependency. In the worst case, omitting this step may also mess up the package database.
Use packages instead of ports when possible. Checkout the -PP and the -P flags in the portupgrade(1) man page, and the examples below.
# Synchronize /usr/ports.
See above.
# Install portupgrade.
pkg_add -r portupgrade
# List outdated ports.
pkg_version -v
# Read /usr/ports/UPDATING to see check which ports need manual attention.
Read it!
# Tell pkg_fetch(1) to get packages from the URLs listed below (change URLs to fit your needs).
export PKG_SITES="ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release \
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable \
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-stable"
# Upgrade all installed ports using packages only. Skip upgrading a port if a package is not available.
portupgrade -aPP
# If you aren't able to upgrade any ports using portupgrade -aPP, there is probably something wrong with the URLs in PKG_SITES environment variable.
You have two alternatives to portupgrade -PP:
# Upgrade all installed ports using packages where available. Build a port from source if a package is not available.
portupgrade -aP
# This will take a longer time than using the -aPP flag, but will guarantee that all your ports will be upgraded.
# Upgrade all installed ports building the ports from source.
portupgrade -a
# This will take a long time.
Troubleshooting:(top) 1.General(top)
In case of an error, always try these steps first:
# Run
pkgdb -F
# if you are prompted to.# Run
pkgdb -fu
# if everything goes wrong.# When a port fails, a solution may be to uninstall it and then reinstall it again manually
cd /usr/ports/failing/port
make deinstall clean
make install clean
# and then run the portupgrade command again.
2.pkg_delete causes 'Segmentation fault (core dumped)' (2009-09-18) - Solution: modify +CONTENTS file(top)
PROBLEM:
During a portupgrade, many ports failed.
I tried to reinstall some of failing ports:
cd /usr/ports/whatever/failing-port
make deinstall clean
Segmentation fault (core dumped)
No luck. Updating the database didn't work either. Then I found this thread with a problem similar to mine.
If the output from
pkg_delete -v failing-port-1.1
shows one or more Dependency: (null) lines, we have confirmed that the package's +CONTENTS file is corrupted.
SOLUTION:
Edit the +CONTENTS file, remove all "Dependencies without dependencies" lines, that is @pkgdep followed by nothing:
@pkgdep
To save time, fix all corrupted +CONTENTS files before you continue with portupgrade.
The command below lists all corrupted +CONTENTS files:
root@jf# /usr/local/sbin/portversion -l "<"
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ... - 515 packages found
(-1 +8) (...)........ done]
(? bsdpan-CGI-Application-Plugin-DBH-3.00)(? bsdpan-DBD-Mock-0.26)(? bsdpan-Pars
e-RecDescent-1.94)/usr/local/lib/ruby/site_ruby/1.8/pkgdb.rb:351:in `deorigin_gl
ob': Invalid argument (PkgDB::DBError)
from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:177:in `config_includ
e?'
from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:172:in `each'
from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:172:in `config_includ
e?'
from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:162:in `config_held?'
from /usr/local/sbin/portversion:310:in `check_pkgs'
from /usr/local/sbin/portversion:300:in `each'
from /usr/local/sbin/portversion:300:in `check_pkgs'
from /usr/local/sbin/portversion:260:in `main'
from /usr/local/sbin/portversion:77:in `initialize'
from /usr/local/sbin/portversion:77:in `new'
from /usr/local/sbin/portversion:77:in `main'
from /usr/local/sbin/portversion:347
root@jf#
Forgot about my own suggestion on this very page :-(
# Run
/usr/local/sbin/pkgdb -fu
# if everything goes wrong.
After that I could run both
/usr/local/sbin/portversion -l "<"
and
/usr/local/sbin/portupgrade -arR
without any problem.
4.'make: don't know how to make describe.' - Solution: refetch and restore the ports collection(top)
Once I was dumb enough to install a port while I was running a portupgrade.
I don't know what part of the ports collection became corrupt, but I tried many ways to fix it.
Error before the fix:
bash-2.05b# /usr/local/sbin/portsdb -Uu
Updating the ports index ... Generating INDEX.tmp - please wait..make: don't know how to make describe. Stop
===> textproc/p5-Sort-PolySort failed
*** Error code 1
1 error
The final solution was to refetch the entire ports collection (almost as quick as a portupgrade, just add the time for downloading the ports collection).
5.'Port info line must consist of 10 fields' - Solution: deinstall/reinstall portupgrade(top)
This problem will probably occur only on old versions on FreeBSD, with an old version of portupgrade installed.
Check your portupgrade version:
pkg_info -I portupgrade\*
If your portupgrade version is more recent or equal to 20040701_3, you can stop read here - your problem isn't version-related.
Long description:
Once I had an unexpected power-off during the portsdb -Uu command.
After rebooting and trying to continue portsdb -Uu, the command failed with the following error:
...
/usr/ports/INDEX-5:3:Port info line must consist of 10 fields.
/usr/ports/INDEX-5:4:Port info line must consist of 10 fields.
/usr/ports/INDEX-5:5:Port info line must consist of 10 fields.
...
I just waited until the next day, ran cvsup again, and some ports were updated, so then I had a new INDEX.
(I also browsed mailing lists confirming that this was the way to go in case of a ports collection out of sync.)
Then I had a new error message:
bash-2.05b# /usr/local/sbin/portsdb -Uu
Updating the ports index ... Generating INDEX.tmp - please wait..make: don't know how to make describe. Stop
===> ftp/lukemftpd failed
*** Error code 1
1 error
********************************************************************
Before reporting this error, verify that you are running a supported
version of FreeBSD (see http://www.FreeBSD.org/ports/) and that you
have a complete and up-to-date ports collection. (INDEX builds are
not supported with partial or out-of-date ports collections -- in
particular, if you are using cvsup, you must cvsup the "ports-all"
collection, and have no "refuse" files.) If that is the case, then
report the failure to ports@FreeBSD.org together with relevant
details of your ports configuration (including FreeBSD version,
your architecture, your environment, and your /etc/make.conf
settings, especially compiler flags and WITH/WITHOUT settings).
Note: the latest pre-generated version of INDEX may be fetched
automatically with "make fetchindex".
********************************************************************
*** Error code 1
Stop in /usr/ports.
*** Error code 1
Stop in /usr/ports.
failed to generate INDEX!
portsdb: index generation error
I tested
cd /usr/ports
make fetchindex
which downloaded a pre-generated INDEX-5 file, but portsdb -Uu still failed with the same error.
I tried cvsup again, and then portsdb -Uu.
This did not work either.
I tried various cvsups and portupgrades without success.
Finally, I found a thread at http://www.mail-archive.com/freebsd-questions@freebsd.org/msg72733.html, which indicated a format change for portupgrade.
The solution was to deinstall the current portupgrade version, and reinstall it.
As the cvsup already had upgraded the port's info, I just did:
cd /usr/ports/sysutils/portupgrade
make deinstall
make install clean
Then /usr/local/sbin/portsdb -Uu worked just fine.
See also:(top)
Read more about portupgrade from
Dru Lavigne's excellent portupgrade tutorial.
The only thing where I do things in a different order from her tutorial, is synchronizing the ports collection before installing the portupgrade port the first time I upgrade the ports collection on a new system.
This has saved me a lot of trouble, as an upgraded version of portupgrade does not suffer from broken dependencies.
