Red Antigua Logo
Yet another piece of web.
Ads by Goooooogle
Search this site (by Google)
Tools    (top)
Your IP
Check a site for broken links
(W3C)

Perl modules    (top)
Tree::Numbered::Tools
(CPAN)
Perl tutorials    (top)
Perl modules
HTML::Template
CGI::Application
Mail::POP3Client
Mail::Send
MIME::Tools
Cookies with CGI::Application
Upload files with CGI::Application
Download files with CGI::Application
Redirect with CGI::Application
CPAN shell
Install DBD::mysql from the CPAN shell
Perl trim function
Validate an IP with Perl
Run suid Perl scripts under Apache
Perl taint mode
Perl date functions with Date::Calc

In Spanish
Curso de Perl

C tutorials    (top)
C - Introduction
C - Absolute beginner's Emacs
C - Examples for beginners
C - Makefile examples
C - Autotools examples
Server configurations    (top)
DNS
Apache
Apache Authentication and Access Control
mod_perl on FreeBSD
MySQL
MySQL add account
phpMyAdmin
Squid
DHCP

UNIX on Windows    (top)
Apache setup on Windows
MySQL setup on Windows
PHP setup on Windows
Perl setup on Windows
Emacs setup on Windows
UnxUtils
PuTTY
WinSCP
GIMP on Windows
MinGW - gcc on Windows
MSYS - UNIX-styled shell on Windows
msysDTK - autotools on Windows
GDB for MinGW on Windows

Misc. FreeBSD/UNIX    (top)
CD and DVD creation on FreeBSD using 'k3b' on FreeBSD
'portupgrade' on FreeBSD
'ipf' on FreeBSD
'pf' on FreeBSD
'su' on FreeBSD
Mount an ISO image under FreeBSD
Load the correct sound driver under FreeBSD without knowing what sound card you are using
Simultaneous sound channels on FreeBSD
FreeBSD network stuff
DOS-to-UNIX file conversion
favicon.ico on UNIX
Emacs tips
Sendmail tips
GKrellm
Command Line Calculator
Save multimedia streams with 'mplayer'
xargs - solution to 'Argument list too long'
Process multiple images from the command line using 'ImageMagick'
Turn the system bell off under X Windows
Process each line in an input file from the command line (or in a shell script)
How to keep a program running in the background using 'nohup'
How to remove symbolic links in the current directory using 'find' and 'rm'
How to remove Emacs backup files in the current directory and all subdirectories using 'find' and 'rm'
How to execute .profile without logging in
Configure X to handle non-English characters
How to move /var to /usr/var

Redirect a web page    (top)
Redirect to another web page
Apache redirect
C redirect
Perl redirect
PHP redirect
HTML redirect
JavaScript redirect

Javascript    (top)
Trim function
Login form
Register form
Popup window

Virus list    (top)
Latest 10 viruses - Sophos

Off topic    (top)
Personal home links

Upgrade FreeBSD ports with portsnap, pkg_version, and portupgrade
Upgrade FreeBSD ports with portsnap, pkg_version, and portupgrade        
Before reading this tutorial, I recommend you to check out Using the Ports Collection from the FreeBSD Handbook, which may have some more up-to-date information that is missing here.
Then you may come back here for some tips and troubleshooting.

Intro (2012-04-17)
All-in-one ports upgrade, using portsnap, pkg_version, and portupgrade (2012-04-17)
Synchronize /usr/ports using csup (2009-09-18)
Quick guide to portupgrade (2009-09-18)
Troubleshooting: (2009-09-18)
- General (2009-09-18)
- pkg_delete causes 'Segmentation fault (core dumped)' (2009-09-18)
- 'PkgDB::DBError' (2005-12-17)
- 'make: don't know how to make describe.' (no date)
- 'Port info line must consist of 10 fields.' (no date - old, before July 2004)
See also
Intro (top) (2012-04-17)


This tutorial is about upgrading the FreeBSD ports collection. I started to write it many years ago, and have added things during the years, but left old things as they were. Thus, the further down you read, bigger the chances are that you find obsolete information, so please check the date when each paragraph was written. Check See also external links for up-to-date information.

If you come from a Linux desktop environment (i.e. Ubuntu), you will probably find the FreeBSD ports upgrade procedure quite painful in comparision to Update Manager.
In the case of FreeBSD, the upgrade procedure is a bit tricky, and we have to stick to the command-line.
The upgrade of the ports collection is done in two steps:
1. Update your ports tree. This basically means downloading the newest available Makefile for each port. Each port's Makefile indicates which version is newest one available (which may or may not be the one already installed). This step is the quick one.
2. Update your ports. This means downloading and installing either precompiled packages (faster, preferrable), or source code to be compiled (slower, but sometimes necessary, see comments below). This step may take a long time. Or a very, very, very long time. Read on.


All-in-one ports upgrade, using portsnap, pkg_version, and portupgrade (top) ((2012-04-17, tested on FreeBSD 9.0)
Step 1.
Update your ports tree using portsnap.
First time you use portsnap, you must run:
portsnap fetch
portsnap extract
To update the ports tree (always, first time included):
portsnap fetch
portsnap update
Step 2.
Upgrade your ports collection using pkg_version and portupgrade.
BATCH=yes portupgrade -arPP
pkg_version -q -l '<' > /tmp/upgrade-me.txt
BATCH=yes portupgrade -rP `cat /tmp/upgrade-me.txt`
pkg_version -q -l '<' > /tmp/upgrade-me.txt
BATCH=yes portupgrade -f `cat /tmp/upgrade-me.txt`

A script to do it all-in-one (yes, as root): download
#!/bin/sh
# Update your ports tree using <code>portsnap</code>.<br>
# First time you use portsnap, you must run: 
portsnap fetch
portsnap extract

# To update the ports tree (always, first time included):
portsnap fetch
portsnap update

# Upgrade the ports using only precompiled packages, never source code to compile (faster).
# The BATCH environment variable avoids the annoying blue popup screens, and the port is installed with the default configuration.
BATCH=yes portupgrade -arPP

# Save a list of the ports which still need to be upgraded (normally ports which only exist as source code, but also precompiled packaged which may have failed to upgrade correctly).
pkg_version -q -l '<' > /tmp/upgrade-me.txt

# Upgrade the ports using precompiled packages if possible, otherwise source code.
BATCH=yes portupgrade -rP `cat /tmp/upgrade-me.txt`

# Save a new, updated list.
pkg_version -q -l '<' > /tmp/upgrade-me.txt

# Force (re)install of ports that still haven't been upgraded correctly.
BATCH=yes portupgrade -f `cat /tmp/upgrade-me.txt`


2009-09-18:
First of all, upgrading your ports collection will take a lot of time.
A FreeBSD system using a desktop environment like GNOME and/or KDE have several hundred ports installed, which easily can take about a week to upgrade.
At the time of this writing, (2012-04-17) there are (AFAK) three different tools to upgrade the FreeBSD ports collection, portupgrade, portmanager, and portmaster, the choice is yours.
I mention cvsup in this tutorial, but nowadays csup is the default tool to synchronize your /usr/ports/ directory with a CVSup server. (You can also synchronize /usr/ports/ using portsnap or sysinstall, but I will not mention these tools here.)

To minimize both headache and time for the upgrade procedure, there are a few tips:
  • Synchronize /usr/ports/ from scratch, from a reliable CVSup server. It will take a little bit longer than just updating an existing /usr/ports/, but it will guarantee you to have a fresh copy of the ports tree.
    These are the steps:
    1. Find a CVSup site near you.
    2. Check the FreeBSD CVSup Modules statistics for the server you have chosen. If your server is marked red, change server.
    3. Delete you existing ports tree and get a fresh tree:
      rm -rf /usr/ports
      # Change cvsup.FreeBSD.org for a server near you to have a faster update.
      csup -L 2 -h cvsup.FreeBSD.org /usr/share/examples/cvsup/ports-supfile
      
  • Read /usr/ports/UPDATING!
    This is probably your biggest time saver, especially if you haven't upgraded you ports collection recently.
    The are ports that portupgrade isn't able to upgrade correctly. There are detailed instructions in /usr/ports/UPDATING how to upgrade these ports manually before upgrading other ports. If you do not take this step, the complete upgrade will most probably fail. Other ports, depending on ports which need to be manually upgraded, will try to build (which may be really time consuming) but will fail due to the port dependency. In the worst case, omitting this step may also mess up the package database.
  • Use packages instead of ports when possible. Checkout the -PP and the -P flags in the portupgrade(1) man page, and the examples below.

Quick guide to portupgrade (top)
# Synchronize /usr/ports.
See above.

# Install portupgrade.
pkg_add -r portupgrade

# List outdated ports.
pkg_version -v

# Read /usr/ports/UPDATING to see check which ports need manual attention.
Read it!

# Tell pkg_fetch(1) to get packages from the URLs listed below (change URLs to fit your needs).
export PKG_SITES="ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release \
  ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable \
  ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-stable"

# Upgrade all installed ports using packages only. Skip upgrading a port if a package is not available.
portupgrade -aPP

# If you aren't able to upgrade any ports using portupgrade -aPP, there is probably something wrong with the URLs in PKG_SITES environment variable.
You have two alternatives to portupgrade -PP:
# Upgrade all installed ports using packages where available. Build a port from source if a package is not available.
portupgrade -aP
# This will take a longer time than using the -aPP flag, but will guarantee that all your ports will be upgraded.
# Upgrade all installed ports building the ports from source.
portupgrade -a
# This will take a long time.
Troubleshooting: (top)

1. General (top)
In case of an error, always try these steps first:
# Run
pkgdb -F
# if you are prompted to.

# Run
pkgdb -fu
# if everything goes wrong.

# When a port fails, a solution may be to uninstall it and then reinstall it again manually
cd /usr/ports/failing/port
make deinstall clean
make install clean
# and then run the portupgrade command again.


2. pkg_delete causes 'Segmentation fault (core dumped)' (2009-09-18) - Solution: modify +CONTENTS file (top)
PROBLEM:
During a portupgrade, many ports failed.
I tried to reinstall some of failing ports:
cd /usr/ports/whatever/failing-port
make deinstall clean
Segmentation fault (core dumped)
pkg_delete failing-port-1.1
Segmentation fault (core dumped)
pkg_delete -f failing-port-1.1
Segmentation fault (core dumped)
No luck. Updating the database didn't work either. Then I found this thread with a problem similar to mine.

If the output from
pkg_delete -v failing-port-1.1
shows one or more Dependency: (null) lines, we have confirmed that the package's +CONTENTS file is corrupted.
SOLUTION:
Edit the +CONTENTS file, remove all "Dependencies without dependencies" lines, that is @pkgdep followed by nothing:
@pkgdep
To save time, fix all corrupted +CONTENTS files before you continue with portupgrade.
The command below lists all corrupted +CONTENTS files:
find /var/db/pkg -name '+CONTENTS'|xargs egrep -l '^@pkgdep[[:blank:]]*$'

3. 'PkgDB::DBError' (2005-12-17) - Solution: /usr/local/sbin/pkgdb -fu (top)
I ran
/usr/local/bin/cvsup -g -L 2 -h cvsup17.freebsd.org /usr/share/examples/cvsup/ports-supfile
and
/usr/local/sbin/portsdb -Uu
without any problem. But
/usr/local/sbin/portversion -l "<"
failed:
root@jf# /usr/local/sbin/portversion -l "<"
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ... - 515 packages found
(-1 +8) (...)........ done]
(? bsdpan-CGI-Application-Plugin-DBH-3.00)(? bsdpan-DBD-Mock-0.26)(? bsdpan-Pars
e-RecDescent-1.94)/usr/local/lib/ruby/site_ruby/1.8/pkgdb.rb:351:in `deorigin_gl
ob': Invalid argument (PkgDB::DBError)
        from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:177:in `config_includ
e?'
        from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:172:in `each'
        from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:172:in `config_includ
e?'
        from /usr/local/lib/ruby/site_ruby/1.8/pkgtools.rb:162:in `config_held?'
        from /usr/local/sbin/portversion:310:in `check_pkgs'
        from /usr/local/sbin/portversion:300:in `each'
        from /usr/local/sbin/portversion:300:in `check_pkgs'
        from /usr/local/sbin/portversion:260:in `main'
        from /usr/local/sbin/portversion:77:in `initialize'
        from /usr/local/sbin/portversion:77:in `new'
        from /usr/local/sbin/portversion:77:in `main'
        from /usr/local/sbin/portversion:347
root@jf#
Forgot about my own suggestion on this very page :-(
# Run
/usr/local/sbin/pkgdb -fu
# if everything goes wrong.
After that I could run both
/usr/local/sbin/portversion -l "<"
and
/usr/local/sbin/portupgrade -arR
without any problem.
4. 'make: don't know how to make describe.' - Solution: refetch and restore the ports collection (top)
Once I was dumb enough to install a port while I was running a portupgrade.
I don't know what part of the ports collection became corrupt, but I tried many ways to fix it.
Error before the fix:
bash-2.05b# /usr/local/sbin/portsdb -Uu
Updating the ports index ... Generating INDEX.tmp - please wait..make: don't know how to make describe. Stop
===> textproc/p5-Sort-PolySort failed
*** Error code 1
1 error
The final solution was to refetch the entire ports collection (almost as quick as a portupgrade, just add the time for downloading the ports collection).
cd /usr
mv ports ports.CORRUPT
fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/ports/ports.tar.gz
gunzip -c ports.tar.gz| tar -xf -
mv /usr/ports.CORRUPT/distfiles /usr/ports/
/usr/local/bin/cvsup -g -L 2 -h cvsup17.freebsd.org /usr/share/examples/cvsup/ports-supfile
/usr/local/sbin/portsdb -Uu
/usr/local/sbin/portversion -l "<"
/usr/local/sbin/portupgrade -arR

5. 'Port info line must consist of 10 fields' - Solution: deinstall/reinstall portupgrade (top)
This problem will probably occur only on old versions on FreeBSD, with an old version of portupgrade installed.
Check your portupgrade version:
pkg_info -I portupgrade\*
If your portupgrade version is more recent or equal to 20040701_3, you can stop read here - your problem isn't version-related.
Long description:
Once I had an unexpected power-off during the portsdb -Uu command.
After rebooting and trying to continue portsdb -Uu, the command failed with the following error:
...
/usr/ports/INDEX-5:3:Port info line must consist of 10 fields.
/usr/ports/INDEX-5:4:Port info line must consist of 10 fields.
/usr/ports/INDEX-5:5:Port info line must consist of 10 fields.
...
I just waited until the next day, ran cvsup again, and some ports were updated, so then I had a new INDEX.
(I also browsed mailing lists confirming that this was the way to go in case of a ports collection out of sync.)
Then I had a new error message:
bash-2.05b# /usr/local/sbin/portsdb -Uu
Updating the ports index ... Generating INDEX.tmp - please wait..make: don't know how to make describe. Stop
===> ftp/lukemftpd failed
*** Error code 1
1 error

********************************************************************
Before reporting this error, verify that you are running a supported
version of FreeBSD (see http://www.FreeBSD.org/ports/) and that you
have a complete and up-to-date ports collection.  (INDEX builds are
not supported with partial or out-of-date ports collections -- in
particular, if you are using cvsup, you must cvsup the "ports-all"
collection, and have no "refuse" files.)  If that is the case, then
report the failure to ports@FreeBSD.org together with relevant
details of your ports configuration (including FreeBSD version,
your architecture, your environment, and your /etc/make.conf
settings, especially compiler flags and WITH/WITHOUT settings).

Note: the latest pre-generated version of INDEX may be fetched
automatically with "make fetchindex".
********************************************************************

*** Error code 1

Stop in /usr/ports.
*** Error code 1

Stop in /usr/ports.
failed to generate INDEX!
portsdb: index generation error
I tested
cd /usr/ports
make fetchindex
which downloaded a pre-generated INDEX-5 file, but portsdb -Uu still failed with the same error.
I tried cvsup again, and then portsdb -Uu.
This did not work either.

I tried various cvsups and portupgrades without success.
Finally, I found a thread at http://www.mail-archive.com/freebsd-questions@freebsd.org/msg72733.html, which indicated a format change for portupgrade.
The solution was to deinstall the current portupgrade version, and reinstall it.
As the cvsup already had upgraded the port's info, I just did:
cd /usr/ports/sysutils/portupgrade
make deinstall
make install clean
Then /usr/local/sbin/portsdb -Uu worked just fine.

See also: (top) Read more about portupgrade from Dru Lavigne's excellent portupgrade tutorial.
The only thing where I do things in a different order from her tutorial, is synchronizing the ports collection before installing the portupgrade port the first time I upgrade the ports collection on a new system.
This has saved me a lot of trouble, as an upgraded version of portupgrade does not suffer from broken dependencies.

Links:
http://wiki.freebsd.org/portupgrade
http://www.freebsd.org/doc/handbook/portsnap.html
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports-using.html
http://www.freebsd.org/cgi/man.cgi?query=pkg_version&apropos=0&sektion=0&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html
Last modified: Tue Apr 17 16:02:40 +0200 2012