OpenBSD Ports - Porting Guide [Handbook Index]

Overview

So you've just compiled your favorite software package on your OpenBSD machine and you want to share your effort by turning it into a standard port. What to do?

The most important thing to do is to communicate. Ask people on ports@openbsd.org if they are working on the same port. Tell the original software authors about it, including problems you may find. If licensing information appears incorrect, tell them. If you had to jump through hoops to make the port build, tell them what can be fixed. If they are only developing on Linux and feel like ignoring the rest of the Unix world, try to make them change their view.

Communication makes the difference between a successful port and a port that will slowly be abandoned by everyone.

First, look at the porting information on this page. Test, then re-test, and finally test again!

OpenBSD fully supports updates. This means that some issues must be taken into account.

Submit the port. Create a gzipped tarball of the port directory. You can then either place it on a public HTTP server, sending its URL to ports@openbsd.org, or send the port MIME encoded to the same address.

Porting some new software takes time. Maintaining it over time is harder. It is quite okay to port software and let other people handle it afterwards. It is also okay to help other people update and maintain other ports, as long as you communicate to avoid doing the same things twice.

In OpenBSD culture, MAINTAINERship is not a status item, but a responsibility. We have CVS and comments to give credit to the person who did the work. A port MAINTAINER is something else: a person who assumes responsibility for the working of the port, and is willing to spend some time ensuring it works as best as can be.

Porting Checklist

The list below is a useful reminder of things to do. This is neither totally accurate nor perfect. Direct comments and questions to ports@openbsd.org.
  1. If you want to be a maintainer, subscribe to ports@openbsd.org.
  2. Being a maintainer means more than just submitting ports. It also means trying to keep them up-to-date, and answering questions about them.

  3. Check out a copy of the ports tree from CVS. You can find instructions on how to do this at the AnonCVS page. As a porter, you should keep your base OS, ports tree, and installed packages up to date.

  4. From the names of the first level subdirectories of /usr/ports/, pick a primary category for your port. Create a new directory below /usr/ports/<category>/ or /usr/ports/mystuff/<category>/ and create the basic infrastructure there. Copy the template Makefile from /usr/ports/infrastructure/templates/Makefile.template. Fill in CATEGORIES with the primary category you have chosen.

  5. Add the fetch portions of the Makefile.

    For more complex ports, you have more options and tools available to you:


  6. Create a checksum in distinfo by typing make makesum. Then verify the checksum is correct by typing make checksum.
  7. Extract the port with make extract. Pay attention to where the base of the sources are. Usually, it's /usr/ports/pobj/${PKGNAME}${FLAVOR_EXT}/${DISTNAME}. You may need to modify the Makefile's WRKDIST variable if it is different.

  8. Read the installation documentation and note what you have to do to build the port and any special options that might be needed.

  9. Now is also a good time to figure out what kind of licensing restrictions apply to your port. Many are freely redistributable, but quite a few are not. We need three questions answered to distribute ports properly. These are the PERMIT_* values in the Makefile.

    Set these values to Yes if it is permitted or to a comment string stating why it is not. Pay attention to any special conditions you may need to fulfill later on. For example, some ports require installing a copy of the license. We recommend you place the license in /usr/local/share/doc/<name>/.

    In addition to the PERMIT_* values, put a license marker like # License above them as a comment, this way we know why the PERMIT_* values are set the way they are.

  10. Add configuration options to Makefile and/or create the configuration script.
  11. Try building the port with make build.
  12. Begin a cycle of make build, generate a patch using make update-patches and make clean patch.
  13. Try setting SEPARATE_BUILD.
  14. Peruse the output (if any) and tweak any options in the Makefile. To repeat issue the command make clean configure.

    Note: make sure host-dependent files go in /etc or /etc/<name>, but NEVER REPLACE OR MODIFY existing files in /etc. Best to have install place them in /usr/local/share/<name> and then copy to /etc or /etc/<name> only if the files do not exist. If the files exist, display a message that says such-and-such files need to be modified. This also guarantees that the files will be included in the package since everything under /usr/local is included in the PLIST. To handle the copying carefully, the @sample keyword is preferably used within the PLIST. After a package has been installed the contents of pkg/MESSAGE will be displayed if it exists.

    The OpenBSD file locations are: 

    user executables:			/usr/local/bin
system admin executables:		/usr/local/sbin
program executables:			/usr/local/libexec
libraries:				/usr/local/lib
architecture dependent data:		/usr/local/lib/<name>
installed include files:		/usr/local/include or
					/usr/local/include/<name>
single-machine data:			/etc or /etc/<name>
local state:				/var/run
games score files:			/var/games
GNU info files:				/usr/local/info
man pages:				/usr/local/man/...
read-only architecture-independent:	/usr/local/share/<name>
misc documentation:			/usr/local/share/doc/<name>
examples files:				/usr/local/share/examples/<name>
  15. Begin a cycle of makes until the port is ready. Patch (see above) clean, and make until the port is generated. Get rid of all warnings if possible, especially security-related warnings.

  16. Control SEPARATE_BUILD semantics. You have to do this only if the port builds with SEPARATE_BUILD defined. Ideally, the port should not modify any file in ${WRKSRC} after make patch. You can check this by making sure you don't have any write access to ${WRKSRC}. Then you can set SEPARATE_BUILD=concurrent. Someone can use the same source tree to build on distinct arches simultaneously. Otherwise, set SEPARATE_BUILD=simple. Building on distinct arches simultaneously may be met with problems, as some source files may be regenerated at awkward moments.

  17. Add COMMENT in Makefile. COMMENT is a SHORT one-line description of the port (max. 60 characters). Do NOT include the package name (or version number of the software) in the comment. Do NOT start with an uppercase letter unless semantically significant, and do NOT end with a period. DON'T EVER START WITH AN INDEFINITE ARTICLE SUCH AS "a" or "an" - remove the article altogether.

  18. Put a longer description of the port into pkg/DESCR. One to a few paragraphs concisely explaining what the port does is sufficient. Lines should be no longer than 80 characters. This can be done by first editing the DESCR file and then running it through fmt -w 80.

  19. If the application needs to create a user or a group, choose the lowest free id from /usr/ports/infrastructure/db/user.list for your port to use and make sure your port gets added to this file at commit time.

  20. Install the application with make fake. Libraries should never be stripped. Executables are stripped by default; this is governed by ${INSTALL_STRIP}. ${INSTALL_PROGRAM} honors this automatically and is preferable to unconditional stripping (e.g., by an install-strip target or by running strip from the Makefile). You can use objdump --syms to determine if a binary is stripped or not. Stripped files have no symbols in the SYMBOL TABLE.

  21. Check port for security holes again. This is especially important for network-facing programs. See our security recommendations for that. Log interesting stuff and fixes in the pkg/SECURITY file. This file should list audited potential problems, along with relevant patches, so that another person can see at first glance what has been done. Example: 
    $OpenBSD$

${WRKDIR}/receiver.c
   call to mktemp (wrapper function do_mktemp) does seem to be correct.

The server makes extensive use of strlcpy/strlcat/snprintf.


  22. Make sure your /etc/mtree directory is up-to-date. (The next step uses the mtree lists to remove existing directories from generated packing-lists). Remember that the OpenBSD (U)pdate does not touch /etc... For automatic updating of /etc, sysmerge(8) may help.

  23. Create pkg/PLIST. After the installation with make fake is complete, use the developer's command make plist, which creates or updates the file PLIST in the pkg directory. This file is a candidate packing list.

    Peruse PLIST and verify that everything was installed and that it was installed in the proper locations. Anything not installed can be added to a port Makefile post-install rule. Note that PLIST annotations are documented in the pkg_create(1) manual.

  24. It is possible some directories do not need to be in the PLIST as they've been installed by a dependency; if you added to LIB_DEPENDS or RUN_DEPENDS, re-run make plist to remove these.

  25. Test the packaging with make package, test installation of the resulting package with make install, and test its removal with make uninstall. When dealing with multi-packages, it may instead be convenient to use pkg_add(1) and pkg_delete(1) directly, setting PKG_PATH to /usr/ports/packages/`arch -s`/all/ in the environment.

  26. Verify dependencies. Peruse your logs to verify the port did detect what is mentioned in DEPENDS, and nothing more. Check names, particularly in the make configure stage, for hidden dependencies (stuff that exists elsewhere in the ports tree and might be detected if the user installs some other ports first).

  27. Verify shared library dependencies. Run make port-lib-depends-check and add every LIB_DEPENDS or WANTLIB annotation that is needed until it runs cleanly. You may want to read the update guidelines to understand why this is so important.

  28. Check for regression tests, and whether they run cleanly. Set NO_TEST=Yes if a port has no test infrastructure. If dependencies are required to run the tests, but not to build the port, list them in TEST_DEPENDS. Please note: do not set NO_TEST if a port has an empty regression test infrastructure.

  29. Test that make clean succeeds. Sometimes the make fake stage creates files in the build directory which will cause this to fail.

  30. Run the /usr/ports/infrastructure/bin/portcheck utility in your port directory and take care of problems it finds, if any.

  31. Mail ports@openbsd.org with a description, the homepage (if any), and a short note asking for comments and testing. Make sure to attach the port/patch to this email, or mention an URL where it can be found, and send it out (mailinglist archives just contain the mails itself).

    Try to get others to test it on a variety of platforms for you.


  32. Incorporate any feedback you get. Test it again on your platform. Get those who gave you feedback to test it again from your new port.

  33. Finally, include it in the "ports" tree. If you do not have CVS access, ask someone on ports@openbsd.org to commit it.

  34. If you are a developer with CVS access, check it in. (Note that addition of new ports requires an explicit OK). Before commit, if using @newuser or @newgroup in the PLIST files, check that no users were added to /usr/ports/infrastructure/db/user.list since the port was created. We normally use cvs import for a new port, rather than adding a zillion (or a dozen) files individually. Import uses "vendor branch" version numbers like 1.1.1.1, but don't worry about that! If you make changes to a specific file (edit, then cvs commit), it will be 1.2, and that will be used.

    In short, import is typically used when a port is created. From that point on, cvs add and cvs rm are typically used to add or remove files, and the normal edit->commit cycle for changes. You might use something like this: 

    $ cd /usr/ports/mystuff/lang/kaffe1
$ make clean	# you really don't want to check in all of work!
$ cvs -d cvs.openbsd.org:/cvs -n import ports/lang/kaffe1 \
	YourName YourName_YYYY-MMM-DD

    As a real example, here is the output of checking in the kaffe1 port, which one of us did on September 8, 1998: 
    $ cd kaffe1
$ make clean >/dev/null
$ cvs import -m 'kaffe1.0(==JDK1.1) port' ports/lang/kaffe1 ian ian_1998-Sep-08
N ports/lang/kaffe1/Makefile
cvs server: Importing /cvs/ports/lang/kaffe1/files
N ports/lang/kaffe1/files/md5
cvs server: Importing /cvs/ports/lang/kaffe1/pkg
N ports/lang/kaffe1/pkg/COMMENT
N ports/lang/kaffe1/pkg/DESCR
N ports/lang/kaffe1/pkg/PLIST

No conflicts created by this import
    In this case, -m was used to specify the commit log message directly on the command line; normally when you import a port, you would do this in the interactive editor rather than specifying it on the command line - include a copy of pkg/DESCR and any other (short) relevant information, such as who has given an OK.

  35. Last but not least, add a one-line entry for the new port in its parent directory's Makefile, e.g., for ports/lang/kaffe1, add it to ports/lang/Makefile. Don't forget to commit any changes made to /usr/ports/infrastructure/db/user.list.

  36. Maintain the port! As time goes by, problems may arise, or new versions of the software may be released. You should strive to keep your port up to date. In other words - iterate, test, test, iterate...

  37. When updating a port, remember to handle dependencies! You shouldn't break any port that depends on yours. In case of problems, communicate with the maintainers of such ports. Likewise, be alert for dependency updates, and check that the maintainer did their job.

Handling Complex Situations

Assume you've managed to build the software, provide required patches, and you want to finish the port.

Know the software

Identify options
The first step is usually to identify build options. You will often have to read the configuration log, see what stuff your port auto-detects. Read the configure script options. Read the port documentation for extra stuff.
Make options work
Recompile your port with various options. Install extra dependencies. Make sure your port detects them correctly. Add supplementary patches to ensure compilation. Test the result, and verify extra stuff does work.
Identify missing software
Some dependencies won't be fulfilled because the missing software has not yet been ported. It is highly recommended to explicitly disable those options. Failure to do that breaks bulk builds all the time: people port new software and import it, and soon after, old ports stop building because they detect the dependency, try to use it, and fail to build or package.
Check run-time dependencies versus build-dependencies
Update your packing-list with make plist. Use make port-lib-depends-check to see what libraries your software needs (that will end up in LIB_DEPENDS or WANTLIB, usually). Identify various files and binaries in the dependencies that have to be present for the port to work.

By this point, you should have a fair understanding of your port's working.

Figure out important options

You won't care about some options. It makes no sense to disable some stuff if it always works, and if the dependencies are quite small. Take special notes of licences on dependencies, especially the PERMIT* stuff. As a rule, even if a dependency is very small, if it affects the licensing of the resulting package, you will have to explicitly take care of it.

Considering all possible options, you should be left with a much smaller set of options for your port, mostly depending on what packages are needed to run the software. For now, do not worry about build dependencies. Remember that the OpenBSD ports system is focused on the end user, and the end user will want to install binary packages, so it doesn't matter if you need a huge piece of software to build your port if it doesn't show up as a library or runtime dependency.

The ideal case: MULTI_PACKAGES and PSEUDO_FLAVORS

By now, you should have a fairly good idea of: In the ideal case, build options will simply create new files, with new dependencies, and not affect other stuff. This is a fairly common scenario for plugin frameworks: you add one library, you end up with a new plugin. This also happens fairly often for core applications with a graphics front-end: the console application is built every time, and the x11 interface shows up as a separate binary.

In this case, try setting the MULTI_PACKAGES variable to a list of -sub packages, add corresponding COMMENTS, and look at your packaging. Basically, MULTI_PACKAGES only affects the packaging: if you have MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist intwo variants: COMMENT-s1 for the first package, COMMENT-s2 for the second package, PLIST-s1, PLIST-s2, DESCR-s1, DESCR-s2. You need to write those COMMENT-s1 and COMMENT-s2 in the Makefile, and to split your PLIST into two parts, and to create DESCR-s1/DESCR-s2. You will also need to specify separate PKGNAMEs for all subpackages.

It is a good idea to start with the minimal framework work required: just copy the existing description and comments, because you will have to fiddle with MULTI_PACKAGES and stuff before you polish this.

Once you've separated the files properly, you will need to check dependencies: LIB_DEPENDS, WANTLIB, and RUN_DEPENDS will be split for each subpackage. It is usually time to check that your multi-packaging "works," and that those nasty dependencies you don't want to force on the user are indeed relegated to a specific subpackage.

Assuming everything works, you're mostly done. Just pick reasonable names for the various packages, and fill in the comments and descriptions. The end-user will be able to just install the package(s) they want.

But wait. What about the build, you say? Well, having a lot of dependencies during build is not a problem. Most packages are built by the OpenBSD team using special build runs (known as bulk builds) where a developer just builds all possible packages on a dedicated machine (or several, for slow architectures). Since everything will get built, having big dependencies is not an issue. Building the same thing several times, is an issue, though, which is why MULTI_PACKAGES are the best way to handle options (when possible): one build, one set of packages to test, better quality overall...

If you also want to help people who build packages themselves, you may consider adding PSEUDO_FLAVORS. A pseudo-flavor is a way to tweak an option (say, disable the graphical interface) that's very similar to actual flavors. In fact, the biggest difference is a functional difference: a pseudo flavor should only affect the set of packages that get built, but it is never allowed to modify the actual package contents.

For instance, assuming you separated the graphical interface into a separate subpackage (MULTI_PACKAGES=-core -x11), you could create a pseudo flavor no_x11 that avoids building the -x11 subpackage. The crucial point is that this flavor should NOT affect the -core package in any way.

You would end up with a Makefile that looks something like this: 

CATEGORIES = app
COMMENT-core = foo core application
COMMENT-x11 = foo graphical interface
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-core-$V
PKGNAME-x11 = foo-x11-$V
PSEUDO_FLAVORS = no_x11
FLAVOR ?=
CONFIGURE_STYLE = gnu

MULTI_PACKAGES = -core
WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt
RUN_DEPENDS-x11 = ::${BASE_PKGPATH},-core

.if ${FLAVOR:L:Mno_x11}
CONFIGURE_ARGS += --disable-x11
.else
MULTI_PACKAGES += -x11
.endif

.include <bsd.port.mk>
Notice that you only have to write a very small conditional section in the Makefile: the system doesn't care at all that you define extra variables.

Interdependencies between subpackages

MULTI_PACKAGES setups used to be asymmetric, with a -main subpackage and other subpackages, with the -main subpackage always built, and other subpackages possibly depending upon it. The current situation is totally symmetric: any subpackage can depend on any other. The infrastructure has specific provisions to avoid looping indefinitely.

The infrastructure takes special care of library inter-dependencies: it can detect which WANTLIB come from external dependencies, and which come from inter-dependencies. While external LIB_DEPENDS and WANTLIB are checked at the start of build, LIB_DEPENDS and WANTLIB that refer to one of the subpackages currently being built, will only be checked at packaging time (and thus packages may be created in a specific order to satisfy interdependencies).

The infrastructure provides specific variables to help in writing inter-dependencies: BUILD_PKGPATH contains the PKGPATH used during building the current packages, taking flavors and pseudo-flavors into account. It is highly recommended to use this variable instead of rolling your own: failure to do so will often trigger rebuilds in interesting flavors situations. For instance: 

...
FLAVORS = a b
FLAVOR ?=
MULTI_PACKAGES = -p1 -p2
WANTLIB-p1 = foo
LIB_DEPENDS-p1 = some/pkgpath,-p2
...
If you go on and build in some/pkgpath with FLAVOR=a, then creating the subpackage for -p1 will trigger a rebuild with FLAVOR=''. You would instead write: 
LIB_DEPENDS-p1 = ${BUILD_PKGPATH},-p2
There is also a BASE_PKGPATH variable, which does not take pseudo-flavors into account. This variable has limited applicability: it corresponds to a transition between old MULTI_PACKAGES and newer ones, where the old main subpackage did not have any marker in its pkgpath, and thus the corresponding package needs a @pkgpath ${BASE_PKGPATH} in its packing-list. (In general, pseudo-flavors are build information, and should not make their way into packages and packing-lists).

True FLAVORS and PKGNAMES

There are some cases where configuration options are too invasive, and you will have to add true flavors to the Makefile: those flavors will command some configuration options, and usually additions to various dependencies. Note that package naming is mostly automatic: the PKGNAME will have an extension built by appending the specified flavors to its name. So, if 
PKGNAME = foo-1.0
FLAVORS = f1 f2 f3
and you build the port with FLAVOR='f3 f1', then FULLPKGNAME=foo-1.0-f1-f3 (FLAVORS is used to reorder specified flavors in a canonical way).

There are sometimes mixed situations, where some packages do depend on the FLAVOR, and some don't. For instance, some ports include a large set of documentation that does not depend on the FLAVOR, and some actual programs that depend on the FLAVOR. In such cases, you can specify the FULLPKGNAME for the documentation subpackage explicitly. Something like this: 

CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto

MULTI_PACKAGES = -core -doc
WANTLIB-core = c m

.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif
As mentioned in the documentation, all package names have the same structure: stem-version-flavor_extension.

By default, packages with the same stem do conflict, and update paths will look at candidates with the same stem. The right package will be the one coming from the exact same PKGPATH, or matching @pkgpath annotation in the packing-list.

Usually, MULTI_PACKAGES should not conflict, so they must have different names (and the infrastructure has no way to build those names). On the other hand, flavors should conflict, and thus have the same name. The flavor information should end at the end of the package name, except for pseudo-flavors, which do not change the way a package is built.

As far as dependencies go, by default, specifying a PKGPATH will just create a stem-* dependency, meaning any package with the right stem will match the dependency. By default, any flavor will match. If only specific flavors are desired, you must include them in your specification, e.g., stem-*-flavor. If some flavors are unwanted, you can remove them from matching packages, e.g., stem-*-!flavor.

Since OpenBSD 4.9, asking for at least some version of a dependency can be directly tacked to the PKGPATH, e.g., dir/foo>=2.0.

Updating ports

The package tools can do updates, so maintainers have to be aware of one simple fact: update is not instantaneous. Even if a user just goes from release to release, each time they run pkg_add -u, the system will replace each package with a new version, one package at a time. So the user will run a mixed system, even if it is for just a few minutes.

There are basically two update models of which maintainers must be aware.

You should note that part of the update process, especially the macroscopic changes for users who update every six months, is not yet automated. The global update mechanism is still a work in progress, and pkg_add will be able to cope with more issues in the future. As of now, you should focus on making sure the system updates correctly, one port at a time, and that one port's update takes other ports into account, as far as conflicts and other issues are concerned.

Update checklist

Part of the work is to be done when making the port itself. Ports often need minor updates without a new version upstream. Part of the work will happen before the update itself. And then the update.

OpenBSD Porting Policy

Security Recommendations

There are many security problems to worry about. If you are not absolutely sure of what you are doing please request help from the ports mailing list.

Generic Porting Hints

Other Helpful Hints

Additional Information