Overview
So you've just compiled your favorite software package on your OpenBSD machine
and you want to share your effort by turning it into a standard port.
What to do?
The most important thing to do is to communicate.
Ask people on ports@openbsd.org if they
are working on the same port.
Tell the original software authors about it, including problems you may
find.
If licensing information appears incorrect, tell them.
If you had to jump through hoops to make the port build, tell them what can be
fixed.
If they are only developing on Linux and feel like ignoring the rest of the
Unix world, try to make them change their view.
Communication makes the difference between a successful port and a port that
will slowly be abandoned by everyone.
First, look at the porting information on this page.
Test, then re-test, and finally test again!
OpenBSD fully supports updates.
This means that some issues must be taken into
account.
Submit the port.
Create a gzipped tarball of the port directory.
You can then either place it on a public HTTP server, sending its URL to
ports@openbsd.org, or send the port
MIME encoded to the same address.
Porting some new software takes time.
Maintaining it over time is harder.
It is quite okay to port software and let other people handle it afterwards.
It is also okay to help other people update and maintain other ports, as long
as you communicate to avoid doing the same things twice.
In OpenBSD culture, MAINTAINERship is not a status item, but a
responsibility.
We have CVS and comments to give credit to the person who did the work.
A port MAINTAINER is something else: a person who assumes
responsibility for the working of the port, and is willing to spend some
time ensuring it works as best as can be.
Porting Checklist
The list below is a useful reminder of things to do.
This is neither totally accurate nor perfect.
Direct comments and questions to
ports@openbsd.org.
-
If you want to be a maintainer, subscribe to
ports@openbsd.org.
-
This is where all ports discussions take place.
-
Reading this list is important since many announcements are made there.
-
You will find a lot of porting-savvy people there.
They can often give you good advice or test ports for you.
-
Being a maintainer means more than just submitting ports.
It also means trying to keep them up-to-date, and answering questions
about them.
-
Check out a copy of the ports tree from CVS.
You can find instructions on how to do this at the
AnonCVS page.
As a porter, you should keep your base OS, ports tree, and installed
packages up to date.
-
From the names of the first level subdirectories of /usr/ports/,
pick a primary category for your port.
Create a new directory below /usr/ports/<category>/
or /usr/ports/mystuff/<category>/ and create the basic
infrastructure there.
Copy the template Makefile from
/usr/ports/infrastructure/templates/Makefile.template.
Fill in CATEGORIES with the primary category you have chosen.
-
Add the fetch portions of the Makefile.
-
Fill in EXTRACT_SUFX if it's anything besides .tar.gz.
Other examples are .tar.Z, or .tgz.
-
Fill in DISTNAME with the name of the file minus the extract suffix.
If you have foo-1.0.tar.gz, DISTNAME is foo-1.0.
-
Fill in MASTER_SITES with a list of URLs to the locations where
the distfile is kept, http://ftp.openbsd.org/pub/OpenBSD/distfiles/ for
example.
Don't forget the trailing slash.
Try to have at least three distinct sites as well.
Place the most easily accessible first as they are traversed in order.
-
Keep in mind that fetch references the file as
${MASTER_SITES}${DISTNAME}${EXTRACT_SUFX}.
All three are used.
Don't set DISTNAME to point to the file directly.
-
You can check to see if you have filled these values in correctly by typing
make fetch-all.
For more complex ports, you have more options and tools available to you:
-
You also have the variable PATCHFILES available.
This is a list of vendor (not OpenBSD) patches to the port.
Common uses are things like security or reliability fixes.
-
If your ports are available over large public mirrors such as GNU,
Sourceforge, or CPAN, we have already provided a list of sites for
your use in
/usr/ports/infrastructure/templates/network.conf.template.
Set MASTER_SITES to ${MASTER_SITE_GNU}, or
${MASTER_SITE_SOURCEFORGE}, etc.
To simplify this process, use the construct ${MASTER_SITE_FOO:=subdir/}
to append the distribution subdirectory.
-
For ports with source distributed from GitHub there are several possibilities.
-
In some cases, for example icinga or darktable, a distribution tar file has
been prepared and is available in a /releases/ directory.
Where available, use these in preference to other options as they usually
include the proper build infrastructure (configure scripts, etc) which are
often missing from "tagged" releases.
Specify as normal, using something like
MASTER_SITES=https://github.com/acct/project/releases/download/relname/
- In other cases, files have been tagged but they rely on github's
on-the-fly archive creation.
These can be specified using the following
bsd.port.mk(5) variables:
GH_ACCOUNT, GH_PROJECT, GH_TAGNAME.
The provided filename is rarely suitable for direct use as a
DISTNAME, frequently only including the version number;
these can be renamed at download time using the "{}" notation in
DISTFILES.
- Occasionally there is no alternative but to have a port reference only a
commit id.
In this case, GH_COMMIT should be used instead of GH_TAGNAME.
-
Ports normally correspond to given versions of software.
Once they are retrieved, files are checksummed and compared to the recorded
checksum(s) in distinfo.
So, to avoid confusion, DISTFILES and PATCHFILES should have
clearly visible version numbers:
don't retrieve foo-latest.tar.gz if it is a link to
foo-1.0.5.tar.gz.
If something is only available in an unversioned file, gently ask the
original program author to make such distinctions clear.
In the meantime, if you must use a file like this, set DIST_SUBDIR to
what a reasonable DISTNAME would be, like foo-1.0.5, such that
different, identically named, versions of the distfile do not clash on the
distribution file mirrors.
-
If a given port needs more than about five DISTFILES and
PATCHFILES to work, use DIST_SUBDIR to avoid cluttering
DISTDIR (/usr/ports/distfiles by default) too much.
In this case, DIST_SUBDIR must not include version numbers.
When the port is updated to a later version, some distfiles may not
change, but will be refetched if DIST_SUBDIR is changed.
Even if all distfiles change, it is easier for the user to track cruft.
-
All DISTFILES and PATCHFILES don't necessarily come from the
same set of MASTER_SITES.
Supplementary sites can be defined using the variables MASTER_SITES0 to
MASTER_SITES9. Just write DISTFILES=foo-1.0.5.tar.gz:5 to
retrieve foo-1.0.5.tar.gz from MASTER_SITES5.
-
Some ports don't always need to retrieve all files in all circumstances.
For instance, some ports may have some compilation options, and associated
files which are only required in such a case.
Or they may need some files for some architectures only.
In such a case, those supplementary optional files must be mentioned in the
SUPDISTFILES variable.
Targets such as makesum or mirror-distfiles will fetch those
supplementary files that the casual user doesn't need.
-
Create a checksum in distinfo by typing make makesum.
Then verify the checksum is correct by typing make checksum.
-
In some rare cases, files checksums can't be verified reliably.
By all means, porters should try to find sites that are reliable.
Communicating with the software author and the archive site maintainer at this
stage is highly desirable.
In the worst case, non-checksummable files can be mentioned in the
IGNOREFILES variable.
-
All files in DISTFILES are usually processed during
make extract.
EXTRACT_ONLY may be used to limit extraction to a subset of files
(possibly empty).
The customary use of this variable is to customize extraction: for instance,
if some DISTFILES need some special treatment, they will be removed
from EXTRACT_ONLY and handled manually at post-extract stage.
For historic reasons, make extract does set up the working directory
first along with extracting files.
Thus, providing a pre-extract or do-extract target is highly
unusual (and fairly suspicious behavior, indicative of a high degree of
obfuscation in the port).
-
Patches that need specific treatment should be mentioned in DISTFILES,
and removed from EXTRACT_ONLY, for historic reasons.
-
Extract the port with make extract.
Pay attention to where the base of the sources are.
Usually, it's /usr/ports/pobj/${PKGNAME}${FLAVOR_EXT}/${DISTNAME}.
You may need to modify the Makefile's WRKDIST variable if it
is different.
-
Read the installation documentation and note what you have to do to build
the port and any special options that might be needed.
-
Now is also a good time to figure out what kind of licensing restrictions
apply to your port.
Many are freely redistributable, but quite a few are not.
We need three questions answered to distribute ports properly.
These are the PERMIT_* values in the Makefile.
-
PERMIT_PACKAGE_CDROM tells us if we can put the package on the cdrom.
-
PERMIT_PACKAGE_FTP tells us if we can put the package on the ftp sites.
-
PERMIT_DISTFILES_FTP tells us if we can mirror the distfiles on the ftp
sites.
Set these values to Yes if it is permitted or to a comment string
stating why it is not.
Pay attention to any special conditions you may need to fulfill later on.
For example, some ports require installing a copy of the license.
We recommend you place the license in
/usr/local/share/doc/<name>/.
In addition to the PERMIT_* values, put a license marker like
# License above them as a comment, this way we know why the
PERMIT_* values are set the way they are.
-
Add configuration options to Makefile and/or create the configuration
script.
-
You can add a port configuration script named configure to a directory named
scripts/.
This will be run before any configuration specified by CONFIGURE_STYLE
is run.
-
If GNU configure is used, you may want to run ./configure --help
to see what options are available.
-
Anything that you may want to override can be changed by adding the
--option flags to the CONFIGURE_ARGS parameter in the
Makefile.
-
Use CONFIGURE_ARGS+= to append to the variable.
CONFIGURE_ARGS= will overwrite it.
-
Try building the port with make build.
-
If you're lucky, the port will go all the way through without errors.
-
If it exits with an error, you will need to generate patches for your port.
Figure out what needs to be changed and make a patch for it.
-
the sequence to make a patch for a file is usually:
-
cd `make show=WRKSRC` ; cp foo/bar.c{,.orig}
-
edit foo/bar.c to fix the error.
-
cd - ; make update-patches
-
this will create patches/patch-foo_bar_c with your modifications.
-
The easiest way to reset the port and test your patches is
make clean patch.
This will delete the work directory, re-extract, and patch your port.
-
Begin a cycle of make build, generate a patch using
make update-patches and make clean patch.
-
Patches go in the directory patches/ and should be named patch-* with
* being something meaningful.
The preferred naming is patch-FILENAME, where FILENAME is
the name of the file it is patching.
(make update-patches does this automatically for you.)
-
Applying PATCHFILES is the first half of the make patch
stage.
It can be invoked separately as make distpatch, which is a convenient
target for porters.
Ignore this if you haven't set it.
-
Only patch one source file per patch file, please.
-
Use make update-patches to generate patches.
-
All patches MUST be relative to ${WRKDIST}.
-
Check that patches DON'T contain tags that cvs will replace.
If they do, your patches won't apply after you check them in.
You can check in your changes with -kk to avoid this.
-
Write a small explanation at the beginning of the patch file about its purpose
(not mandatory, but for example often done for security patches).
-
Please feed your patches back to the author of that software.
-
Try setting SEPARATE_BUILD.
-
If the port can build with object files outside its source tree,
this is cleaner (many programs using CONFIGURE_STYLE=gnu can),
and may help people who mount their ports tree on several arches.
-
This can also spare you some effort, as you will possibly be able to
restart the cycle at configure most of the time.
-
Peruse the output (if any) and tweak any options in the Makefile.
To repeat issue the command make clean configure.
Note: make sure host-dependent files go in /etc or
/etc/<name>, but NEVER REPLACE OR MODIFY
existing files in /etc.
Best to have install place them in /usr/local/share/<name> and
then copy to /etc or /etc/<name> only if the files do
not exist.
If the files exist, display a message that says such-and-such files need to be
modified.
This also guarantees that the files will be included in the package since
everything under /usr/local is included in the PLIST.
To handle the copying carefully, the @sample keyword is preferably used
within the PLIST.
After a package has been installed the contents of pkg/MESSAGE will be
displayed if it exists.
The OpenBSD file locations are:
user executables: /usr/local/bin
system admin executables: /usr/local/sbin
program executables: /usr/local/libexec
libraries: /usr/local/lib
architecture dependent data: /usr/local/lib/<name>
installed include files: /usr/local/include or
/usr/local/include/<name>
single-machine data: /etc or /etc/<name>
local state: /var/run
games score files: /var/games
GNU info files: /usr/local/info
man pages: /usr/local/man/...
read-only architecture-independent: /usr/local/share/<name>
misc documentation: /usr/local/share/doc/<name>
examples files: /usr/local/share/examples/<name>
-
Begin a cycle of makes until the port is ready.
Patch (see above) clean, and make until the port is generated.
Get rid of all warnings if possible, especially security-related warnings.
-
Control SEPARATE_BUILD semantics.
You have to do this only if the port builds with SEPARATE_BUILD defined.
Ideally, the port should not modify any file in ${WRKSRC} after
make patch.
You can check this by making sure you don't have any write access to
${WRKSRC}.
Then you can set SEPARATE_BUILD=concurrent.
Someone can use the same source tree to build on distinct arches simultaneously.
Otherwise, set SEPARATE_BUILD=simple.
Building on distinct arches simultaneously may be met with problems, as some
source files may be regenerated at awkward moments.
-
Add COMMENT in Makefile.
COMMENT is a SHORT one-line description of the port (max. 60
characters).
Do NOT include the package name (or version number of the software) in
the comment.
Do NOT start with an uppercase letter unless semantically significant,
and do NOT end with a period.
DON'T EVER START WITH AN INDEFINITE ARTICLE SUCH AS "a" or "an" - remove
the article altogether.
-
Put a longer description of the port into pkg/DESCR.
One to a few paragraphs concisely explaining what the port does is sufficient.
Lines should be no longer than 80 characters.
This can be done by first editing the DESCR file and then running it
through fmt -w 80.
-
If the application needs to create a user or a group, choose the lowest free
id from /usr/ports/infrastructure/db/user.list for your port to
use and make sure your port gets added to this file at commit time.
-
Install the application with make fake.
Libraries should never be stripped.
Executables are stripped by default; this is governed by
${INSTALL_STRIP}.
${INSTALL_PROGRAM} honors this automatically and is preferable to
unconditional stripping (e.g., by an install-strip target or by
running strip from the Makefile).
You can use objdump --syms to determine if a binary is stripped or not.
Stripped files have no symbols in the SYMBOL TABLE.
-
Check port for security holes again.
This is especially important for network-facing programs.
See our security recommendations for that.
Log interesting stuff and fixes in the pkg/SECURITY file.
This file should list audited potential problems, along with relevant patches,
so that another person can see at first glance what has been done.
Example:
$OpenBSD$
${WRKDIR}/receiver.c
call to mktemp (wrapper function do_mktemp) does seem to be correct.
The server makes extensive use of strlcpy/strlcat/snprintf.
-
Make sure your /etc/mtree directory is up-to-date.
(The next step uses the mtree lists to remove existing directories from
generated packing-lists). Remember that the OpenBSD
(U)pdate does not touch /etc...
For automatic updating of /etc, sysmerge(8) may help.
-
Create pkg/PLIST.
After the installation with make fake is complete, use the
developer's command make plist, which creates or updates
the file PLIST in the pkg directory.
This file is a candidate packing list.
Peruse PLIST and verify that everything was installed and that it was
installed in the proper locations.
Anything not installed can be added to a port Makefile
post-install rule.
Note that PLIST annotations are documented in the
pkg_create(1) manual.
-
It is possible some directories do not need to be in the PLIST
as they've been installed by a dependency; if you added to LIB_DEPENDS
or RUN_DEPENDS, re-run make plist to remove these.
-
Test the packaging with make package, test installation of the
resulting package with make install, and test its removal with
make uninstall.
When dealing with multi-packages, it may instead be convenient to use
pkg_add(1) and
pkg_delete(1) directly,
setting PKG_PATH to /usr/ports/packages/`arch -s`/all/ in the
environment.
-
Verify dependencies.
Peruse your logs to verify the port did detect what is mentioned in
DEPENDS, and nothing more.
Check names, particularly in the make configure stage, for hidden
dependencies (stuff that exists elsewhere in the ports tree and might be
detected if the user installs some other ports first).
-
Verify shared library dependencies.
Run make port-lib-depends-check and add every LIB_DEPENDS
or WANTLIB annotation that is needed until it runs cleanly.
You may want to read the update guidelines to
understand why this is so important.
-
Check for regression tests, and whether they run cleanly.
Set NO_TEST=Yes if a port has no test infrastructure.
If dependencies are required to run the tests, but not to build the port,
list them in TEST_DEPENDS.
Please note: do not set NO_TEST if a port has an empty regression
test infrastructure.
-
Test that make clean succeeds.
Sometimes the make fake stage creates files in the build directory
which will cause this to fail.
-
Run the
/usr/ports/infrastructure/bin/portcheck
utility
in your port directory and take care of problems it finds, if any.
-
Mail ports@openbsd.org with a
description, the homepage (if any), and a short note asking for comments
and testing.
Make sure to attach the port/patch to this email, or mention an URL where
it can be found, and send it out (mailinglist archives just contain the
mails itself).
Try to get others to test it on a variety of platforms for you.
-
The AMD64 systems are good because they are fast, and because
sizeof(int) != sizeof(long) on this platform.
-
Sun SPARC and UltraSPARC are good because they are very common and because
their byte order is the opposite of i386; if you developed on SPARC, of course,
you'd want it tested on i386.
-
Incorporate any feedback you get. Test it again on your platform.
Get those who gave you feedback to test it again from your new port.
-
Finally, include it in the "ports" tree.
If you do not have CVS access, ask someone on
ports@openbsd.org to commit it.
-
If you are a developer with CVS access, check it in.
(Note that addition of new ports requires an explicit OK).
Before commit, if using @newuser or @newgroup in the
PLIST files, check that no users were added to
/usr/ports/infrastructure/db/user.list since the port was created.
We normally use cvs import for a new port, rather than adding a
zillion (or a dozen) files individually.
Import uses "vendor branch" version numbers like 1.1.1.1, but don't worry
about that!
If you make changes to a specific file (edit, then cvs commit),
it will be 1.2, and that will be used.
In short, import is typically used when a port is created.
From that point on, cvs add and cvs rm are typically used
to add or remove files, and the normal edit->commit cycle for changes.
You might use something like this:
$ cd /usr/ports/mystuff/lang/kaffe1
$ make clean # you really don't want to check in all of work!
$ cvs -d cvs.openbsd.org:/cvs -n import ports/lang/kaffe1 \
YourName YourName_YYYY-MMM-DD
-
-d cvs.openbsd.org:/cvs says where cvs lives.
-
-n says to do a 'dry run'; check the output of this, and if things
are OK re-run the command without -n.
-
ports/lang/kaffe1 is the path relative to /cvs where the port
lives.
Don't forget to start it with ports/.
-
YourName (replaced with your login name) is the 'vendor tag'.
You imported it so you are the vendor.
-
YourName_YYYY-MMM-DD (e.g., ian_2000-Jan-01) is the
vendor release tag.
This is as good as any.
As a real example, here is the output of checking in the kaffe1 port,
which one of us did on September 8, 1998:
$ cd kaffe1
$ make clean >/dev/null
$ cvs import -m 'kaffe1.0(==JDK1.1) port' ports/lang/kaffe1 ian ian_1998-Sep-08
N ports/lang/kaffe1/Makefile
cvs server: Importing /cvs/ports/lang/kaffe1/files
N ports/lang/kaffe1/files/md5
cvs server: Importing /cvs/ports/lang/kaffe1/pkg
N ports/lang/kaffe1/pkg/COMMENT
N ports/lang/kaffe1/pkg/DESCR
N ports/lang/kaffe1/pkg/PLIST
No conflicts created by this import
In this case, -m was used to specify the commit log message
directly on the command line; normally when you import a port, you would
do this in the interactive editor rather than specifying it on the command
line - include a copy of pkg/DESCR and any other (short) relevant
information, such as who has given an OK.
-
Last but not least, add a one-line entry for the new port in its parent
directory's Makefile, e.g., for ports/lang/kaffe1,
add it to ports/lang/Makefile.
Don't forget to commit any changes made to
/usr/ports/infrastructure/db/user.list.
-
Maintain the port!
As time goes by, problems may arise, or new versions of the software may be
released.
You should strive to keep your port up to date.
In other words - iterate, test, test, iterate...
-
When updating a port, remember to handle dependencies!
You shouldn't break any port that depends on yours.
In case of problems, communicate with the maintainers of such ports.
Likewise, be alert for dependency updates, and check that the maintainer
did their job.
Handling Complex Situations
Assume you've managed to build the software, provide required patches,
and you want to finish the port.
Know the software
-
Identify options
-
The first step is usually to identify build options.
You will often have to read the configuration log, see what stuff your port
auto-detects.
Read the configure script options.
Read the port documentation for extra stuff.
- Make options work
-
Recompile your port with various options.
Install extra dependencies.
Make sure your port detects them correctly.
Add supplementary patches to ensure compilation.
Test the result, and verify extra stuff does work.
- Identify missing software
- Some dependencies won't be fulfilled because the missing software
has not yet been ported.
It is highly recommended to explicitly disable those options.
Failure to do that breaks bulk builds all the time: people port new software
and import it, and soon after, old ports stop building because they detect
the dependency, try to use it, and fail to build or package.
- Check run-time dependencies versus build-dependencies
-
Update your packing-list with make plist.
Use make port-lib-depends-check to see what libraries your
software needs (that will end up in LIB_DEPENDS or WANTLIB,
usually).
Identify various files and binaries in the dependencies that have to be present
for the port to work.
By this point, you should have a fair understanding of your port's working.
You won't care about some options.
It makes no sense to disable some stuff if it always works, and if the
dependencies are quite small.
Take special notes of licences on dependencies, especially the PERMIT*
stuff.
As a rule, even if a dependency is very small, if it affects the licensing of
the resulting package, you will have to explicitly take care of it.
Considering all possible options, you should be left with a much smaller set
of options for your port, mostly depending on what packages are needed to run
the software.
For now, do not worry about build dependencies.
Remember that the OpenBSD ports system is focused on the end user, and the end
user will want to install binary packages, so it doesn't matter if you need a
huge piece of software to build your port if it doesn't show up as a library
or runtime dependency.
The ideal case: MULTI_PACKAGES and
PSEUDO_FLAVORS
By now, you should have a fairly good idea of:
- which new files show up when you activate each option
- which libraries/runtime files are needed when you activate each option
In the ideal case, build options will simply create new files, with new
dependencies, and not affect other stuff.
This is a fairly common scenario for plugin frameworks: you add one library,
you end up with a new plugin.
This also happens fairly often for core applications with a graphics front-end:
the console application is built every time, and the x11 interface shows up
as a separate binary.
In this case, try setting the MULTI_PACKAGES variable to a list of
-sub packages, add corresponding COMMENTS, and look at your packaging.
Basically, MULTI_PACKAGES only affects the packaging: if you have
MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist
intwo variants:
COMMENT-s1 for the first package,
COMMENT-s2 for the second package,
PLIST-s1, PLIST-s2, DESCR-s1, DESCR-s2.
You need to write those COMMENT-s1 and COMMENT-s2 in the
Makefile, and to split your PLIST into two parts, and to
create DESCR-s1/DESCR-s2.
You will also need to specify separate PKGNAMEs for all subpackages.
It is a good idea to start with the minimal framework work required:
just copy the existing description and comments, because you will have
to fiddle with MULTI_PACKAGES and stuff before you polish this.
Once you've separated the files properly, you will need to check dependencies:
LIB_DEPENDS, WANTLIB, and RUN_DEPENDS will be split
for each subpackage.
It is usually time to check that your multi-packaging "works," and that
those nasty dependencies you don't want to force on the user are indeed
relegated to a specific subpackage.
Assuming everything works, you're mostly done.
Just pick reasonable names for the various packages, and fill in the comments
and descriptions.
The end-user will be able to just install the package(s) they want.
But wait.
What about the build, you say?
Well, having a lot of dependencies during build is not a problem.
Most packages are built by the OpenBSD team using special build runs (known
as bulk builds) where a developer just builds all possible packages on a
dedicated machine (or several, for slow architectures).
Since everything will get built, having big dependencies is not an issue.
Building the same thing several times, is an issue, though, which is why
MULTI_PACKAGES are the best way to handle options (when possible):
one build, one set of packages to test, better quality overall...
If you also want to help people who build packages themselves, you may consider
adding PSEUDO_FLAVORS.
A pseudo-flavor is a way to tweak an option (say, disable the graphical
interface) that's very similar to actual flavors.
In fact, the biggest difference is a functional difference: a pseudo flavor
should only affect the set of packages that get built, but it is never
allowed to modify the actual package contents.
For instance, assuming you separated the graphical interface into a separate
subpackage (MULTI_PACKAGES=-core -x11), you could create a pseudo
flavor no_x11 that avoids building the -x11 subpackage.
The crucial point is that this flavor should NOT affect the -core package
in any way.
You would end up with a Makefile that looks something like this:
CATEGORIES = app
COMMENT-core = foo core application
COMMENT-x11 = foo graphical interface
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-core-$V
PKGNAME-x11 = foo-x11-$V
PSEUDO_FLAVORS = no_x11
FLAVOR ?=
CONFIGURE_STYLE = gnu
MULTI_PACKAGES = -core
WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt
RUN_DEPENDS-x11 = ::${BASE_PKGPATH},-core
.if ${FLAVOR:L:Mno_x11}
CONFIGURE_ARGS += --disable-x11
.else
MULTI_PACKAGES += -x11
.endif
.include <bsd.port.mk>
Notice that you only have to write a very small conditional section in the
Makefile: the system doesn't care at all that you define extra variables.
Interdependencies between subpackages
MULTI_PACKAGES setups used to be asymmetric, with a -main subpackage
and other subpackages, with the -main subpackage always built, and other
subpackages possibly depending upon it.
The current situation is totally symmetric: any subpackage can depend on any
other.
The infrastructure has specific provisions to avoid looping indefinitely.
The infrastructure takes special care of library inter-dependencies: it can
detect which WANTLIB come from external dependencies, and which come
from inter-dependencies.
While external LIB_DEPENDS and WANTLIB are checked at the
start of build, LIB_DEPENDS and WANTLIB that refer to one of
the subpackages currently being built, will only be checked at packaging time
(and thus packages may be created in a specific order to satisfy
interdependencies).
The infrastructure provides specific variables to help in writing
inter-dependencies: BUILD_PKGPATH contains the PKGPATH used
during building the current packages, taking flavors and pseudo-flavors into
account.
It is highly recommended to use this variable instead of rolling your own:
failure to do so will often trigger rebuilds in interesting flavors situations.
For instance:
...
FLAVORS = a b
FLAVOR ?=
MULTI_PACKAGES = -p1 -p2
WANTLIB-p1 = foo
LIB_DEPENDS-p1 = some/pkgpath,-p2
...
If you go on and build in some/pkgpath with FLAVOR=a, then creating the
subpackage for -p1 will trigger a rebuild with FLAVOR=''.
You would instead write:
LIB_DEPENDS-p1 = ${BUILD_PKGPATH},-p2
There is also a BASE_PKGPATH variable, which does not take
pseudo-flavors into account.
This variable has limited applicability: it corresponds to a transition between
old MULTI_PACKAGES and newer ones, where the old main
subpackage did not have any marker in its pkgpath, and thus the corresponding
package needs a @pkgpath ${BASE_PKGPATH} in its packing-list.
(In general, pseudo-flavors are build information, and should not make their
way into packages and packing-lists).
True FLAVORS and PKGNAMES
There are some cases where configuration options are too invasive, and you
will have to add true flavors to the Makefile: those flavors will
command some configuration options, and usually additions to various dependencies.
Note that package naming is mostly automatic: the PKGNAME will have
an extension built by appending the specified flavors to its name.
So, if
PKGNAME = foo-1.0
FLAVORS = f1 f2 f3
and you build the port with FLAVOR='f3 f1', then
FULLPKGNAME=foo-1.0-f1-f3 (FLAVORS is used to reorder
specified flavors in a canonical way).
There are sometimes mixed situations, where some packages do depend on the
FLAVOR, and some don't.
For instance, some ports include a large set of documentation that does not
depend on the FLAVOR, and some actual programs that depend on the
FLAVOR.
In such cases, you can specify the FULLPKGNAME for the documentation
subpackage explicitly.
Something like this:
CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto
MULTI_PACKAGES = -core -doc
WANTLIB-core = c m
.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif
As mentioned in the documentation, all package names have the same structure:
stem-version-flavor_extension.
By default, packages with the same stem do conflict, and update paths will
look at candidates with the same stem.
The right package will be the one coming from the exact same PKGPATH,
or matching @pkgpath annotation in the packing-list.
Usually, MULTI_PACKAGES should not conflict, so they must have
different names (and the infrastructure has no way to build those names).
On the other hand, flavors should conflict, and thus have the same name.
The flavor information should end at the end of the package name, except for
pseudo-flavors, which do not change the way a package is built.
As far as dependencies go, by default, specifying a PKGPATH will just
create a stem-* dependency, meaning any package with the right stem
will match the dependency.
By default, any flavor will match.
If only specific flavors are desired, you must include them in your
specification, e.g., stem-*-flavor.
If some flavors are unwanted, you can remove them from matching packages,
e.g., stem-*-!flavor.
Since OpenBSD 4.9, asking for at least some version of a dependency can be
directly tacked to the PKGPATH, e.g., dir/foo>=2.0.
Updating ports
The package tools can do updates, so maintainers have to be aware of one
simple fact: update is not instantaneous.
Even if a user just goes from release to release, each time they run
pkg_add -u, the system will replace each package with a new version,
one package at a time.
So the user will run a mixed system, even if it is for just a few minutes.
There are basically two update models of which maintainers must be
aware.
-
Some users follow current on a loose basis.
They update their packages every few days/every few weeks.
Either all packages, or only some selected packages.
The update mechanism should work for them: it can force them to update some
specific packages, or to install some new stuff, but it should not fail
silently.
Micro-updates must be tested.
These users will most often be able to cope with small changes, like program
name changes, or configuration adjustments.
-
Some users update every six months with a new release.
They also will download stable updates (security and critical updates).
During six months, a large part of the ports tree changes.
These users do not care about individual software changes.
They just want a system that works.
Changing binary names is completely unfriendly to these users.
Tweaking hundreds of configuration files will be a major pain for them.
Maintainers can help by providing smooth update paths, and major hints
whenever something important changes.
You should note that part of the update process, especially the macroscopic
changes for users who update every six months, is not yet automated.
The global update mechanism is still a work in progress, and pkg_add will be
able to cope with more issues in the future.
As of now, you should focus on making sure the system updates correctly, one
port at a time, and that one port's update takes other ports into account,
as far as conflicts and other issues are concerned.
Update checklist
Part of the work is to be done when making the port itself.
-
Make sure the software authors are aware of your tweaks to make it run on
OpenBSD.
You must endeavor to get OpenBSD patches into the next release of the software.
Otherwise, you can be prepared to rewrite most of your patches from scratch
every release.
-
Make sure the software authors understand version numbering.
If the distfiles do not contain any version numbers, or if they reference
current stuff, communicate with the authors so that you can get some
permanent URLs.
-
Document work-arounds.
This will help you remember to check whether they are still necessary when
you try to update a port.
-
Document dependencies, especially the stuff you don't use.
Some ports can use external software that may not be available at the time
of porting.
Make sure you do not pick it up, and document it, so that you can update
your port when this software becomes available later.
-
If the port uses libtool, copy its log from the file
${WRKBUILD}/shared_libs.log verbatim as a basis for your
SHARED_LIBS setup.
This will help during updates.
-
Use PLIST_DB and build a database of packing-lists.
This is useful to find out about forgotten package name bumps, and also
to check for conflicts with older packages.
-
Make sure dependencies stay as loose as they can be.
By default, RUN_ and LIB_DEPENDS will be satisfied by any
version of a package.
Do not insist on a given version unless you have to. Use minimal versions
whenever possible.
Ports often need minor updates without a new version upstream.
-
Each port update needs a package name bump.
Otherwise, the update mechanism for binary packages won't work.
Anything that affects the binary package implies a bump.
This includes HOMEPAGE, MAINTAINER or description changes.
-
Version numbers always go forward.
If something unexpected happens,and you have to go back, or if the numbering
changes completely so that the version number appears to go backwards, you
must use EPOCH to make sure pkg_add(1) sees the package as a newer
package.
-
If the package does not build, no bump is needed: changes to restore a port
to working order after it got broken do not warrant bumps.
-
Changes to make sure a port does not pick/does pick an external dependency
warrant a bump.
-
Changes in dependencies generally do not affect a port version number.
The package system uses a signature mechanism such that a binary package
is fully identified by its package names, plus the dependencies against
which it was built, plus the version numbers of all shared libraries it
contains.
Part of the work will happen before the update itself.
-
run make patch to have an initial copy of the port before the update.
And then the update.
-
Edit the port's Makefile to grab the new version, run
make makesum and make patch as a starting basis.
-
Once you've fixed patches that failed, run a full diff between the old and
the new version to figure out exactly what changed.
Take notes, be prepared to revisit things later.
-
Do whatever porting work is necessary to get the new version to run under
OpenBSD: adjust dependencies, change package contents, etc.
-
Double check shared library versions.
For libtool-based ports, you have the shared_libs.log to check whether
the software authors did significant changes.
Note well that this is not enough.
Most software authors do not really understand shared library issues.
You have to read the full diff between the old and the new version, and bump
library versions accordingly.
When in doubt, bump the major version.
-
Be aware of conflicts with already-built packages.
For simple updates, nothing needs to be done.
If you split a package into several subpackages, make sure the subpackages
have proper conflict annotations: they should usually conflict with the "old"
single package.
-
Help the user.
If some specific steps must be taken beyond pkg_add -u, make sure
they are visible in the package.
When your packaging policy changes, document it in the package description.
For instance, if you move the documentation into a separate package for
space constraints, the main package description should mention that the
documentation is now available as a separate package.
OpenBSD Porting Policy
-
OpenBSD does NOT use /usr/local/etc/rc.d.
/usr/local is often shared between several machines, thanks to NFS.
For this reason, configuration files that are specific to a given machine
can't be stored under /usr/local.
/etc is the central repository for per machine configuration files.
Moreover, OpenBSD policy is to never update files under /etc
automatically.
Ports that need some specific boot setup should advise the administrator
about what to do instead of blindly installing files.
-
OpenBSD does NOT compress man pages.
-
OpenBSD does NOT require -lcrypt, -ldl, or -lrt.
The functions provided by these libraries are part of libc.
The crypt() function does not support DES, only bcrypt.
-
OpenBSD has a separate namespace for users and groups created by ports.
See /usr/ports/infrastructure/db/user.list for details.
-
OpenBSD is strongly security-oriented.
You should read and understand this page's
security section.
-
Be sure to add the $OpenBSD$ CVS tag to the Makefile.
The account name, version number, etc., will be filled in automatically
by CVS during commit.
-
The goal is to get all ported applications to support OpenBSD.
To achieve this goal you must feed any OpenBSD patches back to the
application maintainer.
Security Recommendations
There are many security problems to worry about.
If you are not absolutely sure of what you are doing please request help from
the ports mailing list.
-
Do not use alpha or beta code when preparing a port.
Use the latest regular or patch release.
-
Any software to be installed as a server should be scanned for buffer
overflows, especially unsafe use of strcat/strcpy/strcmp/sprintf.
In general, sprintf should be replaced with snprintf.
-
Never use filenames instead of true security.
There are numerous race conditions where you don't have proper control.
For instance, an attacker who already has user privileges on your machines
may replace files in /tmp with symbolic links to more strategic
files, such as /etc/master.passwd.
-
For instance, both fopen and freopen create a new file
or open an existing file for writing.
An attacker may create a symbolic link from /etc/master.passwd to
/tmp/addrpool_dump.
The instant you open it, your password file is hosed.
Yes, even with an unlink right before.
You only narrow the window of opportunity.
Use open with O_CREAT|O_EXCL and fdopen instead.
-
Another very common problem is the mktemp function.
Heed the warnings of the bsd linker about its uses.
These must be fixed.
This is not quite as simple as s/mktemp/mkstemp/g.
Refer to mktemp(3) for more
information.
Correct code using mkstemp includes the source to ed or
mail.
A rare instance of code that uses mktemp correctly can be found in
the rsync port.
-
Just because you can read it doesn't mean you should.
A well-known hole of this nature was the startx problem.
As a setuid program, you could launch startx with any file as a script.
If the file was not a valid shell script, a syntax error message would
follow, along with the first line of the offending file, without any
further permission check.
Pretty handy to grab the first line of a shadow passwd file, considering
these often start with root entry.
Do not open your file, and then do an fstat on the open descriptor
to check if you should have been able to open it (or the attacker will play
with /dev/rst0 and rewind your tape) -- open it with the correct
uid/gid/grouplist set.
-
Don't use anything that forks a shell in setuid programs before dropping
your privileges.
This includes popen and system.
Use fork, pipe and execve instead.
-
Pass open descriptors instead of filenames to other programs to avoid race
conditions.
Even if a program does not accept file descriptors, you can still use
/dev/fd/0.
-
Access rights are attached to file descriptors.
If you need setuid rights to open a file, open that file, then drop your
privileges.
You can still access the open descriptor, but you have less to worry about.
This is double-edged: even after dropping privileges, you should still watch
out for those descriptors.
-
Avoid root setuid as much as you can.
Basically, root can do anything, but root rights are very rarely needed,
except maybe to create socket ports with a number under 1024.
It is arguably better to keep that under inetd control and just add
the relevant entries to inetd.conf.
You must know the appropriate magic for writing daemons to achieve that.
It could be argued that you have no business writing setuid programs if you
don't know how to do that.
-
Use setgid instead of setuid.
Apart from those specific files needed by setgid programs, most files are not
group-writable.
Hence, a security problem in a setgid program won't compromise your system as
much: with only group rights, the worst an intruder will be able to do is hack
a games score table or some such.
See the xkobo port for an instance of such a change.
-
Don't trust group-writable files.
Even though they are audited, setgid programs are not perceived as important
potential security holes.
Hence stuff they can tamper with shouldn't be considered sensitive information.
-
Don't trust your environment!
This involves simple things such as your PATH (never use
system with an unqualified name, avoid execvp), but also
more subtle items such as your locale, timezone, termcap, and so on.
Be aware of transitivity: even though you're taking full precautions,
programs you call directly won't necessarily.
Never use system in privileged programs, build your command
line, a controlled environment, and call execve directly.
The perlsec man page is a good tutorial on such problems.
-
Never use setuid shell-scripts.
These are inherently insecure.
Wrap them into some C code that ensures a proper environment.
On the other hand, OpenBSD features secure perl scripts.
-
Beware the dynamic loader.
If you are running setuid, it will only use trusted libraries that were
scanned with ldconfig.
Setgid is not enough.
-
Dynamic libraries are tricky.
C++ code sets a similar problem.
Basically, libraries may take some action based upon your environment before
your main program even gets to check its setuid status.
OpenBSD issetugid addresses this problem, from the library writer
point of view.
Don't try to port libraries unless you understand this issue thoroughly.
-
On OpenBSD, rand(3),
random(3), and the
drand48(3) family return
non-deterministic random numbers by default.
Any specified seed value is ignored by the associated seed function, and
arc4random is used instead.
If the determinstic (i.e: repeatable) behavior must be preserved, use the
OpenBSD extensions:
srand_deterministic, srandom_deterministic,
srand48_deterministic, seed48_deterministic
and lcong48_deterministic.
Generic Porting Hints
-
__OpenBSD__ should be used sparingly, if at all.
Constructs that look like
#if defined(__NetBSD__) || defined(__FreeBSD__)
are often inappropriate.
Don't add blindly __OpenBSD__ to it.
Instead, try to figure out what's going on, and what actual feature is needed.
Manual pages are often useful, as they include historic comments, stating when
a particular feature was incorporated into BSD.
Checking the numeric value of BSD against known releases is often the
right way.
See the
NetBSD pkgsrc guide
for more information.
-
Defining BSD is a bad idea.
Try to include sys/param.h.
This not only defines BSD, it also gives it a proper value.
The right code fragment should look like:
#if (defined(__unix__) || defined(unix)) && !defined(USG)
#include <sys/param.h>
#endif
-
Test for features, not for specific OSes.
In the long run, it is much better to test whether tcgetattr works
than whether you're running under BSD 4.3 or later, or SystemVR4.
These kind of tests just confuse the issue.
The way to go about it is, for instance, to test for one particular system,
set up a slew of HAVE_TCGETATTR defines, then proceed to the next
system.
This technique separates features tests from specific OSes.
In a hurry, another porter can just add the whole set of -DHAVE_XXX
defines to the Makefile.
One may also write or add to a configure script to check for that feature and
add it automatically.
As an example not to follow, check nethack 3.2.2 source: it assumes loads of
things based on the system type.
Most of these assumptions are obsolete and no longer reflect reality: POSIX
functions are more useful than older BSD versus SystemV differences, to the
point that some traditional bsd functions are now only supported through
compatibility libraries.
-
Avoid include files that include other includes that...
First, because this is inefficient.
Your project will end up including a file that includes everything.
Also, because it makes some problems difficult to fix.
It becomes harder to not include one particular file at a given point.
-
Avoid bizarre macro tricks.
Undefining a macro that was defined by a header file is a bad idea.
Defining macros to get some specific behavior from an include file is also
a bad idea: compilation modes should be global.
If you want POSIX behavior, say so, and #define POSIX_C_SOURCE
throughout the whole project, not when you feel like it.
-
Don't ever write system function prototypes.
All modern systems, OpenBSD included, have a standard location for these
prototypes.
Likely places include unistd.h, fcntl.h or termios.h.
The man page frequently states where the prototype can be found.
You might need another slew of HAVE_XXX macros to procure the right
file.
Don't worry about including the same file twice; include files have guards
that prevent all kinds of nastiness.
If some broken system needs you to write the prototype, don't impose on all
other systems.
Roll-your-own prototypes will break because they may use types that are
equivalent on your system, but are not portable to other systems (unsigned
long instead of size_t), or get some const status wrong.
Also, some compilers, such as OpenBSD's own gcc, are able to do a better job
with some very frequent functions such as strlen if you include the
right header file.
-
Carefully check the build log for any compiler warnings.
-
implicit declaration of function foo() indicates that a function
prototype is missing.
This means that the compiler will assume the return type to be an integer.
Where the function actually returns a pointer, on 64-bit platforms it will
be truncated, usually causing a segmentation fault.
-
Don't use the name of a standard system function for anything else.
If you want to write your own function, give it its own name, and call that
function everywhere.
If you wish to revert to the default system function, you just need to comment
your code out, and define your own name to the system function.
Don't do it the other way around.
Code should look like this:
/* prototype part */
#ifdef USE_OWN_GCVT
char *foo_gcvt(double number, size_t ndigit, char *buf);
#else
/* include correct file */
#include <stdlib.h>
/* use system function */
#define foo_gcvt gcvt
#endif
/* definition part */
#ifdef USE_OWN_GCVT
char *foo_gcvt(double number, size_t ndigit, char *buf)
{
/* proper definition */
}
/* typical use */
s = foo_gcvt(n, 15, b);
Other Helpful Hints
-
Recent versions of bsd.port.mk set the installers path.
Specifically, they set /usr/bin and /bin to be searched
before /usr/local/bin and /usr/X11R6/bin.
-
It is OK to rely on a feature that appeared in a recent version of
bsd.port.mk, as people are supposed to update their whole ports tree,
including bsd.port.mk.
NEED_VERSION is now obsolete.
-
Prefer using update-plist to generate and update packing-lists
instead of doing things manually.
You can comment unwanted lines out.
update-plist can detect most file types and copy most extra
annotations correctly.
-
Add USE_SYSTRACE=Yes to /etc/mk.conf to detect misbehaving
scripts, makefiles, etc.
-
In OpenBSD, curses.h/libcurses/libtermlib is the "new curses."
Change:
ncurses.h ==> curses.h
``old (BSD) curses'' is available by defining _USE_OLD_CURSES_ before
including curses.h (usually in a Makefile) and linking with
-locurses.
-
In OpenBSD, terminal discipline has been upgraded from the older BSD
sgtty to the newer POSIX tcgetattr family.
Avoid the older style in new code.
Some code may define tcgetattr to be a synonym for the older
sgtty, but this is at best a stopgap measure on OpenBSD.
The xterm source code is a very good example of what not to do.
Try to get your system functionality right: you want a type that remembers
the state of your terminal (possible typedef), you want a function that
extracts the current state, and a function that sets the new state.
Functions that modify this state are more difficult, as they tend to vary
depending upon the system.
Also, don't forget that you will have to handle cases where you are not
connected to a terminal, and that you need to handle signals: not only
termination, but also background (SIGTSTP).
You should always leave the terminal in a sane state.
Do your tests under an older shell, such as sh, which does not reset the
terminal in any way at program's termination.
-
The newer termcap/terminfo and curses, as included with OpenBSD, include
standard sequences for controlling color terminals.
You should use these if possible, reverting to standard ANSI color sequences
on other systems.
However, some terminal descriptions have not been updated yet, and you may
need to be able to specify these sequences manually.
This is the way vim handles it.
This is not strictly necessary.
Except for privileged programs, it is generally possible to override a termcap
definition through the TERMCAP variable and get it to work properly.
-
Signal semantics are tricky, and vary from one system to another.
Use sigaction to ensure a specific semantics, along with other system
calls referenced in the correspondin man page.
Additional Information
-
The man page bsd.port.mk(5).
This documents the ports infrastructure makefile that is included at the end
of each individual port makefile.
There are still a few comments at the start of the file itself, but most of
the useful information is now documented.
-
The man pages
packages-specs(7) and
library-specs(7).
These explain the exact syntax of package and library dependencies,
respectively.
-
The man page
port-modules(5).
This documents the different port modules which are used throughout
the ports infrastructure, e.g. cpan, java, python, and so on.