OpenBSD Ports - Porting Guide [Handbook Index]



Overview

So you've just compiled your favorite software package on your OpenBSD machine and you want to share your effort by turning it into a standard port. What to do?

The most important thing to do is to communicate. Ask people on [email protected] if they are working on the same port. Tell the original software authors about it, including problems you may find. If licensing information appears incorrect, tell them. If you had to jump through hoops to make the port build, tell them what can be fixed. If they are only developing on Linux and feel like ignoring the rest of the Unix world, try to make them change their view.

Communication makes the difference between a successful port and a port that will slowly be abandoned by everyone.

First, look at the porting information on this page. Test, then re-test, and finally test again!

OpenBSD fully supports updates. This means that some issues must be taken into account.

Submit the port. Create a gzipped tarball of the port directory. You can then either place it on a public HTTP server, sending its URL to [email protected], or send the port MIME encoded to the same address.

Porting some new software takes time. Maintaining it over time is harder. It is quite okay to port software and let other people handle it afterwards. It is also okay to help other people update and maintain other ports, as long as you communicate to avoid doing the same things twice.

In OpenBSD culture, MAINTAINERship is not a status item, but a responsibility. We have CVS and comments to give credit to the person who did the work. A port MAINTAINER is something else: a person who assumes responsibility for the working of the port, and is willing to spend some time ensuring it works as best as can be.

Porting Checklist

The list below is a useful reminder of things to do. This is neither totally accurate nor perfect. Direct comments and questions to [email protected].
  1. If you want to be a maintainer, subscribe to [email protected].
  2. Being a maintainer means more than just submitting ports. It also means trying to keep them up to date, and answering questions about them.

  3. Check out a copy of the ports tree from CVS. You can find instructions on how to do this at the AnonCVS page. As a porter, you should keep your base OS, ports tree, and installed packages up to date.

  4. From the names of the first level subdirectories of /usr/ports/, pick a primary category for your port. Create a new directory below /usr/ports/<category>/ or /usr/ports/mystuff/<category>/ and create the basic infrastructure there. Copy the template Makefile from /usr/ports/infrastructure/templates/Makefile.template. Fill in CATEGORIES with the primary category you have chosen.

  5. Add the fetch portions of the Makefile.

    For more complex ports, you have more options and tools available to you:


  6. Create a checksum in distinfo by typing make makesum.
  7. All files in DISTFILES* are usually processed during make extract. EXTRACT_ONLY may be used to limit extraction to a subset of files (possibly empty). The customary use of this variable is to customize extraction: for instance, if some DISTFILES need some special treatment, they will be removed from EXTRACT_ONLY and handled manually at post-extract stage. For historic reasons, make extract does set up the working directory first along with extracting files. Thus, providing a pre-extract or do-extract target is highly unusual (and fairly suspicious behavior, indicative of a high degree of obfuscation in the port).
  8. Patches that need specific treatment should be mentioned in DISTFILES, and removed from EXTRACT_ONLY, for historic reasons.
  9. Extract the port with make extract. Pay attention to where the base of the sources are. Usually, it's /usr/ports/pobj/${PKGNAME}${FLAVOR_EXT}/${DISTNAME}. You may need to modify the Makefile's WRKDIST variable if it is different.

  10. Read the installation documentation and note what you have to do to build the port and any special options that might be needed.

  11. Now is also a good time to figure out what kind of licensing restrictions apply to your port. Many are freely redistributable, but quite a few are not. We need two questions answered to distribute ports properly. These are the PERMIT_* values in the Makefile.

    PERMIT_PACKAGE tells us whether we can put the package on the mirrors.
    PERMIT_DISTFILES tells us whether we can mirror the distfiles.

    If both are permitted, simply set PERMIT_PACKAGE=Yes. Otherwise, set each variable to either Yes or a comment string indicating the reason distribution is not allowed. Pay attention to any special conditions you may need to fulfill later on. For example, some ports require installing a copy of the license. We recommend you place the license in /usr/local/share/doc/<name>/.

    In addition to the PERMIT_* values, put a license marker like # License above them as a comment, this way we know why the PERMIT_* values are set the way they are. For GPL, specify which version is applicable e.g. "GPLv2 only", "GPLv2+" (meaning "v2 or newer"), "GPLv3 only", etc.

  12. Add configuration options to Makefile and/or create the configuration script.
  13. Try building the port with make build.
  14. Begin a cycle of make build, edit source code, generate a patch using make update-patches and make clean patch.
  15. Try setting SEPARATE_BUILD.
  16. Peruse the output (if any) and tweak any options in the Makefile. To repeat issue the command make clean configure.

    Configuration files will usually be installed somewhere under /usr/local and handled by pkg_add(1) @sample annotations. After a package has been installed the contents of pkg/MESSAGE will be displayed if it exists.

    The OpenBSD file locations are:

    user executables:			/usr/local/bin
    system admin executables:		/usr/local/sbin
    program executables:			/usr/local/libexec
    libraries:				/usr/local/lib
    architecture dependent data:		/usr/local/lib/<name>
    installed include files:		/usr/local/include or
    					/usr/local/include/<name>
    single-machine data:			/etc or /etc/<name>
    local state:				/var/run
    games score files:			/var/games
    GNU info files:				/usr/local/info
    man pages:				/usr/local/man/...
    read-only architecture-independent:	/usr/local/share/<name>
    misc documentation:			/usr/local/share/doc/<name>
    examples files:				/usr/local/share/examples/<name>
    
  17. Begin a cycle of makes until the port is ready. Patch (see above) clean, and make until the port is generated. Get rid of all warnings if possible, especially security-related warnings.

  18. Add COMMENT in Makefile. COMMENT is a SHORT one-line description of the port (max. 60 characters). Do NOT include the package name (or version number of the software) in the comment. Do NOT start with an uppercase letter unless semantically significant, and do NOT end with a period. DON'T EVER START WITH AN INDEFINITE ARTICLE SUCH AS "a" or "an" - remove the article altogether. The main usage of COMMENT is the one-liner from pkg_info(1):
    py3-cairo-1.24.0    cairo bindings for Python
    


  19. Put a longer description of the port into pkg/DESCR. One to a few paragraphs concisely explaining what the port does is sufficient. Lines should be no longer than 80 characters. This can be done by first editing the DESCR file and then running it through fmt.

  20. If the application needs to create a user or a group, choose the lowest free id from /usr/ports/infrastructure/db/user.list for your port to use and don't forget to include that file in your commit.

  21. Install the application with make fake. Libraries should never be stripped. Executables are stripped by default, unless you set DEBUG_PACKAGES; this is governed by ${INSTALL_STRIP}. ${INSTALL_PROGRAM} honors this automatically and is preferable to unconditional stripping (e.g., by an install-strip target or by running strip from the Makefile). You can use objdump --syms to determine if a binary is stripped or not. Stripped files have no symbols in the SYMBOL TABLE.

  22. Check port for security holes again. This is especially important for network-facing programs. See our security recommendations for that.

  23. Make sure your /etc/mtree directory is up to date. (The next step uses the mtree lists to remove existing directories from generated packing-lists). Remember that the OpenBSD (U)pdate does not touch /etc... For automatic updating of /etc, sysmerge(8) may help.

  24. Create pkg/PLIST. After the installation with make fake is complete, use the developer's command make update-plist, which creates or updates the file PLIST in the pkg directory. This file is a candidate packing list.

    Peruse PLIST and verify that everything was installed and that it was installed in the proper locations. Anything not installed can be added to a port Makefile post-install rule. Note that PLIST annotations are documented in the pkg_create(1) manual, and that update-plist(1) is also documented.

  25. It is possible some directories do not need to be in the PLIST as they've been installed by a dependency; if you added to LIB_DEPENDS or RUN_DEPENDS, run make update-plist to remove these.

  26. Test the packaging with make package, test installation of the resulting package with make install, and test its removal with make uninstall. When dealing with multi-packages, it may instead be convenient to use pkg_add(1) and pkg_delete(1) directly, setting TRUSTED_PKG_PATH to /usr/ports/packages/%a/all/ in the environment.

  27. Verify dependencies. Peruse your logs to verify the port did detect what is mentioned in *DEPENDS, and nothing more. Check names, particularly in the make configure stage, for hidden dependencies (stuff that exists elsewhere in the ports tree and might be detected if the user installs some other ports first).

  28. Verify shared library dependencies. Run make port-lib-depends-check and add every LIB_DEPENDS or WANTLIB annotation that is needed until it runs cleanly. You may want to read the update guidelines to understand why this is so important.

  29. Check for regression tests, and whether they run cleanly. Set NO_TEST=Yes if a port has no test infrastructure. If dependencies are required to run the tests, but not to build the port, list them in TEST_DEPENDS. Please note: do not set NO_TEST if a port has an empty regression test infrastructure.

  30. Test that make clean succeeds. Sometimes the make fake stage creates files in the build directory which will cause this to fail.

  31. Run the /usr/ports/infrastructure/bin/portcheck utility in your port directory and take care of problems it finds, if any.

  32. Mail [email protected] with a description, the homepage (if any), and a short note asking for comments and testing. Make sure to attach the port/patch to this email (or include a URL where it can be found) and send it out.

    Try to get others to test it on a variety of platforms for you.


  33. Incorporate any useful feedback you get. Test it again on your platform. Get those who gave you feedback to test it again from your new port.

  34. Get the port committed to CVS.

    If you are not a developer with CVS access, then you will have to find an OpenBSD developer to do the following steps for you (ask on [email protected]).

    Before you import anything, get at least one "OK" from another ports developer (the more the better).

    If using @newuser or @newgroup in the PLIST files, check that no users were added to /usr/ports/infrastructure/db/user.list since the port was created.

    For new ports we use cvs import, rather than adding new files individually. For example, to import a new lang/kaffe1 port, first do a dry-run import:

    $ cd /usr/ports/lang/kaffe1
    $ cvs -ndcvs.openbsd.org:/cvs import ports/lang/kaffe1 username username_yyyymmdd
    
    Where username is your account username, and yyyymmdd is the current date. If this succeeds, then you can remove -n to import for real. Your editor will be invoked so that you can enter a commit message. In the commit message, at least state what you are importing and which developers provided OKs. Make sure the import path is correct; it always starts with ports/.

    Alternatively, you can use ports/infrastructure/bin/portimport to import new ports.

  35. Last but not least, add a one-line entry for the new port in its parent directory's Makefile, e.g., for ports/lang/kaffe1, add it to ports/lang/Makefile. Don't forget to commit any changes made to /usr/ports/infrastructure/db/user.list.

  36. Maintain the port! As time goes by, problems may arise, or new versions of the software may be released. You should strive to keep your port up to date. In other words - iterate, test, test, iterate...

  37. When updating a port, remember to handle dependencies! You shouldn't break any port that depends on yours. The sqlports package includes show-reverse-deps, making it trivial to know the full tree of ports that depend directly or indirectly on a given port. In case of problems, communicate with the maintainers of such ports. Likewise, be alert for dependency updates, and check that the maintainer did their job.

Handling Complex Situations

Assume you've managed to build the software, provide required patches, and you want to finish the port.

Know the Software

Identify options
The first step is usually to identify build options. You will often have to read the configuration log, see what stuff your port auto-detects. Read the configure script options. Read the port documentation for extra stuff.
Make options work
Recompile your port with various options. Install extra dependencies. Make sure your port detects them correctly. Add supplementary patches to ensure compilation. Test the result, and verify extra stuff does work.
Identify missing software
Some dependencies won't be fulfilled because the missing software has not yet been ported. It is highly recommended to explicitly disable those options. Failure to do that breaks bulk builds all the time: people port new software and import it, and soon after, old ports stop building because they detect the dependency, try to use it, and fail to build or package.
Check run-time dependencies versus build-dependencies
Update your packing-list with make update-plist. Use make port-lib-depends-check to see what libraries your software needs (that will end up in LIB_DEPENDS or WANTLIB, usually). Identify various files and binaries in the dependencies that have to be present for the port to work.

By this point, you should have a fair understanding of your port's working.

Figure Out Important Options

You won't care about some options. It makes no sense to disable some stuff if it always works, and if the dependencies are quite small. Take special notes of licences on dependencies, especially the PERMIT* stuff. As a rule, even if a dependency is very small, if it affects the licensing of the resulting package, you will have to explicitly take care of it.

Considering all possible options, you should be left with a much smaller set of options for your port, mostly depending on what packages are needed to run the software. For now, do not worry about build dependencies. Remember that the OpenBSD ports system is focused on the end user, and the end user will want to install binary packages, so it doesn't matter if you need a huge piece of software to build your port if it doesn't show up as a library or runtime dependency.

The Ideal Case: MULTI_PACKAGES and PSEUDO_FLAVORS

By now, you should have a fairly good idea of: In the ideal case, build options will simply create new files, with new dependencies, and not affect other stuff. This is a fairly common scenario for plugin frameworks: you add one library, you end up with a new plugin. This also happens fairly often for core applications with a graphics front-end: the console application is built every time, and the x11 interface shows up as a separate binary.

In this case, try setting the MULTI_PACKAGES variable to a list of -sub packages, add corresponding COMMENTS, and look at your packaging. Basically, MULTI_PACKAGES only affects the packaging: if you have MULTI_PACKAGES=-s1 -s2 all stuff relevant to the package will exist in two variants: COMMENT-s1 for the first package, COMMENT-s2 for the second package, PLIST-s1, PLIST-s2, DESCR-s1, DESCR-s2. You need to write those COMMENT-s1 and COMMENT-s2 in the Makefile, split your PLIST into two parts, and create DESCR-s1/DESCR-s2.

It is a good idea to start with the minimal framework work required: just copy the existing description and comments, because you will have to fiddle with MULTI_PACKAGES and stuff before you polish this.

Once you've separated the files properly, you will need to check dependencies: LIB_DEPENDS, WANTLIB, and RUN_DEPENDS will be split for each subpackage. It is usually time to check that your multi-packaging "works," and that those nasty dependencies you don't want to force on the user are indeed relegated to a specific subpackage.

Assuming everything works, you're mostly done. Just pick reasonable names for the various packages, and fill in the comments and descriptions. The end-user will be able to just install the package(s) they want.

But wait. What about the build, you say? Well, having a lot of dependencies during build is not a problem. Most packages are built by the OpenBSD team using special build runs (known as bulk builds) where a developer just builds all possible packages on a dedicated machine (or several, for slow architectures). Since everything will get built, having big dependencies is not an issue. Building the same thing several times, is an issue, though, which is why MULTI_PACKAGES are the best way to handle options (when possible): one build, one set of packages to test, better quality overall...

If you also want to help people who build packages themselves, you may consider adding PSEUDO_FLAVORS. A pseudo-flavor is a way to tweak an option (say, disable the graphical interface) that's very similar to actual flavors. In fact, the biggest difference is a functional difference: a pseudo flavor should only affect the set of packages that get built, but it is never allowed to modify the actual package contents.

For instance, assuming you separated the graphical interface into a separate subpackage (MULTI_PACKAGES=-main -x11), you could create a pseudo flavor no_x11 that avoids building the -x11 subpackage. The crucial point is that this flavor should NOT affect the -main package in any way.

You would end up with a Makefile that looks something like this:

COMMENT-main =	foo core application
COMMENT-x11 =	foo graphical interface

V =		1.0
DISTNAME =	foo-$V
CATEGORIES =	app

MULTI_PACKAGES =	-main -x11
PSEUDO_FLAVORS =	no_x11
FLAVOR ?=

WANTLIB = c m crypto ssl
WANTLIB-x11 = ${WANTLIB} X11 Xt

RUN_DEPENDS-x11 =	${BASE_PKGPATH},-main>=$V

CONFIGURE_STYLE =	gnu

.include <bsd.port.arch.mk>

.if !${BUILD_PACKAGES:M-x11}
CONFIGURE_ARGS += --disable-x11
.endif

.include <bsd.port.mk>
Notice that you only have to write a very small conditional section in the Makefile: the system doesn't care at all that you define extra variables.

Interdependencies Between Subpackages

MULTI_PACKAGES setups used to be asymmetric, with a -main subpackage and other subpackages, with the -main subpackage always built, and other subpackages possibly depending upon it. The current situation is totally symmetric: any subpackage can depend on any other. The infrastructure has specific provisions to avoid looping indefinitely.

The infrastructure takes special care of library inter-dependencies: it can detect which WANTLIB come from external dependencies, and which come from inter-dependencies. While external LIB_DEPENDS and WANTLIB are checked at the start of build, LIB_DEPENDS and WANTLIB that refer to one of the subpackages currently being built will only be checked at packaging time (and thus packages may be created in a specific order to satisfy interdependencies).

The infrastructure provides specific variables to help in writing inter-dependencies: BUILD_PKGPATH contains the PKGPATH used during building the current packages, taking flavors and pseudo-flavors into account. It is highly recommended to use this variable instead of rolling your own: failure to do so will often trigger rebuilds in interesting flavors situations. For instance:

...
FLAVORS = a b
FLAVOR ?=
MULTI_PACKAGES = -p1 -p2
WANTLIB-p1 = foo
LIB_DEPENDS-p1 = some/pkgpath,-p2
...
If you go on and build in some/pkgpath with FLAVOR=a, then creating the subpackage for -p1 will trigger a rebuild with FLAVOR=''. You would instead write:
LIB_DEPENDS-p1 = ${BUILD_PKGPATH},-p2
There is also a BASE_PKGPATH variable, which does not take pseudo-flavors into account. This variable has limited applicability: it corresponds to a transition between old MULTI_PACKAGES and newer ones, where the old main subpackage did not have any marker in its pkgpath, and thus the corresponding package needs a @pkgpath ${BASE_PKGPATH} in its packing-list. (In general, pseudo-flavors are build information, and should not make their way into packages and packing-lists).

True FLAVORS and PKGNAMES

There are some cases where configuration options are too invasive, and you will have to add true flavors to the Makefile: those flavors will command some configuration options, and usually additions to various dependencies. Note that package naming is mostly automatic: the PKGNAME will have an extension built by appending the specified flavors to its name. So, if
PKGNAME = foo-1.0
FLAVORS = f1 f2 f3
and you build the port with FLAVOR='f3 f1', then FULLPKGNAME=foo-1.0-f1-f3 (FLAVORS is used to reorder specified flavors in a canonical way).

There are sometimes mixed situations, where some packages do depend on the FLAVOR, and some don't. For instance, some ports include a large set of documentation that does not depend on the FLAVOR, and some actual programs that depend on the FLAVOR. In such cases, you can specify the FULLPKGNAME for the documentation subpackage explicitly. Something like this:

CATEGORIES = app
COMMENT-core = foo application
COMMENT-doc = foo documentation
V = 1.0
DISTNAME = foo-1.0
PKGNAME-core = foo-$V
FULLPKGNAME-doc = foo-doc-$V
FLAVORS = crypto

MULTI_PACKAGES = -core -doc
WANTLIB-core = c m

.if ${FLAVOR:L:Mcrypto}
WANTLIB-core += ssl crypto
CONFIGURE_ARGS += --enable-crypto
.endif
As mentioned in the documentation, all package names have the same structure: stem-version-flavor_extension.

By default, packages with the same stem do conflict, and update paths will look at candidates with the same stem. The right package will be the one coming from the exact same PKGPATH, or matching @pkgpath annotation in the packing-list.

Usually, MULTI_PACKAGES should not conflict, so they must have different names (and the infrastructure has no way to build those names). On the other hand, flavors should conflict, and thus have the same name. The flavor information should end at the end of the package name, except for pseudo-flavors, which do not change the way a package is built.

As far as dependencies go, by default, specifying a PKGPATH will just create a stem-* dependency, meaning any package with the right stem will match the dependency. By default, any flavor will match. If only specific flavors are desired, you must include them in your specification, e.g., stem-*-flavor. If some flavors are unwanted, you can remove them from matching packages, e.g., stem-*-!flavor.

Since OpenBSD 4.9, asking for at least some version of a dependency can be directly tacked to the PKGPATH, e.g., dir/foo>=2.0.

Updating Ports

The package tools can do updates, so maintainers have to be aware of one simple fact: update is not instantaneous. Even if a user just goes from release to release, each time they run pkg_add -u, the system will replace each package with a new version, one package at a time. So the user will run a mixed system, even if it is for just a few minutes.

There are basically two update models of which maintainers must be aware.

You should note that part of the update process, especially the macroscopic changes for users who update every six months, is not yet automated. The global update mechanism is still a work in progress, and pkg_add will be able to cope with more issues in the future. As of now, you should focus on making sure the system updates correctly, one port at a time, and that one port's update takes other ports into account, as far as conflicts and other issues are concerned.

Update Checklist

Part of the work is to be done when making the port itself. Ports often need minor updates without a new version upstream. Part of the work will happen before the update itself. And then the update.

Committing Port Updates

When a port update is ready, get it in CVS. If you do not have a CVS account, then you will need to find a developer to do the following for you (ask on [email protected]).
  1. Get at least one "OK" from another ports developer.

  2. Use cvs add to add any new files.

  3. Use cvs rm to remove files that are no longer needed (e.g. upstreamed patches).

  4. Check the output of cvs -n up -d. New files should be marked A, deleted files should be marked D, and changed files should be marked M. Look for files marked ? - did you mean to cvs add them?

  5. If all is well, commit the new/deleted/changed files using cvs commit. When invoking cvs commit, you can either list the files individually, or if you provide no filenames, CVS will recursively commit (be careful with this feature). You will be asked to enter a commit message, in which you should state which port is being updated, and who provided an OK.

OpenBSD Porting Policy

Security Recommendations

There are many security problems to worry about. If you are not absolutely sure of what you are doing please request help from the ports mailing list.

Generic Porting Hints

Other Helpful Hints

Additional Information