Creating a Custom Package Repository

From PFSenseDocs
Jump to: navigation, search
This article was contributed or cited from an outside source. The style and formatting may not match other articles.
Important30.pngThis article is part of the DEVELOPMENT series. Any features, steps, or general information contained in this article is 100% unsupported and may break your system. Use at your own risk!Important30.png

These directions will walk through setting up a custom package repository, in order to host packages locally or develop/test package changes.

Step 1: Web Server

Set up a web server (which must be visible from the pfSense box), and install PHP. After installing PHP, enable it in the web server. On an Apache 2.x server, add this to httpd.conf:

AddHandler php5-script php
AddType text/html php


On a Lighttpd server, enable the server module "mod_fastcgi" and add this to lighttpd.conf:

       fastcgi.server = ( ".php" =>
                           ( "127.0.0.1" =>
                               (   
                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",
                                   "bin-path" => "/usr/local/bin/php-cgi",
                                   "max-procs" => 2,
                                   "bin-environment" => (
                                       "PHP_FCGI_CHILDREN" => "16",
                                       "PHP_FCGI_MAX_REQUESTS" => "10000"
                                   ),
                                   "bin-copy-environment" => (
                                       "PATH", "SHELL", "USER"
                                   ),
                                   "broken-scriptfilename" => "enable"
                               )
                           )
                       )


This server can be located on a WAN, LAN, or even a VPN as long as it is reachable by the pfSense box without hitting some kind of authenticating proxy or portal.

It can't easily be done from a pfSense box, itself without custom modifications to the XML-RPC code. It should be a completely separate server.

Step 2: Packages

Next, download the package server XML-RPC code, and the package repository itself.

Install git on the server or download the repositories to a workstation and upload them with FTP.

If a UNIX-based web server is being used, go to the root of the web server (e.g. /usr/local/www/) and run the following commands:

# git clone git://github.com/pfsense/xmlrpc-server.git pfSense

This will download the XML-RPC package code into the pfSense folder (the default location looked for by pfSense installations). This path may be changed, it is governed by the xmlrpcpath setting in /etc/inc/globals.inc located on the pfSense firewall.

# git clone git://github.com/pfsense/pfsense-packages.git packages

This will download the actual packages repository, which contains the individual package GUI elements, pfSense-related package code, and associated XML files. Beware, this repository is very large, and will consume a lot of disk space. This does not contain the actual binary package archives.

pfSense Versions

For the following pfSense Versions the Web Server must be setup in the following way...

2.1

The repo needs to have a server root with 2 subdirectories and nothing else: /pfSense and /packages (both in the root of repo)

http://repo/pfSense
http://repo/packages

2.1.2

The /pfSense content must be the root and a subdirectory /packages.

http://repo/
http://repo/packages

XMLRPC Modifications

If XMLRPC is still not working check the following in xmlrpc.php:

xmlrpc.php, line 45:

$path_to_files = './xmlrpc/';

xmlrpc.php, line 124:

$path_to_files = './packages/';

Cloning on a Workstation

If git and/or shell access is not available on the web server, the packages may be still be hosted. Clone those repositories on a workstation or another server first.

Use gitbash/msysgit on Windows, or git packages for any other OS. After cloning the repositories on a local system, upload the files to the proper places on the web server.

There is no need to upload the .git folders from these local repositories, but do not delete them.

To edit the package repository files as described in later steps, edit them on the workstation and then upload them after they have been changed.

Step 3: Test

Both URLs should show a PHP array if everything is working properly.

The web server may need configured so that the packages directory does not have PHP code processed underneath that directory tree. Many packages try to download .php files and if this is not done, .php files that packages attempt to download will be interpreted by the web server instead of downloaded intact.

Create a .htaccess file in the packages directory or the config directory under that, containing:

RemoveHandler .php
RemoveType .php

Step 4: Have pfSense use the repository

NOTE: "reposerver" is only the hostname, NOT a full URL.

Screenshot of Package Settings:

Package Settings
Package Settings

Step 5: Check it out

  • Go to the GUI under System -> Packages
  • The packages listed should be served by the custom repository

Step 6: Edit Package List

To limit the packages which are shown in this list, edit the file in the packages directory for the appropriate platform. For pfSense 2.0/2.1 it is pkg_config.8.xml. For pfSense 2.0/2.1 64-bit packages, this is pkg_config.8.xml.amd64. For pfSense 2.2, both are in one file: pkg_config.10.xml.

Remove the entries from the file that are not wanted, or set their required_version much higher (3, 4, etc)

Change paths for packages intended to be useed so that they point to a custom server as well. For example, on the System Patches package, the stock pkg_config.10.xml looks like this for the config_file URL:

<config_file>https://packages.pfsense.org/packages/config/systempatches/systempatches.xml</config_file>

Change that to:

<config_file>https://reposerver/packages/config/systempatches/systempatches.xml</config_file>

And then edit packages/config/systempatches/systempatches.xml on the server for the next step.

It is usually OK to leave the FreeBSD-style package URLs the same (<blah>.pbi or .tbz) unless the custom server will mirror the entire set of binary packages as well.

Step 7: Edit Package URLs

As with above, for editing and testing of packages, edit the package's XML file and alter the paths so that they point to the same files on the custom server instead of files on pfSense servers. Check the package's .inc files if it has any, as the installation routine may also download additional files.

Step 8: Patch and play

Now patch the package configuration files, and install additional modified packages! Or leave it alone and run a simple local repository.

Step 9: Report bugs

If a change fixed a problem, open a ticket at http://redmine.pfsense.org with the patched changes so they can be fixed for everyone else.

Maintaining the Package Repository

To keep the repository up-to-date, change to the packages directory and then run:

# git pull

That will update the repository with any changes made since the initial clone. If changes were made, it may be possible to keep them if they are committed locally first:

# git commit -a -m 'My Local Changes'
# git pull

Changes should be reapplied after the new code is updated, provided none of the changes conflict.

Making New Packages

To make new packages, or for more information about the format of the files, see Developing Packages

Copied (with some modifications) from http://forum.pfsense.org/index.php/topic,25519.0.html by kaarposoft