Support #737
Updated by Daniel Curtis over 6 years ago
{{>toc}}
This is a guide on using nginx to create a reverse proxy package cache for FreeBSD pkgng, ports, installer packages. The intent is to allow development of commonly used applications during times that I am not connected to the internet.
h2. Prepare the Environment
* Make sure the system is up to date:
<pre>
pkg update && pkg upgrade
</pre>
h2. Install Nginx
* Install nginx:
<pre>
pkg install nginx
</pre>
* Enable nginx at boot:
<pre>
sysrc nginx_enable=YES
</pre>
* Create a configuration directory to make managing individual server blocks easier
<pre>
mkdir /usr/local/etc/nginx/conf.d
</pre>
* Edit the main nginx config file:
<pre>
vi /usr/local/etc/nginx/nginx.conf
</pre>
#* And strip down the config file and add the include statement at the end to make it easier to handle various server blocks:
<pre>
worker_processes 1;
error_log /var/log/nginx-error.log;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# nginx may need to resolve domain names at run time
resolver 208.67.222.222 208.67.220.220;
# Load config files from the /etc/nginx/conf.d directory
include /usr/local/etc/nginx/conf.d/*.conf;
}
</pre>
h2. Caching pkgng
h3. Pkgng Nginx Cache Config
* Create the directory for the cache and adjust the permissions so nginx can write files to it:
<pre>
mkdir /var/cache/pkgmirror
chown www:www /var/cache/pkgmirror
</pre>
* Configure nginx as our dynamic cache:
<pre>
vi /etc/nginx/conf.d/pkgmirror.example.com.conf
</pre>
#* And add the following:
<pre>
server {
listen 80;
server_name pkgmirror.example.com;
root /var/cache/pkgmirror;
autoindex on;
# Requests for actual packages should be served directly from cache if available.
# If not available, retrieve and save the package from an upstream mirror.
location / {
try_files $uri @pkg_mirror;
}
# Retrieve package from upstream mirrors and cache for future requests
location @pkg_mirror {
proxy_store on;
proxy_redirect off;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://pkg-mirrors$request_uri;
}
}
# Upstream FreeBSD Mirrors
upstream pkg-mirrors {
server localhost:8007;
server localhost:8008 backup;
server localhost:8009 backup;
}
# FreeBSD Global Mirror Proxy Configuration
server {
listen 8007;
server_name localhost;
location / {
proxy_pass http://pkg.FreeBSD.org$request_uri;
proxy_set_header Host pkg.FreeBSD.org;
}
}
# FreeBSD US West Mirror Proxy Configuration
server {
listen 8008;
server_name localhost;
location / {
proxy_pass http://pkg.us-west.FreeBSD.org$request_uri;
proxy_set_header Host pkg.us-west.FreeBSD.org;
}
}
# FreeBSD US East Mirror Proxy Configuration
server {
listen 8009;
server_name localhost;
location / {
proxy_pass http://pkg.us-east.FreeBSD.org$request_uri;
proxy_set_header Host pkg.us-east.FreeBSD.org;
}
}
</pre>
* Restart nginx:
<pre>
service restart nginx
</pre>
h3. Using a Private Update System Repo
* Edit the pkg repo config file:
<pre>
mkdir -p /usr/local/etc/pkg/repos
vi /usr/local/etc/pkg/repos/FreeBSD.conf
</pre>
* Add the following line to use the this new proxy cache with quarterly updates
<pre>
FreeBSD: {
url: "pkg+http://pkgmirror.example.com/${ABI}/quarterly", "pkg+http://pkgmirror.example.com/${ABI}/latest",
mirror_type: "srv",
enabled: yes
}
</pre>
h3. Cache Cleaning
* The following command will clean the package cache:
<pre>
setenv PKG_CACHEDIR /var/cache/pkgmirror
pkg clean
</pre>
h2. Caching the Ports Tree
h3. Pkgng Nginx Cache Config
* Create the directory for the cache and adjust the permissions so nginx can write files to it:
<pre>
mkdir /var/cache/portsmirror
chown www:www /var/cache/portsmirror
</pre>
* Configure nginx as our dynamic cache:
<pre>
vi /etc/nginx/conf.d/portsmirror.example.com.conf
</pre>
#* And add the following:
<pre>
# nginx may need to resolve domain names at run time
server {
listen 80;
server_name portsmirror.example.com;
root /var/cache/portsmirror;
autoindex on;
# Requests for actual packages should be served directly from cache if available.
# If not available, retrieve and save the package from an upstream mirror.
location / {
try_files $uri @ports_mirror;
}
# Retrieve package from upstream mirrors and cache for future requests
location @ports_mirror {
proxy_store on;
proxy_redirect off;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://ports-mirrors$request_uri;
}
}
# Upstream FreeBSD Mirrors
upstream ports-mirrors {
server localhost:8010;
server localhost:8011 backup;
server localhost:8012 backup;
}
# FreeBSD Global Mirror Proxy Configuration
server {
listen 8010;
server_name localhost;
location / {
proxy_pass http://your-org.portsnap.freebsd.org$request_uri;
proxy_set_header Host your-org.portsnap.freebsd.org;
}
}
# FreeBSD US West Mirror Proxy Configuration
server {
listen 8011;
server_name localhost;
location / {
proxy_pass http://sourcefire.portsnap.freebsd.org$request_uri;
proxy_set_header Host sourcefire.portsnap.freebsd.org;
}
}
# FreeBSD US East Mirror Proxy Configuration
server {
listen 8012;
server_name localhost;
location / {
proxy_pass http://isc.portsnap.freebsd.org$request_uri;
proxy_set_header Host isc.portsnap.freebsd.org;
}
}
</pre>
* Restart nginx:
<pre>
service restart nginx
</pre>
h3. Download Ports Tree
* Edit the portsnap config file:
<pre>
vi /etc/portsnap
</pre>
#* And change the SERVERNAME variable to the local proxy:
<pre>
SERVERNAME=portsmirror.example.com
</pre>
* Set the http_proxy environment variable:
<pre>
setenv http_proxy http://portsmirror.example.com
</pre>
* Then fetch the latest ports tree:
<pre>
portsnap fetch extract
</pre>
h3. FreeBSD Install Cache Config
* Create the directory for the cache and adjust the permissions so nginx can write files to it:
<pre>
mkdir /var/cache/freebsdmirror
chown www:www /var/cache/freebsdmirror
</pre>
* Configure nginx as our dynamic cache:
<pre>
vi /etc/nginx/conf.d/freebsdmirror.example.com.conf
</pre>
#* And add the following:
<pre>
server {
listen 80;
server_name freebsdmirror.example.com;
root /var/cache/freebsdmirror;
autoindex on;
# Requests for actual packages should be served directly from cache if available.
# If not available, retrieve and save the package from an upstream mirror.
location ~ \.txz$ {
try_files $uri @freebsd_mirror;
}
# Retrieve package from upstream mirrors and cache for future requests
location @freebsd_mirror {
proxy_store on;
proxy_redirect off;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://freebsd-mirrors$request_uri;
}
}
# Upstream FreeBSD Mirrors
upstream freebsd-mirrors {
server localhost:8013;
server localhost:8014 backup;
server localhost:8015 backup;
}
# FreeBSD Global Mirror Proxy Configuration
server {
listen 8013;
server_name localhost;
location / {
proxy_pass http://ftp.FreeBSD.org$request_uri;
proxy_set_header Host ftp.FreeBSD.org;
}
}
# FreeBSD Mirror Proxy 2 Configuration
server {
listen 8014;
server_name localhost;
location / {
proxy_pass http://ftp2.FreeBSD.org$request_uri;
proxy_set_header Host ftp2.FreeBSD.org;
}
}
# FreeBSD Mirror Proxy 3 Configuration
server {
listen 8015;
server_name localhost;
location / {
proxy_pass http://ftp3.FreeBSD.org$request_uri;
proxy_set_header Host ftp3.FreeBSD.org;
}
}
</pre>
* Restart nginx:
<pre>
service restart nginx
</pre>
* Now during an installation of FreeBSD, during the mirror selection phase select *Other* and use the new installer URL:
<pre>
http://freebsdmirror.example.com/pub/FreeBSD/releases/amd64/amd64/10.2-RELEASE
</pre>
h2. -Pkg Metadata Backup-
*NOTE*: The pkg metadata rotation method is a much more robust solution
* Install rename:
<pre>
pkg install rename
</pre>
* Create the pkg metadata backup script:
<pre>
vi /usr/local/bin/backup-pkg-meta.sh
</pre>
#* And add the following:
<pre>
#!/bin/sh
cd /var/cache/pkgmirror
find . -name "meta.txz" -exec renamex -s/\.txz/\.txz\.bak/g {} \;
find . -name "packagesite.txz" -exec renamex -s/\.txz/\.txz\.bak/g {} \;
</pre>
#* Make the script executable:
<pre>
chmod +x /usr/local/bin/backup-pkg-meta.sh
</pre>
* Since the nginx proxy cache config is set to cache the pkg metadata files, they need to be moved so the newer metadata files can be downloaded and cached. Run the script frequently or cron job to run the script:
<pre>
backup-pkg-meta.sh
</pre>
h2. Pkg Metadata Rotation
* Create a pkg metadata rotation script:
<pre>
vi /usr/local/bin/rotate-pkgmirror-meta.sh
</pre>
#* And add the following:
<pre>
#!/bin/sh
# Rotate the metadata archives for the pkg mirror
IP_HOST=8.8.8.8
META_PACKAGE=meta.txz
META_BACKUP="${META_PACKAGE}.bak"
META_TMP="${META_PACKAGE}.tmp"
PACKAGESITE_PACKAGE=packagesite.txz
PACKAGESITE_BACKUP="${PACKAGESITE_PACKAGE}.bak"
LOCAL_PATH=/var/cache/pkgmirror/FreeBSD:11:amd64/quarterly/ LOCAL_PATH=/var/cache/pkgmirror/FreeBSD:11:amd64/latest/
LOCAL_META_PACKAGE_PATH="${LOCAL_PATH}${META_PACKAGE}"
LOCAL_META_BACKUP_PATH="${LOCAL_PATH}${META_BACKUP}"
LOCAL_META_TMP_PATH="${LOCAL_PATH}${META_TMP}"
LOCAL_PACKAGESITE_PACKAGE_PATH="${LOCAL_PATH}${PACKAGESITE_PACKAGE}"
LOCAL_PACKAGESITE_BACKUP_PATH="${LOCAL_PATH}${PACKAGESITE_BACKUP}"
REMOTE_PATH=http://pkg.freebsd.org/FreeBSD:11:amd64/quarterly/
REMOTE_META_PATH="${REMOTE_PATH}${META_PACKAGE}"
REMOTE_PACKAGESITE_PATH="${REMOTE_PATH}${PACKAGESITE_PACKAGE}"
if ping -q -c1 -W1 ${IP_HOST} >/dev/null ; then
echo "IPv4 is up. Fetching current meta.txz."
fetch -qo ${LOCAL_META_TMP_PATH} ${REMOTE_META_PATH}
if `diff ${LOCAL_META_PACKAGE_PATH} ${LOCAL_META_TMP_PATH} >/dev/null` ; then
echo "meta.txz is current. Cleaning up and not rotating."
rm ${LOCAL_META_TMP_PATH}
else
echo "meta.txz is different. Updating and rotating meta.txz/packagesite.txz."
mv ${LOCAL_META_PACKAGE_PATH} ${LOCAL_META_BACKUP_PATH}
mv ${LOCAL_PACKAGESITE_PACKAGE_PATH} ${LOCAL_PACKAGESITE_BACKUP_PATH}
mv ${LOCAL_META_TMP_PATH} ${LOCAL_META_PACKAGE_PATH}
fetch -o ${LOCAL_PACKAGESITE_PACKAGE_PATH} ${REMOTE_PACKAGESITE_PATH}
fi
else
echo "IPv4 is down. Not rotating."
fi
</pre>
* Make the script executable:
<pre>
chmod +x /usr/local/bin/rotate-pkgmirror-meta.sh
</pre>
* Edit the crontab:
<pre>
crontab -e
</pre>
#* And add the following to run the script every half hour:
<pre>
*/30 * * * * /usr/local/bin/rotate-pkgmirror-meta.sh >/tmp/rotate-pkgmirror.log
</pre>