Support #736
Updated by Daniel Curtis over 8 years ago
{{>toc}}
This is a guide on how I setup a dynamic reverse proxy package cache using Nginx on Arch Linux.
*+WARNING+*: This method has a limitation. You must use mirrors that use the same relative path to package files and you must configure your cache to use that same path. In this example, we are using mirrors that use the relative path @/archlinux/$repo/os/$arch@ and our cache's Server setting in mirrorlist is configured similarly.
h2. Prepare the Environment
* Make sure everything is up to date using the following command:
<pre>
pacman -Syu
</pre>
h2. Install Nginx
* Install Nginx
<pre>
pacman -S nginx
</pre>
* Start and enable nginx at boot:
<pre>
systemctl enable nginx
systemctl start nginx
</pre>
* Create a configuration directory to make managing individual server blocks easier
<pre>
mkdir /etc/nginx/conf.d
</pre>
* Edit the main nginx config file:
<pre>
vi /etc/nginx/nginx.conf
</pre>
#* And strip down the config file and add the include statement at the end to make it easier to handle various server blocks:
<pre>
worker_processes 1;
error_log /var/log/nginx-error.log;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}
</pre>
h3. Package Cache Config
* Create the directory for the cache and adjust the permissions so nginx can write files to it:
<pre>
mkdir /srv/http/pacmirror
chown http:http /srv/http/pacmirror
</pre>
* Configure nginx as our dynamic cache:
<pre>
vi /etc/nginx/conf.d/pacmirror.example.com.conf /etc/nginx/confd/pacmirror.example.com.conf
</pre>
#* And add the following:
<pre>
# nginx may need to resolve domain names at run time
resolver 208.67.222.222 208.67.220.220;
server {
listen 80;
server_name pacmirror.example.com;
root /srv/http/pacmirror;
autoindex on;
# Requests for package db and signature files should redirect upstream without caching
location ~ \.(db|sig)$ {
proxy_pass http://mirrors$request_uri;
}
# Requests for actual packages should be served directly from cache if available.
# If not available, retrieve and save the package from an upstream mirror.
location ~ \.tar\.xz$ {
try_files $uri @pkg_mirror;
}
# Retrieve package from upstream mirrors and cache for future requests
location @pkg_mirror {
proxy_store on;
proxy_redirect off;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://mirrors$request_uri;
}
}
# Upstream Arch Linux Mirrors
upstream mirrors {
server localhost:8001;
server localhost:8002 backup;
server localhost:8003 backup;
}
# Arch Mirror 1 Proxy Configuration
server {
listen 8001;
server_name localhost;
location / {
proxy_pass http://mirror.us.leaseweb.net$request_uri;
proxy_set_header Host mirror.us.leaseweb.net;
}
}
# Arch Mirror 2 Proxy Configuration
server {
listen 8002;
server_name localhost;
location / {
proxy_pass http://mirror.rit.edu$request_uri;
proxy_set_header Host mirror.rit.edu;
}
}
# Arch Mirror 3 Proxy Configuration
server {
listen 8003;
server_name localhost;
location / {
proxy_pass http://lug.mtu.edu$request_uri;
proxy_set_header Host lug.mtu.edu;
}
}
</pre>
* Restart nginx:
<pre>
systemctl restart nginx
</pre>
h3. Update System Mirrorlist
* Edit the mirrorlist file:
<pre>
vi /etc/pacman.d/mirrorlist
</pre>
#* Add the following line to use this new cache
<pre>
Server = http://pacmirror.example.com/archlinux/$repo/os/$arch
</pre>
h3. Manual Cache Cleaning
*NOTE*: You will need to create a method to clear old packages, as this directory will continue to grow over time. paccache (which is included with pacman) can be used to automate this using retention criteria of your choosing. For example the following command will keep the last 3 versions of packages in your cache directory.:
<pre>
find /var/cache/pacmirror/ -type d -exec paccache -v -r -k 3 -c {} \;
</pre>
h2. Resources
* https://wiki.archlinux.org/index.php/Pacman/Tips_and_tricks#Dynamic_reverse_proxy_cache_using_nginx