In this post I will showcase my deployment pipeline setup to deliver updates to my blog an portfolio websites with the help of GitHub webhooks. I keep it as generalized as possible, however, some specifics are inevitable, though adaptable to other systems and tooling.

The problem#

I needed a way to instantly update my websites on my self-hosted werver, as it got really tedious to go in and pull in the changes from remote, then reload the webserver and do this for more and more sites (currently I run 3).

The solution#

I read up on this, asked AI and found that there is a pretty straightforward way to do this with the help of GitHub Webhooks. Once set up properly the only thing you need is to push to your remotes main branch (or whichever branch you prefer) and it the changes will be automatically applied and set up on the server side.

The setup#

  1. webhook.py — a Python HTTP listener that receives GitHub push events, verifies the HMAC signature, and triggers the deploy script
    • Residing at: /opt/scripts/webhook.py
  2. autodeploy.sh — pulls the latest changes from git and reloads Caddy, logs to a daily file in ~/logs/
    • Residing at: /opt/scripts/autodeploy.sh
  3. Caddy — proxies /deploy to the Python listener over HTTPS
    • My Caddyfile sits in my home directory, it could be different for you.
  4. systemd — keeps the webhook listener running as yourorany-user with the secret in the environment
    • webhook.service - a webhook service specific to the application it is set up for is running and gets triggered once the webhook sends a request to the server
  5. GitHub webhook — sends a signed POST request to your server on every push to main
    • Residing at: /etc/systemd/system/webhook.service

The whole flow: push to GitHub → GitHub POSTs to your server → signature verified → git pull → Caddy reloads. After the reload, your changes are live.

webhook.py#

This is the generic orchestrater for all webhooks on the server. It checks the webhooks signature for validity and uses the deploy scripts to perform the proper actions for each deployment pipeline. It is a very generic script, it doesn’t know the details of the specific repositories and their actions, it only connects the webhooks with the scripts, which then handle the rest.

The webhook port is set by the service later, however, the script defaults to a basic port, this is what line 6 shows.

# /opt/scripts/webhook.py
from http.server import BaseHTTPRequestHandler, HTTPServer
import hmac, hashlib, subprocess, os

SECRET = os.environ.get("WEBHOOK_SECRET", "").encode()
PORT = int(os.environ.get("WEBHOOK_PORT", 9101))

class WebhookHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        if self.path != "/deploy":
            self.send_response(404)
            self.end_headers()
            return

        length = int(self.headers.get("Content-Length", 0))
        body = self.rfile.read(length)

        sig = self.headers.get("X-Hub-Signature-256", "")
        digest = "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest()
        if not hmac.compare_digest(sig, digest):
            self.send_response(403)
            self.end_headers()
            return

        args = os.environ.get("DEPLOY_ARGS", "").split()
        subprocess.Popen([os.environ.get("DEPLOY_SCRIPT")] + args)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"OK")

    def log_message(self, format, *args):
        print(f"{self.address_string()} - {format % args}")

if __name__ == "__main__":
    server = HTTPServer(("127.0.0.1", PORT), WebhookHandler)
    print(f"Webhook listener started on port {PORT}")
    server.serve_forever()

Permissions#

Make sure the user you are using to run these webhook setups is set properly. In this case, this user needs to own the webhook script with the following permissions:

ls -al /opt/scripts/webhook.py
-rw-r----- 1 <your-user> <your-user> 1255 Mar 16 20:37 /opt/scripts/webhook.py

I like to use the octal notation, here it comes in handy as we need to be specific:

chmod 640 /opt/scripts/webhook.py

A very useful sheet, where this notation is explained can be found here: Linux permissions cheat sheet

We also need it to be owned by the given user:

chown <your-user>:<your-user> /opt/scripts/webhook.py

autodeploy.sh#

This is the other generic script, it is specific to caddy and to my systems Caddyfile location. It also keeps a log of each change detection. Useful for finding errors, if deployment were to fail somewhere along the way.

#!/bin/bash
# /opt/scripts/autodeploy.sh

REPO_DIR="$1"
LOG_NAME="$2"
LOG_FILE="/home/your-user/logs/autodeploy-${LOG_NAME}_$(date +%Y-%m-%d).log"

cd "$REPO_DIR" || exit 1

git fetch origin main

LOCAL=$(git rev-parse HEAD)
REMOTE=$(git rev-parse origin/main)

if [ "$LOCAL" != "$REMOTE" ]; then
    echo "[$(date +%H:%M:%S)] Changes detected, pulling and reloading..." >> "$LOG_FILE"
    git pull origin main >> "$LOG_FILE" 2>&1
    cd ~
    caddy reload
    echo "[$(date +%H:%M:%S)] Caddy reloaded successfully." >> "$LOG_FILE"
fi

Permissions#

This file is special, as both root and my user needed access to this script. I needed some time to figure this out, the reason is, caddy is interacting with this script, so I had to make sure access is granted to caddy. Caddy works with the regular user here, as I set it up with that.

ls -al /opt/scripts/autodeploy.sh
-r-xr-x--- 1 root <your-user> 535 Mar 16 20:35 /opt/scripts/autodeploy.sh

Notice the permissions: read and execute. To achieve this use the command below:

chmod 550 /opt/scripts/autodeploy.sh

To make sure ownership is correct, run this:

chown root:<your-user> /opt/scripts/autodeploy.sh

Caddy#

For each of my subdomains, I added a /deploy handle that should be triggered by the webhook:

blog.panther.fun {
		# ...

        handle /deploy {
                reverse_proxy localhost:<port-1>
        }
}

me.panther.fun {
		# ...
		
        handle /deploy {
                reverse_proxy localhost:<port-2>
        }
}

Make sure the ports aren’t matching, that will cause conflicts.

webhook.service#

This service is specific to every app you use. You need to create one such service for each application. In my case I needed one for my blog, and one for my portfolio site. I could replicate this for any webpage I would like to auto-refresh on git push.

If you copy this, make sure to replace the details with your credentials.

# /etc/systemd/system/deploy-your-app.service
[Unit]
Description=GitHub Webhook Listener

[Service]
ExecStart=/usr/bin/python3 /opt/scripts/webhook.py
Environment="WEBHOOK_SECRET=<MYVERYBIGSECRET>"
Environment="DEPLOY_SCRIPT=/opt/scripts/autodeploy.sh"
Environment="DEPLOY_ARGS=</home/your-user/your-page> <your-page>
Environment="WEBHOOK_PORT=<whatever port you are using>"
Restart=always
User=<your-user>

[Install]
WantedBy=multi-user.target

This service needs to be ran after set up correctly, make sure to select the proper name and run the below command:

sudo systemctl start <deploy-your-app> # Whatever your service name is

Sometimes you may need to test and rerun this service, what I did is reload the daemon and restart:

sudo systemctl daemon-reload
sudo systemctl restart webhook
sudo systemctl show webhook --property=Environment # To check if the credentials and other parameters are correct

Permissions#

This is the goal state:

ls -al /etc/systemd/system/deploy-yourapp.service
-rw-r----- 1 root root 439 Mar 19 17:48 /etc/systemd/system/deploy-yourapp.service

Of course, first you need to edit this file, then copy it to the systemd directory, or edit it there directly. To make sure permissions are set afterwards, run the chmod 644 command if they aren’t set already, this time on the webhook:

chmod 640 /etc/systemd/system/deploy-your-app.service

Webhook setup - GitHub#

GitHub has a good tutorial on how to add a webhook. Make sure to select the repo where you want the automation pipeline to be and follow the instructions of this link: GitHub Docs | Creating webhooks

Important to note, I highly recommend not leaving the secrets field empty, the url for the deploy path will be exposed and can be abused, make sure to keep it safe.

Conclusion#

I played around and tested a lot until I got this running, but I’m glad I did. Now I can just replicate the service, run it and any new site I build can use the same structure with its own deploy-pipeline. There are a dozen other ways to do this, for me, this hacky way was just right and exactly what I needed. Hope you got something out of this too!