Netzwerktester

The Problem: Recurring Network Complaints With No Data

Some customers state their internet is bad, failing, slow and so on. When we visit them, everything seems normal. Without historical records and continuous monitoring, there is no way to determine whether the issue occurred once or throughout the week or how long it occurs, whether it influences downloads or only latency, or whether it originates at the local router or higher in the ISP network.

We need a solution to show objectively, via time-series data on network quality at customer locations, without depending on customers to conduct tests themselves. Furthermore, we could use this data to fix their infrastructure or file complaints to their ISP.

The Solution: A Raspberry Pi Appliance

The approach involved a compact Raspberry Pi installed at the customer site that operates continuously. No customer configuration is necessary. It monitors the connection continuously, transmits metrics to our central Grafana system via a WireGuard VPN connection, and remains in place while the complaint continues.

The device operates a Python application named net-tester, deployed and managed completely through Ansible. With Ansible, we can easily deploy new devices at other customer locations.

What It Measures

Every 15 seconds, the device pings each configured destination and records average latency, minimum and maximum values, and packet loss. External destinations include Google (8.8.8.8, 8.8.4.4) and Cloudflare (1.1.1.1).

On startup, the application performs a traceroute to identify local hops and assigns descriptive labels automatically: the first hop becomes Gateway, any following private IP becomes Modem, and the first public IP receives the label ISP. The Grafana dashboard instantly displays distinct latency graphs for each network segment without manual configuration.

Every 30 minutes, a speedtest executes via the Ookla CLI and records download, upload, latency, and jitter. A manual activation option is also accessible from the web dashboard.

WireGuard

The Pi consistently establishes a WireGuard tunnel to the Wentocon server network during boot. It employs a static VPN address (10.100.0.100/24) and a split-tunnel setup: only traffic destined for the internal 10.100.0.0/24 subnet traverses the VPN. All remaining traffic, including pings and speedtests, relies directly on the customer’s internet connection. This approach guarantees that measurements represent the genuine customer experience rather than the VPN route.

Hotspot and Captive Portal

When WiFi is the sole option, the Pi activates a NetworkManager-driven hotspot fallback. If it boots without Ethernet and without a recognised WiFi network, it launches an access point named WNT-NetTest. Connecting prompts the built-in captive portal on iOS, Android, and Windows, redirecting to a configuration page where we identify networks and connect the Pi appropriately. After connection, the hotspot deactivates automatically. Ethernet remains the preferred option when accessible alongside WiFi.

Tech Stack and Deployment

The net-tester app functions as a Flask application offering a Prometheus metrics endpoint at port 9100. Prometheus retrieves data via WireGuard, Grafana presents visualisations, and Promtail forwards systemd logs to Loki. The web dashboard displays live ping outcomes, the most recent speedtest, active interfaces, public IP, and WireGuard status.

The complete configuration deploys from scratch through a single make deploy using Ansible. Secrets are safeguarded with Ansible Vault -- to which i don't know the password anymore.