Skip to content

Secure Access

Servala clusters live fully in private networks with no direct access to the Kubernetes API and nodes from the Internet. This page describes how the Servala Portal and Platform Engineers reach the clusters securely. The full evaluation and decision behind this model is in ADR 003.

Netbird VPN

All cluster access is routed through Netbird, a WireGuard-based peer-to-peer mesh VPN. Netbird was chosen for its multi-tenant capabilities, full open-source availability (BSD-3-Clause), and EU-based company (Berlin, Germany).

Two Netbird mechanisms are in use, for different purposes:

  • The Netbird Talos Extension on each cluster node provides management access for platform engineers and peers traffic for the Servala Portal.
  • The Netbird Kubernetes Operator on each cluster connects platform workloads to the mesh, currently for shipping observability data (see below).

Node Access: Talos Extension

We use the Netbird Talos Extension installed on each cluster node. This provides infrastructure-level access without dependencies on a running cluster.

How it works:

  1. The Netbird Talos Extension is included in the Talos machine configuration
  2. Upon booting, nodes self-register with Netbird and are automatically placed in the cluster group
  3. A Netbird network is created with the Kubernetes API resource, using all nodes as routing peers to access the Kubernetes API VIP
  4. This provides fault-tolerant access at both the Netbird level (multiple routing peers) and Kubernetes level (HA control plane)

Access Policies

Resource Port Access
Kubernetes API 6443 Clients group via node routing peers
Talos Machine API 50000 Clients group direct to each node
Private network n/a Routed through peers for transparent access

The private network routing gives engineers and the Servala Portal transparent access as if they were in the same network as the cluster.

Benefits

  • No bastion host or LBaaS required for cluster bootstrapping
  • Machines boot with Netbird pre-configured via CloudInit and self-register automatically
  • After boot, only talosctl bootstrap is needed to initialize the cluster
  • Consistent access model across all environments and CSPs
  • Provides reliable access for both automation (Servala Portal) and Platform Engineers

Requirements for Servala engineers:

  • Netbird account
  • Device membership in the servala-engineers group

Netbird Kubernetes Operator

Each cluster runs the Netbird Kubernetes Operator. Its current use is the observability data path:

  • On the management cluster, the operator exposes the Mimir and Loki remote write endpoints as Netbird network resources, reachable in the mesh under the servala.internal domain.
  • On the workload clusters, a Netbird SidecarProfile injects a Netbird sidecar into the Prometheus and Vector pods. Metrics and logs are shipped through the mesh to Mimir and Loki on the management cluster.

Long-term, the Servala Portal should also use a SidecarProfile instead of its current SOCKS5 proxy sidecar (see below).

Servala Portal Connectivity

The Servala Portal requires access to all workload cluster Kubernetes APIs to manage AppCat instances and customer resources. This is achieved using a Netbird sidecar running in netstack mode.

Architecture

Portal Connectivity

How It Works

  1. Netbird sidecar in netstack mode: The Netbird agent runs as a sidecar container using userspace WireGuard (netstack mode). This requires no elevated privileges, making it compatible with restricted environments like OpenShift.

  2. SOCKS5 proxy: The sidecar exposes a SOCKS5 proxy on localhost:1080. The Portal application configures its Kubernetes client to route all traffic through this proxy.

  3. Direct VIP access via node network routing: Netbird routes each cluster's node CIDR (172.16.x.0/24) through the cluster's nodes as routing peers. The Kubernetes API VIP lives on this network, so the Portal can reach it directly: DNS resolves the API FQDN to the VIP, and Netbird handles the routing transparently.

Authorization

The Portal authenticates to each workload cluster with a dedicated servala-portal ServiceAccount, bound to a scoped ClusterRole. The role permits:

  • Managing namespaces and AppCat resources (appcat.vshn.io, vshn.appcat.vshn.io)
  • Reading pods, pod logs, nodes, events, PVCs, and metrics
  • Reading secrets (to retrieve connection credentials for service instances)

Authentication

  • Each cluster has its own Keycloak client (public client)
  • Kubernetes API is configured for OIDC authentication
  • Engineers authenticate via kubectl oidc-login
  • A ClusterRoleBinding grants cluster-admin to OIDC-authenticated Servala engineers