Up until recently I had been accessing my Proxmox managment console like https://10.0.78.2:8006. When that node is down, I would access the next one like https://10.0.78.3:8006, and so on…
Wouldn’t it be nice if you could just hit an url like https://pmx.internal.net and automatically get redirected to an available node in the cluster?
I came across HAProxy that made this possible.
HAProxy configuration
HAProxy is a free open-source software that provides a high availability load balancer, which I found suitable for this task. It also has an official docker image , which I used to deploy it in my kubernetes cluster.
This is the configuration I used:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
global
stats timeout 30s
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
frontend web
bind *:443
mode tcp
option tcplog
default_backend pveweb
backend pveweb
mode tcp
balance source
server opx01 10.0.78.2:8006 check
server opx02 10.0.78.3:8006 check
server opx03 10.0.78.4:8006 check
server opx04 10.0.78.5:8006 check
server opx05 10.0.78.6:8006 check
frontend spice
bind *:3128
mode tcp
option tcplog
default_backend pvespice
backend pvespice
mode tcp
balance source
server opx01 10.0.78.2:3128 check
server opx02 10.0.78.3:3128 check
server opx03 10.0.78.4:3128 check
server opx04 10.0.78.5:3128 check
server opx05 10.0.78.6:3128 check
This configuration would load balance both the Proxmox web interface and the SPICE console connections across the nodes in the cluster.
I did not go into getting the SSL certificates right, so keep in mind that this configuration would cause certificate warnings in browsers.
Deploying HAProxy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
apiVersion : v1
kind : ConfigMap
metadata :
name : pmx-proxy
namespace : network
data :
haproxy.cfg : |
global
stats timeout 30s
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
frontend web
bind *:443
mode tcp
option tcplog
default_backend pveweb
backend pveweb
mode tcp
balance source
server opx01 10.0.78.2:8006 check
server opx02 10.0.78.3:8006 check
server opx03 10.0.78.4:8006 check
server opx04 10.0.78.5:8006 check
server opx05 10.0.78.6:8006 check
frontend spice
bind *:3128
mode tcp
option tcplog
default_backend pvespice
backend pvespice
mode tcp
balance source
server opx01 10.0.78.2:3128 check
server opx02 10.0.78.3:3128 check
server opx03 10.0.78.4:3128 check
server opx04 10.0.78.5:3128 check
server opx05 10.0.78.6:3128 check
---
apiVersion : apps/v1
kind : Deployment
metadata :
name : pmx-proxy
namespace : network
labels :
app : pmx-proxy
spec :
replicas : 1
selector :
matchLabels :
app : pmx-proxy
template :
metadata :
labels :
app : pmx-proxy
spec :
containers :
- name : pmx-proxy
image : haproxy:bookworm
ports :
- containerPort : 80
- containerPort : 443
volumeMounts :
- name : config-volume
mountPath : /usr/local/etc/haproxy/haproxy.cfg
subPath : haproxy.cfg
volumes :
- name : config-volume
configMap :
name : pmx-proxy
---
apiVersion : v1
kind : Service
metadata :
name : pmx-proxy
namespace : network
spec :
type : LoadBalancer
loadBalancerIP : 10.0.78.252
ports :
- name : http
protocol : TCP
port : 80
targetPort : 80
- name : https
protocol : TCP
port : 443
targetPort : 443
selector :
app : pmx-proxy
Exposing the service
Since this is a service for internal use only, I exposed it in my internal network, which is set up here: Setting up an internal network and DNS with Kubernetes, traefik, and bind9
Simply adding an A record in my DNS zone config (in my case, for domain junyi.me) did the trick:
Now the Proxmox cluster can be accessed at https://pmx.i.junyi.me from my internal network.
