KVM 위 Kubernetes 클러스터 외부 접근 구성.

WiFi 환경이라 VM에 외부 IP를 직접 붙이기 어려움.

OpenVPN + MetalLB + Ingress 조합으로 처리.

구성.

호스트 서버
WiFi 192.168.200.100
virbr0 192.168.122.1/24

KVM VM.

k8s-master
192.168.122.10

k8s-worker1
192.168.122.11

k8s-worker2
192.168.122.12

OpenVPN.

VPN 대역:
192.168.255.0/24

K8s VM용 OpenVPN 클라이언트 발급.

bash scripts/create-client.sh k8s-master
bash scripts/create-client.sh k8s-worker1
bash scripts/create-client.sh k8s-worker2

CCD 고정 IP.

# /etc/openvpn/ccd/k8s-master
ifconfig-push 192.168.255.10 255.255.255.0

# /etc/openvpn/ccd/k8s-worker1
ifconfig-push 192.168.255.11 255.255.255.0

# /etc/openvpn/ccd/k8s-worker2
ifconfig-push 192.168.255.12 255.255.255.0

openvpn.conf 추가.

client-config-dir /etc/openvpn/ccd
topology subnet

VM에 OpenVPN 설치.

sudo apt-get install -y openvpn
sudo cp k8s-master.ovpn /etc/openvpn/client/k8s-master.conf
sudo systemctl enable openvpn-client@k8s-master
sudo systemctl start openvpn-client@k8s-master

노드 IP 확인.

kubectl get nodes -o wide

INTERNAL-IP.

k8s-master
192.168.255.10

k8s-worker1
192.168.255.11

k8s-worker2
192.168.255.12

MetalLB 설치.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.9/config/manifests/metallb-native.yaml

MetalLB IP pool.

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: vpn-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.255.100-192.168.255.150
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: vpn-l2advert
  namespace: metallb-system
spec:
  ipAddressPools:
  - vpn-pool
  interfaces:
  - tun0

tun 인터페이스는 ARP가 안 돼서 MetalLB L2만으로는 안 됨.

OpenVPN iroute 추가.

# /etc/openvpn/ccd/k8s-master
ifconfig-push 192.168.255.10 255.255.255.0
iroute 192.168.255.96 255.255.255.224

openvpn.conf route 추가.

route 192.168.255.96 255.255.255.224

Ingress Controller Service를 LoadBalancer로 변경.

kubectl patch svc ingress-nginx-controller -n ingress-nginx \
  -p '{"spec":{"type":"LoadBalancer"}}'

서비스 배포.

apiVersion: v1
kind: Service
metadata:
  name: nginx-demo-svc
spec:
  selector:
    app: nginx-demo
  ports:
  - port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: apache-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: apache-demo
  template:
    metadata:
      labels:
        app: apache-demo
    spec:
      containers:
      - name: apache
        image: httpd:alpine
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: apache-demo-svc
spec:
  selector:
    app: apache-demo
  ports:
  - port: 80

nginx-demo Deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-demo
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80

Ingress 규칙.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-demo-svc
            port:
              number: 80
      - path: /apache
        pathType: Prefix
        backend:
          service:
            name: apache-demo-svc
            port:
              number: 80

상태 확인.

kubectl get pods -A | grep -E "apache|nginx"

kubectl get ingress

테스트.

curl -v http://192.168.255.100/

curl -v http://192.168.255.100/apache

결과.

GET /
nginx-demo

GET /apache
apache-demo

최종 구조.

VPN 클라이언트
-> 192.168.255.100
-> Ingress Controller
-> / nginx-demo
-> /apache apache-demo