Bài viết này là một phần của chùm bài viết: Private cloud OpenShift.
Một số từ viết tắt:
| OKD | OpenShift Community [Origin] |
| LB | Load Balancing |
| GW | Gateway |
| NFS | Network File System |
| DNS | Domain Name System |
| FCOS 33 | Fedora CoreOS 33 |
| CentOS 8 | Operation System |
Để đọc bài viết này hiệu quả, yêu cầu có kiến thức về: Linux, Webserver, Load Balancing, DNS, ssh key, Docker
Sơ qua về tiến trình cài đặt: Boostrap sẽ cài đặt cho các Master. Master sẽ cài đặt cho các Worker.
Nên nhớ Boostrap chỉ dùng để cài đặt cho các Master. Xong việc nó sẽ tự động hủy.
Mô hình sẽ cài đặt: 3 node Master và 2 node Worker.
Danh sách các server cần phải chuẩn bị: [Giả sử môi trường đang thực hiện có lớp mạng 192.168.99.x, có gateway 192.168.99.1]
| okd4-services | DNS / Web / admin | CentOS 8 | 192.168.99.51 | 16G RAM, 4 core cpu, 200G HDD |
| okd4-lb | LB | CentOS 8 | 192.168.99.52 | 8G RAM, 2 core cpu, 120G HDD |
| okd4-nfs | NFS | CentOS 8 | 192.168.99.53 | 24G RAM, 4 core cpu, 120G HDD [OS], 2T HDD data |
| okd4-bootstrap | Bootstrap | FCOS 33 | 192.168.99.60 | 16G RAM, 4 core cpu, 200G HDD |
| okd4-control-plane-1 | Master | FCOS 33 | 192.168.99.61 | 16G RAM, 4 core cpu, 200G HDD |
| okd4-control-plane-2 | Master | FCOS 33 | 192.168.99.62 | 16G RAM, 4 core cpu, 200G HDD |
| okd4-control-plane-3 | Master | FCOS 33 | 192.168.99.63 | 16G RAM, 4 core cpu, 200G HDD |
| okd4-compute-1 | Worker | FCOS 33 | 192.168.99.64 | 128G RAM, 40 core cpu, 300G HDD |
| okd4-compute-2 | Worker | FCOS 33 | 192.168.99.65 | 128G RAM, 40 core cpu, 300G HDD |
Version các phần mềm
| okd | Openshift Community | 4.6.0-0.okd-2021-02-14-205305 |
| oc | Openshitf Client | 4.6.0-0.okd-2021-02-14-205305 |
| Fedora CoreOS 33 [FCOS 33] | Operation System: FCOS 33 | fedora-coreos-33.20210201.3.0-live.x86_64 |
| Kubernetes | kubernetes | 1.19.4 |
| CentOS 8 | Operation System: CentOS 8 | CentOS-8.3.2011-x86_64 |
Lưu ý: version của phần mềm sẽ thích hợp với nhau. Ví dụ okd4.6 sẽ đi với FCOS33
Bài viết này cài đặt okd 4.6 mà không phải version cuối cùng với mục đích là để trải nghiệm nguyên hệ cluster nó upgrade như thế nào.
Thông tin khái quát ở trên đã xong, tiến hành cài đặt thôi nào!!!
I/ Install okd4-services
1. Tiền xử lý
Cài hệ điều hành CentOS-8.3.2011-x86_64. Đặt ip là 192.168.99.51 và để tạm
DNS:8.8.8.8
Cài dịch vụ để đồng bộ ngày giờ: chrony
2. Chuẩn bị
Tạo ssh-key$ ssh-keygen -t rsa
Tạo sẵn các file cấu hìnhGenerating public/private rsa key pair.
Enter file in which to save the key [/root/.ssh/idrsa]:
Enter passphrase [empty for no passphrase]:
Enter same passphrase again:
Your identification has been saved in /root/.ssh/idrsa.
Your public key has been saved in /root/.ssh/idrsa.pub.
Tạo thư mục okd4_files
$ mkdir /root/okd4_files
Tạo các file cho cấu hình DNS:
+Tạo file /root/okd4_files/named.conf với nội dung bên dưới:
//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named[8] DNS
// server as a caching only nameserver [as a localhost DNS resolver only].
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
// See the BIND Administrator's Reference Manual [ARM] for details about the
// configuration located in /usr/share/doc/bind-{version}/Bv9ARM.html
options {
listen-on port 53 { 127.0.0.1; 192.168.99.51; };
# listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
recursing-file "/var/named/data/named.recursing";
secroots-file "/var/named/data/named.secroots";
allow-query { localhost; 192.168.99.0/24; };
/*
- If you are building an AUTHORITATIVE DNS server, do NOT enable recursion.
- If you are building a RECURSIVE [caching] DNS server, you need to enable
recursion.
- If your recursive DNS server has a public IP address, you MUST enable access
control to limit queries to your legitimate users. Failing to do so will
cause your server to become part of large scale DNS amplification
attacks. Implementing BCP38 within your network would greatly
reduce such attack surface
*/
recursion yes;
forwarders {
8.8.8.8;
8.8.4.4;
};
dnssec-enable yes;
dnssec-validation yes;
/* Path to ISC DLV key */
bindkeys-file "/etc/named.root.key";
managed-keys-directory "/var/named/dynamic";
pid-file "/run/named/named.pid";
session-keyfile "/run/named/session.key";
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
zone "." IN {
type hint;
file "named.ca";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
include "/etc/named/named.conf.local";
+Tạo file /root/okd4_files/named.conf.local với nội dung bên dưới:
zone "okd.local" {
type master;
file "/etc/named/zones/db.okd.local"; # zone file path
};
zone "99.168.192.in-addr.arpa" {
type master;
file "/etc/named/zones/db.192.168.99"; # 192.168.99/24 subnet
};
+Tạo file /root/okd4_files/db.okd.local với nội dung bên dưới:
$TTL 604800
@ IN SOA okd4-services.okd.local. admin.okd.local. [
1 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ; Negative Cache TTL
]
; name servers - NS records
IN NS okd4-services
; name servers - A records
okd4-services.okd.local. IN A 192.168.99.51
okd4-lb.okd.local. IN A 192.168.99.52
okd4-nfs.okd.local. IN A 192.168.99.53
; OpenShift Container Platform Cluster - A records
okd4-bootstrap.cloud.okd.local. IN A 192.168.99.60
okd4-control-plane-1.cloud.okd.local. IN A 192.168.99.61
okd4-control-plane-2.cloud.okd.local. IN A 192.168.99.62
okd4-control-plane-3.cloud.okd.local. IN A 192.168.99.63
okd4-compute-1.cloud.okd.local. IN A 192.168.99.64
okd4-compute-2.cloud.okd.local. IN A 192.168.99.65
; OpenShift internal cluster IPs - A records
api.cloud.okd.local. IN A 192.168.99.52
api-int.cloud.okd.local. IN A 192.168.99.52
*.apps.cloud.okd.local. IN A 192.168.99.52
etcd-0.cloud.okd.local. IN A 192.168.99.61
etcd-1.cloud.okd.local. IN A 192.168.99.62
etcd-2.cloud.okd.local. IN A 192.168.99.63
console-openshift-console.apps.cloud.okd.local. IN A 192.168.99.52
oauth-openshift.apps.cloud.okd.local. IN A 192.168.99.52
; OpenShift internal cluster IPs - SRV records
_etcd-server-ssl._tcp.cloud.okd.local. 86400 IN SRV 0 10 2380 etcd-0.cloud
_etcd-server-ssl._tcp.cloud.okd.local. 86400 IN SRV 0 10 2380 etcd-1.cloud
_etcd-server-ssl._tcp.cloud.okd.local. 86400 IN SRV 0 10 2380 etcd-2.cloud
+Tạo file /root/okd4_files/db.192.168.99 với nội dung bên dưới:
$TTL 604800
@ IN SOA okd4-services.okd.local. admin.okd.local. [
6 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ; Negative Cache TTL
]
; name servers - NS records
IN NS okd4-services.okd.local.
; name servers - PTR records
51 IN PTR okd4-services.okd.local.
52 IN PTR okd4-lb.okd.local.
53 IN PTR okd4-nfs.okd.local.
; OpenShift Container Platform Cluster - PTR records
60 IN PTR okd4-bootstrap.cloud.okd.local.
61 IN PTR okd4-control-plane-1.cloud.okd.local.
62 IN PTR okd4-control-plane-2.cloud.okd.local.
63 IN PTR okd4-control-plane-3.cloud.okd.local.
64 IN PTR okd4-compute-1.cloud.okd.local.
65 IN PTR okd4-compute-2.cloud.okd.local.
52 IN PTR api.cloud.okd.local.
52 IN PTR api-int.cloud.okd.local.
3. Cài đặt DNS
$ sudo dnf -y install bind bind-utils
cd /root/okd4_files
sudo cp named.conf /etc/named.conf
sudo cp named.conf.local /etc/named/
sudo mkdir /etc/named/zones
sudo cp db.okd.local /etc/named/zones
sudo cp db.192.168.100 /etc/named/zones
sudo systemctl enable named
sudo systemctl stop named
sudo systemctl start named
sudo systemctl status named
sudo firewall-cmd --permanent --add-port=53/udp
sudo firewall-cmd --reload
sudo nmcli connection modify ens192 ipv4.dns "127.0.0.1"
$ dig okd.local
; DiG 9.11.20-RedHat-9.11.20-5.el8 okd.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADERHEADER