We will install Elasticsearch and Kibana as well as set up basic security for the Elastic Stack plus secured HTTPS traffic.
Pre-requisites
We are using our Kubernetes homelab in this article.
Configuration files used in this article can be found on GitHub. Clone the following repository:
$ git clone https://github.com/lisenet/kubernetes-homelab.git $ cd ./kubernetes-homelab/logging/
The Plan
- Install Helm.
- Create an internal Certificate Authority (CA).
- Create a wildcard certificate for Elasticsearch signed by the CA.
- Install Elasticsearch 7.17 using Helm.
- Install Kibana 7.17 using Helm.
Install Helm
On a Debian-based OS, do the following:
$ curl https://baltocdn.com/helm/signing.asc | sudo apt-key add - $ sudo apt-get install -y apt-transport-https $ echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list $ sudo apt-get update $ sudo apt-get install -y helm
Add Helm repository:
$ helm repo add elastic https://helm.elastic.co
Create Internal Certificate Authority (CA)
This section covers steps required to create a Root CA. Note that we have done this for the homelab environment here.
Generate a Root CA that is valid for 10 years:
$ openssl req -newkey rsa:2048 -keyout homelab-ca.key -nodes -x509 -days 3650 -out homelab-ca.crt
Verify X509v3 extensions:
$ openssl x509 -text -noout -in homelab-ca.crt | grep CA
CA:TRUE
Create a wildcard certificate signed by the Root CA to be used with Elasticsearch and Kibana:
$ DOMAIN="wildcard.hl.test"
$ openssl genrsa -out "${DOMAIN}".key 2048 && chmod 0600 "${DOMAIN}".key
Generate a Certificate Sign Request (CSR):
$ openssl req -new -sha256 -key "${DOMAIN}".key -out "${DOMAIN}".csr
Sign the request with the Root CA:
$ openssl x509 -req -in "${DOMAIN}".csr -CA homelab-ca.crt -CAkey homelab-ca.key -CAcreateserial -out "${DOMAIN}".crt -days 1825 -sha256
Optional: import the Root CA in to your browser.
Install Elasticsearch on Kubernetes
Create logging namespace:
$ kubectl create namespace logging
Create a secret to store Elasticsearch credentials:
$ kubectl apply -f ./elastic-credentials-secret.yml
Create a secret to store Elasticsearch SSL certificates. We are using the Root CA to sign the certificate.
$ kubectl apply -f ./elastic-certificates-secret.yml
By default, the Elasticsearch security features are disabled when we have a basic license. To enable security features, we will use the xpack.security.enabled setting.
In order to enable TLS/SSL on the HTTP networking layer, which Elasticsearch uses to communicate with other clients, we will use the xpack.security.http.ssl.enabled setting.
Create a values file values-elasticsearch.yml for Elasticsearch:
---
clusterName: "elasticsearch"
nodeGroup: "master"
roles:
master: "true"
ingest: "true"
data: "true"
remote_cluster_client: "true"
ml: "true"
replicas: 1
minimumMasterNodes: 1
protocol: https
httpPort: 9200
imagePullPolicy: "IfNotPresent"
extraEnvs:
- name: "ELASTIC_PASSWORD"
valueFrom:
secretKeyRef:
name: "elastic-credentials"
key: "password"
- name: "ELASTIC_USERNAME"
valueFrom:
secretKeyRef:
name: "elastic-credentials"
key: "username"
esConfig:
elasticsearch.yml: |
xpack.security.enabled: "true"
xpack.security.transport.ssl.enabled: "true"
xpack.security.transport.ssl.supported_protocols: "TLSv1.2"
xpack.security.transport.ssl.client_authentication: "none"
xpack.security.transport.ssl.key: "/usr/share/elasticsearch/config/certs/tls.key"
xpack.security.transport.ssl.certificate: "/usr/share/elasticsearch/config/certs/tls.crt"
xpack.security.transport.ssl.certificate_authorities: "/usr/share/elasticsearch/config/certs/homelab-ca.crt"
xpack.security.transport.ssl.verification_mode: "certificate"
xpack.security.http.ssl.enabled: "true"
xpack.security.http.ssl.supported_protocols: "TLSv1.2"
xpack.security.http.ssl.client_authentication: "none"
xpack.security.http.ssl.key: "/usr/share/elasticsearch/config/certs/tls.key"
xpack.security.http.ssl.certificate: "/usr/share/elasticsearch/config/certs/tls.crt"
xpack.security.http.ssl.certificate_authorities: "/usr/share/elasticsearch/config/certs/homelab-ca.crt"
secretMounts:
- name: "elastic-certificates"
secretName: "elastic-certificates"
path: "/usr/share/elasticsearch/config/certs"
defaultMode: "0755"
resources:
requests:
cpu: "250m"
memory: "2Gi"
limits:
cpu: "1000m"
memory: "4Gi"
volumeClaimTemplate:
accessModes: ["ReadWriteOnce"]
storageClassName: "freenas-nfs-csi"
resources:
requests:
storage: 64Gi
service:
enabled: true
labels: {}
labelsHeadless: {}
type: LoadBalancer
nodePort: ""
annotations: {}
httpPortName: https
transportPortName: transport
loadBalancerIP: "10.11.1.59"
loadBalancerSourceRanges: []
externalTrafficPolicy: ""
clusterHealthCheckParams: "wait_for_status=yellow&timeout=2s"
Deploy a single node Elasticsearch with authentication, certificates for TLS and custom values:
$ helm upgrade --install elasticsearch \ elastic/elasticsearch \ --namespace logging \ --version "7.17.1" \ --values ./values-elasticsearch.yml
Elasticsearch endpoint will be available at https://10.11.1.59:9200/.
You can test it by using curl:
$ curl -sk -u "username:password" https://10.11.1.59:9200/ | jq
{
"name": "elasticsearch-master-0",
"cluster_name": "elasticsearch",
"cluster_uuid": "t6rPuP6NSn6IDaW98J0VWw",
"version": {
"number": "7.17.1",
"build_flavor": "default",
"build_type": "docker",
"build_hash": "e5acb99f822233d62d6444ce45a4543dc1c8059a",
"build_date": "2022-02-23T22:20:54.153567231Z",
"build_snapshot": false,
"lucene_version": "8.11.1",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"
}
Install Kibana on Kubernetes
Create a values file values-kibana.yml for Kibana:
---
elasticsearchHosts: "https://elasticsearch-master:9200"
replicas: 1
protocol: https
httpPort: 5601
imagePullPolicy: "IfNotPresent"
extraEnvs:
- name: "NODE_OPTIONS"
value: "--max-old-space-size=1800"
- name: "ELASTICSEARCH_USERNAME"
valueFrom:
secretKeyRef:
name: "elastic-credentials"
key: "username"
- name: "ELASTICSEARCH_PASSWORD"
valueFrom:
secretKeyRef:
name: "elastic-credentials"
key: "password"
kibanaConfig:
kibana.yml: |
server.ssl:
enabled: "true"
key: "/usr/share/kibana/config/certs/tls.key"
certificate: "/usr/share/kibana/config/certs/tls.crt"
certificateAuthorities: [ "/usr/share/kibana/config/certs/homelab-ca.crt" ]
clientAuthentication: "none"
supportedProtocols: [ "TLSv1.2", "TLSv1.3" ]
elasticsearch.ssl:
certificateAuthorities: [ "/usr/share/kibana/config/certs/homelab-ca.crt" ]
verificationMode: "certificate"
newsfeed.enabled: "false"
telemetry.enabled: "false"
telemetry.optIn: "false"
secretMounts:
- name: "elastic-certificates"
secretName: "elastic-certificates"
path: "/usr/share/kibana/config/certs"
defaultMode: "0755"
resources:
requests:
cpu: "55m"
memory: "512Mi"
limits:
cpu: "1000m"
memory: "2Gi"
service:
type: LoadBalancer
loadBalancerIP: "10.11.1.58"
port: 5601
nodePort: ""
labels: {}
annotations: {}
loadBalancerSourceRanges: []
httpPortName: http
Deploy Kibana using authentication and TLS to connect to Elasticsearch:
$ helm upgrade --install kibana \ elastic/kibana \ --namespace logging \ --version "7.17.1" \ --values ./values-kibana.yml
Kibana endpoint will be available at https://10.11.1.58:5601/.
Verify that pods are running:
$ kubectl get po -n logging NAME READY STATUS RESTARTS AGE elasticsearch-master-0 1/1 Running 0 23h kibana-kibana-5d8dc78bfb-4fqr2 1/1 Running 0 23h
Verify services:
$ kubectl get svc -n logging NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elasticsearch-master LoadBalancer 10.105.182.194 10.11.1.59 9200:31657/TCP,9300:32405/TCP 3d22h elasticsearch-master-headless ClusterIP None none 9200/TCP,9300/TCP 3d22h kibana-kibana LoadBalancer 10.105.176.223 10.11.1.58 5601:31251/TCP 3d21h
References
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/configuring-stack-security.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-settings.html
Brilliant, thank you for posting this!