We will install Elasticsearch and Kibana as well as set up basic security for the Elastic Stack plus secured HTTPS traffic.
Pre-requisites
We are using our Kubernetes homelab in this article.
Configuration files used in this article can be found on GitHub. Clone the following repository:
$ git clone https://github.com/lisenet/kubernetes-homelab.git $ cd ./kubernetes-homelab/logging/
The Plan
- Install Helm.
- Create an internal Certificate Authority (CA).
- Create a wildcard certificate for Elasticsearch signed by the CA.
- Install Elasticsearch 7.17 using Helm.
- Install Kibana 7.17 using Helm.
Install Helm
On a Debian-based OS, do the following:
$ curl https://baltocdn.com/helm/signing.asc | sudo apt-key add - $ sudo apt-get install -y apt-transport-https $ echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list $ sudo apt-get update $ sudo apt-get install -y helm
Add Helm repository:
$ helm repo add elastic https://helm.elastic.co
Create Internal Certificate Authority (CA)
This section covers steps required to create a Root CA. Note that we have done this for the homelab environment here.
Generate a Root CA that is valid for 10 years:
$ openssl req -newkey rsa:2048 -keyout homelab-ca.key -nodes -x509 -days 3650 -out homelab-ca.crt
Verify X509v3 extensions:
$ openssl x509 -text -noout -in homelab-ca.crt | grep CA CA:TRUE
Create a wildcard certificate signed by the Root CA to be used with Elasticsearch and Kibana:
$ DOMAIN="wildcard.hl.test" $ openssl genrsa -out "${DOMAIN}".key 2048 && chmod 0600 "${DOMAIN}".key
Generate a Certificate Sign Request (CSR):
$ openssl req -new -sha256 -key "${DOMAIN}".key -out "${DOMAIN}".csr
Sign the request with the Root CA:
$ openssl x509 -req -in "${DOMAIN}".csr -CA homelab-ca.crt -CAkey homelab-ca.key -CAcreateserial -out "${DOMAIN}".crt -days 1825 -sha256
Optional: import the Root CA in to your browser.
Install Elasticsearch on Kubernetes
Create logging namespace:
$ kubectl create namespace logging
Create a secret to store Elasticsearch credentials:
$ kubectl apply -f ./elastic-credentials-secret.yml
Create a secret to store Elasticsearch SSL certificates. We are using the Root CA to sign the certificate.
$ kubectl apply -f ./elastic-certificates-secret.yml
By default, the Elasticsearch security features are disabled when we have a basic license. To enable security features, we will use the xpack.security.enabled
setting.
In order to enable TLS/SSL on the HTTP networking layer, which Elasticsearch uses to communicate with other clients, we will use the xpack.security.http.ssl.enabled
setting.
Create a values file values-elasticsearch.yml
for Elasticsearch:
--- clusterName: "elasticsearch" nodeGroup: "master" roles: master: "true" ingest: "true" data: "true" remote_cluster_client: "true" ml: "true" replicas: 1 minimumMasterNodes: 1 protocol: https httpPort: 9200 imagePullPolicy: "IfNotPresent" extraEnvs: - name: "ELASTIC_PASSWORD" valueFrom: secretKeyRef: name: "elastic-credentials" key: "password" - name: "ELASTIC_USERNAME" valueFrom: secretKeyRef: name: "elastic-credentials" key: "username" esConfig: elasticsearch.yml: | xpack.security.enabled: "true" xpack.security.transport.ssl.enabled: "true" xpack.security.transport.ssl.supported_protocols: "TLSv1.2" xpack.security.transport.ssl.client_authentication: "none" xpack.security.transport.ssl.key: "/usr/share/elasticsearch/config/certs/tls.key" xpack.security.transport.ssl.certificate: "/usr/share/elasticsearch/config/certs/tls.crt" xpack.security.transport.ssl.certificate_authorities: "/usr/share/elasticsearch/config/certs/homelab-ca.crt" xpack.security.transport.ssl.verification_mode: "certificate" xpack.security.http.ssl.enabled: "true" xpack.security.http.ssl.supported_protocols: "TLSv1.2" xpack.security.http.ssl.client_authentication: "none" xpack.security.http.ssl.key: "/usr/share/elasticsearch/config/certs/tls.key" xpack.security.http.ssl.certificate: "/usr/share/elasticsearch/config/certs/tls.crt" xpack.security.http.ssl.certificate_authorities: "/usr/share/elasticsearch/config/certs/homelab-ca.crt" secretMounts: - name: "elastic-certificates" secretName: "elastic-certificates" path: "/usr/share/elasticsearch/config/certs" defaultMode: "0755" resources: requests: cpu: "250m" memory: "2Gi" limits: cpu: "1000m" memory: "4Gi" volumeClaimTemplate: accessModes: ["ReadWriteOnce"] storageClassName: "freenas-nfs-csi" resources: requests: storage: 64Gi service: enabled: true labels: {} labelsHeadless: {} type: LoadBalancer nodePort: "" annotations: {} httpPortName: https transportPortName: transport loadBalancerIP: "10.11.1.59" loadBalancerSourceRanges: [] externalTrafficPolicy: "" clusterHealthCheckParams: "wait_for_status=yellow&timeout=2s"
Deploy a single node Elasticsearch with authentication, certificates for TLS and custom values:
$ helm upgrade --install elasticsearch \ elastic/elasticsearch \ --namespace logging \ --version "7.17.1" \ --values ./values-elasticsearch.yml
Elasticsearch endpoint will be available at https://10.11.1.59:9200/.
You can test it by using curl:
$ curl -sk -u "username:password" https://10.11.1.59:9200/ | jq { "name": "elasticsearch-master-0", "cluster_name": "elasticsearch", "cluster_uuid": "t6rPuP6NSn6IDaW98J0VWw", "version": { "number": "7.17.1", "build_flavor": "default", "build_type": "docker", "build_hash": "e5acb99f822233d62d6444ce45a4543dc1c8059a", "build_date": "2022-02-23T22:20:54.153567231Z", "build_snapshot": false, "lucene_version": "8.11.1", "minimum_wire_compatibility_version": "6.8.0", "minimum_index_compatibility_version": "6.0.0-beta1" }, "tagline": "You Know, for Search" }
Install Kibana on Kubernetes
Create a values file values-kibana.yml
for Kibana:
--- elasticsearchHosts: "https://elasticsearch-master:9200" replicas: 1 protocol: https httpPort: 5601 imagePullPolicy: "IfNotPresent" extraEnvs: - name: "NODE_OPTIONS" value: "--max-old-space-size=1800" - name: "ELASTICSEARCH_USERNAME" valueFrom: secretKeyRef: name: "elastic-credentials" key: "username" - name: "ELASTICSEARCH_PASSWORD" valueFrom: secretKeyRef: name: "elastic-credentials" key: "password" kibanaConfig: kibana.yml: | server.ssl: enabled: "true" key: "/usr/share/kibana/config/certs/tls.key" certificate: "/usr/share/kibana/config/certs/tls.crt" certificateAuthorities: [ "/usr/share/kibana/config/certs/homelab-ca.crt" ] clientAuthentication: "none" supportedProtocols: [ "TLSv1.2", "TLSv1.3" ] elasticsearch.ssl: certificateAuthorities: [ "/usr/share/kibana/config/certs/homelab-ca.crt" ] verificationMode: "certificate" newsfeed.enabled: "false" telemetry.enabled: "false" telemetry.optIn: "false" secretMounts: - name: "elastic-certificates" secretName: "elastic-certificates" path: "/usr/share/kibana/config/certs" defaultMode: "0755" resources: requests: cpu: "55m" memory: "512Mi" limits: cpu: "1000m" memory: "2Gi" service: type: LoadBalancer loadBalancerIP: "10.11.1.58" port: 5601 nodePort: "" labels: {} annotations: {} loadBalancerSourceRanges: [] httpPortName: http
Deploy Kibana using authentication and TLS to connect to Elasticsearch:
$ helm upgrade --install kibana \ elastic/kibana \ --namespace logging \ --version "7.17.1" \ --values ./values-kibana.yml
Kibana endpoint will be available at https://10.11.1.58:5601/.
Verify that pods are running:
$ kubectl get po -n logging NAME READY STATUS RESTARTS AGE elasticsearch-master-0 1/1 Running 0 23h kibana-kibana-5d8dc78bfb-4fqr2 1/1 Running 0 23h
Verify services:
$ kubectl get svc -n logging NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elasticsearch-master LoadBalancer 10.105.182.194 10.11.1.59 9200:31657/TCP,9300:32405/TCP 3d22h elasticsearch-master-headless ClusterIP None none 9200/TCP,9300/TCP 3d22h kibana-kibana LoadBalancer 10.105.176.223 10.11.1.58 5601:31251/TCP 3d21h
References
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/configuring-stack-security.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-settings.html
Brilliant, thank you for posting this!