Overview of third-party addons for EKS (Teleport)
- Oleksii Bebych
- Mar 7, 2023
- 8 min read
In the previous post we checked several EKS addons (Kubecost, Dynarace and Istio), but we still have others. In this post we will look at Teleport. Teleport is an identity-aware, multi-protocol access proxy which understands SSH, HTTPS, RDP, Kubernetes API, MySQL, MongoDB and PostgreSQL wire protocols.
Add-on installation
Teleport can be installed into your EKS clusted as add-on.
Once the add-on is installed, you need to find a load balancer, that was created:
$ kubectl get services -n teleport
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
teleport LoadBalancer 172.20.41.184 ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com 443:31946/TCP,3023:32150/TCP,3026:32689/TCP,3024:31432/TCP,3036:31808/TCP 44sThen you need to add the load balancer endpoint to the ConfigMap "teleport":
$ kubectl edit cm -n teleport teleport
apiVersion: v1
data:
teleport.yaml: |
teleport:
log:
severity: INFO
output: stderr
format:
output: text
extra_fields: ["timestamp","level","component","caller"]
auth_service:
enabled: true
cluster_name: teleport.teleport.svc.cluster.local
authentication:
type: "local"
local_auth: true
second_factor: "otp"
proxy_service:
public_addr: ['ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com', 'teleport.teleport.svc.cluster.local:443', '127.0.0.1:3080']
kube_listen_addr: 0.0.0.0:3026
mysql_listen_addr: 0.0.0.0:3036
enabled: true
ssh_service:
enabled: false
kind: ConfigMap
metadata:
creationTimestamp: "2023-02-27T12:10:47Z"
name: teleport
namespace: teleport
resourceVersion: "17380"
uid: 2da3f3b3-297f-45a8-b8b7-8d155d23ba76Recreate all Teleport pods:
$ kubectl delete pods -n teleport --all
pod "teleport-b99977958-6hwdw" deleted
$ k get po -n teleport -w
NAME READY STATUS RESTARTS AGE
teleport-b99977958-tkrxk 1/1 Running 0 7sCreate a new "admin" user. You will recieve a link for setting up the password and MFA (required).
$ kubectl exec -n teleport deployment/teleport -- tctl users add admin --roles=editor,access,auditor
User "admin" has been created but requires a password. Share this URL with the user to complete user setup, link is valid for 1h:
https://ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com:3080/web/invite/f9ba7c6013e29276700140fc0f042875
Server Access
Teleport Server Access consolidates SSH access across all environments, decreases configuration complexity, supports industry best practices and compliance while giving complete visibility over all sessions and events.
Teleport Server Access is designed for the following kinds of scenarios:
When up to a vast number of clusters must be managed using the command-line (tsh) or programmatically (through the Teleport API) and you want to simplify your stack, security, and configuration complexity.
When security team members must track and audit every user session.
When Teleport users require a complete, dedicated, and secure SSH option (Teleport Node running in SSH mode) and more than a certificate authority (Teleport Auth) with proxy (Teleport Proxy).
When resource and network security must be maximized: SSH certificates over secret keys, Two-Factor Authentication (2FA), Single Sign-On (SSO), and short-lived certificates.
Here is an example of manual adding a new VM. I chose Amazon Linux 2 EC2 instance, but Teleport supports various OSs:
A bootstrap script is generated with a short-term access token:
Script installs required packages and configure them:
sudo bash -c "$(curl -kfsSL https://ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com/scripts/3bd4fa2bd8b232302aa465f51ddee489/install-node.sh)"
2023-02-27 13:09:57 UTC [teleport-installer] TELEPORT_VERSION: 10.3.1
2023-02-27 13:09:57 UTC [teleport-installer] TARGET_HOSTNAME: ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com
2023-02-27 13:09:57 UTC [teleport-installer] TARGET_PORT: 3080
2023-02-27 13:09:57 UTC [teleport-installer] JOIN_TOKEN: 3bd4fa2bd8b232302aa465f51ddee489
2023-02-27 13:09:57 UTC [teleport-installer] CA_PIN_HASHES: sha256:47525f0632e6382036180cbeb9bd975383d3ca58fbb8f7f3f69023d2996c64d7
2023-02-27 13:09:57 UTC [teleport-installer] Checking TCP connectivity to Teleport server (ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com:3080)
2023-02-27 13:09:57 UTC [teleport-installer] Couldn't find nc, telnet or /dev/tcp to do a connection test
2023-02-27 13:09:57 UTC [teleport-installer] Going to blindly continue without testing connectivity
2023-02-27 13:09:57 UTC [teleport-installer] Detected host: linux-gnu, using Teleport binary type linux
2023-02-27 13:09:57 UTC [teleport-installer] Detected arch: x86_64, using Teleport arch amd64
2023-02-27 13:09:57 UTC [teleport-installer] Detected distro type: "centos rhel fedora"
2023-02-27 13:09:57 UTC [teleport-installer] Using Teleport distribution: rpm
2023-02-27 13:09:57 UTC [teleport-installer] Created temp dir /tmp/teleport-1ITr6q3fQS
2023-02-27 13:09:57 UTC [teleport-installer] Found 'yum' package manager, using it
2023-02-27 13:09:57 UTC [teleport-installer] Downloading Teleport rpm release 10.3.1
2023-02-27 13:09:57 UTC [teleport-installer] Running curl -fsSL --retry 5 --retry-delay 5 https://get.gravitational.com/teleport-10.3.1-1.x86_64.rpm
2023-02-27 13:09:57 UTC [teleport-installer] Downloading to /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm
2023-02-27 13:10:07 UTC [teleport-installer] Downloaded file size: 112045092 bytes
2023-02-27 13:10:07 UTC [teleport-installer] Will use sha256sum to validate the checksum of the downloaded file
2023-02-27 13:10:08 UTC [teleport-installer] The downloaded file's checksum validated correctly
2023-02-27 13:10:08 UTC [teleport-installer] Installing Teleport release from /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm using yum -y localinstall
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Examining /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm: teleport-10.3.1-1.x86_64
Marking /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package teleport.x86_64 0:10.3.1-1 will be installed
--> Finished Dependency Resolution
amzn2-core/2/x86_64 | 3.7 kB 00:00:00
Dependencies Resolved
==========================================================================
Package Arch Version Repository Size
==========================================================================
Installing:
teleport x86_64 10.3.1-1 /teleport-10.3.1-1.x86_64 370 M
Transaction Summary
==========================================================================
Install 1 Package
Total size: 370 M
Installed size: 370 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : teleport-10.3.1-1.x86_64 1/1
Verifying : teleport-10.3.1-1.x86_64 1/1
Installed:
teleport.x86_64 0:10.3.1-1
Complete!
2023-02-27 13:10:15 UTC [teleport-installer] Found: Teleport v10.3.1 git:v10.3.1-0-g2fa9454 go1.18.6
2023-02-27 13:10:15 UTC [teleport-installer] Writing Teleport node service config to /etc/teleport.yaml
Wrote config to file "/etc/teleport.yaml". Now you can start the server. Happy Teleporting!
2023-02-27 13:10:15 UTC [teleport-installer] Host is using systemd
2023-02-27 13:10:15 UTC [teleport-installer] Starting Teleport via systemd. It will automatically be started whenever the system reboots.
Created symlink from /etc/systemd/system/multi-user.target.wants/teleport.service to /usr/lib/systemd/system/teleport.service.
Teleport has been started.
View its status with 'sudo systemctl status teleport.service'
View Teleport logs using 'sudo journalctl -u teleport.service'
To stop Teleport, run 'sudo systemctl stop teleport.service'
To start Teleport again if you stop it, run 'sudo systemctl start teleport.service'
You can see this node connected in the Teleport web UI or 'tsh ls' with the name 'ip-10-0-151-153.ec2.internal'
Find more details on how to use Teleport here: https://goteleport.com/docs/user-manual/Choose the OS user that is present in the VM and will be used by Teleport:
And test a connectivity:
After that you can see the VM in the list:
You don't need to register all EC2 instances manually, there is a native capability to do it automatically.
The Teleport Discovery Service can connect to Amazon EC2 and automatically discover and enroll EC2 instances matching configured labels. It will then execute an install script on these discovered instances using AWS Systems Manager that will install Teleport, start it and join the cluster.
You can get a CLI via your web browser:
or your local terminal:
$ tctl version
Teleport v12.0.5 git:api/v12.0.5-0-g818318c go1.20.1
$ tsh version
Teleport v12.0.5 git:api/v12.0.5-0-g818318c go1.20.1
$ tsh login --proxy=teleport.ait-demo1.com --user=admin
Enter password for Teleport user admin:
Enter an OTP code from a device:
> Profile URL: https://teleport.ait-demo1.com:443
Logged in as: admin
Cluster: teleport.ait-demo1.com
Roles: access, editor
Logins: ec2-user, -teleport-internal-join
Kubernetes: enabled
Valid until: 2023-03-05 09:23:03 +0200 EET [valid for 12h0m0s]
Extensions: login-ip, permit-agent-forwarding, permit-port-forwarding, permit-pty, private-key-policy
$ tsh ls
Node Name Address Labels
-----------------------------------------------------------------------------------
ip-10-0-151-153.ec2.internal <- Tunnel hostname=ip-10-0-151-153.ec2.internal,teleport.internal/resource-id=39162ea2-7e84-4be1-a417-d416f2d3b...
$ tsh ssh ec2-user@ip-10-0-151-153.ec2.internal
[ec2-user@ip-10-0-151-153 ~]$ ls -la
total 12
drwx------ 3 ec2-user ec2-user 74 Feb 27 12:23 .
drwxr-xr-x 4 root root 38 Feb 27 13:08 ..
-rw-r--r-- 1 ec2-user ec2-user 18 Jul 15 2020 .bash_logout
-rw-r--r-- 1 ec2-user ec2-user 193 Jul 15 2020 .bash_profile
-rw-r--r-- 1 ec2-user ec2-user 231 Jul 15 2020 .bashrc
drwx------ 2 ec2-user ec2-user 29 Feb 27 12:23 .sshOther users can join the active session as "Observer", "Moderator" or "Peer"
peer: Can join and collaborate in a session. They can view output and send input.
moderator: Can join and watch a session. They can view output and forcefully terminate the session at will.
observer: Can join and watch a session. They cannot control the session in any way.
Access logs usually contain information like:
IP address or hostname of the client
Timestamps of operations or access attempts
Event metadata like “severity”
Result of an attempted operation (success/failure)
Log messaging output from the operation
It’s incredibly important to understand what types of data are being radiated and recorded by your system so that you can stay on top of your infrastructure.
Kubernetes access
Teleport provides secure access to Kubernetes clusters:
Users can access Kubernetes clusters with Single Sign-On (SSO) providers like Okta and switch between clusters without logging in twice.
Operators can implement granular role-based access controls, including limiting access to specific Kubernetes clusters or even specific pods within a namespace.
Organizations can achieve compliance by recording kubectl sessions.
I chose EKS cluster for demo. You will need to install the Teleport agent Helm chart, values file will be generated for you:
Values file contains a shotr-term auth token:
$ helm install teleport-agent teleport/teleport-kube-agent -f prod-cluster-values.yaml --create-namespace --namespace teleport
$ kubectl get po teleport-agent-0 -n teleport
NAME READY STATUS RESTARTS AGE
teleport-agent-0 1/1 Running 0 28sWhen your Kubernetes cluster is identified, you can connect:
$ tsh kube login demo1
Logged into Kubernetes cluster "demo1". Try 'kubectl version' to test the connection.
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
development loadbalancer-6df4b7468c-wzbmx 1/1 Running 0 17m
development webapp-66b94c9c64-ms4r6 1/1 Running 0 17m
kube-system aws-node-87f82 1/1 Running 0 5d9h
kube-system aws-node-vq5ft 1/1 Running 0 5d9h
kube-system coredns-79989457d9-jw88j 1/1 Running 0 5d9h
kube-system coredns-79989457d9-p5p6j 1/1 Running 0 5d9h
kube-system ebs-csi-controller-68fb47d4f5-4gmqt 6/6 Running 1 (5h42m ago) 5d9h
kube-system ebs-csi-controller-68fb47d4f5-qtm2l 6/6 Running 0 5d9h
kube-system ebs-csi-node-jkjd2 3/3 Running 0 5d9h
kube-system ebs-csi-node-zclxv 3/3 Running 0 5d9h
kube-system kube-proxy-5lsc6 1/1 Running 0 5d9h
kube-system kube-proxy-gb5fb 1/1 Running 0 5d9h
production loadbalancer-6df4b7468c-7khv6 1/1 Running 0 17m
production webapp-66b94c9c64-p5n5c 1/1 Running 0 17mEKS clusters can be registered in Teleport automatially as well EKS Auto-Discovery can automatically discover any EKS cluster and enroll it in Teleport if its tags match the configured labels.
Teleport Kubernetes Auto-Discovery involves two components.
The first, the Discovery Service, is responsible for watching your cloud provider and checking if there are any new clusters or if there have been any modifications to previously discovered clusters. The second, the Kubernetes Service, monitors the clusters created by the Discovery Service. It proxies communications between users and the API servers of these clusters.
Database Access
Teleport can provide secure connections to your databases while improving both access control and visibility.
Some of the things you can do with Database Access:
Enable users to retrieve short-lived database certificates using a Single Sign-On flow, thus maintaining their organization-wide identity.
Configure role-based access controls for databases and implement custom Access Request workflows.
Capture database activity in the Teleport audit log.
Teleport supports many types of databases:
Active Directory SQL Server: Connect Microsoft SQL Server with Active Directory authentication.
AWS DynamoDB: Connect AWS DynamoDB.
AWS ElastiCache & MemoryDB: Connect AWS ElastiCache or AWS MemoryDB for Redis database.
AWS RDS & Aurora: Connect AWS RDS or Aurora PostgreSQL, MariaDB or MySQL database.
AWS RDS Proxy: Connect AWS RDS Proxy instances to Teleport.
AWS Redshift: Connect AWS Redshift database.
AWS Redshift Serverless: Connect to AWS Redshift serverless.
AWS Keyspaces (Apache Cassandra): Connect to an AWS Keyspaces database.
Azure PostgreSQL & MySQL: Connect Azure PostgreSQL or MySQL.
Azure Cache for Redis: Connect Azure Cache for Redis.
Azure SQL Server: Connect Azure SQL Server with Azure Active Directory authentication.
GCP Cloud SQL MySQL: Connect GCP Cloud SQL MySQL database.
GCP Cloud SQL PostgreSQL: Connect GCP Cloud SQL PostgreSQL database.
MongoDB Atlas: Connect MongoDB Atlas cluster.
Self-hosted CockroachDB: Connect self-hosted CockroachDB database.
Self-hosted Elasticsearch
Self-hosted MongoDB: Connect self-hosted MongoDB database.
Self-hosted MySQL & MariaDB: Connect self-hosted MySQL or MariaDB database.
Self-hosted PostgreSQL: Connect self-hosted PostgreSQL database.
Self-hosted Redis Cluster: Connect a self-hosted Redis Cluster.
Self-hosted Redis: Connect self-hosted Redis.
Self-Hosted Cassandra & ScyllaDB: Connect self-hosted Cassandra or ScyllaDB.
Snowflake: Connect Snowflake.
For this demo I chose AWS RDS for MySQL. A high-level diagram if the following:
Add the database as all other services:
Choose the required type:
Enter the DB connection endpoint, AWS account ID and RDS Resource ID:
The generated bootstrap script should be executed on the host that has a network connectivity wiyh your RDS:
RDS IAM Authentication will be used for login, so the intermediate host must have such permissions:
DB user and database should be created in advance. AWSAuthenticationPlugin should be enabled and appropriate permissions granted
CREATE USER demouser IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
GRANT ALL ON `%`.* TO 'demouser'@'%';
Then you can use your local terminal to connect to the database, even if it is deployed in a private subnet.
$ tsh db connect demodb --db-user=demouser --db-name=demodb
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| demodb |
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.13 sec)Pricing
There are two pricing models, Community and Enterprise:
Conclusion
Teleport is the identity-native infrastructure access platform for engineers and machines. By replacing insecure secrets like passwords, keys and tokens with true identity based on biometrics and security modules, Teleport delivers phishing-proof zero trust for every engineer and service connected to your global infrastructure. The open-source Teleport Access Platform consolidates connectivity, authentication, authorization and audit into a single source of truth for access policy across your entire infrastructure while delivering a frictionless developer experience. Teleport replaces VPNs, shared credentials, secrets vaults and legacy PAM solutions, improving security and engineering productivity.
Available in the AWS Marketplace as EKS add-on
Komentar