niedziela, 31 maja 2026

Claude Code plugins

  1. https://github.com/anthropics/claude-code/tree/main/plugins/ralph-wiggum:
    loops through tasks, commits to Git and resets context between tasks,
  2. https://claude.com/plugins/context7:
    pulls live, version specific library documentations into your session (stops Claude hallucinating from outdated training data),
  3. https://github.com/anthropics/claude-code/tree/main/plugins/feature-dev:
    launches a workflow using explorer, architect and reviewer agents to analyze code base, design and review the work,
  4. https://github.com/anthropics/claude-code/tree/main/plugins/code-review:
    reviews changed files for quality, security and test coverage before you commit,
  5. https://claude.com/plugins/playwright:
    navigates URLs, fills forms, takes screenshots and run end to end test,
  6. https://claude.com/plugins/superpowers:
    planning, debugging and code review all in one,
  7. https://claude.com/plugins/marketing:
    builds performance reports, runs SEO audits and designs mail sequences.
Autor: o

niedziela, 24 maja 2026

Claude Code slash commands

Claude Code modes (toggle with Shift + Tab):

  • >>> normal (reads, writes and runs commands),
  • >> auto accept (like normal but skips permission prompts),
  • || plan (read only, explores but never edits).
Slash commands:
  • /model choose model you want to work with,
  • /status - version, model, account, usage and so on,
  • /help - all available commands and skills,
  • /diff - review all file changes Claude just made,
  • /skills - all installed skills,
  • /usage - check your token usage and spending,
  • /compact - summarize conversation to free up context,
  • /init - create a "CLAUDE.md" project guide,
  • /context - show Claude context window utilization.

Autor: o

poniedziałek, 10 listopada 2025

Remove a versioned S3 bucket

Let's imagine you have a S3 bucket that is versioned. It contains thousands of files. If you try to delete it you're warned because of the versions. You cannot select all files in AWS Console so a way to do this is from your command line:
aws s3 rm s3://YOUR_BUCKET --recursive

But what about the versions and maybe delete markers. We can use the following Python script:

Let's imagine you have a S3 bucket that is versioned. It contains thousands of files. If you try to delete it you're warned because of the versions. You cannot select all files in AWS Console so a way to do this is from your command line:
aws s3 rm s3://YOUR_BUCKET --recursive

But what about the versions and maybe delete markers. We can use the following Python script:

#!/usr/bin/env python3

import boto3
from itertools import islice
from typing import Iterable, Dict

bucket = 'YOUR_BUCKET'
s3_client = boto3.client('s3')

def batched(iterable: Iterable, n: int) -> Iterable[list]:
    """Yield lists of size n from iterable."""
    it = iter(iterable)

    while True:
        batch = list(islice(it, n))
        if not batch:
            break

        yield batch

def iter_all_object_versions(bucket: str, prefix: str | None = None) -> Iterable[Dict[str, str]]:
    """Iterate all versions and delete markers in a S3 bucket (optionally under a prefix)."""
    paginator = s3_client.get_paginator('list_object_versions')
    params = {'Bucket': bucket}

    if prefix:
        params['Prefix'] = prefix

    for page in paginator.paginate(**params):
        for v in page.get('Versions', []):
            yield {'Key': v['Key'], 'VersionId': v['VersionId'], 'IsDeleteMarker': False}
        for dm in page.get('DeleteMarkers', []):
            yield {'Key': dm['Key'], 'VersionId': dm['VersionId'], 'IsDeleteMarker': True}

def remove_s3_object_versions(bucket: str, prefix: str | None = None, dry_run: bool = False, batch_size: int = 1000):
    total_versions = 0
    total_delete_markers = 0
    to_delete = []

    for entry in iter_all_object_versions(bucket, prefix):
        if entry['IsDeleteMarker']:
            total_delete_markers += 1
        else:
            total_versions += 1
        to_delete.append({'Key': entry['Key'], 'VersionId': entry['VersionId']})

    print(f"[*] Discovered {total_versions} versions and {total_delete_markers} delete markers (total {total_versions + total_delete_markers}) in bucket "{bucket}"{f" with prefix '{prefix}'" if prefix else ''}.")

    if dry_run:
        print("[!] Dry run enabled. No deletions performed.")
        return

    deleted_count = 0

    for batch in batched(to_delete, batch_size):
        s3_client.delete_objects(Bucket=bucket, Delete={'Objects': batch, 'Quiet': True})

        deleted_count += len(batch)

        if deleted_count % 5000 == 0:
            print(f"[*]Progress: deleted {deleted_count} items.")

    print(f"[*] Deletion complete. Removed {deleted_count} versioned entries from bucket "{bucket}".")

if __name__ == '__main__':
    remove_s3_object_versions(bucket=bucket, dry_run=False)

Autor: o

poniedziałek, 6 października 2025

S3 cross account replication

I was moving AWS resources from one account to separate staging and production accounts. One of the steps was to migrate S3 buckets. A solution was cross account replication. Because S3 cross region replication moves only new files we have to create a S3 Batch Operation to move existing objects.

S3 cross account replication and Batch Operation

As following:

  • enable S3 bucket versioning on your buckets,
  • in source account prepare an IAM policy as a part of an IAM role to be used by the S3 replication:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetReplicationConfiguration",
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging",
                "s3:GetObjectRetention",
                "s3:GetObjectLegalHold"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET",
                "arn:aws:s3:::SOURCE_BUCKET/*",
                "arn:aws:s3:::TARGET_BUCKET",
                "arn:aws:s3:::TARGET_BUCKET/*"
            ]
        },
        {
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ReplicateTags",
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET/*",
                "arn:aws:s3:::TARGET_BUCKET/*"
            ]
        }
    ]
}
  • in source account create an IAM role that includes above policy (trusted entity type = AWS service, Use case = S3),
  • in source account prepare an IAM role for a S3 Batch Operation (trusted entity type = AWS service, Use case = S3 Batch Operations):
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "GetSourceBucketConfiguration",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:GetBucketAcl",
                "s3:GetReplicationConfiguration",
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging",
                "s3:PutInventoryConfiguration",
                "s3:GetInventoryConfiguration",
                "s3:PutObject",
                "s3:GetObject",
                "s3:InitiateReplication",
                "s3:AbortMultipartUpload"
            ],
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET",
                "arn:aws:s3:::SOURCE_BUCKET/*"
            ]
        },
        {
            "Sid": "ReplicateToDestinationBuckets",
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:*Object",
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ReplicateTags"
            ],
            "Resource": [
                "arn:aws:s3:::TARGET_BUCKET",
                "arn:aws:s3:::TARGET_BUCKET/*"
            ]
        },
        {
            "Sid": "PermissionToOverrideBucketOwner",
            "Effect": "Allow",
            "Action": [
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Resource": [
                "arn:aws:s3:::TARGET_BUCKET",
                "arn:aws:s3:::TARGET_BUCKET/*"
            ]
        }
    ]
}
  • in target account update S3 bucket policy:
{
    "Version": "2012-10-17" ,
    "Id": "",
    "Statement": [
        {
            "Sid": "Set-permissions-for-objects",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::SOURCE_ACCOUNT_NUMBER:role/service-role/REPLICATION_IAM_ROLE_NAME"
            },
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete"
            ],
            "Resource": "arn:aws:s3:::TARGET_BUCKET/*"
        },
        {
            "Sid": "Set permissions on bucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::SOURCE_ACCOUNT_NUMBER:role/service-role/REPLICATION_IAM_ROLE_NAME"
            },
            "Action": [
                "s3:GetBucketVersioning",
                "s3:PutBucketVersioning"
            ],
            "Resource": "arn:aws:s3:::TARGET_BUCKET"
        },
        {
            "Sid": "Permissions on objects and buckets",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::SOURCE_ACCOUNT_NUMBER:role/BATCH_OPERATIONS_IAM_ROLE_NAME"
            },
            "Action": [
                "s3:List*",
                "s3:GetBucketVersioning",
                "s3:PutBucketVersioning",
                "s3:ReplicateDelete",
                "s3:ReplicateObject"
            ],
            "Resource": [
                "arn:aws:s3:::TARGET_BUCKET",
                "arn:aws:s3:::TARGET_BUCKET/*"
            ]
        },
        {
            "Sid":"1",
            "Effect":"Allow",
            "Principal":{"AWS":"arn:aws:iam::SOURCE_ACCOUNT_NUMBER:role/service-role/REPLICATION_IAM_ROLE_NAME"},
            "Action":["s3:ObjectOwnerOverrideToBucketOwner"],
            "Resource":"arn:aws:s3:::TARGET_BUCKET/*"
        },
        {
            "Sid": "Permission to override bucket owner",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::SOURCE_ACCOUNT_NUMBER:role/BATCH_OPERATIONS_IAM_ROLE_NAME"
            },
            "Action": "s3:ObjectOwnerOverrideToBucketOwner",
            "Resource": "arn:aws:s3:::TARGET_BUCKET/*"
        }
    ]
}
  • go to your source S3 bucket, then "Management" bookmark and click on "Create replication rule":
    • give a name,
    • status = "Enabled",
    • role scope = "Apply to all objects in the bucket",
    • choose your destination bucket (mark "Change object ownership to destination bucket owner"),
    • choose your S3 replication IAM role,
    • mark "Change the storage class for the replicated objects with Standard storage class",
    • mark "Delete marker replication" as a additional replication option,
  • in your account go to "S3", open "Batch Operations" and push "Create job":
    • object list = "Generate an object list based on a replication configuration" (it will check S3 replication rule we created previously),
    • choose your source S3 bucket,
    • click "Next",
    • operation = "Replicate",
    • click "Next",
    • put a name,
    • unmark "Generate completion report",
    • choose your S3 Batch Operations IAM role,
    • click "Next",
    • check settings and click "Submit".
Autor: o

poniedziałek, 14 lipca 2025

Include Terraform dependency lock file

Why? Because in the beginning of an initialization it save modules and providers checksums. Thanks to this you can track if anything changed in the version you used.

Source: https://www.hashicorp.com/en/blog/terraform-security-5-foundational-practices

Autor: o

czwartek, 27 lutego 2025

A cost optimized AWS environment

Costs saving:

  • Saving Plans,
  • Reserved Instances,
  • change your default payment method to avoid currency conversion,
  • Spot Instances (a development environment),
  • Data Lifecycle Management for EBSes (remove unneeded EBSes),
  • S3:
    • a lifecycle policy for a bucket (move your data into a cheaper storage class),
    • compress objects to save space,
    • S3 Requester Pays,
  • use VPC endpoints (AWS charges for outbound data transfer),
  • use Graviton instance type,
  • use Lambda to switch off your instances (for example EC2, RDS) out of working hours on your development environments.
  • choose a right region because a resource can be cheaper in a different region,
  • Parameter Store instead of Secrets Manager if you don't need a versioning or rotation,
  • ElastiCache for Redis:
    • consider using ElastiCache for Valkey,
  • CloudWatch:
    • logs retention,
  • NAT Gateway:
  • Route 53:
    • check your records TTLs - the lower TTL the less you pay.

Monitoring:

  • Cost Explorer,
  • Cost and Usage Reports,
  • Cost Anomaly Detection,
  • Budgets,
  • Trusted Advisor,
  • cost allocation tags,
  • AWS Compute Optimizer,
  • S3 Storage Lens.

Autor: o

niedziela, 24 listopada 2024

WireGuard instead of AWS Client VPN

Let's pretend your client wants to have an access to your private EKS cluster but don't want to pay much for AWS Client VPN. A solution is to establish an EC2 instance (for example t3.micro with 10 GB storage) based on Amazon Linux in a public Subnet with Elastic IP. Also Instance's Security Group must have open 51820 UDP port.

The server is created so let's install WireGuard (as root):
yum update -y
amazon-linux-extras enable epel
yum install epel-release -y
yum install wireguard-tools -y

Then we have to generate a key pair of WireGuard server:
cd /etc/wireguard
umask 077
wg genkey > privatekey
wg pubkey < privatekey > publickey

Now open “/etc/wireguard/wg0.conf” file and put:
[Interface]
Address = 10.100.0.1/24 # Choose a different range than your VPC CIDR.
SaveConfig = true
ListenPort = 51820
PrivateKey = GENERATED_PRIVATE_KEY
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = GENERATED_PUBLIC_KEY_OF_YOUR_CLIENT # Described below.
AllowedIPs = 10.100.0.2/32 # Put an IP you want to assign to your client.

Start WireGuard:
systemctl enable wg-quick@wg0
systemctl start wg-quick@wg0

Check if IP forwarding is enabled (if it’s not then enable):
sysctl net.ipv4.ip_forward
echo "net.ipv4.ip_forward=1" | tee -a /etc/sysctl.conf
sysctl -p

To change a configuration and apply new changes:
systemctl reload wg-quick@wg0

Now install a client on your favourite system. Then you have to add a new configuration (an empty tunnel). It will generate a private key and a public key for you. Put this public key in an additional [Peer] section on the server in “/etc/wireguard/wg0.conf” file. Now we have edit the client configuration to look like this:
[Interface]
PrivateKey = GENERATED_PRIVATE_KEY # Don't touch.
Address = 10.100.0.2/32 # IP you want to assign.

[Peer]
PublicKey = SERVER_PUBLIC_KEY
AllowedIPs = 10.100.0.0/24, 10.21.0.0/16 # VPN CIDR, VPC CIDR
Endpoint = 13.50.30.59:51820 # VPN address.
PersistentKeepalive = 25

Autor: o

niedziela, 17 listopada 2024

EKS private access without a VPN

When you create a new EKS cluster Amazon also creates an endpoint for the managed Kubernetes API server that you use to communicate with your K8s (using Kubernetes management tools such as kubectl). By default this API server endpoint is public to the Internet and access to the API server is secured using a combination of IAM and native Kubernetes Role Based Access Control.

You can enable both private and public access. Thanks to this you have the remote access but all communication between your nodes and the API server stays within your VPC. This guide assume your cluster has public access disabled.

You must have an EC2 bastion host created within the VPC where your EKS cluster is established. Also your instance must have SSH server run and SSH key added to a user (use Key Pair).

I assume you have an AWS profile configured ("AWS_PROFILE" variable exporter or a similar) and Session Manager plugin installed.

Let's test a connection with our instance (in this case we have Amazon Linux):
ssh -o ProxyCommand="sh -c 'aws ssm start-session --region YOUR_REGION --target YOUR_EC2_INSTANCE_IDENTIFIER --document-name AWS-StartSSHSession --parameters portNumber=22'" ec2-user@YOUR_EC2_INSTANCE_IDENTIFIER -i YOUR_PRIVATE_KEY

Let's imagine your don't have a VPN or you don't want to use it. Install sshuttle.

Open a terminal window and put (don't close the terminal):
aws ssm start-session --region=YOUR_REGION  --target YOUR_EC2_INSTANCE_IDENTIFIER --document-name AWS-StartPortForwardingSession --parameters "localPortNumber=2222,portNumber=22"

Open a second terminal (don't close the terminal):
sshuttle --dns -NHr ec2-user@localhost:2222 -e 'ssh -i EC2_BASTION_PRIVATE_KEY_PATH' YOUR_VPC_CIDR

Then from a third terminal you can connect with your private EKS cluster without a VPN.

There is an another way. In a terminal (don't close):
ssh -o ProxyCommand="sh -c 'aws ssm start-session --region YOUR_REGION --target YOUR_EC2_INSTANCE_IDENTIFIER --document-name AWS-StartSSHSession --parameters portNumber=22'" -q -D 6669 ec2-user@YOUR_EC2_INSTANCE_IDENTIFIER -i YOUR_PRIVATE_KEY

In a terminal where you want to connect with your EKS cluster:
export http_proxy=socks5://127.0.0.1:6669
export https_proxy=socks5://127.0.0.1:6669

Autor: o

poniedziałek, 11 listopada 2024

YubiKey and SSH key

On macOS (put your FIDO2 PIN when asked):
ssh-keygen -t ed25519-sk -C "mailbox@address" -f ~/.ssh/id_ed25519-sk

Probably you'll see this error:
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
No FIDO SecurityKeyProvider specified
Key enrollment failed: invalid format

You have to install SSH via Homebrew:
brew install openssh

Autor: o

sobota, 9 listopada 2024

YubiKey and AWS CLI

First you have to install YubiKey Manager CLI. For me on macOS:
brew install ykman

Add a virtual device for your IAM user:


In the next wizard window click on "Show secret key":


Copy the QR code and don't close the card in your browser.

Put your YubiKey in your hardware and:
ykman oath accounts add -t YOUR_LABEL YOUR_QR_CODE

Now ask the YubiKey twice to get a code:
ykman oath accounts code YOUR_LABEL

Put the codes in the right fields and finish the configuration.

Now we can configure our terminal to communicate with AWS. Set your AWS profile before ("AWS_PROFILE" variable).

For example a "~/.aws/config":
[profile yubikey-test]
region = eu-central-1

"~/.aws/credentials":
[yubikey-test]
aws_access_key_id = YOUR_IAM_USER_SECRET_KEY
aws_secret_access_key = YOUR_IAM_USER_ACCESS_KEY

The following command will give your a set of needed variables to get control on your IAM user:
aws sts get-session-token --serial-number ARN_OF_THE_DEVICE --token-code ASK_YUBIKEY_TOKEN --output json

Write down the values and export these environment variables:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_SESSION_TOKEN
  • AWS_DEFAULT_REGION
To make it easier add at your home directory in ".bash_profile" file (or in ".zshrc" depending on your shell)
function aws-get-yubikey-mfa-code {
    ykman oath accounts code YOUR_LABEL 2>/dev/null | sed -E 's/(None:)?AWS[[:space:]]+([[:digit:]]+)/\2/'
}

Close the editor and put
source ~/.bash_profile

Now using aws-get-yubikey-mfa-code command you can get a code using YubiKey.

We don't want to set manually every time all needed variables so let's create another function in "~/.zshrc" file (my case):
AWS_MFA_SERIAL="YOUR_VIRTUAL_DEVICE_ARN"

function aws-yubikey-mfa-session {
    STS_CREDENTIALS=$(aws sts get-session-token --serial-number "$AWS_MFA_SERIAL" --token-code "$1" --output json)

   if [ "$?" -eq "0" ]
    then
        export AWS_ACCESS_KEY_ID=$(echo $STS_CREDENTIALS | jq -r '.Credentials.AccessKeyId')
        export AWS_SECRET_ACCESS_KEY=$(echo $STS_CREDENTIALS | jq -r '.Credentials.SecretAccessKey')
        export AWS_SECURITY_TOKEN=$(echo $STS_CREDENTIALS | jq -r '.Credentials.SessionToken')
        export AWS_SESSION_TOKEN=$(echo $STS_CREDENTIALS | jq -r '.Credentials.SessionToken')
        export AWS_SESSION_EXPIRY=$(echo $STS_CREDENTIALS | jq -r '.Credentials.Expiration')

        echo "[*] Session credentials set. Expires at $AWS_SESSION_EXPIRY."
    else
        echo "[!] Failed to obtain temporary credentials."
    fi
}

Now put
source ~/.zshrc

Install jq command. On macOS it will be:
brew install jq

Now you're ready:
aws-yubikey-mfa-session GET_A_YUBIKEY_TOKEN

Autor: o

niedziela, 3 listopada 2024

YubiKey and AWS Console

Your company wants you to start using YubiKey. You have an AWS account and it's time to reconfigure the access. I recommend to do this easily. I mean if you already have configured a MFA device as an authenticator application you can add another device and test in a first few days simultaneously.

Add new MFA device in your Console:


Plug in your YubiKey and tap:


Put your FIDO2 PIN:


Now you can log into Console using a given method:


Autor: o

poniedziałek, 23 września 2024

DevOps engineer's starting pack

Let's pretend you have a new computer after you joined to a new company and you have to setup everything from scratch.

Usually I use macOS but it suits to a Linux machine even.

This is how my starting pack looks like:

  • Chrome browser (on my private machine I use Vivaldi but this is because many companies has Google Workspace),
  • Homebrew (a package manager),
  • KeePassXC (a password manager),
  • iTerm2 (a terminal emulator with a terminal split),
  • Oh My Zsh (manage your Zsh shell):
    • Kubernetes prompt:
      brew update
      brew install kube-ps1
      and add to your "~/.zshrc"
      plugins=(kube-ps1)
      PROMPT='$(kube_ps1)'$PROMPT
    • AWS profile:
      add to your "~/.zshrc"
      plugins=(aws)
  • Visual Studio Code:
    • Terraform extension,
    • YAML extension,
    • Hashicorp HCL extension,
    • GitHub Copilot,
    • insert a final line at the end of the file when saving it,
  • desktop Claude,
  • a GitHub account (I don't use my personal account but I create an account per company because sometime they want you to setup something additional what you maybe don't want),
  • generate SSH keys (add these keys to your GitHub account if you use):
    • ssh-keygen -t ed25519 -C "YOUR_COMPANY_EMAIL",
    • ssh-keygen -t rsa -C "YOUR_COMPANY_EMAIL",
  • AWS CLI,
  • Granted (switching between AWS profiles),
  • kubectl:
  • kubectx (switch between your Kubernetes clusters),
  • Lens,
  • Helm,
  • Python pip,
  • tfenv to switch between Terraform versions (you don't have to separately install Terraform),
  • Docker,
  • Docker Compose,
  • Vim:
    • add to ".vimrc" file in your home directory (start from the last position):
      au BufReadPost * if line("'\"") > 0 && line("'\"") <= line("$") | exe "normal! g`\"" | endif

Autor: o

niedziela, 15 września 2024

LXC - let's get started

I know that everyone uses Docker but sometimes you have to take care of some legacy server where you have, for example, LXC container in which your client runs a Docker container (I know it's weird).

Le'ts imagine you have in you LXC container a device which is mounted as "/var/lib/docker" path. You don't have enough space on it and you don't want to restart the server or even LXC daemon. Let's just replace it.

Add a new device:
lxc storage create new-device btrfs size=100GB

Add a new volume to the device:
lxc storage volume create new-device volume

Then we have get into the LXC container and stop the Docker daemon:
lxc exec lxc-container -- /bin/bash
systemctl stop docker
exit

Disable the previous device from the LXC container:
lxc config device remove lxc-container old-device

Add the new device:
lxc config device add lxc-container some-name disk pool=new-device path=/var/lib/docker source=volume

Run the Docker daemon in the container.

Some additional useful commands:

  • storage list: "lxc storage list",
  • what connected device your container has: "lxc config show container-name --expanded",
  • run a LXC container with a given profile and image: "lxc launch ubuntu:20.04 container-name -p default",
  • run containers: "lxc list",
  • stop and remove a container: "lxc stop container-name", "lxc delete container-name",
  • information about a storage: "lxc storage show storage-name",
  • list the volumes in a storage: "lxc storage volume list storage-name",
  • remove a volume from a storage: "lxc storage volume delete storage-name volume-name",
  • remove a storage: "lxc storage delete storage-name".

Autor: o

czwartek, 21 marca 2024

Mount your Amazon EBS in an EKS Pod

If you want to use EBS in your Kubernetes Persistent Volume.

EC2

Dynamic provisioning:

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
  encrypted: "true"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-sc
  resources:
    requests:
      storage: 4Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
  - name: app
    image: centos
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: ebs-claim

Fargate

You can't mount Amazon EBS volumes to Fargate Pods.

More examples here.

Autor: o

środa, 20 marca 2024

Mount your Amazon EFS in an EKS Pod

If you have already created an EFS and you want to mount it in a Kubernetes Pod you have to add the EKS' Security Group to each mount target.

EC2

This example shows how to make a static provisioned EFS persistent volume (PV) mounted inside container with encryption in transit configured:

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
mountOptions:
  - tls
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-03e456ec05d6df74e # Replace with you EFS identifier.
    volumeAttributes:
      encryptInTransit: "true"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: efs-app
spec:
  containers:
  - name: app
    image: centos
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim

Fargate

A Pod running on AWS Fargate automatically mounts an Amazon EFS file system without needing the driver installation.

You can't use dynamic provisioning for persistent volumes with Fargate nodes but you can use static provisioning. An example how to use:

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  storageClassName: efs-sc
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-03e456ec05d6df74e # Replace with you EFS identifier.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: efs-app
spec:
  containers:
  - name: app
    image: centos
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim

More examples here.

Autor: o

wtorek, 12 grudnia 2023

Vim tip and tricks

In the normal mode:

  • go to the top of the file: press "gg". 

Open ".vimrc" in your home directory to set permanently:

  • "set number": add the line numbers,
  • "set mouse+=a": easy copy without the line numbers.

Autor: o

wtorek, 19 września 2023

Konfiguracja BIND

Post pisany w oparciu o Debiana 11. 

Następnie otwieramy "/etc/bind/named.conf.options":

acl "trusted" {

        ADRES_IP; # Serwer podstawowy

        ADRES_IP; # Serwer zapasowy

};


options {

directory "/var/cache/bind";

        recursion yes; # Pozwol na rekursywne zapytania.

        allow-recursion { trusted; }; # Pozwol na rekursywne zapytania z listy zaufanych klientow.

        allow-transfer { none; }; # Wylacz domyslny transfer strefy.

dnssec-validation auto;

auth-nxdomain no;

listen-on-v6 { any; };


        forwarders {

                8.8.8.8;

                8.8.4.4;

        };

};

Teraz "/etc/bind/named.conf.local":

zone "yolandi.pl" {

    type master;

    file "/etc/bind/db.yolandi.pl"; # Sciezka do pliku strefy

    allow-transfer { ADRES_IP; }; # IP serwera zapasowego

};

 Tworzymy "/etc/bind/zones/db.yolandi.pl":

$TTL 604800

@ IN SOA yolandi.pl. admin.yolandi.pl. (

2021051601 ; Serial

604800 ; Refresh

  86400 ; Retry

2419200 ; Expire

604800 ) ; Negative Cache TTL

    IN NS TWOJA_GLOWNA_DOMENA.

    IN NS DOMENA_SERWERA_ZAPASOWEGO.

yolandi.pl. IN A ADRES_IP

Autor: o

poniedziałek, 8 maja 2023

EFS as a storage for an EKS deployment

Let's imagine you have an EKS cluster with a Grafana deployment. You cluster has as a default storage EBS. Unfortunately EBS doesn't allow you to create more replicas of your deployment because the EBS is already attached to the instance. To workaround this we can use EFS.

Before you start changing your workloads you install Amazon EFS CSI driver as an addon.

Then create an EFS disk and write down the identifier. It's better to have an EFS per deployment (better maintenance for example).

Now we have a possibility to create a StorageClass in our Helm chart:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-storageclass
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap
  fileSystemId: {{ .Values.grafana_custom.StorageClass.efs_id | quote }}
  directoryPerms: "755"
  basePath: "/grafana"

When a PVC is created with a StorageClass that has a basePath specified, the dynamically provisioned PV will be created with a subdirectory path under the specified basePath. This can be useful for organizing and managing the storage resources in a cluster.

It is better to first deploy the StorageClass before you change the Grafana storage type.

Change the StorageClass name in the Helm values: https://github.com/grafana/helm-charts/blob/main/charts/grafana/values.yaml#L327

Change the access mode to "ReadWriteMany": https://github.com/grafana/helm-charts/blob/main/charts/grafana/values.yaml#L328

Enter the needed disk size: https://github.com/grafana/helm-charts/blob/main/charts/grafana/values.yaml#L329

Autor: o

poniedziałek, 12 grudnia 2022

Create a certificate by cert-manager and AWS Private CA

In our EKS ecosystem there is a possibility to create a certificate issued via ACM Private CA. To do this we use (already implemented) https://github.com/cert-manager/cert-manager with https://github.com/cert-manager/aws-privateca-issuer module.

Let’s pretend we have an EKS cluster with some custom deployment. Usually to issue a certificate you can use Issuer or ClusterIssuer Kubernetes resource. The different is that ClusterIssuer you can use from any namespace. If we use aws-privateca-issuer module we must use AWSPCAIssuer or AWSPCAClusterIssuer.

On our platform the AWSPCAClisterIssuer already exists:

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: YOUR_NAME
spec:
  arn: PRIVATE_CA_ARN
  region: YOUR_REGION

But how to create a certificate? To do this we use a Certificate:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: MY_SUBDOMAIN
spec:
  commonName: MY_SUBDOMAIN
  dnsNames:
    - MY_SUBDOMAIN
  duration: 2160h0m0s
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: YOUR_NAME
  renewBefore: 360h0m0s
  secretName: MY_SUBDOMAIN
  usages:
    - server auth
    - client auth
  privateKey:
    algorithm: "RSA"
    size: 2048

Use "kubectl -n MY_NAMESPACE get certificate" and check the result:

NAME                      READY   SECRET                    AGE
MY_SUBDOMAIN True         MY_SUBDOMAIN   12s

The certificate is stored in a Secret. To view the details:

kubectl get secret MY_SUBDOMAIN -n MY_NAMESPACE -o 'go-template={{index .data "tls.crt"}}' | base64 --decode | openssl x509 -noout -text

Autor: o

piątek, 2 grudnia 2022

EKS upgrade

Why do we need to upgrade an EKS? First of all every new version of EKS (Kubernetes) provides a new features and adds some fixes, patches. Thanks to this your EKS cluster is safe and tour workloads can be more sophisticated. Remember also that one day your EKS will not be supported by AWS. Check when your version starts to be unsupported.

Before you start to upgrade an EKS cluster please check the available versions at https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html and if you upgrade to 1.22 you must complete the 1.22 prerequisites. But if your current cluster version is 1.21 and you want to upgrade to 1.23 you must first update your cluster to 1.22 and then to 1.23.

Check the requirements here: https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html#update-existing-cluster

You must check what was changed in the version you want upgrade to: https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html

First check the changelog: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#changelog-since-v1220

Then read the blogs:

In these blogs you can read about the potential risks and how to handle them.

One of the important things can happen is that the API versions of your resources (describe in the manifest) can be deprecated. That’s why before you upgrade compare your Helm Charts and (or) resources with the Kubernetes changelog. And if in a current version an API is deprecated it means in a future it will be deleted and you will not able to use it.

In my case I used a module. But we can imagine using directly the resources. Then you must upgrade separately the main node and then the workers. In the beginning add the same number of worker nodes to the cluster. Then upgrade the main node and next the worker nodes you added. At the and remove the old worker nodes.

Thanks to this we have the development and production environments we can test an upgrade before we go production.

As a default Kubernetes uses the rolling update strategy when a Pod is deploying. Thanks to this for an user an upgrade should be invisible.

After you read all requirements and changelog and you’re still not sure if your application will work on a new EKS version. You can create a separate EKS and deploy your application (as an Helm Chart for example) to check if it’s up and running.

Check what version of the add-ons you must put when you upgrade. For instance:

Be sure if your Terraform provider of Kubernetes has right API version. You can check it in the prerequisites. But don’t change the providers setup before you applied Terraform to deploy new cluster version (it won’t work because Terraform firs uses the old configuration).

After it’s done check the server version:

kubectl version --short

And then check the versions of your nodes:

kubectl get nodes

Autor: o
Subskrybuj: Posty (Atom)