7th March 2022

Coding⚑

Python⚑

Boto3⚑

• New: List the files of a bucket.

FastAPI⚑

• New: Resolve the 409 error.

  Probably an exception was raised in the backend, use pdb to follow the trace and catch where it happened.

Python Snippets⚑

• New: How to Find Duplicates in a List in Python.

  numbers = [1, 2, 3, 2, 5, 3, 3, 5, 6, 3, 4, 5, 7]
  
  duplicates = [number for number in numbers if numbers.count(number) > 1]
  unique_duplicates = list(set(duplicates))
  

  If you want to count the number of occurrences of each duplicate, you can use:

  from collections import Counter
  numbers = [1, 2, 3, 2, 5, 3, 3, 5, 6, 3, 4, 5, 7]
  
  counts = dict(Counter(numbers))
  duplicates = {key:value for key, value in counts.items() if value > 1}
  

  To remove the duplicates use a combination of list and set:

  unique = list(set(numbers))
  

• New: How to decompress a gz file.

  import gzip
  import shutil
  with gzip.open('file.txt.gz', 'rb') as f_in:
      with open('file.txt', 'wb') as f_out:
          shutil.copyfileobj(f_in, f_out)
  

• New: How to compress/decompress a tar file.

  def compress(tar_file, members):
      """
      Adds files (`members`) to a tar_file and compress it
      """
      tar = tarfile.open(tar_file, mode="w:gz")
  
      for member in members:
          tar.add(member)
  
      tar.close()
  
  def decompress(tar_file, path, members=None):
      """
      Extracts `tar_file` and puts the `members` to `path`.
      If members is None, all members on `tar_file` will be extracted.
      """
      tar = tarfile.open(tar_file, mode="r:gz")
      if members is None:
          members = tar.getmembers()
      for member in members:
          tar.extract(member, path=path)
      tar.close()
  

DevOps⚑

Infrastructure Solutions⚑

Krew⚑

• New: Introduce krew.

  Krew is a tool that makes it easy to use kubectl plugins. Krew helps you discover plugins, install and manage them on your machine. It is similar to tools like apt, dnf or brew.

Ksniff⚑

• New: Introduce Ksniff.

  Ksniff is a Kubectl plugin to ease sniffing on kubernetes pods using tcpdump and wireshark.

Mizu⚑

• New: Introduce mizu.

  Mizu is an API Traffic Viewer for Kubernetes, think TCPDump and Chrome Dev Tools combined.

Debugging⚑

• New: How to debug kubernetes network traffic.

  Sometimes you need to monitor the network traffic that goes between pods to solve an issue. There are different ways to see it:

  • Using Mizu
  • Running tcpdump against a running container
  • Using ksniff
  • Using ephemeral debug containers

  Of all the solutions, the cleaner and easier is to use Mizu.

Operative Systems⚑

Linux⚑

Linux Snippets⚑

• Correction: Clean snap data.

  If you're using snap you can clean space by:

  • Reduce the number of versions kept of a package with snap set system refresh.retain=2

  • Remove the old versions with clean_snap.sh

    #!/bin/bash
    #Removes old revisions of snaps
    #CLOSE ALL SNAPS BEFORE RUNNING THIS
    set -eu
    LANG=en_US.UTF-8 snap list --all | awk '/disabled/{print $1, $3}' |
        while read snapname revision; do
            snap remove "$snapname" --revision="$revision"
        done)
    

• Correction: Clean journalctl data.

  • Check how much space it's using: journalctl --disk-usage
  • Rotate the logs: journalctl --rotate

  Then you have three ways to reduce the data:

  1. Clear journal log older than X days: journalctl --vacuum-time=2d
  2. Restrict logs to a certain size: journalctl --vacuum-size=100M
  3. Restrict number of log files: journactl --vacuum-files=5.

  The operations above will affect the logs you have right now, but it won't solve the problem in the future. To let journalctl know the space you want to use open the /etc/systemd/journald.conf file and set the SystemMaxUse to the amount you want (for example 1000M for a gigabyte). Once edited restart the service with sudo systemctl restart systemd-journald.