Skip to content

March of 2022

Life Management⚑

Task Management⚑

Task Management Workflows⚑

  • Correction: Update the task workflow of the month, and week plannings.

Coding⚑

  • New: Sum up the VueJS tutorial.

Generic Coding Practices⚑

Use warnings to evolve your code⚑

  • New: Using warnings to evolve your package.

    Regardless of the versioning system you're using, once you reach your first stable version, the commitment to your end users must be that you give them time to adapt to the changes in your program. So whenever you want to introduce a breaking change release it under a new interface, and in parallel, start emitting DeprecationWarning or UserWarning messages whenever someone invokes the old one. Maintain this state for a defined period (for example six months), and communicate explicitly in the warning message the timeline for when users have to migrate.

    This gives everyone time to move to the new interface without breaking their system, and then the library may remove the change and get rid of the old design chains forever. As an added benefit, only people using the old interface will ever see the warning, as opposed to affecting everyone (as seen with the semantic versioning major version bump).

Python⚑

  • New: Add humanize library.

    humanize: This modest package contains various common humanization utilities, like turning a number into a fuzzy human-readable duration ("3 minutes ago") or into a human-readable size or throughput.

Boto3⚑

Type Hints⚑

  • New: Suggest to use Sequence over List.

    Because using List could lead to some unexpected errors when combined with type inference. For example:

    class A: ...
class B(A): ...

lst = [A(), A()]  # Inferred type is List[A]
new_lst = [B(), B()]  # inferred type is List[B]
lst = new_lst  # mypy will complain about this, because List is invariant

    Possible strategies in such situations are:

    • Use an explicit type annotation:

      new_lst: List[A] = [B(), B()]
lst = new_lst  # OK

    • Make a copy of the right hand side:

      lst = list(new_lst) # Also OK

    • Use immutable collections as annotations whenever possible:

      def f_bad(x: List[A]) -> A:
    return x[0]
f_bad(new_lst) # Fails

def f_good(x: Sequence[A]) -> A:
    return x[0]
f_good(new_lst) # OK

  • New: Overloading the methods.

    Sometimes the types of several variables are related, such as “if x is type A, y is type B, else y is type C”. Basic type hints cannot describe such relationships, making type checking cumbersome or inaccurate. We can instead use @typing.overload to represent type relationships properly.

    from __future__ import annotations

from collections.abc import Sequence
from typing import overload

@overload
def double(input_: int) -> int:
    ...

@overload
def double(input_: Sequence[int]) -> list[int]:
    ...

def double(input_: int | Sequence[int]) -> int | list[int]:
    if isinstance(input_, Sequence):
        return [i * 2 for i in input_]
    return input_ * 2

    This looks a bit weird at first glance—we are defining double three times! Let’s take it apart.

    The first two @overload definitions exist only for their type hints. Each definition represents an allowed combination of types. These definitions never run, so their bodies could contain anything, but it’s idiomatic to use Python’s ... (ellipsis) literal.

    The third definition is the actual implementation. In this case, we need to provide type hints that union all the possible types for each variable. Without such hints, Mypy will skip type checking the function body.

    When Mypy checks the file, it collects the @overload definitions as type hints. It then uses the first non-@overload definition as the implementation. All @overload definitions must come before the implementation, and multiple implementations are not allowed.

    When Python imports the file, the @overload definitions create temporary double functions, but each is overridden by the next definition. After importing, only the implementation exists. As a protection against accidentally missing implementations, attempting to call an @overload definition will raise a NotImplementedError.

    @overload can represent arbitrarily complex scenarios. For a couple more examples, see the function overloading section of the Mypy docs.

  • Correction: Debug the Start request repeated too quickly error.

    Use journalctl -eu docker to debug

  • Correction: Update TypeVars nomenclature.

    Using UserT is not supported by pylint, use UserT instead.

Code Styling⚑

  • New: Creating your custom factories.

    If your model has an attribute that is not supported by pydantic-factories and it depends on third party libraries, you can create your custom extension subclassing the ModelFactory, and overriding the get_mock_value method to add your logic.

    from pydantic_factories import ModelFactory

class CustomFactory(ModelFactory[Any]):
    """Tweak the ModelFactory to add our custom mocks."""

    @classmethod
    def get_mock_value(cls, field_type: Any) -> Any:
        """Add our custom mock value."""
        if str(field_type) == "my_super_rare_datetime_field":
            return cls._get_faker().date_time_between()

        return super().get_mock_value(field_type)

    Where cls._get_faker() is a faker instance that you can use to build your returned value.

  • New: Solve W1514 pylint error.

    with open('file.txt', 'r', encoding='utf-8'):

FastAPI⚑

  • New: Resolve the 422 error.

    You're probably passing the wrong arguments to the POST request, to solve it see the text attribute of the result. For example:

    result = client.post(
    "/source/add",
    json={"body": body},
)

result.text

    The error is telling us that the required url parameter is missing.

  • New: Resolve the 409 error.

    Probably an exception was raised in the backend, use pdb to follow the trace and catch where it happened.

Pytest⚑

  • New: Run tests in a random order.

    pytest-random-order is a pytest plugin that randomises the order of tests. This can be useful to detect a test that passes just because it happens to run after an unrelated test that leaves the system in a favourable state.

    To use it add the --random-order to your pytest run.

    It can't yet be used with pytest-xdist though :(.

Python Snippets⚑

  • New: Add the Warning categories.

    Class Description
    Warning This is the base class of all warning category classes.
    UserWarning The default category for warn().
    DeprecationWarning Warn other developers about deprecated features.
    FutureWarning Warn other end users of applications about deprecated features.
    SyntaxWarning Warn about dubious syntactic features.
    RuntimeWarning Warn about dubious runtime features.
    PendingDeprecationWarning Warn about features that will be deprecated in the future (ignored by default).
    ImportWarning Warn triggered during the process of importing a module (ignored by default).
    UnicodeWarning Warn related to Unicode.
    BytesWarning Warn related to bytes and bytearray.
    ResourceWarning Warn related to resource usage (ignored by default).

  • New: How to Find Duplicates in a List in Python.

    numbers = [1, 2, 3, 2, 5, 3, 3, 5, 6, 3, 4, 5, 7]

duplicates = [number for number in numbers if numbers.count(number) > 1]
unique_duplicates = list(set(duplicates))

    If you want to count the number of occurrences of each duplicate, you can use:

    from collections import Counter
numbers = [1, 2, 3, 2, 5, 3, 3, 5, 6, 3, 4, 5, 7]

counts = dict(Counter(numbers))
duplicates = {key:value for key, value in counts.items() if value > 1}

    To remove the duplicates use a combination of list and set:

    unique = list(set(numbers))

  • New: How to decompress a gz file.

    import gzip
import shutil
with gzip.open('file.txt.gz', 'rb') as f_in:
    with open('file.txt', 'wb') as f_out:
        shutil.copyfileobj(f_in, f_out)

  • New: How to compress/decompress a tar file.

    def compress(tar_file, members):
    """
    Adds files (`members`) to a tar_file and compress it
    """
    tar = tarfile.open(tar_file, mode="w:gz")

    for member in members:
        tar.add(member)

    tar.close()

def decompress(tar_file, path, members=None):
    """
    Extracts `tar_file` and puts the `members` to `path`.
    If members is None, all members on `tar_file` will be extracted.
    """
    tar = tarfile.open(tar_file, mode="r:gz")
    if members is None:
        members = tar.getmembers()
    for member in members:
        tar.extract(member, path=path)
    tar.close()

Pydantic⚑

  • New: Use mypy pydantic's plugin.

    If you use mypy I highly recommend you to activate the pydantic plugin by adding to your pyproject.toml:

    [tool.mypy]
plugins = [
  "pydantic.mypy"
]

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true

questionary⚑

  • New: Exit when using control + c.

    If you want the question to exit when it receives a KeyboardInterrupt event, use unsafe_ask instead of ask.

HTML⚑

  • New: Sum up the W3 HTML tutorial.

CSS⚑

  • New: Sum up the W3 CSS tutorial.

Javascript⚑

  • New: Use ternary conditional operator.

    It's defined by a condition followed by a question mark ?, then an expression to execute if the condition is truthy followed by a colon :, and finally the expression to execute if the condition is falsy.

    condition ? exprIfTrue : exprIfFalse

    function getFee(isMember) {
  return (isMember ? '$2.00' : '$10.00');
}

console.log(getFee(true));
// expected output: "$2.00"

console.log(getFee(false));
// expected output: "$10.00"

console.log(getFee(null));
// expected output: "$10.00"

  • New: Filter the contents of an array.

    The filter() method creates a new array filled with elements that pass a test provided by a function.

    The filter() method does not execute the function for empty elements.

    The filter() method does not change the original array.

    For example:

    const ages = [32, 33, 16, 40];
const result = ages.filter(checkAdult);

function checkAdult(age) {
  return age >= 18;
}

  • New: Interacting with HTML.

DevOps⚑

Infrastructure Solutions⚑

Krew⚑

  • New: Find if external IP belongs to you.

    You can list the network interfaces that match the IP you're searching for

    aws ec2 describe-network-interfaces --filters Name=association.public-ip,Values="{{ your_ip_address}}"

  • New: Introduce krew.

    Krew is a tool that makes it easy to use kubectl plugins. Krew helps you discover plugins, install and manage them on your machine. It is similar to tools like apt, dnf or brew.

Ksniff⚑

Mizu⚑

  • New: Introduce mizu.

    Mizu is an API Traffic Viewer for Kubernetes, think TCPDump and Chrome Dev Tools combined.

Debugging⚑

  • New: How to extract information from AWS WAF.

    AWS WAF is a web application firewall that helps protect your web applications or APIs against common web exploits and bots that may affect availability, compromise security, or consume excessive resources. AWS WAF gives you control over how traffic reaches your applications by enabling you to create security rules that control bot traffic and block common attack patterns, such as SQL injection or cross-site scripting. You can also customize rules that filter out specific traffic patterns.

    In the article there are many queries you can do on it's data and a workflow to understand your traffic.

  • New: How to debug kubernetes network traffic.

    Sometimes you need to monitor the network traffic that goes between pods to solve an issue. There are different ways to see it:

    Of all the solutions, the cleaner and easier is to use Mizu.

Continuous Integration⚑

Flakeheaven⚑

  • New: Introduce Flakeheaven.

    Flakeheaven is a Flake8 wrapper to make it cool. The community maintained fork of flakeheaven.

Automating Processes⚑

cruft⚑

Monitoring⚑

AlertManager⚑

Operative Systems⚑

Linux⚑

Linux Snippets⚑

  • New: Install one package from Debian unstable.
  • New: Monitor outgoing traffic.

  • Correction: Clean snap data.

    If you're using snap you can clean space by:

    • Reduce the number of versions kept of a package with snap set system refresh.retain=2

    • Remove the old versions with clean_snap.sh

      #!/bin/bash
#Removes old revisions of snaps
#CLOSE ALL SNAPS BEFORE RUNNING THIS
set -eu
LANG=en_US.UTF-8 snap list --all | awk '/disabled/{print $1, $3}' |
    while read snapname revision; do
        snap remove "$snapname" --revision="$revision"
    done)

  • Correction: Clean journalctl data.

    • Check how much space it's using: journalctl --disk-usage
    • Rotate the logs: journalctl --rotate

    Then you have three ways to reduce the data:

    1. Clear journal log older than X days: journalctl --vacuum-time=2d
    2. Restrict logs to a certain size: journalctl --vacuum-size=100M
    3. Restrict number of log files: journactl --vacuum-files=5.

    The operations above will affect the logs you have right now, but it won't solve the problem in the future. To let journalctl know the space you want to use open the /etc/systemd/journald.conf file and set the SystemMaxUse to the amount you want (for example 1000M for a gigabyte). Once edited restart the service with sudo systemctl restart systemd-journald.

  • New: Set up docker logs rotation.

    By default, the stdout and stderr of the container are written in a JSON file located in /var/lib/docker/containers/[container-id]/[container-id]-json.log. If you leave it unattended, it can take up a large amount of disk space.

    If this JSON log file takes up a significant amount of the disk, we can purge it using the next command.

    truncate -s 0 <logfile>

    We could setup a cronjob to purge these JSON log files regularly. But for the long term, it would be better to setup log rotation. This can be done by adding the following values in /etc/docker/daemon.json.

    {
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "10"
  }
}

  • New: Clean old kernels.

    The full command is

    dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get -y purge

    To test what packages will it remove use:

    dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get --dry-run remove

    Remember that your running kernel can be obtained by uname -r.

  • Correction: Clean old kernels warning.

    I don't recommend using this step, rely on apt-get autoremove, it's safer.

ffmpeg⚑

  • New: Convert VOB to mkv.

    • Unify your VOBs 

      cat *.VOB > output.vob

    • Identify the streams

      ffmpeg -analyzeduration 100M -probesize 100M -i output.vob

      Select the streams that you are interested in, imagine that is 1, 3, 4, 5 and 6.

    • Encoding

      ffmpeg \
  -analyzeduration 100M -probesize 100M \
  -i output.vob \
  -map 0:1 -map 0:3 -map 0:4 -map 0:5 -map 0:6 \
  -metadata:s:a:0 language=ita -metadata:s:a:0 title="Italian stereo" \
  -metadata:s:a:1 language=eng -metadata:s:a:1 title="English stereo" \
  -metadata:s:s:0 language=ita -metadata:s:s:0 title="Italian" \
  -metadata:s:s:1 language=eng -metadata:s:s:1 title="English" \
  -codec:v libx264 -crf 21 \
  -codec:a libmp3lame -qscale:a 2 \
  -codec:s copy \
  output.mkv

Rtorrent⚑

  • New: Debug rtorrent docker problems.

Wireshark⚑

  • New: Introduce Wireshark, it's installation and basic usage.

    Wireshark is the world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on your network at a microscopic level and is the de facto (and often de jure) standard across many commercial and non-profit enterprises, government agencies, and educational institutions.

Arts⚑

Aerial Silk⚑

  • New: Introduce Aerial Silk, some warmups and some figures.

    Aerial Silk is a type of performance in which one or more artists perform aerial acrobatics while hanging from a fabric. The fabric may be hung as two pieces, or a single piece, folded to make a loop, classified as hammock silks. Performers climb the suspended fabric without the use of safety lines and rely only on their training and skill to ensure safety. They use the fabric to wrap, suspend, drop, swing, and spiral their bodies into and out of various positions. Aerial silks may be used to fly through the air, striking poses and figures while flying. Some performers use dried or spray rosin on their hands and feet to increase the friction and grip on the fabric.

Book Binding⚑

  • New: Introduce book binding.

    Book binding is the process of physically assembling a book of codex format from an ordered stack of paper sheets that are folded together into sections called signatures or sometimes left as a stack of individual sheets. Several signatures are then bound together along one edge with a thick needle and sturdy thread.