Skip to content

40th Week of 2022

Activism⚑

Antifascism⚑

Antifascist Actions⚑

Life Management⚑

Calendar Management⚑

  • New: Introduce Calendar Management.

    Since the break of my taskwarrior instance I've used a physical calendar to manage the tasks that have a specific date. Can't wait for the first version of pydo to be finished.

    The next factors made me search for a temporal solution:

    • It's taking longer than expected.
    • I've started using a nextcloud calendar with some friends.
    • I frequently use Google calendar at work.
    • I'm sick of having to log in Nexcloud and Google to get the day's appointments.

    To fulfill my needs the solution needs to:

    • Import calendar events from different sources, basically through the CalDAV protocol.
    • Have a usable terminal user interface
    • Optionally have a command line interface or python library so it's easy to make scripts.
    • Optionally it can be based in python so it's easy to contribute
    • Support having a personal calendar mixed with the shared ones.
    • Show all calendars in the same interface

    Looking at the available programs I found khal, which looks like it may be up to the task.

    Go through the installation steps and configure the instance to have a local calendar.

    If you want to sync your calendar events through CalDAV, you need to set vdirsyncer.

Coding⚑

Languages⚑

Python⚑

Python Snippets⚑

  • New: Get an instance of an Enum by value.

    If you want to initialize a pydantic model with an Enum but all you have is the value of the Enum then you need to create a method to get the correct Enum. Otherwise mypy will complain that the type of the assignation is str and not Enum.

    So if the model is the next one:

    class ServiceStatus(BaseModel):
    """Model the docker status of a service."""

    name: str
    environment: Environment

    You can't do ServiceStatus(name='test', environment='production'). you need to add the get_by_value method to the Enum class:

    class Environment(str, Enum):
    """Set the possible environments."""

    STAGING = "staging"
    PRODUCTION = "production"

    @classmethod
    def get_by_value(cls, value: str) -> Enum:
        """Return the Enum element that meets a value"""
        return [member for member in cls if member.value == value][0]

    Now you can do:

    ServiceStatus(
    name='test',
    environment=Environment.get_by_value('production')
)

Typer⚑

  • New: Get the command line application directory.

    You can get the application directory where you can, for example, save configuration files with typer.get_app_dir():

    from pathlib import Path

import typer

APP_NAME = "my-super-cli-app"

def main() -> None:
    """Define the main command line interface."""
    app_dir = typer.get_app_dir(APP_NAME)
    config_path: Path = Path(app_dir) / "config.json"
    if not config_path.is_file():
        print("Config file doesn't exist yet")

if __name__ == "__main__":
    typer.run(main)

    It will give you a directory for storing configurations appropriate for your CLI program for the current user in each operating system.

  • New: Exiting with an error code.

    typer.Exit() takes an optional code parameter. By default, code is 0, meaning there was no error.

    You can pass a code with a number other than 0 to tell the terminal that there was an error in the execution of the program:

    import typer

def main(username: str):
    if username == "root":
        print("The root user is reserved")
        raise typer.Exit(code=1)
    print(f"New user created: {username}")

if __name__ == "__main__":
    typer.run(main)

DevOps⚑

Storage⚑

  • New: Introduce storage.

    I have a server at home to host some services for my closest ones. The server is an Intel NUC which is super awesome in terms of electric consumption, CPU and RAM versus cost. The downside is that it has no hard drive to store the services data. It does have some USB ports to connect external hard drives though. As the data kept growing I started buying bigger drives. While it was affordable I purchased two so as to have one to store the backup of the data. The problem came when it became unaffordable for me. Then I took the good idea to assume that I could only have one drive of 16TB with my data. Obviously the inevitable happened. The hard drive died and those 10TB of data that were not stored in any backup were lost.

    Luckily enough, it was not unique data like personal photos. The data could be regenerated by manual processes at the cost of precious time (I'm still suffering this :(). But every cloud has a silver lining, this failure gave me the energy and motivation to improve my home architecture. To prevent this from happening again, the solution needs to be:

    • Robust: If disks die I will have time to replace them before data is lost.
    • Flexible: It needs to expand as the data grows.
    • Not very expensive.
    • Easy to maintain.

    There are two types of solutions to store data:

    • On one host: All disks are attached to a server and the storage capacity is shared to other devices by the local network.
    • Distributed: The disks are attached to many servers and they work together to provide the storage through the local network.

    A NAS server represents the first solution, while systems like Ceph or GlusterFS over Odroid HC4 fall into the second.

    Both are robust and flexible but I'm more inclined towards building a NAS because it can hold the amount of data that I need, it's easier to maintain and the underlying technology has been more battle proven throughout the years.

NAS⚑

  • New: Introduce NAS.

    Network-attached storage or NAS, is a computer data storage server connected to a computer network providing data access to many other devices. Basically a computer where you can attach many hard drives.

    I've done an analysis to choose what solution I'm going to build in terms of:

    More will come in the next days.

OpenZFS⚑

  • New: Learning.

    I've found that learning about ZFS was an interesting, intense and time consuming task. If you want a quick overview check this video. If you prefer to read, head to the awesome Aaron Toponce articles and read all of them sequentially, each is a jewel. The docs on the other hand are not that pleasant to read. For further information check JRS articles.

  • New: Storage planning.

    There are many variables that affect the number and type of disks, you first need to have an idea of what kind of data you want to store and what use are you going to give to that data.

  • New: Choosing the disks to hold data.

    Analysis on how to choose the disks taking into account:

    The conclusions are that I'm more interested on the 5400 RPM drives, but of all the NAS disks available to purchase only the WD RED of 8TB use it, and they use the SMR technology, so they aren't a choice.

    The disk prices offered by my cheapest provider are:

    Disk Size Price
    Seagate IronWolf 8TB 225$
    Seagate IronWolf Pro 8TB 254$
    WD Red Plus 8TB 265$
    Seagate Exos 7E8 8TB 277$
    WD Red Pro 8TB 278$

    WD Red Plus has 5,640 RPM which is different than the rest, so it's ruled out. Between the IronWolf and IronWolf Pro, they offer 180MB/s and 214MB/s respectively. The Seagate Exos 7E8 provides much better performance than the WD Red Pro so I'm afraid that WD is out of the question.

    There are three possibilities in order to have two different brands. Imagining we want 4 disks:

    Combination Total Price
    IronWolf + IronWolf Pro 958$
    IronWolf + Exos 7E8 1004$ (+46$ +4.5%)
    IronWolf Pro + Exos 7E8 1062$ (+54$ +5.4%)

    In terms of:

    • Consumption: both IronWolfs are equal, the Exos uses 2.7W more on normal use and uses 0.2W less on rest.
    • Warranty: IronWolf has only 3 years, the others 5.
    • Speed: Ironwolf has 210MB/s, much less than the Pro (255MB/s) and Exos (249MB/s), which are more similar.
    • Sostenibility: The Exos disks are much more robust (more workload, MTBF and Warranty).

    I'd say that for 104$ it makes sense to go with the IronWolf Pro + Exos 7E8 combination.

  • New: Choosing the disks for the cache.

    Using a ZLOG greatly improves the writing speed, equally using an SSD disk for the L2ARC cache improves the read speeds and improves the health of the rotational disks.

    The best M.2 NVMe SSD for NAS caching are the ones that have enough capacity to actually make a difference to overall system performance. It also requires a good endurance rating for better reliability and longer lifespan, and you should look for a drive with a specific NAND technology if possible.

    I've made an analysis based on:

    As conclusion, I’d recommend the Western Digital Red SN700, which has a good 1 DWPD endurance rating, is available in sizes up to 4TB, and is using SLC NAND technology, which is great for enhancing reliability through heavy caching workloads. A close second place goes to the Seagate IronWolf 525, which has similar specifications to the SN700 but utilizes TLC.

    Disk Size Speed Endurance Warranty Tech Price
    WD Red SN700 500 GB 3430MB/s 1 DWPD 5 years SLC 73$
    SG IronWolf 525 500 GB 5000MB/s 0.8 DWPD 5 years TLC ?
    WD Red SN700 1 TB 3430MB/s 1 DWPD 5 years SLC 127$
    SG IronWolf 525 1 TB 5000MB/s 0.8 DWPD 5 years TLC ?

Operating Systems⚑

Linux⚑

Khal⚑

  • New: Introduce khal.

    khal is a standards based Python CLI (console) calendar program, able to synchronize with CalDAV servers through vdirsyncer.

    Features:

    • Can read and write events/icalendars to vdir, so vdirsyncer can be used to synchronize calendars with a variety of other programs, for example CalDAV servers.
    • Fast and easy way to add new events
    • ikhal (interactive khal) lets you browse and edit calendars and events.

    Limitations:

    • Only rudimentary support for creating and editing recursion rules
    • You cannot edit the timezones of events

LUKS⚑

vdirsyncer⚑

  • New: Introduce vdirsyncer.

    vdirsyncer is a Python command-line tool for synchronizing calendars and addressbooks between a variety of servers and the local filesystem. The most popular usecase is to synchronize a server with a local folder and use a set of other programs such as khal to change the local events and contacts. Vdirsyncer can then synchronize those changes back to the server.

    However, vdirsyncer is not limited to synchronizing between clients and servers. It can also be used to synchronize calendars and/or addressbooks between two servers directly.

    It aims to be for calendars and contacts what OfflineIMAP is for emails.