Skip to content

47th Week of 2022

Coding⚑

Languages⚑

PDM⚑

  • Correction: Solve circular dependencies by manual constraining.

    It also helps to run pdm update with the -v flag, that way you see which are the candidates that are rejected, and you can put the constrain you want. For example, I was seeing the next traceback:

    pdm.termui: Conflicts detected:
  pyflakes>=3.0.0 (from <Candidate autoflake 2.0.0 from https://pypi.org/simple/autoflake/>)
  pyflakes<2.5.0,>=2.4.0 (from <Candidate flake8 4.0.1 from unknown>)

    So I added a new dependency to pin it:

    [tool.pdm.dev-dependencies]
dependencies = [
    # Until flakeheaven supports flake8 5.x
    # https://github.com/flakeheaven/flakeheaven/issues/132
    "flake8>=4.0.1,<5.0.0",
    "pyflakes<2.5.0",
]

    If none of the above works, you can override them:

    [tool.pdm.overrides]
"importlib-metadata" = ">=3.10"

Click⚑

  • New: File System Isolation.

    For basic command line tools with file system operations, the CliRunner.isolated_filesystem() method is useful for setting the current working directory to a new, empty folder.

    from click.testing import CliRunner
from cat import cat

def test_cat():
    runner = CliRunner()
    with runner.isolated_filesystem():
        with open("hello.txt", "w") as f:
            f.write("Hello World!")

        result = runner.invoke(cat, ["hello.txt"])
        assert result.exit_code == 0
        assert result.output == "Hello World!\n"

    Pass temp_dir to control where the temporary directory is created. The directory will not be removed by Click in this case. This is useful to integrate with a framework like Pytest that manages temporary files.

    def test_keep_dir(tmp_path):
    runner = CliRunner()

    with runner.isolated_filesystem(temp_dir=tmp_path) as td:
        ...

Pytest⚑

  • New: The tmp_path fixture.

    You can use the tmp_path fixture which will provide a temporary directory unique to the test invocation, created in the base temporary directory.

    tmp_path is a pathlib.Path object. Here is an example test usage:

    def test_create_file(tmp_path):
    d = tmp_path / "sub"
    d.mkdir()
    p = d / "hello.txt"
    p.write_text(CONTENT)
    assert p.read_text() == CONTENT
    assert len(list(tmp_path.iterdir())) == 1
    assert 0

  • Correction: Deprecate the tmpdir fixture.

    Warning: Don't use tmpdir use tmp_path instead because tmpdir uses py which is unmaintained and has unpatched vulnerabilities.

Python Snippets⚑

  • Correction: Deprecate tmpdir in favour of tmp_path.

  • New: Pad integer with zeros.

    >>> length = 1
>>> print(f'length = {length:03}')
length = 001

  • New: Pathlib make parent directories if they don't exist.

    pathlib.Path("/tmp/sub1/sub2").mkdir(parents=True, exist_ok=True)

    From the docs:

    • If parents is true, any missing parents of this path are created as needed; they are created with the default permissions without taking mode into account (mimicking the POSIX mkdir -p command).

    • If parents is false (the default), a missing parent raises FileNotFoundError.

    • If exist_ok is false (the default), FileExistsError is raised if the target directory already exists.

    • If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file.

  • New: Pathlib touch a file.

    Create a file at this given path.

    pathlib.Path("/tmp/file.txt").touch(exist_ok=True)

    If the file already exists, the function succeeds if exist_ok is true (and its modification time is updated to the current time), otherwise FileExistsError is raised.

    If the parent directory doesn't exist you need to create it first.

    global_conf_path = xdg_home / "autoimport" / "config.toml"
global_conf_path.parent.mkdir(parents=True)
global_conf_path.touch(exist_ok=True)

Elasticsearch⚑

  • New: Get documents that match a string.

    curl \
    -H 'Content-Type: application/json' \
    -XPOST "https://localhost:9200/_search" \
    -d' { "query": { "query_string": {"query": "test company"} }}'

DevOps⚑

Continuous Integration⚑

Operating Systems⚑

Linux⚑

Linux Snippets⚑

  • New: Df and du showing different results.

    Sometimes on a linux machine you will notice that both df command (display free disk space) and du command (display disk usage statistics) report different output. Usually, df will output a bigger disk usage than du.

    The du command estimates file space usage, and the df command shows file system disk space usage.

    There are many reasons why this could be happening:

  • New: Clean up docker data.

    To remove unused docker data you can run docker system prune -a. This will remove:

    • All stopped containers
    • All networks not used by at least one container
    • All images without at least one container associated to them
    • All build cache

    Sometimes that's not enough, and your /var/lib/docker directory still weights more than it should. In those cases:

    • Stop the docker service.
    • Remove or move the data to another directory
    • Start the docker service.

    In order not to loose your persisted data, you need to configure your dockers to mount the data from a directory that's not within /var/lib/docker.

aleph⚑

  • New: Problems accessing redis locally.

    If you're with the VPN connected, turn it off.

  • New: PDB behaves weird.

    Sometimes you have two traces at the same time, so each time you run a PDB command it jumps from pdb trace. Quite confusing. Try to c the one you don't want so that you're left with the one you want. Or put the pdb trace in a conditional that only matches one of both threads.

Other⚑