August of 2022

Projects⚑

• New: Version Update Manager.

  Keeping software updated is not easy because:

  • There are many technologies involved: package managers (apt, yum, pip, yarn, npm, ...), programming languages (python, java, ruby, ...), operative systems (Debian, Ubuntu, ...), deployment technologies (OS install, Docker, Kubernetes, Ansible, Helm), template software (cruft).
  • Each software maintainers use a different version system.
  • Even a small increase in a version may break everything.
  • Sometimes only the latest version is the supported version.
  • It's not easy to check if the update went well.
  • You not only need the desired package to be updated, but also it's dependencies.

  I'd like to find a solution that:

  • Gives an overall insight of the update status of a system.
  • Automates the update process.
  • Support both single system installation or aggregator of multiple systems.

Activism⚑

Antifascism⚑

Antifascist Actions⚑

• New: An open data initiative to map spanish fascist icons.

  The project DeberíaDesaparecer have created an open database of the remains of the spanish fascist regime icons. The visualization they've created is astonishing, and they've provided a form so that anyone can contribute to the dataset.

Life Management⚑

Task Management⚑

Task Management Workflows⚑

• Correction: Update the workflows.

  To suggest to use a script to follow them

Health⚑

Sleep⚑

• New: How your brain generates sleep.

  Brainwave activity of REM sleep looks similar to the one you have when you're awake. They cycle (going up and down) at a fast frequency of thirty or forty times per second in an unreliable pattern. This behaviour is explained by the fact that different parts of your waking brain are processing different pieces of information at different moments in time and in different ways.

Coding⚑

Learning to code⚑

• New: Suggest a workflow to learn Git.

  Git is a software for tracking changes in any set of files, usually used for coordinating work among programmers collaboratively developing source code during software development. Its goals include speed, data integrity, and support for distributed, non-linear workflows (thousands of parallel branches running on different systems).

  Git is a tough nut to crack, no matter how experience you are you'll frequently get surprised. Sadly it's one of the main tools to develop your code, so you must master it as soon as possible.

  I've listed you some resources here on how to start. From that article I think it's also interesting that you read about:

  • Pull Request process
  • Git workflow

Languages⚑

Bash snippets⚑

• New: Do the remainder or modulus of a number.

  expr 5 % 3
  

• New: Update a json file with jq.

  Save the next snippet to a file, for example jqr and add it to your $PATH.

  query="$1"
  file=$2
  
  temp_file="$(mktemp)"
  
  jq "$query" $file > "$temp_file"
  
  cmp -s "$file" "$temp_file"
  if [[ $? -eq 0 ]] ; then
    /bin/rm "$temp_file"
  else
    /bin/mv "$temp_file" "$file"
  fi
  

  Imagine you have the next json file:

  {
    "property": true,
    "other_property": "value"
  }
  

  Then you can run:

  jqr '.property = false' status.json
  

  And then you'll have:

  {
    "property": false,
    "other_property": "value"
  }
  

Dash⚑

• Correction: Deprecate in favour of Streamlit.

  Streamlit is a much more easy, beautiful and clean library for the same purpose.

• New: Running process in background.

  By default, each running command blocks until completion. If you have a long-running command, you can put it in the background with the _bg=True special kwarg:

  sleep(3)
  print("...3 seconds later")
  
  p = sleep(3, _bg=True)
  print("prints immediately!")
  p.wait()
  print("...and 3 seconds later")
  

  You’ll notice that you need to call RunningCommand.wait() in order to exit after your command exits.

  Commands launched in the background ignore SIGHUP, meaning that when their controlling process (the session leader, if there is a controlling terminal) exits, they will not be signalled by the kernel. But because sh commands launch their processes in their own sessions by default, meaning they are their own session leaders, ignoring SIGHUP will normally have no impact. So the only time ignoring SIGHUP will do anything is if you use _new_session=False, in which case the controlling process will probably be the shell from which you launched python, and exiting that shell would normally send a SIGHUP to all child processes.

  If you want to terminate the process use p.kill().

• New: Output callbacks.

  In combination with _bg=True, sh can use callbacks to process output incrementally by passing a callable function to _out and/or _err. This callable will be called for each line (or chunk) of data that your command outputs:

  from sh import tail
  
  def process_output(line):
      print(line)
  
  p = tail("-f", "/var/log/some_log_file.log", _out=process_output, _bg=True)
  p.wait()
  

  To “quit” your callback, simply return True. This tells the command not to call your callback anymore. This does not kill the process though see Interactive callbacks for how to kill a process from a callback.

  The line or chunk received by the callback can either be of type str or bytes. If the output could be decoded using the provided encoding, a str will be passed to the callback, otherwise it would be raw bytes.

Qwik⚑

• New: Introduce Qwik.

  Qwik is a new kind of web framework that can deliver instantly load web applications at any size or complexity. Your sites and apps can boot with about 1kb of JS (regardless of application complexity), and achieve consistent performance at scale.

  You can see a good overview in the Qwik presentation.

Pytest⚑

• New: Change log level of a dependency.

  caplog.set_level(logging.WARNING, logger="urllib3")
  

• New: Show logging messages on the test run.

  Add to your pyproject.toml:

  [tool.pytest.ini_options]
     log_cli = true
     log_cli_level = 10
  

  Or run it in the command itself pytest -o log_cli=true --log-cli-level=10 func.py.

  Remember you can change the log level of the different components in case it's too verbose.

Python Snippets⚑

• New: Initialize a dataclass with kwargs.

  If you care about accessing attributes by name, or if you can't distinguish between known and unknown arguments during initialisation, then your last resort without rewriting __init__ (which pretty much defeats the purpose of using dataclasses in the first place) is writing a @classmethod:

  from dataclasses import dataclass
  from inspect import signature
  
  @dataclass
  class Container:
      user_id: int
      body: str
  
      @classmethod
      def from_kwargs(cls, **kwargs):
          # fetch the constructor's signature
          cls_fields = {field for field in signature(cls).parameters}
  
          # split the kwargs into native ones and new ones
          native_args, new_args = {}, {}
          for key, value in kwargs.items():
              if key in cls_fields:
                  native_args[key] = value
              else:
                  new_args[key] = value
  
          # use the native ones to create the class ...
          ret = cls(**native_args)
  
          # ... and add the new ones by hand
          for new_key, new_value in new_args.items():
              setattr(ret, new_key, new_value)
          return ret
  

  Usage:

  params = {'user_id': 1, 'body': 'foo', 'bar': 'baz', 'amount': 10}
  Container(**params)  # still doesn't work, raises a TypeError
  c = Container.from_kwargs(**params)
  print(c.bar)  # prints: 'baz'
  

• New: Replace a substring of a string.

  txt = "I like bananas"
  
  x = txt.replace("bananas", "apples")
  

• New: Create random number.

  import random
  a=random.randint(1,10)
  

• New: Check if local port is available or in use.

  Create a temporary socket and then try to bind to the port to see if it's available. Close the socket after validating that the port is available.

  def port_in_use(port):
      """Test if a local port is used."""
      sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      with suppress(OSError):
          sock.bind(("0.0.0.0", port))
          return True
      sock.close()
      return False
  

Pydantic⚑

• New: Ignore a field when representing an object.

  Use repr=False. This is useful for properties that don't return a value quickly, for example if you save an sh background process.

  class Temp(BaseModel):
      foo: typing.Any
      boo: typing.Any = Field(..., repr=False)
  

sh⚑

• New: Avoid exception logging when killing a background process.

  In order to catch this exception execute your process with _bg_exec=False and execute p.wait() if you want to handle the exception. Otherwise don't use the p.wait().

  p = sh.sleep(100, _bg=True, _bg_exc=False)
  try:
      p.kill()
      p.wait()
  except sh.SignalException_SIGKILL as err:
      print("foo")
  
  foo
  

Git⚑

• Correction: Update the git flow to match my current one.

DevOps⚑

Infrastructure Solutions⚑

Kubectl⚑

• New: Port forward / Tunnel to an internal service.

  If you have a service running in kubernetes and you want to directly access it instead of going through the usual path, you can use kubectl port-forward.

  kubectl port-forward allows using resource name, such as a pod name, service replica set or deployment, to select the matching resource to port forward to. For example, the next commands are equivalent:

  kubectl port-forward mongo-75f59d57f4-4nd6q 28015:27017
  kubectl port-forward deployment/mongo 28015:27017
  kubectl port-forward replicaset/mongo-75f59d57f4 28015:27017
  kubectl port-forward service/mongo 28015:27017
  

  The output is similar to this:

  Forwarding from 127.0.0.1:28015 -> 27017
  Forwarding from [::1]:28015 -> 27017
  

  If you don't need a specific local port, you can let kubectl choose and allocate the local port and thus relieve you from having to manage local port conflicts, with the slightly simpler syntax:

  $: kubectl port-forward deployment/mongo :27017
  
  Forwarding from 127.0.0.1:63753 -> 27017
  Forwarding from [::1]:63753 -> 27017
  

• New: Run a command against a specific context.

  If you have multiple contexts and you want to be able to run commands against a context that you have access to but is not your active context you can use the --context global option for all kubectl commands:

  kubectl get pods --context <context_B>
  

  To get a list of available contexts use kubectl config get-contexts

Monitoring⚑

Prometheus⚑

• New: Accessing Prometheus metrics through python.

  import requests
  
  response = requests.get(
      "http://127.0.0.1:9090/api/v1/query",
      params={"query": "container_cpu_user_seconds_total"},
  )
  

Operating Systems⚑

Linux⚑

Linux Snippets⚑

• New: Scan a physical page in Linux.

  Install xsane and run it.

Pipx⚑

• New: Introduce pipx.

  Pipx is a command line tool to install and run Python applications in isolated environments.

  Very useful not to pollute your user or device python environments.

  Install it with:

  pip install pipx
  

Arts⚑

Cooking⚑

Cooking software⚑

• New: Analysis of existing recipe manager software.

  List the expected features from the recipe manager and add links of the software found after an analysis of the state of the art, it's still a work in progress

Other⚑

• Reorganization: Reorder the programming languages under a Languages section.

• New: Bear with me or Bare with me.

  "Bear with me" is the correct form.