August of 2022
Projects⚑
-
Keeping software updated is not easy because:
- There are many technologies involved: package managers (apt, yum, pip, yarn, npm, ...), programming languages (python, java, ruby, ...), operative systems (Debian, Ubuntu, ...), deployment technologies (OS install, Docker, Kubernetes, Ansible, Helm), template software (cruft).
- Each software maintainers use a different version system.
- Even a small increase in a version may break everything.
- Sometimes only the latest version is the supported version.
- It's not easy to check if the update went well.
- You not only need the desired package to be updated, but also it's dependencies.
I'd like to find a solution that:
- Gives an overall insight of the update status of a system.
- Automates the update process.
- Support both single system installation or aggregator of multiple systems.
Activism⚑
Antifascism⚑
Antifascist Actions⚑
-
New: An open data initiative to map spanish fascist icons.
The project DeberíaDesaparecer have created an open database of the remains of the spanish fascist regime icons. The visualization they've created is astonishing, and they've provided a form so that anyone can contribute to the dataset.
Life Management⚑
Task Management⚑
Task Management Workflows⚑
-
Correction: Update the workflows.
To suggest to use a script to follow them
Health⚑
Sleep⚑
-
New: How your brain generates sleep.
Brainwave activity of REM sleep looks similar to the one you have when you're awake. They cycle (going up and down) at a fast frequency of thirty or forty times per second in an unreliable pattern. This behaviour is explained by the fact that different parts of your waking brain are processing different pieces of information at different moments in time and in different ways.
Coding⚑
Learning to code⚑
-
New: Suggest a workflow to learn Git.
Git is a software for tracking changes in any set of files, usually used for coordinating work among programmers collaboratively developing source code during software development. Its goals include speed, data integrity, and support for distributed, non-linear workflows (thousands of parallel branches running on different systems).
Git is a tough nut to crack, no matter how experience you are you'll frequently get surprised. Sadly it's one of the main tools to develop your code, so you must master it as soon as possible.
I've listed you some resources here on how to start. From that article I think it's also interesting that you read about:
Languages⚑
Bash snippets⚑
-
New: Do the remainder or modulus of a number.
expr 5 % 3
-
New: Update a json file with jq.
Save the next snippet to a file, for example
jqr
and add it to your$PATH
.query="$1" file=$2 temp_file="$(mktemp)" jq "$query" $file > "$temp_file" cmp -s "$file" "$temp_file" if [[ $? -eq 0 ]] ; then /bin/rm "$temp_file" else /bin/mv "$temp_file" "$file" fi
Imagine you have the next json file:
{ "property": true, "other_property": "value" }
Then you can run:
jqr '.property = false' status.json
And then you'll have:
{ "property": false, "other_property": "value" }
Dash⚑
-
Correction: Deprecate in favour of Streamlit.
Streamlit is a much more easy, beautiful and clean library for the same purpose.
-
New: Running process in background.
By default, each running command blocks until completion. If you have a long-running command, you can put it in the background with the
_bg=True
special kwarg:sleep(3) print("...3 seconds later") p = sleep(3, _bg=True) print("prints immediately!") p.wait() print("...and 3 seconds later")
You’ll notice that you need to call
RunningCommand.wait()
in order to exit after your command exits.Commands launched in the background ignore
SIGHUP
, meaning that when their controlling process (the session leader, if there is a controlling terminal) exits, they will not be signalled by the kernel. But becausesh
commands launch their processes in their own sessions by default, meaning they are their own session leaders, ignoringSIGHUP
will normally have no impact. So the only time ignoringSIGHUP
will do anything is if you use_new_session=False
, in which case the controlling process will probably be the shell from which you launched python, and exiting that shell would normally send aSIGHUP
to all child processes.If you want to terminate the process use
p.kill()
. -
New: Output callbacks.
In combination with
_bg=True
,sh
can use callbacks to process output incrementally by passing a callable function to_out
and/or_err
. This callable will be called for each line (or chunk) of data that your command outputs:from sh import tail def process_output(line): print(line) p = tail("-f", "/var/log/some_log_file.log", _out=process_output, _bg=True) p.wait()
To “quit” your callback, simply
return True
. This tells the command not to call your callback anymore. This does not kill the process though see Interactive callbacks for how to kill a process from a callback.The line or chunk received by the callback can either be of type str or bytes. If the output could be decoded using the provided encoding, a str will be passed to the callback, otherwise it would be raw bytes.
Qwik⚑
-
New: Introduce Qwik.
Qwik is a new kind of web framework that can deliver instantly load web applications at any size or complexity. Your sites and apps can boot with about 1kb of JS (regardless of application complexity), and achieve consistent performance at scale.
You can see a good overview in the Qwik presentation.
Pytest⚑
-
New: Change log level of a dependency.
caplog.set_level(logging.WARNING, logger="urllib3")
-
New: Show logging messages on the test run.
Add to your
pyproject.toml
:[tool.pytest.ini_options] log_cli = true log_cli_level = 10
Or run it in the command itself
pytest -o log_cli=true --log-cli-level=10 func.py
.Remember you can change the log level of the different components in case it's too verbose.
Python Snippets⚑
-
New: Initialize a dataclass with kwargs.
If you care about accessing attributes by name, or if you can't distinguish between known and unknown arguments during initialisation, then your last resort without rewriting
__init__
(which pretty much defeats the purpose of using dataclasses in the first place) is writing a@classmethod
:from dataclasses import dataclass from inspect import signature @dataclass class Container: user_id: int body: str @classmethod def from_kwargs(cls, **kwargs): # fetch the constructor's signature cls_fields = {field for field in signature(cls).parameters} # split the kwargs into native ones and new ones native_args, new_args = {}, {} for key, value in kwargs.items(): if key in cls_fields: native_args[key] = value else: new_args[key] = value # use the native ones to create the class ... ret = cls(**native_args) # ... and add the new ones by hand for new_key, new_value in new_args.items(): setattr(ret, new_key, new_value) return ret
Usage:
params = {'user_id': 1, 'body': 'foo', 'bar': 'baz', 'amount': 10} Container(**params) # still doesn't work, raises a TypeError c = Container.from_kwargs(**params) print(c.bar) # prints: 'baz'
-
New: Replace a substring of a string.
txt = "I like bananas" x = txt.replace("bananas", "apples")
-
import random a=random.randint(1,10)
-
New: Check if local port is available or in use.
Create a temporary socket and then try to bind to the port to see if it's available. Close the socket after validating that the port is available.
def port_in_use(port): """Test if a local port is used.""" sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) with suppress(OSError): sock.bind(("0.0.0.0", port)) return True sock.close() return False
Pydantic⚑
-
New: Ignore a field when representing an object.
Use
repr=False
. This is useful for properties that don't return a value quickly, for example if you save ansh
background process.class Temp(BaseModel): foo: typing.Any boo: typing.Any = Field(..., repr=False)
sh⚑
-
New: Avoid exception logging when killing a background process.
In order to catch this exception execute your process with
_bg_exec=False
and executep.wait()
if you want to handle the exception. Otherwise don't use thep.wait()
.p = sh.sleep(100, _bg=True, _bg_exc=False) try: p.kill() p.wait() except sh.SignalException_SIGKILL as err: print("foo") foo
Git⚑
- Correction: Update the git flow to match my current one.
DevOps⚑
Infrastructure Solutions⚑
Kubectl⚑
-
New: Port forward / Tunnel to an internal service.
If you have a service running in kubernetes and you want to directly access it instead of going through the usual path, you can use
kubectl port-forward
.kubectl port-forward
allows using resource name, such as a pod name, service replica set or deployment, to select the matching resource to port forward to. For example, the next commands are equivalent:kubectl port-forward mongo-75f59d57f4-4nd6q 28015:27017 kubectl port-forward deployment/mongo 28015:27017 kubectl port-forward replicaset/mongo-75f59d57f4 28015:27017 kubectl port-forward service/mongo 28015:27017
The output is similar to this:
Forwarding from 127.0.0.1:28015 -> 27017 Forwarding from [::1]:28015 -> 27017
If you don't need a specific local port, you can let
kubectl
choose and allocate the local port and thus relieve you from having to manage local port conflicts, with the slightly simpler syntax:$: kubectl port-forward deployment/mongo :27017 Forwarding from 127.0.0.1:63753 -> 27017 Forwarding from [::1]:63753 -> 27017
-
New: Run a command against a specific context.
If you have multiple contexts and you want to be able to run commands against a context that you have access to but is not your active context you can use the
--context
global option for allkubectl
commands:kubectl get pods --context <context_B>
To get a list of available contexts use
kubectl config get-contexts
Monitoring⚑
Prometheus⚑
-
New: Accessing Prometheus metrics through python.
import requests response = requests.get( "http://127.0.0.1:9090/api/v1/query", params={"query": "container_cpu_user_seconds_total"}, )
Operating Systems⚑
Linux⚑
Linux Snippets⚑
-
New: Scan a physical page in Linux.
Install
xsane
and run it.
Pipx⚑
-
New: Introduce pipx.
Pipx is a command line tool to install and run Python applications in isolated environments.
Very useful not to pollute your user or device python environments.
Install it with:
pip install pipx
Arts⚑
Cooking⚑
Cooking software⚑
-
New: Analysis of existing recipe manager software.
List the expected features from the recipe manager and add links of the software found after an analysis of the state of the art, it's still a work in progress
Other⚑
- Reorganization: Reorder the programming languages under a Languages section.
-
New: Bear with me or Bare with me.
"Bear with me" is the correct form.