Skip to content

Projects

There is an ever growing pool of ideas where I want to invest my time. Sadly time is a finite currency, and even though I am lucky enough to be able to put my focus on maximizing it, it's never enough.

I understand projects as a mental tool that groups ideas, processes and tools to achieve a specific goal. Following the digital garden metaphor, projects are plants in different phases of development where I've spent a different amount of effort.

The development phases are:

  • Seeds: Are raw, basic ideas of projects that may once be.

  • Seedlings: Are projects that don't yet have their first stable version, but the drafts of the ADR and some code is already written.

  • Growing: Projects that have a stable release and are under active development.

  • Dormant: Projects whose growth has temporally stopped. I still believe they are useful and even though I don't want to work on them at the moment, I see myself doing it in the future.

    I still maintain them by answering to issues, reviewing pull requests, keeping the continuous integration pipelines alive and developing fixes to important issues.

  • Dying: Projects that I know are going to be deprecated soon, and I'm looking for alternatives.

  • Dead: Project no longer used.

Growing plants⚑

Blue book⚑

What you're reading right now. I'm storing most of the new knowledge I learn every day. At the same time I'm migrating the notes of the previous version of this digital garden which consists on 7422 articles, almost 50 million lines.

Repository ORM⚑

I'm creating a Python library to make it easier to use the repository pattern in new projects.

I monthly spin up new ideas for programs, and managing the storage of the information is cumbersome and repeating. My idea is to refactor that common codebase into a generic library that anyone can use.

Pynbox⚑

I wanted a system to improve the management of ideas, tasks, references, suggestions when I'm not in front of the computer. Right now I've got Markor for Android to register these quicknotes, but the reality is that I don't act upon them, so it's just a log of tasks that never get done, and ideas, references and suggestions that aren't registered in my knowledge or media management systems.

On the computer there are also cases of tasks that are not worth registering in the task management system, or ideas that I get at a moment but don't have time to process at the moment.

The idea then is to automatically sync the Android quicknote with syncthing, and have a special format for the file that allows pynbox to extract the elements from that file to the "inbox system". For example:

t. buy groceries
tv. IT crowd
i. Improve the inbox management

I want a system to improve ...

Gets introduced in the "inbox system" as a task, a TV suggestion and an idea.

Dormant Plants⚑

mkdocs-newsletter⚑

MkDocs plugin to show the changes of documentation repositories in a user friendly format, at the same time that it's easy for the authors to maintain.

It creates daily, weekly, monthly and yearly newsletter articles with the changes of each period. Those pages, stored under the Newsletters section, are filled with the changes extracted from the commit messages of the git history. The changes are grouped by categories, subcategories and then by file using the order of the site's navigation structure. RSS feeds are also created for each newsletter type, so it's easy for people to keep updated with the evolution of the site.

I use it for this site newsletters.

Autoimport⚑

Throughout the development of a python program you continuously need to manage the python import statements either because you need one new object or because you no longer need it. This means that you need to stop writing whatever you were writing, go to the top of the file, create or remove the import statement and then resume coding.

This workflow break is annoying and almost always unnecessary. autoimport solves this problem if you execute it whenever you have an import error, for example by configuring your editor to run it when saving the file.

The reasons why it is dormant are:

Clinv⚑

As part of my DevSecOps work, I need to have an updated inventory of cloud assets organized under a risk management framework.

As you can see in how do you document your infrastructure?, there is still a void on how to maintain an inventory of dynamic resources with a DevSecOps point of view.

  • Manage a dynamic inventory of risk management resources (Projects, Services, Information, People) and infrastructure resources (EC2, RDS, S3, Route53, IAM users, IAM groups…).
  • Add risk management metadata to your AWS resources.
  • Monitor if there are resources that are not inside your inventory.
  • Perform regular expression searches on all your resources.
  • Get all your resources information.
  • Works from the command line.

So I started building clinv.

yamlfix⚑

A simple opinionated yaml formatter that keeps your comments.

The reasons why it is dormant are:

  • The current features cover most of needs.
  • My hype is elsewhere.

bruty⚑

Python program to bruteforce dynamic web applications with Selenium.

Cookiecutter Python template⚑

Following the same reasoning as the previous section, I've spent a lot of time investigating quality measures for python projects, such as project structure, ci testing, ci building, dependency management, beautiful docs or pre-commits. With the cookiecutter template, it is easy to create a new project with the best quality measures with zero effort. Furthermore, with cruft I can keep all the projects generated with the template updated with the best practices.

Mediarss⚑

I've always wanted to own the music I listen, because I don't want to give my data to the companies host the streaming services, nor I trust that they'll keep on giving the service.

So I started building some small bash scrappers (I wasn't yet introduced to Python) to get the media. That's when I learned to hate the web developers for their constant changes and to love the API.

Then I discovered youtube-dl, a Python command-line program to download video or music from streaming sites. But I still laked the ability to stay updated with the artist channels.

So mediarss was born. A youtube-dl wrapper to periodically download new content.

This way, instead of using Youtube, Soundcloud or Bandcamp subscriptions, I've got a YAML with all the links that I want to monitor.

Playlist_generator⚑

When my music library started growing due to mediarss, I wanted to generate playlists filtering my content by:

  • Rating score fetched with mep.
  • First time/last listened.
  • Never listened songs.

The playlists I usually generate with these filters are:

  • Random unheard songs.
  • Songs discovered last month/year with a rating score greater than X.
  • Songs that I haven't heard since 20XX with a rating score greater than X (this one gave me pleasant surprises ^^).

Media indexation⚑

I've got a music collection of more than 136362 songs, belonging to mediarss downloads, bought CDs rips and friend library sharing. It is more less organized in a directory tree by genre, but I lack any library management features. I've got a lot of duplicates, incoherent naming scheme, no way of filtering or intelligent playlist generation.

playlist_generator helped me with the last point, based on the metadata gathered with mep, but it's still not enough.

So I'm in my way of migrate all the library to beets, and then I'll deprecate mep in favor to a mpd client that allows me to keep on saving the same metadata.

Once it's implemented, I'll migrate all the metadata to the new system.

Home Stock inventory⚑

I try to follow the idea of emptying my mind as much as possible, so I'm able to spend my CPU time wisely.

Keeping track of what do you have at home or what needs to be bought is an effort that should be avoided.

So I've integrated Grocy in my life.

Drode⚑

drode is a wrapper over the Drone and AWS APIs to make deployments more user friendly.

It assumes that the projects are configured to continuous deliver all master commits to staging. Then those commits can be promoted to production or to staging for upgrades and rollbacks.

It has the following features:

  • Prevent failed jobs to be promoted to production.
  • Promote jobs with less arguments than the drone command line.
  • Wait for a drone build to end, then raise the terminal bell.

Create an ordered list of digital gardens⚑

A best-of-lists compilation of awesome list of digital gardens.

Seedlings⚑

Learn about antifascism⚑

I'm summing up the key insights of Mark's and Pol's awesome books in this article

Automate email management⚑

Most of the emails I receive require repetitive actions that can be automated, in the Email management article I'm gathering the steps to setup such an infrastructure.

Learn about sleep⚑

I'm reading Matthew Walker's awesome book Why we sleep and summing up the key insights in this article

Life⚑

Life is a real time sandbox role game where you play as yourself surviving in today's world.

Self hosted map⚑

I love maps, as well as traveling and hiking. This project aims to create a web interface that let's me interact with the data gathered throughout my life. I'd like to:

  • Browse the waypoints and routes that I've done.
  • Create routes and export the gpx.
  • Be able to search through the data
  • Plan trips

All the data must live in my servers.

I first started with umap but it stopped being responsive when you add many points, and it's not easy to self-host. Then I went with folium, but it lacked the interactively I wanted, so I ended up using dash leaflet.

The first phase of the project is to be able to import and browse the existing data. A second phase will be to add the routing functionality, maybe with Leaflet Routing Machine, which will probably need a self-hosted OSRM server. Meanwhile I'm using the brouter-web that uses brouter behind the scenes.

Seeds⚑

This is a gathering of tools, ideas or services that I'd like to enjoy.

If you have any lead, as smallest as it may be on how to fulfill them, please contact me.

Beancount forecast⚑

I'd like to see a forecast of the evolution of my accounts given an amount of time. Maybe by doing seasonality analysis and forecast in time series as stated here and here.

It will also be interesting to see for a given account the evolution of the subaccounts.

Life warnings⚑

I've always tackled the pursuit of the peace of mind by improving in task management, for example trying to predict when I have to do something in order to avoid a nuisance. Maybe it's more interesting to monitor and visibilice the warnings that are affecting you.

Once a warning is active you need to keep track of its evolve throughout time maybe with a simple questionary. This track could be both quantitative (bother level from 1 to 10), and qualitative (a description of the state and evolution).

The user will have a view (probably command line interface) of the active warnings, their priority, bother level, bother level change, last update, number of days it's been active, predicted end. Another interesting view can be the evolution of solved warnings and new warnings.

Warnings are not completely solved, as they can happen again. That way we can see how often do they reactivate, and maybe we can get an estimate of the next occurrence. It can also be interesting to see how long did the last warnings last and the effectiveness of the different actions taken to solve it.

We'll need an easy way to filter and merge warnings so as not to repeat similar ones. I think it's better to prioritize tags over categories to avoid hierarchy problems. For example, we can track a headache with the tags health and head.

The activation/deactivation of warnings will change with the warning type:

  • Manual: User triggers the state change through the program's command line

  • Script: A python or bash script that automatically gathers the data and decides if the warning is still active or inactive.

    This method can give a huge boost in the motivation of self logging through quantified self.

Pomodoro command line⚑

Command line to help with the pomodoro workflow, besides the basic stuff it will interact with the task manager, activitywatch and the notifications system so that:

  • If you are in focus mode, the notifications will be deactivated, once the pomodoro cycle ends, the notifications will show up.
  • If you are in focus mode, and you check the notification applications, a notification warning will be shown.
  • As you'll check the notification systems between pomodoro cycles, unless you start the pomodoro cycle in focus mode, it's assumed that you may need to interact with them, but if X amount of minutes has passed since the start of the cycle and you haven't seen them, then it's assumed that you are in focus mode, and therefore the notifications will be deactivated.
  • When you start a pomodoro cycle it will let you activate one of the task manager tasks, so it will track the time spent in that task. If you change the window manager focus to a window that is not related to the task at hand it will stop recording and show you a warning.

Version Update Manager⚑

Keeping software updated is not easy because:

  • There are many technologies involved: package managers (apt, yum, pip, yarn, npm, ...), programming languages (python, java, ruby, ...), operative systems (Debian, Ubuntu, ...), deployment technologies (OS install, Docker, Kubernetes, Ansible, Helm), template software (cruft).
  • Each software maintainers use a different version system.
  • Even a small increase in a version may break everything.
  • Sometimes only the latest version is the supported version.
  • It's not easy to check if the update went well.
  • You not only need the desired package to be updated, but also it's dependencies.

I'd like to find a solution that:

  • Gives an overall insight of the update status of a system.
  • Automates the update process.
  • Support both single system installation or aggregator of multiple systems.

Shared accounting⚑

I use beancount for my personal accounting, I'd like to have a system that integrates more less easily with beancount and let's do a shared accounting with other people, for example in trips. I've used settle up in the past but it requires access to their servers, and an account linked to google, facebook or one you register in their servers.

I've looked at facto but it uses a logic that doesn't apply to my case, it does a heavy use on a common account, instead of minimizing the transactions between the people. I also tried tabby, even though they still don't support Docker, but it doesn't suit my case either :(.

Until a new solution shows up, I'll go with Tricky Tripper available in F-Droid, and manage the expenses myself and periodically send the html reports to the rest of the group.

Improve the reliability of the Open Science collections⚑

The current free knowledge efforts are based on the health of a collection of torrents.

Something that is needed is a command line tool that reads the list of ill torrents, and downloads the ones that have a low number of seeders and DHT peers. The number of torrents to download could be limited by the amount the user wants to share. A second version could have an interaction with the torrent client so that when a torrent is no longer ill, it's automatically replaced with one that is.

Once the tool is built:

Monitor and notify on disk prices⚑

Diskprices.com sorts the prices of the disks on the different amazon sites based on many filters. It will be interesting to have a service that monitors the data on this site and alerts the user once there is a deal that matches its criteria.

Once it's done, promote it in the DataHoarder reddit.

Switch to a better browser⚑

Firefox + Trydactil has it's limits, and Firefox has been following an ill path for a while, qutebrowser looks like the perfect replacement.

I've just stumbled upon nyxt (code), and it looks superb.

Automating computer file management⚑

Today I've stumbled upon organize looks good for automating processes on files. Maybe it's interesting to run it with inotifywait instead of with a cron job.

Self hosted search engine⚑

It would be awesome to be able to self host a personal search engine that performs prioritized queries in the data sources that I choose.

This idea comes from me getting tired of:

  • Forgetting to search in my gathered knowledge before going to the internet.
  • Not being able to prioritize known trusted sources.

Some sources I'd like to query:

Each source should be added as a plugin to let people develop their own.

I'd also like to be able to rate in my browsing the web pages so they get more relevance in future searches. That rating can also influence the order of the different sources.

It will archive the rated websites to avoid link rot maybe with ipwb.

If we use a knowledge graph, we could federate to ask other nodes and help discover or prioritize content.

The browsing could be related with knowledge graph tags.

We can also have integration with Anki after a search is done.

A possible architecture could be:

  • A flask + Reactjs frontend.
  • An elasticsearch instance for persistence.
  • A Neo4j or knowledge graph to get relations.

It must be open sourced and Linux compatible. And it would be awesome if I didn't have to learn how to use another editor.

Maybe meilisearch (follow their blog) or searx could be a solution. Following another approach, archivy looks good too.

Or I could use Jina is a search library linked to pydantic, or maybe quickwit if they're more stable and mature than right now.

If I've already started the quantified self project, maybe adri's memex is a good solution.

Quantified self⚑

I've been gathering data about myself for a long time, but I don't have a centralized solution that lets me extract information.

There are already good solutions to start with, being the best HPI:

For the interface adri's memex looks awesome! It's inspired in the Andrew Louis talk Building a Memex whose blog posts seems to be a gold mine.

Also look at hpi's compilation and the awesome-quantified-self resources.

Improve the way of launching applications with i3wm⚑

In the past I tried installing rofi without success, I should try again. If the default features are not enough, check adi1090x's custom resources.

Improve the notification management in Linux⚑

I want to be able to group and silence the notifications under a custom logic. For example:

  • If I want to focus on a task, only show the most important ones.
  • Only show alerts once every X minutes. Or define that I want to receive them the first 10 minutes of every hour.
  • If I'm not working, silence all work alerts.

From what I see dunst notification manager supports rules and filters, if it's not powerful enough, I may use it with a custom script that uses apprise.

Check Archlinux dunst wiki page and the source code too.

Improve the hard drive monitor system⚑

Use something like scrutiny (there's a linuxserver image) to collect and display the information. For alerts, use one of their supported providers.

Improve the periodic tasks and application metrics monitoring⚑

Setup an healthchecks instance with the linuxserver image to monitor cronjobs.

For the notifications either use the prometheus metrics or an apprise compatible system.

See the source code here.

Aggregate all notifications⚑

Instead of reading the email, github, gitlab, discourse, reddit notifications, aggregate all in one place and show them to the user in a nice command line interface.

For the aggregator server, my first choice would be gotify.

Decentralized encrypted end to end VOIP and video software⚑

I'd like to be able to make phone and video calls keeping in mind that:

  • Every connection must be encrypted end to end.
  • I trust the security of a linux server more than a user device. This rules out distributed solutions such as tox that exposes the client IP in a DHT table.
  • The server solution should be self hosted.
  • It must use tested cryptography, which again rolls out tox.

These are the candidates I've found:

  • Riot. You'll need to host your own Synapse server.
  • Jami. I think it can be configured as decentralized if you host your own DHTproxy, bootstrap and nameserver, but I need to delve further into how it makes a call. I'm not sure, but you'll probably need to use push notifications so as not to expose a service from the user device.
  • Linphone. If we host our Flexisip server, although it asks for a lot of permissions.

Jitsi Meet it's not an option as it's not end to end encrypted. But if you want to use it, please use Disroot service or host your own.

Self hosted voice assistant⚑

Host a virtual assistant in my servers to help me automate repetitive stuff.

For offline voice recognition, vosk-api can be used. Or voiceliner once it supports offline voice recognition.

Others⚑

  • A tool or service to follow the updates of software, right now I subscribe to the releases of the github repositories, but I'd like it to be provider agnostic, and cleaner than the emails github sends.
  • Movie/serie/music rating self hosted solution that based on your ratings discovers new content.
  • Hiking route classifier and rating self hosted web application.
  • A command line friendly personal CRM like Monica that is able to register the time length and rating of interactions to do data analysis on my relations.
  • Digital e-ink note taking system that is affordable, self hosted and performs character recognition.
  • A git issue tracker that keeps the context of why I subscribed to them. Until I find one, I'll use the issues document.
  • An easy way of creating markdown links to other file's sections. Similar to mkdx functionality. I tried using it but it didn't work for me, and I don't know if it works for other files.
  • A markdown formatter that fixes the indentation of lists.
  • An e-reader support that could be fixed to the wall.

Dying plants⚑

mep⚑

I started life logging with mep. One of the first programs I wrote when learning Bash.

It is a mplayer wrapper that allows me to control it with i3 key bindings and store metadata of the music I listen.

I don't even publish it because it's a horrible program that would make your eyes bleed. 600 lines of code, only 3 functions, 6 levels of nested ifs, no tests at all, but hey, the functions have docstrings! (ｏ・_・)ノ”(ᴗ_ ᴗ。)

The thing is that it works, so I everyday close my eyes and open my ears, waiting for a solution that gives me the same features with mpd.

faker-optional⚑

Wrapper over other Faker providers to return their value or None. Useful to create data of type Optional[Any].

Not needed anymore as I use pydantic factories now.