18th Week of 2023

Coding⚑

Languages⚑

Logql ⚑

New: Introduce LogQL.

LogQL is Grafana Loki’s PromQL-inspired query language. Queries act as if they are a distributed grep to aggregate log sources. LogQL uses labels and operators for filtering.

There are two types of LogQL queries:
- Log queries: Return the contents of log lines.
- Metric queries: Extend log queries to calculate values based on query results.
New: Apply a pattern to the value of a label.

Some logs are sent in json and then one of their fields can contain other structured data. You may want to use that structured data to further filter the logs.
```
{app="ingress-nginx"} | json | line_format `{{.log}}` | pattern `<_> - - <_> "<method> <_> <_>" <status> <_> <_> "<_>" <_>` | method != `GET`
```
- {app="ingress-nginx"}: Show only the logs of the ingress-nginx.
- | json: Interpret the line as a json.
- ``| line_format}| pattern<> - - <> " <> <>" <> <> "<>" <>```: interpret thelog` json field of the trace with the selected pattern
- ``| method !=GET````: Filter the line using a key extracted by the pattern.
New: Count the unique values of a label.

Sometimes you want to alert on the values of a log. For example if you want to make sure that you're receiving the logs from more than 20 hosts (otherwise something is wrong). Assuming that your logs attach a host label you can run
```
sum(count by(host) (rate({host=~".+"} [24h])) > bool 0)
```
This query will: - {host=~".+"}: Fetch all log lines that contain the label host - count by(host) (rate({host=~".+"} [24h]): Calculates the number of entries in the last 24h. - count by(host) (rate({host=~".+"} [24h])) > bool 0: Converts to 1 all the vector elements that have more than 1 message. - sum(count by(host) (rate({host=~".+"} [24h])) > bool 0): Sums all the vector elements to get the number of hosts that have more than one message.

journald promtail parser is known to fail between upgrades, it's useful too to make an alert to make sure that all your hosts are sending the traces. You can do it with: sum(count by(host) (rate({job="systemd-journal"} [24h])) > bool 0)

DevOps⚑

Infrastructure as Code⚑

Gitea ⚑

New: Gitea actions overview.

We've been using Drone as CI runner for some years now as Gitea didn't have their native runner. On Mar 20, 2023 however Gitea released the version 1.19.0 which promoted to stable the Gitea Actions which is a built-in CI system like GitHub Actions. With Gitea Actions, you can reuse your familiar workflows and Github Actions in your self-hosted Gitea instance. While it is not currently fully compatible with GitHub Actions, they intend to become as compatible as possible in future versions. The typical procedure is as follows:
- Register a runner (at the moment, act runners are the only option). This can be done on the following scopes:
- site-wide (by site admins)
- organization-wide (by organization owners)
- repository-wide (by repository owners)
- Create workflow files under .gitea/workflows/<workflow name>.yaml or .github/workflows/<workflow name>.yaml. The syntax is the same as the GitHub workflow syntax where supported.
Gitea Actions advantages are:
- Uses the same pipeline syntax as Github Actions, so it's easier to use for new developers
- You can reuse existent Github actions.
- Migration from Github repositories to Gitea is easier.
- You see the results of the workflows in the same gitea webpage, which is much cleaner than needing to go to drone
- Define the secrets in the repository configuration.
Drone advantages are:
- They have the promote event. Not critical as we can use other git events such as creating a tag.
- They can be run as a service by default. The gitea runners will need some work to run on instance restart.
- Has support for running kubernetes pipelines. Gitea actions doesn't yet support this
New: Setup Gitea actions.

You need a Gitea instance with a version of 1.19.0 or higher. Actions are disabled by default (as they are still an feature-in-progress), so you need to add the following to the configuration file to enable it:
```
[actions]
ENABLED=true
```
Even if you enable at configuration level you need to manually enable the actions on each repository until this issue is solved.

So far there is only one possible runner which is based on docker and act. Currently, the only way to install act runner is by compiling it yourself, or by using one of the pre-built binaries. There is no Docker image or other type of package management yet. At the moment, act runner should be run from the command line. Of course, you can also wrap this binary in something like a system service, supervisord, or Docker container.

Before running a runner, you should first register it to your Gitea instance using the following command:
```
./act_runner register --no-interactive --instance <instance> --token <token>
```
There are two arguments required, instance and token.

instance refers to the address of your Gitea instance, like http://192.168.8.8:3000. The runner and job containers (which are started by the runner to execute jobs) will connect to this address. This means that it could be different from the ROOT_URL of your Gitea instance, which is configured for web access. It is always a bad idea to use a loopback address such as 127.0.0.1 or localhost, as we will discuss later. If you are unsure which address to use, the LAN address is usually the right choice.

token is used for authentication and identification, such as P2U1U0oB4XaRCi8azcngmPCLbRpUGapalhmddh23. It is one-time use only and cannot be used to register multiple runners. You can obtain tokens from your_gitea.com/admin/runners.

After registering, a new file named .runner will appear in the current directory. This file stores the registration information. Please do not edit it manually. If this file is missing or corrupted, you can simply remove it and register again.

Finally, it’s time to start the runner.
```
./act_runner daemon
```
New: Use the gitea actions.

Even if Actions is enabled for the Gitea instance, repositories still disable Actions by default. Enable it on the settings page of your repository.

You will need to study the workflow syntax for Actions and write the workflow files you want.

However, we can just start from a simple demo:
```
name: Gitea Actions Demo
run-name: ${{ gitea.actor }} is testing out Gitea Actions
on: [push]
jobs:
  Explore-Gitea-Actions:
    runs-on: ubuntu-latest
    steps:
      - run: echo "The job was automatically triggered by a ${{ gitea.event_name }} event."
      - run: echo "This job is now running on a ${{ runner.os }} server hosted by Gitea!"
      - run: echo "The name of your branch is ${{ gitea.ref }} and your repository is ${{ gitea.repository }}."
      - name: Check out repository code
        uses: actions/checkout@v3
      - run: echo "The ${{ gitea.repository }} repository has been cloned to the runner."
      - run: echo "The workflow is now ready to test your code on the runner."
      - name: List files in the repository
        run: |
          ls ${{ gitea.workspace }}
      - run: echo "This job's status is ${{ gitea.status }}."
```
You can upload it as a file with the extension .yaml in the directory .gitea/workflows/ or .github/workflows of the repository, for example .gitea/workflows/demo.yaml.

You may be aware that there are tens of thousands of marketplace actions in GitHub. However, when you write uses: actions/checkout@v3, it actually downloads the scripts from gitea.com/actions/checkout by default (not GitHub). This is a mirror of github.com/actions/checkout, but it’s impossible to mirror all of them. That’s why you may encounter failures when trying to use some actions that haven’t been mirrored.

The good news is that you can specify the URL prefix to use actions from anywhere. This is an extra syntax in Gitea Actions. For example:
- uses: https://github.com/xxx/xxx@xxx
- uses: https://gitea.com/xxx/xxx@xxx
- uses: http://your_gitea_instance.com/xxx@xxx
Be careful, the https:// or http:// prefix is necessary!

Storage⚑

OpenZFS ⚑

New: Get compress ratio of a filesystem.
```
zfs get compressratio {{ filesystem }}
```

Authentication⚑

Authentik ⚑

Correction: Finish the installation of prometheus.

Operating Systems⚑

Linux⚑

Jellyfin ⚑

New: Deceptive site ahead.

It seems that Google is marking the domains that host Jellyfin as deceptive. If it happens to you, your users won't be able to access your instance with Firefox, Chrome nor the Android app. Nice uh? It's kind of scary how google is able to control who can access what in the internet without you signing for it.

If you search the problem online they suggest that you log in with your google account into the Search Console and see the reasons behind it. Many people did this and reported in the issue that they didn't get any useful information through this process. It's a privacy violation though, as now google is able to tie your identity (as your google account is linked to your phone number) with your Jellyfin domain. Completely disgusting.

To solve this issue you need to file a case with google and wait for them to unban you. It's like asking them for permission so that they let your users access your system. The disgust levels keep on growing. Don't waste your time being creative in the Comments of the request either, it looks like they don't even read them.

The problem is that until the people from Jellyfin finds a solution, after following this ugly process, you may be flagged again any time in the future (ranging from days to months).

A mitigation of the problem is to have an alternative domain that your users can use (for example in duckdns.org). You may be lucky that google doesn't block both domains at the same time.

For more information follow the Jellyfin issue or the Jellyfin reddit thread.

Vim ⚑

New: Configure Telescope to follow symbolic links.

By default symbolic links are not followed either for files or directories, to enable it use

  require('telescope').setup {
    pickers = {
      find_files = {
        follow = true
      }
    }
  }

Android⚑

LibreTube ⚑

New: Introduce Libretube.

Libretube is an alternative frontend for YouTube, for Android.

YouTube has an extremely invasive privacy policy which relies on using user data in unethical ways. They store a lot of your personal data - ranging from ideas, music taste, content, political opinions, and much more than you think.

This project is aimed at improving the users' privacy by being independent from Google and bypassing their data collection.

Therefore, the app is using the Piped API, which uses proxies to circumvent Google's data collection and includes some other additional features.

Differences to NewPipe:

With NewPipe, the extraction is done locally on your phone, and all the requests sent towards YouTube/Google are done directly from the network you're connected to, which doesn't use a middleman server in between. Therefore, Google can still access information such as the user's IP address. Aside from that, subscriptions can only be stored locally.

LibreTube takes this one step further and proxies all requests via Piped (which uses the NewPipeExtractor). This prevents Google servers from accessing your IP address or any other personal data. Apart from that, Piped allows syncing your subscriptions between LibreTube and Piped, which can be used on desktop too.

If the NewPipeExtractor breaks, it only requires an update of Piped and not LibreTube itself. Therefore, fixes usually arrive faster than in NewPipe.

While LibreTube only supports YouTube, NewPipe also allows the use of other platforms like SoundCloud, PeerTube, Bandcamp and media.ccc.de. Both are great clients for watching YouTube videos. It depends on the individual's use case which one fits their needs better.

Other software that uses Piped:
- Yattee - an alternative frontend for YouTube, for IOS.
- Hyperpipe - an alternative privacy respecting frontend for YouTube Music.
- Musicale - an alternative to YouTube Music, with style.
- ytify - a complementary minimal audio streaming frontend for YouTube.
- PsTube - Watch and download videos without ads on Android, Linux, Windows, iOS, and Mac OSX.
- Piped-Material - A fork of Piped, focusing on better performance and a more usable design.
- ReacTube - Privacy friendly & distraction free Youtube front-end using Piped API.

18th Week of 2023

Coding⚑

Languages⚑

Logql⚑

DevOps⚑

Infrastructure as Code⚑

Gitea⚑

Storage⚑

OpenZFS⚑

Authentication⚑

Authentik⚑

Operating Systems⚑

Linux⚑

Jellyfin⚑

Vim⚑

Android⚑

LibreTube⚑

Logql ⚑

Gitea ⚑

OpenZFS ⚑

Authentik ⚑

Jellyfin ⚑

Vim ⚑

LibreTube ⚑