DEV Community: Yujin

Installing Psycopg2 in Macos

Yujin — Fri, 29 Jul 2022 00:45:15 +0000

TL;DR

env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install psycopg2


# OR

env LDFLAGS="-I/usr/local/opt/openssl@3/include -L/usr/local/opt/openssl@3/lib" pip install psycopg2

# OR

# https://github.com/brianmario/mysql2/issues/795#issuecomment-337006164

env LIBRARY_PATH="/usr/local/opt/openssl@3/lib" pip install psycopg2 --no-cache

Source: https://stackoverflow.com/questions/26288042/error-installing-psycopg2-library-not-found-for-lssl/39244687#39244687

The reason is that probably it might work faster in my machine if it was installed using the not binary version

Make your Django project newbie contributor friendly with pre-commit

Yujin — Sun, 04 Jul 2021 15:03:11 +0000

It’s really worth investing time configuring your project and make it easy for other developers to contribute.

One way of enabling this is a clean, organized, and well-formatted code.

This is really helpful especially for first time developers or contributors as it makes pull request reviews less painful (e.g. trailing white space, unorganized imports, debug statements)

We can easily prevent this by using a tool called pre-commit. Let’s get started!

Installation

Installing pre-commit

Just pick one installation method

Install using their official installer if you’re on any unix based distribution website

$ curl https://pre-commit.com/install-local.py | python -

Install with pip OR just add pre-commit to your requirements.txt or requirements-dev.txt.

$ pip install pre-commit

Installing Django (optional)

If you already have an existing repository, you can skip this step.

This isn’t gonna be a Django tutorial so we’ll just do the bare minimum setup

$ mkdir mysite
$ python -m venv venv # Create a virtualenv called `venv`
$ source venv/bin/activate
$ pip install Django
$ django-admin startproject .

Let’s initialize it as a git repository since pre-commit only works with git repositories.

$ git init .
$ echo "venv" >> .gitignore
$ git add .
$ git commit -m "Initial setup"

Configuring pre-commit

Now, on your root directory, let’s create file called .pre-commit-config.yaml. pre-commit looks for this file so it knows what kind of hooks to run when we are committing.

$ touch .pre-commit-config.yaml

Copy and paste this inside the newly created file

repos:
  - repo: 'https://github.com/pre-commit/pre-commit-hooks'
    rev: v3.4.0
    hooks:
      - id: trailing-whitespace
      - id: check-yaml
      - id: check-merge-conflict
      - id: debug-statements
      - id: check-added-large-files
      - id: requirements-txt-fixer
  - repo: local
    hooks:
      - id: django-check
        name: Check django project for potential problems
        entry: sh -c 'python manage.py check'
        types:
          - python
        pass_filenames: false
        language: system
      - id: django-check-migrations
        name: Check django project for missing migrations.
        entry: sh -c 'python manage.py makemigrations --check --dry-run'
        files: models.py
        types:
          - python
        pass_filenames: false
        language: system
  - repo: 'https://gitlab.com/pycqa/flake8'
    rev: 3.9.0
    hooks:
      - id: flake8
  - repo: 'https://github.com/pycqa/isort'
    rev: 5.8.0
    hooks:
      - id: isort
  - repo: 'https://github.com/python/black'
    rev: 20.8b1
    hooks:
      - id: black

It might look overwhelming but you just need to focus on 2 things here: the repo and hooks. If you read it carefully, the hooks are kinda self-explanatory. These hooks gets run in order every time you commit. Let’s see it in action.

Let’s install the hooks and add .pre-commit-config.yaml to git:

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit
$ git add .pre-commit-config.yaml
$ git commit -m "Add pre-commit config"
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.
[INFO] Initializing environment for https://github.com/pycqa/isort.
[INFO] Initializing environment for https://github.com/python/black.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://gitlab.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pycqa/isort.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/python/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Trim Trailing Whitespace.................................................Passed
Check Yaml...............................................................Passed
Check for merge conflicts................................................Passed
Debug Statements (Python)............................(no files to check)Skipped
Check for added large files..............................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Check django project for potential problems..........(no files to check)Skipped
Check django project for missing migrations..........(no files to check)Skipped
flake8...............................................(no files to check)Skipped
isort................................................(no files to check)Skipped
black................................................(no files to check)Skipped

Notice that pre-commit says no files to check. It only checks for files included in our commit and skips the hooks that aren’t applicable.

You might be asking, what if we just wanted to run pre-commit without actually committing. Well, that’s actually possible especially for existing repositories. We can run pre-commit against all files by passing the --all-files argument

$ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for merge conflicts................................................Passed
Debug Statements (Python)................................................Passed
Check for added large files..............................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Check django project for potential problems..............................Passed
Check django project for missing migrations..........(no files to check)Skipped
flake8...................................................................Passed
isort....................................................................Passed
black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted /Users/trafalgar/Sources/github.com/yujinyuz/mysite/mysite/asgi.py
reformatted /Users/trafalgar/Sources/github.com/yujinyuz/mysite/manage.py
reformatted /Users/trafalgar/Sources/github.com/yujinyuz/mysite/mysite/urls.py
reformatted /Users/trafalgar/Sources/github.com/yujinyuz/mysite/mysite/wsgi.py
reformatted /Users/trafalgar/Sources/github.com/yujinyuz/mysite/mysite/settings.py
All done! ✨ 🍰 ✨
5 files reformatted, 1 file left unchanged.

If you notice, our black formatter has failed. This is because it performed changes to our files. And if you run the previous command again, everything should now pass

$ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for merge conflicts................................................Passed
Debug Statements (Python)................................................Passed
Check for added large files..............................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Check django project for potential problems..............................Passed
Check django project for missing migrations..........(no files to check)Skipped
flake8...................................................................Passed
isort....................................................................Passed
black....................................................................Passed

Aaaaand that’s it! You can now add the files and commit them to have a better and cleaner repository.

$ git add .
$ git commit -m "Lint files"
Trim Trailing Whitespace.................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for merge conflicts................................................Passed
Debug Statements (Python)................................................Passed
Check for added large files..............................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Check django project for potential problems..............................Passed
Check django project for missing migrations..........(no files to check)Skipped
flake8...................................................................Passed
isort....................................................................Passed
black....................................................................Passed
[main 725df70] Lint files
 5 files changed, 41 insertions(+), 41 deletions(-)

Configuring your formatters (isort, flake8, and black)

It’s always common to have an agreed set of rules when working with a team. Since our pre-commit config is using isort, flake8, and black, we might want to add configurations to them such as maximum line length, import ordering, etc.

Let’s create a file called setup.cfg. This gets picked up by our formatters and uses the settings when formatting our code:

$ touch setup.cfg

And paste the following:

[flake8]
ignore = W503,E501,E226,E702,E731
max-line-length = 88
exclude = migrations

[isort]
profile = black
multi_line_output = 3
force_grid_wrap = 0
use_parentheses = True
ensure_newline_before_comments = True
line_length = 88
known_django = django
sections = FUTURE,STDLIB,DJANGO,THIRDPARTY,FIRSTPARTY,LOCALFOLDER
skip = wsgi.py,setup.py

This is the config that I use. You can change it to whatever floats your boat.

Conclusion

pre-commit has been a great tool and helped me reduce time reviewing and nitpicking unformatted code and focus on the important things.

Hope you find this helpful. Have a great day!

Changing caps lock key to Escape when pressed alone and Control when pressed with another

Yujin — Sat, 02 Jan 2021 18:43:28 +0000

Like you before, I pretty much didn’t give a damn about remmaping my mapping keyboard because I was worried about what if I’m using someone elses keyboard? I’d be too dependent about my current setup.

But let me ask you. When was the last time you used someone elses keyboard?

After some time living inside the terminal, I noticed that it’s quite hard to press the control key especially when you are using a Mac keyboard layout.

It was kinda annoying to reach the control key when I want to stop a running program via <Control C> or clearing the screen with <Control L>.

It was when I was browsing through reddit and found a post where people using Vim were suggesting to remap Caps Lock to Escapeand some were also suggesting to remap Caps Lock to Control. My current configuration in Vim already mapped jk to Escapeso I took the route of mapping Caps Lock to Contol. Also, I don’t use caps lock much often.

After a few weeks of getting used to my new Control, everything felt easier! I almost even questioned why the Caps Lock key was positioned in that beautiful spot and we don’t even use it that often.

So, you might be asking what happened to my Caps Lock key? I found a tool called Karabiner Elements and it basically lets me create key combinations to execute another key. I trigger my Caps Lock key by pressing Right Command + Right Shift key together.

Thanks to karabiner, I also converted my Caps Lock key to be Escape when pressed alone and Control when pressed with another key.

Here’s how to configure it via Karabiner Elements:

Download and install it first here: https://karabiner-elements.pqrs.org/docs/getting-started/installation/
Next, open Karabiner Elements and click Complex Modifications, then click on Add Rule.
Click Import rules from the Internet (open a web browser)
Search for caps lock and then look for Change caps_lock key (rev 4) then click on Import.
Something similar to this should appear and you should click Enable:

And we’re done!

Faking google cloud storage in Python

Yujin — Sun, 27 Dec 2020 20:49:11 +0000

I had a problem where I couldn’t register to google cloud and use the google cloud storage service. Tried searching online and found this repository: https://github.com/fsouza/fake-gcs-server

It has great documentation and you can also run it if you have docker installed:

$ docker run -d --name fake-gcs-server -p 4443:4443 fsouza/fake-gcs-server

I had a task to upload files to google cloud storage but I wasn’t given access to the bucket. We were using docker and I was able to easily add it to our docker-compose.yml file:

version: '2'
services:
  mygooglecloudstorage:
    container_name: mygooglecloudstorage
    image: fsouza/fake-gcs-server
    ports:
      - "4443:4443"
    volumes:
      - ./gcs/data:/data # This just prepopulates data

And since I wanted to use it locally first, I created a file called fake_gcs.py with the following content:

def get_fake_client():
    import requests
    import urllib3
    from google.api_core.client_options import ClientOptions
    from google.auth.credentials import AnonymousCredentials
    from google.cloud import storage
    my_http = requests.Session()
    my_http.verify = False # disable SSL validation
    urllib3.disable_warnings(
        urllib3.exceptions.InsecureRequestWarning
    ) # disable https warnings for https insecure certs

    client = storage.Client(
        credentials=AnonymousCredentials(),
        project="test",
        _http=my_http,
        client_options=ClientOptions(api_endpoint='https://127.0.0.1:4443'),
    )
    return client

I did this so that I could easily replace client = storage.Client() with client = get_fake_client() later when I push my code.

storage.Client() package uses the GOOGLE_APPLICATION_CREDENTIALS environment variable so there wasn’t much of a problem switching between the original and the fake client.

Now, you can perform regular operations in your fake server like you normally would in the original server.

from fake_gcs import get_fake_client

client = get_fake_client()
bucket = client.bucket('sample-bucket')
blob = bucket.get_blob('test_file.txt')

How I used Case..When in Django

Yujin — Sun, 20 Dec 2020 21:01:25 +0000

I was working on a multi-tenant project and encountered a bug when using Django’s GenericForeignKey with django-tenants. It was using the public schema’s contenttype_id instead of the tenant schema’s contenttype_id.

So, if I have a model of Comment, my django_content_type table would have something like

public.django_content_type

id	app_label	model
15	comments	comment

tenant.django_content_type

id	app_label	model
19	comments	comment

There shouldn’t be a problem here since django-tenants should handle this because it chooses the id of the tenant first and then only use the public as a fall back value. But for some reason, it was sometimes using the public id so comments aren’t appearing at all!

In order to fix this, I opted to remove django_content_type table from all of my tenants and should only use the public’s django_content_type values.

What I had to do was to update the contents inside my models that were using GenericForeignKeys, which in my case is the comments table.

Here’s the model:

# comments/models.py

class Comment(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
    parent = models.ForeignKey("self", null=True, blank=True, on_delete=models.SET_NULL)
    path = models.CharField(max_length=350)
    text = models.TextField()
    timestamp = models.DateTimeField(auto_now_add=True)
    updated = models.DateTimeField(auto_now=True)
    active = models.BooleanField(default=True)
    flagged = models.BooleanField(default=False)

    target_content_type = models.ForeignKey(ContentType, null=True, blank=True, on_delete=models.SET_NULL)
    target_object_id = models.PositiveIntegerField(null=True, blank=True)
    target_object = GenericForeignKey("target_content_type", "target_object_id")

I need to update the target_content_type so that it uses the public id which is 15 instead of 19.

What I needed to do was:

Determine which target ids need to be updated. For example, a comment can be in an Announcement or in a Post. So, we’d have to determine the content_type_id for Announcement and Post in the tenant’s schema and update its value so it uses the one in public.
Update the values using Case..When.
Drop the tenant.django_content_type table so it would always use public.django_content_type.

For number 1, I had to do a GROUP BY to determine which id’s I need to update then get its equivalent in the public schema.

For number 2, I had to use the Case..When syntax. So, for example when the target_content_type_id is 19, then update its value to 15.when the target_content_type_id is 20, then update its value to 12.

I think I need not explain number 3 since it only drops the table.

I created a management command for this so it can be easily executed in production. Here’s the code:

from django.apps import apps
from django.contrib.contenttypes.models import ContentType
from django.core.management.base import BaseCommand
from django.db import connection
from django.db.models import Case, F, Value, When

from django_tenants.utils import schema_context

from tenant.models import Tenant

def group_by_sql(schema, table, column):
    sql = f"""
        SELECT {column} FROM {schema}.{table}
        GROUP BY {column}
    """
    print(sql)
    return sql

class Command(BaseCommand):

    help = "One time management command execution to update tenant's content_type_ids"

    def handle(self, *args, **options):

        has_gfk_models = [
            {
                'app_label': 'comments',
                'model': 'comment',
                'col': 'target_content_type_id'
            },
            {
                'app_label': 'notifications',
                'model': 'notification',
                'col': 'target_content_type_id'
            },
            {
                'app_label': 'notifications',
                'model': 'notification',
                'col': 'action_content_type_id',
            },
            {
                'app_label': 'prerequisites',
                'model': 'prereq',
                'col': 'parent_content_type_id',
            },
        ]

        for tenant in Tenant.objects.exclude(schema_name='public'):
            for has_gfk_model in has_gfk_models:
                app_label, model, col = has_gfk_model.values()

                # Number 1
                with connection.cursor() as cursor:
                    cursor.execute(group_by_sql(
                        schema=tenant.schema_name,
                        table=f"{app_label}_{model}",
                        column=col))

                    # Remove null ids
                    tenant_target_content_type_ids = [_id[0] for _id in cursor.fetchall() if _id[0]]
                    # print(tenant_target_content_type_ids)

                    # tenant content_type_id : public content_type_id
                    ct_ids_map = {}
                    for ct_id in tenant_target_content_type_ids:
                        # Get what kind of model the given ID is
                        with schema_context(tenant.schema_name):
                            ct_tenant_app = ContentType.objects.get(id=ct_id)
                        # ... then fetch its equivalent in the public tenant
                        try:
                            ct_public = ContentType.objects.get(app_label=ct_tenant_app.app_label, model=ct_tenant_app.model)
                            ct_ids_map[ct_id] = ct_public.id
                        except ContentType.DoesNotExist:
                            # Just skip the apps that aren't installed anymore
                            print(f'{ct_tenant_app} has been removed from settings.APPS')
                            continue

                    # Number 2
                    Model = apps.get_model(app_label, model)
                    with schema_context(tenant.schema_name):
                        # Using CASE..WHEN is much faster compared to bulk_update in this case
                        # https://docs.djangoproject.com/en/dev/ref/models/conditional-expressions/#conditional-update
                        whens = []

                        for tenant_ct_id, public_ct_id in ct_ids_map.items():
                            # Build query
                            # when target_content_type_id is 19 then update it to 15
                            # When(target_content_type_id={tenant_ct_id}, then=Value({public_ct_id}))
                            when = {
                                col: tenant_ct_id,
                                'then': Value(public_ct_id),
                            }
                            whens.append(When(**when))

                        # If we are currently updating comments, the query would look something like
                        # Comment.objects.update(
                        # target_content_type_id=Case(
                        # When(target_content_type_id=17, then=Value(25)),
                        # When(...),
                        # default=F(target_content_type_id)))
                        # )
                        case_when = {
                            # When statements should be wrapped in a `Case` so we need to unpack the list `*whens`
                            col: Case(*whens, default=F(col)),
                        }

                        # Filter out the queryset so we don't bother updating other target ids
                        # The `default` is useless in this case because we are only updating the ids that are needed
                        # so it's safe to remove the `default=F(col)`.
                        qs = Model.objects.filter(**{f'{col}__in': ct_ids_map.keys()})
                        qs.update(**case_when)
                        print(connection.queries)

            # Drop the table so it only uses public.django_content_type
            drop_contenttype_table = f"DROP TABLE IF EXISTS {tenant.schema_name}.django_content_type CASCADE"
            with connection.cursor() as cursor:
                cursor.execute(drop_contenttype_table)

The code above could still be improved but it did the job for me. Also, here’s a link to a GitHub gist: https://gist.github.com/yujinyuz/d257d3ce978deb0bc7a1fecbd3f8d101

Using pg_dump and pg_restore to backup and restore PostgreSQL database

Yujin — Sun, 13 Dec 2020 21:47:26 +0000

So, I was trying to reproduce some issues and bugs that only happened in production. I needed an exact copy of the production database and run it locally.

Here’s how I did it using pg_dump and pg_restore.

$ pg_dump -U postgres -Fc -Z 9 -j 8 production.dump -d postgres

Here’s the breakdown for the arguments:

U means username. It’s used to connect to your postgres database. In this case, my username is postgres.
F means format. It’s usually combined with c that means c ustom
Z means compress. It can have values from 0 to 9. 0 being no compression and 9 being the maximum compression. It’s supposed to make the dump file smaller. I used 9 here because I know that my database doesn’t contain much data and I want its size to be small as possible. You might be asking why not use 9 all the time? The answer to that is it takes a while to restore compressed dumps. So if you want to restore it faster, use a lower value.
j means jobs. It’s the number of jobs that we want to run. You can think of it as how many persons are going to dump your databse. More jobs means faster dump.
d means database. It’s the name of the database you want to dump. The name of the database I also want to dump is postgres.

After that, I copied it from my remote ssh machine to my local machine

$ scp yujinyuz@myremotesite:/usr/share/nginx/repo/production.dump ~/Downloads/

And then restored it locally:

$ pg_restore -U postgres -Fc -j 8 production.dump -d postgres -c

Notice that most of the arguments are almost the same except for the new -c.

c means clean. It will drop database objects before creating them. So, if I have existing tables in my local, it would just drop them and recreate it so I would have the exact copy of my production database.

That’s it! Hope that also helps you, too!

Using find and execute a command

Yujin — Mon, 16 Nov 2020 14:36:28 +0000

I mostly work on django projects and as a dev who likes shortcuts, I wanted to have an alias of pm for python manage.py.

I could’ve just added

alias pm="python manage.py"

inside of my config but I was working on different projects where manage.py was located in different folders. Typical projects have it located under root while some projects have it under src/manage.py.

What I needed was a way to find where the manage.py was located and use that so I can do

$ pm runserver # OR
$ pm shell # OR
$ pm <command>

Here’s my first attempt:

$ find . -type f -name "manage.py" -exec python {} \; -quit

What this does is find under the current directory . a type of file -type f with a name of manage.py -name "manage.py". And then, for the file you found execute a command -exec The {} is the current file name returned by find command.

In our case it’s gonna be:

$ python ./src/manage.py

The -quit is optional in most cases of using the find exec command since it it exits immediately after the first match.

But there’s a problem with that approach. If we just do:

$ alias pm="find . -type f -name "manage.py" -exec python {} \; -quit"
$ pm runserver

We get:

$ find: runserver: unknown primary or operator

In order to fix that, we have to wrap it inside a function so that it could take arguments:

$ alias pm='f(){ find . -type f -name "manage.py" -exec python {} "$@" \; -quit; unset -f f; }; f'

The $@ stands for all other arguments you passed. You can add that inside of your .bashrc or .zshrc.

If you are using fish like me, I have this file inside my ~/.config/fish/functions/pm.fish

function pm --description "Find and execute manage.py"
    command find . -type f -name "manage.py" -exec python {} $argv \; -quit
end

Now it works fine!

$ pm runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (2 silenced).
November 16, 2020 - 06:16:50
Django version 2.2.17, using settings 'myproject.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Switching from pyenv, rbenv, goenv and nvm to asdf

Yujin — Sun, 19 Jul 2020 19:57:19 +0000

So, there was a time when I was only developing applications using Python. And so I found out about virtual environments.And then after a couple of months, I discovered pyenv.

It also came to a time I had to work on multiple projects that uses different versions of nodejs and searched something similarso I installed nvm.

Then, I was required to work on a Ruby project so I installed rbenv.

And since golang was one of the new shiny objects that kinda got my interest, I went on and installed goenv.

Everything was fine until I kinda felt my config.fish file got bloated and somehow nvm was kinda slowing down my shell startup.

I did do some optimizations such as lazy loading nvm and used a fish plugin called fish-nvm.

And then one day, I found out about asdf-vm.

At first, I was quite hesitant to install it since it would kinda disrupt my work flow with python projects since I heavily use pyenv virtualenv <version> <name>.

But I did feel that the benefits of using asdf-vm outweighs the cons so I went ahead and proceeded.

Installation

$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.7.8

asdf has a pretty good documentation to be honest. Despite brew being my go to package manager, I chose git as my installation method.My reason is that asdf requires me to add this line in my config.fish

source (brew --prefix asdf)/asdf.fish

and the brew --prefix adsf is just too slow compared to just having this

source $HOME/.asdf/asdf.fish

Next, for the auto completions just run this command in your terminal:

$ mkdir -p ~/.config/fish/completions; and cp ~/.asdf/completions/asdf.fish ~/.config/fish/completions

Restart your shell so that PATH changes take effect!

You can checkout a more detailed documentation here: https://asdf-vm.com/#/core-manage-asdf-vm?id=install

Before, my config.fish looked something like this:

# pyenv
set -gx PYENV_ROOT $HOME/.pyenv
set -gx PYTHON_BUILD_ARIA2_OPTS "-x 10 -k 1M" # Use aria2c when downloading
contains $PYENV_ROOT/bin $fish_user_paths; or set -Ua fish_user_paths $PYENV_ROOT/bin
status --is-interactive; and pyenv init - | source
status --is-interactive; and pyenv virtualenv-init - | source

# goenv
set -gx GOENV_GOPATH_PREFIX $HOME/.go
status --is-interactive; and goenv init - | source

# rbenv
status --is-interactive; and rbenv init - | source

# Set nvm aliases and add to path
set -gx nvm_alias_output $HOME/.node_aliases
contains $nvm_alias_output $fish_user_paths; or set -Ua fish_user_paths $nvm_alias_output

After

# pyenv (asdf still uses pyenv under the hood)
set -gx PYTHON_BUILD_ARIA2_OPTS "-x 10 -k 1M" # Use aria2c when downloading

# asdf
source $HOME/.asdf/asdf.fish

Installing pyenv, rbenv, goenv, and nvm replacements

Again, most of the things I put here just came from the documentation: https://asdf-vm.com/#/core-manage-plugins?id=add

pyenv replacement

$ asdf plugin add python
$ asdf install python latest:3 # At the moment of writing this, it installed 3.8.4
$ asdf global python 3.8.4 # This sets python 3.8.4 as our default python version

rbenv replacement

$ asdf plugin add ruby
$ asdf install ruby latest # We can omit the version number. Currently installs 2.7.1
$ asdf global ruby 2.7.1

goenv replacement

$ asdf plugin add golang
$ asdf install golang latest # 1.14.6
$ asdf global golang 1.14.6

nvm replacement

$ asdf plugin add nodejs
$ asdf install nodejs 12.18.2
$ asdf global nodejs 12.18.2

Using asdf global creates a file under your HOME directory called .tool-versions

$ cat ~/.tool-versions
python 3.8.4
ruby 2.7.1
golang 1.14.6
nodejs 12.18.2

This lets asdf know which versions to use. And of course, in contrast to global,there is also the local keyword that creates another .tool-versions.This is useful when projects require different version.

$ cd ~/project1
$ asdf local python 3.7.5
$ python --version # Uses the python version specified in .tool-versions
3.7.5
$ cd ~/project2
$ python --version # Uses the python version specified in ~/.tool-versions
3.8.4

asdf has a lot of plugins available. You can check them out here: https://github.com/asdf-vm/asdf-plugins

Extras

In order to accommodate my work flow when I was still using pyenv virtualenv,I created a function that behaves quite similar to pyenv virtualenv.

$ touch ~/.config/fish/functions/venv.fish

And paste the following:

function venv --argument-names 'python_version' --description 'Create virtualenv named the same as current directory'
  set -l python_bin

  if not test -n "$python_version"
    # Use default python version set by asdf
    set python_bin ($HOME/.asdf/bin/asdf which python)
  else
    set python_bin $ASDF_DIR/installs/python/$python_version/bin/python
  end

  set -l venv_name (basename $PWD | tr . -)

  echo
  if not test -e $python_bin
    echo "Python version `$python_version` is not installed."
    return 1
  end

  echo Creating virtualenv `$venv_name`
  $python_bin -m venv $HOME/.virtualenvs/$venv_name
  source $HOME/.virtualenvs/$venv_name/bin/activate.fish
end

Whenever I’m inside a python project, I just need to type venv or venv <python_version>and it will automatically create a virtualenv under ~/.virtualenvs using the current directory name.

In order to automatically activate the virtualenv when cding to a project, do the following:

$ touch ~/.config/fish/conf.d/__auto_venv.fish

And paste the following:

function __auto_venv --on-variable PWD --description "Automatically activate python venv"
  set -l venv_name (basename $PWD | tr . -)

  if test -d $HOME/.virtualenvs/$venv_name
    source $HOME/.virtualenvs/$venv_name/bin/activate.fish
  end
end

Cool! Not only did I reduce the lines in my config.fish, I could also notice a decreased startup time which is a good thing!

You can also check out my dotfiles in my GitHub repository: https://github.com/yujinyuz/dotfiles

I hope this article helped you! You can leave a comment below and I’ll try to answer them as fast as I can.

Thanks for reading! 🎉

Custom colors in oh-my-zsh themes

Yujin — Fri, 27 Dec 2019 13:37:54 +0000

Custom colors in oh-my-zsh themes

So, I was trying to configure my custom zsh theme. I wanted something minimal like displaying the current user, a shortened path and the git info of a directory (if there is).

After a few hours of tinkering, I pretty much got what I wanted but the only thing I didn't like was the colors available.

I wasn't sure which file to look for that contains the fg_bold and fg.

I did try doing a ripgrep: rg fg_bold hoping that I would find a list of available colors. But all I found were .zsh-theme files that were using a fixed set of colors. Magenta, green, blue, yellow, cyan were the ones I found.

I wasn't satisfied with the colors available and I did some trial and error by doing $fg_bold[orange], $fg[pink], etc. But I had no luck.

I did some google search about defining custom colors in zsh and it led me to this GitHub issue: https://github.com/ohmyzsh/ohmyzsh/issues/1101#issuecomment-5450278

It seems that oh-my-zsh uses Spectrum under the hood. The source code can be found under ~/.oh-my-zsh/lib/spectrum.zsh.

Upon checking the source code, I tried running spectrum_ls on my terminal and I was amazed by the output :asto

I can now use custom colors on my custom zsh theme via $FG[<0-255>] e.g. $FG[021]

So, that's it! I just wanted to share how I managed to define custom colors in my zsh setup.

I mainly use Python and Javascript so I did add a few customizations like displaying the node and python version of the current environment.

I made a minimal version of the theme I was working on for those who might be interested: https://github.com/yujinyuz/dotfiles/blob/master/zsh/themes/jpro-minimal.zsh-theme

You can check out the documentation on adding themes to oh-my-zsh: https://github.com/ohmyzsh/ohmyzsh/wiki/Customization#overriding-and-adding-themes

Thanks for reading! It was a great experience writing my first article here on dev.to after being a member for about 2 years now 😅