Merge pull request #24 from Nukesor/tooling

Tooling
Nukesor · Oct 27, 2022 · 261ff3f · 261ff3f
2 parents 0d46381 + 5657b6d
commit 261ff3f
Show file tree

Hide file tree

Showing 20 changed files with 702 additions and 326 deletions.
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -0,0 +1,56 @@
+name: Lint
+
+on:
+  push:
+    paths:
+      - ".github/**/*"
+      - "archivebot/**"
+  pull_request:
+    paths:
+      - ".github/**/*"
+      - "archivebot/**"
+
+defaults:
+  run:
+    working-directory: archivebot
+
+jobs:
+  lint:
+    runs-on: ubuntu-22.04
+    strategy:
+      matrix:
+        python-version: ["3.10"]
+      fail-fast: false
+
+    steps:
+      - name: Clone the repository
+        uses: actions/checkout@v3
+
+      # This has been done according to https://docs.makedeb.org/prebuilt-mpr/getting-started/
+      # This is needed for packages: just
+      - name: Add makedb debian user repository
+        run: |
+          curl -q 'https://proget.makedeb.org/debian-feeds/prebuilt-mpr.pub' | gpg --dearmor | sudo tee /usr/share/keyrings/prebuilt-mpr-archive-keyring.gpg 1> /dev/null
+          echo "deb [signed-by=/usr/share/keyrings/prebuilt-mpr-archive-keyring.gpg] https://proget.makedeb.org prebuilt-mpr $(lsb_release -cs)" | sudo tee /etc/apt/sources.list.d/prebuilt-mpr.list
+          sudo apt update
+
+      - name: Install dev dependencies
+        run: sudo apt install just
+
+      - name: Install Poetry
+        uses: snok/install-poetry@v1
+
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: "poetry"
+
+      - name: Install Python dependencies
+        if: steps.cache.outputs.cache-hit != 'true'
+        run: |
+          poetry install
+
+      - name: Lint
+        run: |
+          just lint
diff --git a/Justfile b/Justfile
@@ -0,0 +1,30 @@
+default: run
+
+run:
+    poetry run python main.py run
+
+initdb *args:
+    poetry run python main.py initdb {{ args }}
+
+setup:
+    poetry install
+
+lint:
+    poetry run black --check archivebot
+    poetry run isort --check-only archivebot
+    poetry run flake8 archivebot
+
+format:
+    # remove unused imports
+    poetry run autoflake \
+        --remove-all-unused-imports \
+        --recursive \
+        --in-place archivebot
+    poetry run black archivebot
+    poetry run isort archivebot
+
+
+# Watch for something
+# E.g. `just watch lint` or `just watch test`
+watch *args:
+    watchexec --clear 'just {{ args }}'
diff --git a/Makefile b/Makefile
diff --git a/README.md b/README.md
@@ -7,7 +7,7 @@
 A handy bot which enables to download files from telegram chats to your server.
 
 It features a full backup of all files posted in a chat and a continuous backup of incoming new media.
-A zip archive can then be created and downloaded from the Telegram chat with a single command at any time.
+A 7z archive can then be created and downloaded from the Telegram chat with a single command at any time.
 
 For instance, this is great to collect images and videos from the members of your last holiday trip or to simply push backups or interesting files from your telegram chats to your server.
 
@@ -16,16 +16,16 @@ For instance, this is great to collect images and videos from the members of you
 </p>
 
 To send multiple uncompressed pictures and videos with your phone:
-1. Click the share button
-2. Select `File`
-3. Select Gallery (To send images without compression)
 
+1. Click the share button
+1. Select `File`
+1. Select Gallery (To send images without compression)
 
 **WARNING:**  
 This is a hobby side project of mine. It has been developed for use on Linux, but might work on windows as well.
 There might be some bugs, but I didn't find them yet. I'm happy about any PR's, feel free to help out!  
 
-## Features:
+## Features
 
 - Zip all files and post it into the chat with the simple `/zip` command.
 - Clear all files from the server with a simple `/clear_history` command.
@@ -36,34 +36,32 @@ There might be some bugs, but I didn't find them yet. I'm happy about any PR's,
 - Properly handles forwarded messages (If `sort_by_user` is enabled, the original sender will be used).
 - Verbose option for notifying users of duplicates or compressed images.
 
-
-## Stuff that's not working:
+## Stuff that's not working
 
 - I don't handle all media files yet. Feel free to create a pull request for this.
 
-## Installation and starting:
+## Installation and starting
+
 **This bot is developed for Linux.** Windows isn't tested, but it shouldn't be too hard to make it compatible. Feel free to create a PR.
 
+Dependencies:
 
-Dependencies: 
-- `poetry` to setup a virtualenv and run everything conveniently. (If you don't want poetry, you need to install the dependencies defined in `pyproject.toml` by hand.)
-- A sql database. archivebot uses sqlite by default
+- `poetry` to setup a virtualenv and run everything conveniently.
+    (If you don't want poetry, you need to install the dependencies defined in `pyproject.toml` by hand.)
+- [Just](https://github.com/casey/just) for convience.
 - `7zip` if you want to use the zip feature
+- Sqlite
 
+1. Clone the repository:
+    ```sh
+    git clone git@github.com:nukesor/archivebot && cd archivebot
+    ```
 
-1. Clone the repository: 
-
-        % git clone git@github.com:nukesor/archivebot && cd archivebot
-
-2. Execute the following commands to install all dependencies and to initialize the database:
-
-        % poetry install
-        % poetry run python ./initdb.py
-
-3. Either start the archivebot once or copy the `archivebot.toml` manually to `~/.config/archivebot.toml` and adjust all necessary values.
-
-4. Start the bot: `poetry run python ./main.py`
-
+1. Run `just setup` to setup the project
+1. Run `just initdb` to initialize the database
+1. Run `just run` to create a default config `~/.config/archivebot.toml`
+1. Adjust the configuration
+1. Start the bot via `just run`
 
 ## Configuration
 
@@ -72,43 +70,46 @@ If run as a normal telegram bot, archivebot is unable to scan the whole chat his
 Thereby `/scan_chat` doesn't work as well as the `/zip` command, since normal bots can't upload files larger than 20MB.
 Userbots on the other hand can upload files up to 1.5GB.
 
-
 **If you run the bot as a normal Telegram bot, disable the privacy mode for your bot via the BotFather menu!** Telegram bots can't read group messages by default.
 
 In case you decide to run it as a userbot to access all features, set the userbot flag to `true` and add your phone number to the configuration.
 You will receive a login code, which has to be entered on the first start and every time your session expires (which happens pretty much never).
 
+## Commands
 
-## Commands:
 In group chats the bot expects a command in combination with its username. (In userbot mode that's your own username)
 E.g. `/start@some_bot_name` or `/start@Nukesor`
 
-    /start Start the bot
-    /stop Stop the bot
-    /clear_history Clear all files from the server.
-    /zip Create a zip file of all files on the server
-    /set_name Set the name for this chat. This also determines the name of the target folder on the server.
-    /scan_chat Scan the whole chat history for files to back up.
-    /accept ['document', 'photo'] Specify the accepted media Example: '/accept document photo'
-    /verbose ['true', 'false'] The bot will complain if there are duplicate files or uncompressed images are sent, whilst not being accepted.
-    /sort_by_user ['true', 'false'] Incoming files will be sorted by user in the server directory for this chat.
-    /allow_duplicates ['true', 'false'] Allow to save files with duplicate names.
-    /info Show current settings.
-    /help Show this text
-
-
-### Botfather commands:
+```txt
+/start Start the bot
+/stop Stop the bot
+/clear_history Clear all files from the server.
+/zip Create a zip file of all files on the server
+/set_name Set the name for this chat. This also determines the name of the target folder on the server.
+/scan_chat Scan the whole chat history for files to back up.
+/accept ['document', 'photo'] Specify the accepted media Example: '/accept document photo'
+/verbose ['true', 'false'] The bot will complain if there are duplicate files or uncompressed images are sent, whilst not being accepted.
+/sort_by_user ['true', 'false'] Incoming files will be sorted by user in the server directory for this chat.
+/allow_duplicates ['true', 'false'] Allow to save files with duplicate names.
+/info Show current settings.
+/help Show this text
+```
+
+### Botfather commands
+
 These are the command descriptions formatted for the botfather, in case you want to host your own bot
 
-    start - Start archiving Files for this chat
-    stop - Stop archiving Files for this chat
-    clear_history - Clear all files from the server.
-    zip - Create a zip file of all files on the server.
-    set_name - Set the name for this chat. This also determines the name of the target folder on the server.
-    scan_chat - Scan the whole chat history for files to back up.
-    accept - ['document', 'photo'] Specify the allowed media types. Example: `/accept document photo`
-    sort_by_user - ['true', 'false'] Incoming files will be sorted by user in the server directory for this chat.
-    verbose - ['true', 'false'] The bot will complain if there are duplicate files or uncompressed images are sent, whilst not being accepted.
-    allow_duplicates - ['true', 'false'] Allow to save files with duplicate names.
-    info - Show current settings.
-    help - Show the help text.
+```txt
+start - Start archiving Files for this chat
+stop - Stop archiving Files for this chat
+clear_history - Clear all files from the server.
+zip - Create a zip file of all files on the server.
+set_name - Set the name for this chat. This also determines the name of the target folder on the server.
+scan_chat - Scan the whole chat history for files to back up.
+accept - ['document', 'photo'] Specify the allowed media types. Example: `/accept document photo`
+sort_by_user - ['true', 'false'] Incoming files will be sorted by user in the server directory for this chat.
+verbose - ['true', 'false'] The bot will complain if there are duplicate files or uncompressed images are sent, whilst not being accepted.
+allow_duplicates - ['true', 'false'] Allow to save files with duplicate names.
+info - Show current settings.
+help - Show the help text.
+```
diff --git a/archivebot.toml b/archivebot.toml
diff --git a/archivebot/archivebot.py b/archivebot/archivebot.py
@@ -1,34 +1,26 @@
 """A bot which downloads various files from chats."""
 import os
+import shutil
 import time
+
 from telethon import TelegramClient, events, types
 from telethon.errors import BadMessageError, FloodWaitError
-import shutil
 
 from archivebot.config import config
-from archivebot.sentry import sentry
-from archivebot.models import (  # noqa
-    File,
-    Subscriber,
-)
-
 from archivebot.helper import (
-    get_peer_information,
+    UnknownUser,
     get_info_text,
     get_option_for_subscriber,
+    get_peer_information,
+    get_username,
     help_text,
     possible_media,
     should_accept_message,
-    get_username,
-    UnknownUser,
 )
+from archivebot.helper.file import create_file, create_zips, get_chat_path, init_zip_dir
 from archivebot.helper.session import session_wrapper
-from archivebot.helper.file import (
-    create_file,
-    create_zips,
-    get_chat_path,
-    init_zip_dir,
-)
+from archivebot.models import File, Subscriber  # noqa
+from archivebot.sentry import sentry
 
 if config["telegram"]["userbot"]:
     NAME = "archivebot"
@@ -247,7 +239,7 @@ async def zip(event, session):
 
     zip_dir = init_zip_dir(subscriber.chat_name)
 
-    text = f"Zipping started, this might take some time. Please don't issue this command again until I'm finished."
+    text = "Zipping started, this might take some time. Please don't issue this command again until I'm finished."
     await event.respond(text)
 
     create_zips(subscriber.chat_name, zip_dir, chat_path)

diff --git a/archivebot/config.py b/archivebot/config.py
@@ -1,6 +1,7 @@
 """Config values for archivebot."""
 import os
 import sys
+
 import toml
 
 default_config = {
@@ -11,13 +12,20 @@
         "app_api_id": 0,
         "app_api_hash": "apihash",
     },
-    "database": {"sql_uri": "sqlite:///archivebot.db",},
-    "logging": {"sentry_enabled": False, "sentry_token": "",},
+    "database": {
+        "sql_uri": "sqlite:///archivebot.db",
+    },
+    "logging": {
+        "sentry_enabled": False,
+        "sentry_token": "",
+    },
     "download": {
         "allowed_types": ["document", "photo"],
         "target_dir": "/home/user/archivebot/",
     },
-    "zip": {"volume_size": "1400m",},
+    "zip": {
+        "volume_size": "1400m",
+    },
 }
 
 config_path = os.path.expanduser("~/.config/archivebot.toml")

diff --git a/archivebot/db.py b/archivebot/db.py
@@ -1,9 +1,10 @@
 """Helper class to create a database engine and to get a session."""
-from archivebot.config import config
 from sqlalchemy import create_engine
+from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.orm import scoped_session
 from sqlalchemy.orm.session import sessionmaker
-from sqlalchemy.ext.declarative import declarative_base
+
+from archivebot.config import config
 
 engine = create_engine(config["database"]["sql_uri"])
 base = declarative_base(bind=engine)

diff --git a/archivebot/helper/__init__.py b/archivebot/helper/__init__.py
@@ -3,7 +3,6 @@
 
 from archivebot.models import Subscriber
 
-
 possible_media = ["document", "photo", "sticker"]
 
 help_text = f"""A handy telegram bot which allows to store files on your server, which are posted in a chat.
Original file line number	Diff line number	Diff line change
Expand Up		@@ -3,7 +3,6 @@

		from archivebot.models import Subscriber


		possible_media = ["document", "photo", "sticker"]

		help_text = f"""A handy telegram bot which allows to store files on your server, which are posted in a chat.
Expand Down