The HMAC File Server ensures secure file uploads and downloads using HMAC authentication. It incorporates rate limiting, CORS support, retries, file versioning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and a graceful shutdown mechanism ensure reliable and efficient file handling.

Special thanks to Thomas Leister for inspiration drawn from [prosody-filer](https://github.com/ThomasLeister/prosody-filer).

• File deduplication
• Configurable TTL for automatic file cleanup
• Secure HMAC-based authentication
• Chunked uploads and downloads
• Virus scanning via ClamAV
• Prometheus metrics integration
• Customizable worker management
• Support ISO-based storage

1. Installation
2. Configuration
3. Usage
4. Setup
   • Reverse Proxy
   • Systemd Service
5. Building
6. Changelog
7. License

• Go 1.18 or higher
• Redis server (optional, for caching)
• ClamAV (optional, for virus scanning)

1. Clone the repository:

   git clone https://github.com/PlusOne/hmac-file-server.git
   cd hmac-file-server

2. Build the server:

   go build -o hmac-file-server

3. Create necessary directories:

   mkdir -p /path/to/hmac-file-server/data/
   mkdir -p /path/to/hmac-file-server/deduplication/
   mkdir -p /path/to/hmac-file-server/iso/

4. Copy and edit the configuration file:

   cp config.example.toml config.toml

5. Start the server:

   ./hmac-file-server -config config.toml

The server is configured via a config.toml file. Key settings include:

• Server Settings: Port, logging, metrics
• Security: HMAC secret, TLS options
• File Management: TTL, deduplication, uploads, and downloads
• ISO: Generation and mounting settings
• Workers: Adjust thread management

For detailed configuration options, refer to the Wiki.

Start the server and access it on the configured port. Use curl or a client library to interact with the API.

Upload a file:

curl -X POST -F 'file=@example.jpg' http://localhost:8080/upload

Set up a reverse proxy using Apache2 or Nginx to handle requests.

<VirtualHost *:80>
    ServerName your-domain.com

    ProxyPreserveHost On
    ProxyPass / http://localhost:8080/
    ProxyPassReverse / http://localhost:8080/
</VirtualHost>

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Create a systemd service file for the HMAC File Server:

[Unit]
Description=HMAC File Server
After=network.target

[Service]
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
WorkingDirectory=/path/to/hmac-file-server
Restart=always
User=www-data
Group=www-data

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server
sudo systemctl start hmac-file-server

To build for different architectures:

• Linux (x86_64):

  GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64

• macOS (x86_64):

  GOOS=darwin GOARCH=amd64 go build -o hmac-file-server-darwin-amd64

• Windows (x86_64):

  GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe

• Deduplication Support: Automatically remove duplicate files based on SHA256 hashing to save storage space.
• ISO Container Management: Create and mount ISO containers for specialized storage needs, enhancing flexibility in file management.
• Prometheus Metrics Enhancements: Added detailed metrics for deduplication and ISO container operations to improve monitoring and observability.
• Redis Integration Improvements: Enhanced caching mechanisms using Redis for faster access to file metadata and application states.
• Precaching Feature: Implemented precaching of file structures on startup to reduce access times for frequently used files.
• Configuration Options: Updated config.toml to include new settings for deduplication, ISO management, and worker scaling.

• Worker Pool Scaling: Implemented dynamic adjustment of worker threads based on system resources to optimize performance.
• Logging Enhancements: Improved logging for file operations, including detailed information on file extensions and MIME types during uploads.
• Temporary Path Configuration: Replaced hardcoded temporary upload directories with a configurable TempPath parameter in config.toml for greater flexibility.

• Temporary File Handling: Resolved issues where temporary .tmp files caused "Unsupported file type" warnings by enhancing MIME type detection logic.
• MIME Type Detection: Improved MIME type detection to ensure better compatibility and accuracy during file uploads.

• Thumbnail Support (Previous Implementation): Dropped the previous thumbnail support mechanism. This feature will not return in future releases.

Note: Ensure to update your config.toml with the new configuration options to take full advantage of the added features. Refer to the Configuration section in the documentation for detailed settings.

• GitHub Release: Create a corresponding GitHub release tagged v2.6.0 with these updated release notes to inform users of the new features and changes.
• Migration Steps: If upgrading from a previous version, review the configuration changes and deprecated features to adjust your setup accordingly.

[server]
listenport = "8080"
unixsocket = false
storagepath = "./uploads"
metricsenabled = true
metricsport = "9090"
filettl = "8760h"
minfreebytes = "100MB"
autoadjustworkers = true
networkevents = false
temppath = "/tmp"
loggingjson = false
pidfilepath = "/var/run/hmacfileserver.pid"
cleanuponexit = true
precaching = true
filettlenabled = true
deduplicationenabled = true
globalextensions = [".txt", ".pdf", ".png", ".jpg"]

[logging]
level = "info"
file = "/var/log/hmac-file-server.log"
max_size = 100
max_backups = 7
max_age = 30
compress = true

[deduplication]
enabled = true
directory = "/deduplication"

[iso]
enabled = true
size = "1GB"
mountpoint = "/mnt/iso"
charset = "utf-8"
containerfile = "/mnt/iso/container.iso"

[timeouts]
readtimeout = "4800s"
writetimeout = "4800s"
idletimeout = "4800s"

[security]
secret = "changeme"

[versioning]
enableversioning = false
maxversions = 1

[uploads]
resumableuploadsenabled = true
chunkeduploadsenabled = true
chunksize = "64MB"
allowedextensions = [".txt", ".pdf", ".png", ".jpg"]

[downloads]
resumabledownloadsenabled = true
chunkeddownloadsenabled = true
chunksize = "64MB"
allowedextensions = [".txt", ".pdf", ".png", ".jpg"]

[clamav]
clamavenabled = true
clamavsocket = "/var/run/clamav/clamd.ctl"
numscanworkers = 2
scanfileextensions = [".txt", ".pdf", ".png", ".jpg"]

[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"

[workers]
numworkers = 4
uploadqueuesize = 50

[build]
version = "2.5"

MIT License

Copyright (c) 2025 Alexander Renz

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.