Rspamd 3.11 Release Notes

Breaking Changes

• Elasticsearch/OpenSearch Plugin: Major rework with breaking changes:
  • Added support for Elasticsearch 8 & OpenSearch 2
  • Added index policy with logs retention
  • Updated configuration format

Major Features

• Added LRU cache for last filled ratelimit buckets
• Added utilities to manage ratelimit buckets
• Added include/exclude logic for headers processing
• Improved address rotation algorithm for upstream selection
• Added rspamadm mime strip command for attachments removal
• Added new message anonymization capabilities
• Replaced fastutf with simdutf for better architecture support and performance
• Added more ways to extend Rspamd configuration, including lua.local.d folder

Important Fixes

• Fixed ARC-Seal signing issues
• Fixed RFC 2047 header encoding
• Fixed issues with dynamic keys in fuzzy storage
• Fixed TCP connection handling with cumulative timeouts
• Improved handling of DNS limits in SPF module
• Fixed multiple phishing detection false positives
• Fixed DMARC structured headers encoding

WebUI Improvements

• Reworked symbol description display on hover
• Improved keyboard accessibility
• Enhanced symbol rendering

Other Changes

• Skip extra RBL checks when Received IP matches From IP
• Multimap now uses only distinct text parts for content matching
• Improved GPT module JSON parsing
• Various configuration and logging improvements
• Multiple performance optimizations

The release includes significant improvements in configuration flexibility, security features, and overall stability. Users should pay special attention to the breaking changes in the Elasticsearch/OpenSearch plugin when upgrading.

Full Changelog: 3.10.0...3.11.0

• [CritFix] Fix ARC-Seal signing
• [Fix] add EOF to openmetrics response in proxy and server

Full Changelog: 3.10.1...3.10.2

What's Changed

• [Fix] Use correct type for keylen in lua_ucl_newindex by @arkamar in #5169
• Update hiredis library removing all hacks by @vstakhov in #5167
• Remove proxy from url_redirector.conf as it not the option by @dragoangel in #5164
• chore(publicsuffix): update effective_tld_names.dat by @wdhdev in #5176
• [Fix] Avoid null-bytes in Log-Tag header value by @smarsching in #5179
• [Fix] Do not abort when OpenSSL is broken, report that to a user by @vstakhov in #5188
• Some build fixes by @vstakhov in #5189
• Some more fixes by @vstakhov in #5190

New Contributors

• @wdhdev made their first contribution in #5176
• @smarsching made their first contribution in #5179

Full Changelog: 3.10.0...3.10.1

What's Changed

• [FIX] Arc regex to no longer match on dmarc= by @JasonStephenson in #4960
• [Conf] Add SenderScore RPBL return codes by @moisseev in #5077
• Add name in statistic.conf by @dragoangel in #5079
• [Minor] Improve errors reporting for lua_redis by @vstakhov in #5086
• [Feature] Allow to set negative group score limit via min_score by @vstakhov in #5084
• [Fix] Fix Redis scripts uploading when Redis is not ready by @vstakhov in #5091
• [Conf] Add SenderScore Reputationlist RBL by @moisseev in #5088
• [Test] Add tests for milter_headers by @fatalbanana in #5083
• [Project] Allow to change log tag from HTTP request by @vstakhov in #5092
• [Rework] Resolve rdns in a separate function by @vstakhov in #5093
• [Dep] Update doctest to 2.4.11 by @vstakhov in #5094
• Pull Request for OpenSSL 3.0 support by @left-try in #5072
• Remove NIST mode from cryptobox by @vstakhov in #5099
• Fix keypairs encryption/decryption by @vstakhov in #5100
• [Project] Allow manipulations with opaque UCL objects by @vstakhov in #5104
• Provide OpenSSL 3.0 support for lua_rsa by @left-try in #5101
• Fix issue with broken HTTP message to learn endpoints by @vstakhov in #5106
• [Test] Improve grow_factor tests by @fatalbanana in #5110
• Set expiration for history_redis to 5 days by default by @dragoangel in #5115
• Fix for issue #5089 - learning of the empty tasks by @vstakhov in #5118
• Update URIBL_RED/GREY scores by @slavkoja in #5119
• [Fix] Preserve the previous behaviour of RDNS_* checks by @vstakhov in #5121
• [Test] Improve startup error handling by @fatalbanana in #5114
• [Fix] Unify lua symbols registration by @vstakhov in #5125
• [PATCH] ICAP: encode constructed path to be URL safe. by @oliyks in #5111
• [Rework] Change the logic of skipping symbols by @vstakhov in #5126
• Draft pull request for issue #5095 by @left-try in #5116
• Rework fuzzy symbols by @vstakhov in #5127
• [Fix] Another fix for Redis schema by @vstakhov in #5129
• [Feature] Allow differen modes for fuzzy rules by @vstakhov in #5131
• Implement individual limits for fuzzy keys by @vstakhov in #5135
• [Fix] Rework DMARC grammar to correctly handle spaces in values by @spacefreak86 in #5139
• [Test] Upgrade linters by @moisseev in #5133
• [Feature] DMARC: Implement reporting.only_domains setting by @fatalbanana in #5142
• Draft pull request for PR#5116 by @left-try in #5132
• Detect CPU using __builtin_cpu_supports where it's possible by @vstakhov in #5140
• [Rework] Allow more flexible keypair encoding by @vstakhov in #5144
• Add more build targets by @vstakhov in #5153
• Fixes for OpenSSL by @vstakhov in #5154
• [Fix] Iterate over dynamic keys in fuzzy storage by @vstakhov in #5156
• [Rework] Remove control block support by @vstakhov in #5160
• MIME UTF8 support by @vstakhov in #5056
• [Fix] Fix DCC rep handling by @vstakhov in #5161

New Contributors

• @slavkoja made their first contribution in #5119
• @oliyks made their first contribution in #5111

Full Changelog: 3.9.1...3.10.0

What's Changed

• [Test] Use actions/setup-node by @moisseev in #5058
• [Feature] Treat SPF +all in a special way by @vstakhov in #5059
• [Test] Migrate ESLint to 9.7.0 by @moisseev in #5063
• Some tunes and fixes for GPT by @vstakhov in #5068

Full Changelog: 3.9.0...3.9.1

What's Changed

• [Feature] Add extraction type for from maps by @vstakhov in #4794
• [Rework] Breaking: Do not report module as action by @moisseev in #4795
• [Minor] Move server selection logic to common.js by @moisseev in #4796
• [WebUI] Show pass-through module in History by @moisseev in #4798
• [Feature] Allow to add templates to redis history prefix by @vstakhov in #4801
• Deal with Connection and Host headers on proxying by @vstakhov in #4802
• [Minor] configwizard: really use LOCAL_CONFDIR by @fatalbanana in #4804
• [Fix] Resolve issue with bayes stat in rspamadm mode by @vstakhov in #4805
• [Minor] rspamd_task API docs: fix broken link by @fatalbanana in #4807
• [Minor] Add more returnbits to surbl configuration by @fatalbanana in #4803
• [Feature] Support JSON logging when in syslog mode by @vstakhov in #4813
• Fix issues with URL fragment in HTTP requests by @vstakhov in #4814
• [Fix] Encode headers in metadata exporter by @vstakhov in #4816
• Improve Kaspersky AS integration by @vstakhov in #4806
• Allow dynamic keypairs loading in fuzzy storage by @vstakhov in #4818
• Build tests with CTest in Cmake by @vstakhov in #4822
• [Fix] Fix history key, as we use {= and not {{ in templates by @vstakhov in #4826
• Revert "[Fix] Fix history key, as we use {= and not {{ in templates" by @vstakhov in #4829
• [Fix] Allow different template expansion parameters by @vstakhov in #4830
• [WebUI] Disable buttons until tables are ready by @moisseev in #4838
• Rework jinja_template functions to avoid LUPA global state dependency by @vstakhov in #4839
• Support libarchive by @vstakhov in #4843
• [Test] Use Robot-native process management by @fatalbanana in #4834
• [Minor] Add HAS_FILE_URL rule for messages containing a file:// URL by @twesterhever in #4846
• [Test] rspamadm tests: override DBDIR by @fatalbanana in #4847
• Sync public suffix list by @fatalbanana in #4845
• [WebUI] Add file input and drop area to scan tab by @moisseev in #4849
• Update nconns in proxy by @vstakhov in #4852
• [Minor] Improve minWidth calculation by @moisseev in #4853
• [Minor] Align items in card headers and buttons by @moisseev in #4855
• [Minor] Fix status tables borders by @moisseev in #4856
• [Project] Optimization of userdata hashing by @vstakhov in #4857
• Further optimization to the hot path of lua userdata checks by @vstakhov in #4860
• [Minor] Serve SVG with the correct CT by @moisseev in #4861
• [Minor] Add background icon to drop area by @moisseev in #4862
• [Minor] Add HTTP_MAGIC_SVG key by @moisseev in #4863
• [WebUI] Add check/uncheck all checkboxes buttons by @moisseev in #4864
• Vault signing improvements by @vstakhov in #4866
• [Feature] Support reply in message pack format by @vstakhov in #4867
• Fix parsing of maps definitions by @vstakhov in #4869
• Use zstd compression in client all the time by @vstakhov in #4870
• Remove demjson dependency as it is clearly outdated and broken by @vstakhov in #4871
• Migrate to Github Actions by @fatalbanana in #4823
• [Test] Use actions/checkout by @fatalbanana in #4874
• Do not save multipatterns to FS in certain cases by @vstakhov in #4873
• [Minor] README: update build status badge by @fatalbanana in #4875
• [Test] Really fix saving test logs... by @fatalbanana in #4876
• [WebUI] Add column display mode settings by @moisseev in #4877
• [Minor] Change FooTable breakpoints to match Bootstrap 5 by @moisseev in #4878
• [Rework] Remove some of the GLib types in lieu of standard ones by @vstakhov in #4880
• Fix inconsistent nesting in mime parts by @vstakhov in #4882
• Protect regexp matcher from regexps with empty patterns by @vstakhov in #4887
• [Minor] Fix format of example setting (rspamd/rspamd.com#736) by @fatalbanana in #4891
• [Enhancement] Add more symbols for Reply-To header characteristics by @twesterhever in #4889
• [Minor] Add rule for localhost HELOs in Received headers by @twesterhever in #4890
• Several improvements to the Cloudmark integration by @vstakhov in #4894
• Allow custom milter quarantine and tempfail messages by @vstakhov in #4901
• Fix shared memory proxying when compression is set by @vstakhov in #4903
• [Test] Try break greylisting tests by @fatalbanana in #4911
• Update greylisting threshold on each scan as it can be easily dynamic by @vstakhov in #4910
• Rework grow factor by @vstakhov in #4912
• [WebUI] Escape HTML characters in errors history by @moisseev in #4913
• [Test] Fix uploading of test logs when tests are failed by @fatalbanana in #4917
• Add tests for grow_factor by @fatalbanana in #4918
• [Minor] Increase RDNS_NONE score to 2.0 by @twesterhever in #4916
• [Enhancement] Exclude false positives of MIME_DOUBLE_BAD_EXTENSION by @dragoangel in #4920
• [Minor] Cloudmark scores_symbols: convert keys to numeric values by @fatalbanana in #4924
• [WebUI] Add missing handler for file input by @moisseev in #4922
• [WebUI] Prevent navbar layout shift by @moisseev in #4927
• [Minor] Add selector to get rspamd_hostname by @lucasRolff in #4926
• [Fix] Relearn messages in bayes filter by @aduernberger in #4930
• Fix statfile symbols disabling by @vstakhov in #4932
• Support UTF8 flag for the received headers by @vstakhov in #4933
• Fix error in headers_checks.lua by @dragoangel in #4934
• [Minor] Update Bootstrap by @moisseev in #4935
• Serialize control commands by @vstakhov in #4937
• Implement backpressure for fuzzy check by @vstakhov in #4939
• Do not read out-of-boundary when doing base64 encoding by @vstakhov in #4942
• Rework composites for spam injected into compromised accounts by @twesterhever in #4914
• [Minor] Add bad file types commonly found in archives attached to malspam by @twesterhever in #4949
• Apply detection phase if fasttext could not detect language by @vstakhov in #4951
• Add detection for freemail and disposable e-mail usage for message delivery notification by @twesterhever in #4915
• Add Content-Description header rules by @twesterhever in #4950
• Implement DNSBL checks for Message-ID RHS by @twesterhever in #4888
• [Minor] Remove redundant sub-property by @moisseev in #4954
• [Fix] SUBJ_ALL_CAPS is overkill with some unicase scripts by @ikedas in #4953
• [Minor] Properly make use of selectors for checking MID RHSs against DNSBLs by @twesterhever in #4955
• Enable fasttext support on debianalikes by @fatalbanana in #4961
• [Test] Disable @stylistic/multiline-comment-style by @moisseev in #4966
• Revert "[Minor] debian: enable fasttext" by @fatalbanana in #4963
• [Fix] Check nconns when firing the final termination event by @vstakhov in https://github....

What's Changed

• [Fix] Allow different template expansion parameters by @vstakhov in #4830

Full Changelog: 3.8.3...3.8.4

This release includes a fix for a regression in redis_history which caused history to be recorded to a misnamed key. No other changes were introduced in this release

Full Changelog: 3.8.2...3.8.3

What's Changed

• [Feature] Add extraction type for from maps by @vstakhov in #4794
• [Rework] Breaking: Do not report module as action by @moisseev in #4795
• [Minor] Move server selection logic to common.js by @moisseev in #4796
• [WebUI] Show pass-through module in History by @moisseev in #4798
• [Feature] Allow to add templates to redis history prefix by @vstakhov in #4801
• Deal with Connection and Host headers on proxying by @vstakhov in #4802
• [Minor] configwizard: really use LOCAL_CONFDIR by @fatalbanana in #4804
• [Fix] Resolve issue with bayes stat in rspamadm mode by @vstakhov in #4805
• [Minor] rspamd_task API docs: fix broken link by @fatalbanana in #4807
• [Minor] Add more returnbits to surbl configuration by @fatalbanana in #4803
• [Feature] Support JSON logging when in syslog mode by @vstakhov in #4813
• Fix issues with URL fragment in HTTP requests by @vstakhov in #4814
• [Fix] Encode headers in metadata exporter by @vstakhov in #4816
• Improve Kaspersky AS integration by @vstakhov in #4806
• Allow dynamic keypairs loading in fuzzy storage by @vstakhov in #4818
• Build tests with CTest in Cmake by @vstakhov in #4822

Full Changelog: 3.8.1...3.8.2

What's Changed

• [Fix] Replace comment types by @arkamar in #4785
• [Test] statistics: per-user by @fatalbanana in #4789
• [Minor] rbl: fix warning by @fatalbanana in #4790

Full Changelog: 3.8.0...3.8.1