Rspamd Feature und Bugfix Release 3.3 veröffentlicht

Die Open-Source Lösung zur Bekämpfung von Spam, Junk, Phishing und sonstigen unerwünschten E-Mails, erhielt das Update 3.3. Dies ist hauptsächlich ein Bugfix Release, dass jedoch auch einige neue Features und Stabilitätsverbesserungen bringt.

Migration to Rspamd 3.3

Please be cautious if you migrate to Rspamd 3.3 when you use custom passthrough rules (meaning the most of plugins that define action and not least action). Prior to 3.3, you could still process other rules, whilst in 3.3+ passthrough means exactly what it means: set the final action and skip directly to the idempotent stage.

Users of the neural plugin can have a significant Redis storage leak introduced in the version 3.2. This issue is fixed in the version 3.3 since the following commit, however, this fix will not remove old stale keys. Unfortunately, those keys do not have any expire either. One of the possible sollutions to clean the database up is to remove all keys starting with rn_ prefix. There are multiple options available to perform this action, so you can take a look at the following conversation on the Stackoverflow.

Rspamd also requires C++20 compatible compiler and toolchain to be built from this version.

Rspamd 3.3 Release Notes

Reworked and redesigned symbols cache

Symbols cache is responsible for rules exectuion and planning. In this release, there was a major rework of it’s logic and functionality. For example, it can now keep track of timeouts, plan fast events before slow and implement real passthrough for the rules that define such a behaviour. It is useful, when you want some rule to be executed as quick as possible to block or pass evident spam/ham without wasting network/cpu resources. The major drawback of such a rework is that passthrough rules are now really passthrough and can prevent other rules from being executed (that is expected from the design, but it could be not the case before).

Critical fix in the neural network module

There was a regression introduced in the version 3.2 that prevented old keys in Redis to be cleaned that caused infinite Redis database growth. This is fixed in the release 3.3 and the mitigation of this bug are described in the upgrade guide.

DKIM parser now ignores unknown tags

By standard, DKIM checker must ignore unknown tags for forward compatibility. Rspamd will now behave properly and ignore unknown tags as specified in RFC.

Upstreams support in lua_http and lua_tcp modules

It is possible now to use the functionality of the upstreams directly in Lua modules that use lua_http and lua_tcp libraries. It allows better support of the names resolution, IPv6 support for resolving the hostnames and internal handling of the upstreams logic by C code automatically.

CNAME records support in the DNS resolver

Rspamd DNS resolver now supports querying and parsing of the CNAME records. This technique might be useful for fighting some specific spam patterns.

Various memory leaks detected and plugged

In this release, we have found and fixed a good bunch of memory leaks and memory corruptions in the code.

All significant changes

Here is the list of the important changes:

• [Conf] Add missing groups for whitelist module symbols
• [CritFix] Neural: Fix keys regression after #3968
• [Feature] Accept upstream in lua_tcp
• [Feature] Add ability to statically maintain disabled/enabled patterns
• [Feature] Add function to store upstreams for HTTP urls
• [Feature] Allow augmentations set in Lua API
• [Feature] Allow lua_http module to accept upstreams
• [Feature] Allow to limit write access to fuzzy storage by key
• [Feature] Allow to sort symbols output
• [Feature] Check content for binary stuff before dumping it to Lua
• [Feature] Implement symbols augmentations
• [Fix] Add missing flags
• [Fix] Add more sanity checks for rua in dmarc_report
• [Fix] Adjust length of the fuzzy checks for short text parts
• [Fix] Another try to fix add headers compatibility logic
• [Fix] Another try to fix race condition in the runtime destruction
• [Fix] Avoid cyclic references in symcache and fix memory leaks
• [Fix] Avoid overriding IP with Sender IP
• [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes
• [Fix] Bind AF_UNIX DGRAM client connection to annonymous address
• [Fix] Disable IPv6 lookups for Blocklist.de RBL
• [Fix] Distinguish dynamic and static items
• [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC
• [Fix] Dmarc report: Use local timezone instead of GMT
• [Fix] Do not exclude authenticated users from URIBL lookups
• [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.
• [Fix] External_relay: Restore the originating hostname check
• [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64
• [Fix] Fix copying of sockaddr_un addresses
• [Fix] Fix crash with cname replies
• [Fix] Fix dependencies propagation
• [Fix] Fix iteration over milter headers
• [Fix] Fix ordering when sorting symcache
• [Fix] Fix reading of the cached maps
• [Fix] Fix several issues with the HTTP keepalive parsing
• [Fix] Fix stack smashing
• [Fix] Fix synchronous auth/select in lua_redis
• [Fix] Fix various symcache issues
• [Fix] Ignore all (I hope) unknown DKIM signature KV pairs
• [Fix] Ignore directories in RarV5 archives
• [Fix] Libucl: avoid memory leak on objects merging
• [Fix] Lua_tcp: Another try to fix closing logic
• [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size
• [Fix] Only check allowed fuzzy worker update ips for non-unix sockets
• [Fix] Plug memory leak in regexp destruction with pcre2
• [Fix] Properly check the original email flag
• [Fix] Properly deal with get_symbol/get_metric_symbol ambiguity
• [Fix] Properly parse expressions atoms
• [Fix] Properly set Host in rspamd_proxy
• [Fix] Rbl: Fix received positioned checks
• [Fix] Remove check for a score with no symbol being registered
• [Fix] Same fix for lua_tcp
• [Fix] Skip cname records when processing SPF records
• [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241
• [Fix] Stop slow timer on task destruction
• [Fix] Symcache: Do not use C style comparators in C++ sorts
• [Fix] Try to avoid a corner case for @ pattern
• [Fix] Try to fix dkim reputation adjustements
• [Fix] Try to fix passthrough results processing logic
• [Fix] Try to fix the mess with read only flag
• [Fix] Upstreams: Don’t ignore revive_time config option
• [Fix] Use proper format string, sigh…
• [Fix] Use space category in ragel automata to resolve space characters
• [Fix] Zstd: Fix compression with the new Zstd API
• [Fix] milter_headers: Header fields may be inserted at wrong position.
• [Project] Rework symbols cache
• [Rework] Rewrite rspamc in C++

Quelle: Rspamd 3.3 has been released