From: <your@apache.org address here>
To: <your@apache.org address here>
Bcc: users@spamassassin.apache.org, dev@spamassassin.apache.org, announce@spamassassin.apache.org, announce@apache.org
Reply-to: dev@spamassassin.apache.org
Subject: ANNOUNCE: Apache SpamAssassin 4.0.0 available

Release Notes -- Apache SpamAssassin -- Version 4.0.0


Introduction
------------

Apache SpamAssassin 4.0.0 contains numerous tweaks and bug fixes over
the past releases. In particular, it includes major changes that
significantly improve the handling of text in international language.

As with any major release, there are countless functional patches and
improvements to upgrade to 4.0.0. Apache SpamAssassin 4.0.0 includes
several years of fixes that significantly improve classification and
performance. It has been thoroughly tested in production systems. We
strongly recommend upgrading as soon as possible.


Important Notes
—--------------

*** On March 1, 2020, we stopped publishing rulesets with SHA-1
  signatures. If you do not update to 3.4.2 or later, you will be
  stuck at the last ruleset with SHA-1 signatures. Such an upgrade
  should be to 3.4.6 to obtain the contained security fixes ***

*** Ongoing development on the 3.4 branch has ceased. All future
  releases and bug fixes will be on the 4.0 series, unless a new
  security issue is found that necessitates a 3.4.7 release. ***


Thanks
—-----

Many thanks to the committers (see CREDITS file), contributors, rule
testers, mass checkers, and code testers who have made this release
possible. We would also like to thank cPanel for their continued
support of new features.

Notable features:
=================


New plugins
-----------

There are three new plugins added with this release:


#1 Mail::SpamAssassin::Plugin::ExtractText

This plugin uses external tools to extract text from message parts,
and then sets the text as the rendered part. All SpamAssassin rules
that apply to the rendered part will run on the extracted text as
well.


#2 Mail::SpamAssassin::Plugin::DMARC

This plugin checks if emails match DMARC policy after parsing DKIM and
SPF results.


#3 Mail::SpamAssassin::Plugin::DecodeShortURLs

This plugin looks for URLs shortened by a list of URL shortening
services. Upon finding a matching URL, plugin will send a HTTP request
to the shortening service and retrieve the Location-header which
points to the actual shortened URL. It then adds this URL to the list
of URIs extracted by SpamAssassin which can then be accessed by uri
rules and plugins such as URIDNSBL.


Removed plugin
--------------

HashCash module, formerly deprecated, has now been removed completely


Notable changes
---------------

This release includes fixes for the following:

  - Support for international text such as UTF-8 rules has been
     completed and significantly improved to include native UTF-8
     processing

  - Bayes plugin has been improved to skip common words aka noise
     words written in languages other than English

  - OLEVBMacro plugin has been improved in order to detect more
     Microsoft Office macros and dangerous content. It has also been
     improved to extract URIs from Office documents for automatic
     inclusion in rules such as RBL lookups.

  - You can now use Captured Tags to use tags “captured” in one rule
     inside other rules

  - sa-update(1) tool has been improved with three new options:

    #1 forcemirror: forces sa-update to use a specific mirror server,

    #2 score-multiplier: adjust all scores from update channel by a
      given multiplier to quickly level set scores to match your
      preferred threshold

    #3 score-limit adjusts all scores from update channel over a
      specified limit to a new limit

* SSL client certificate support has been improved and made easier to
   implement with spamc/spamd

* DKIM plugin can now detect ARC signatures

* More work on improving the configuration and internal coding to use
  more inclusive and less divisive language

* spamc(1) speed has been improved when both SSL and compression are
  used

* The normalize_charset option is now enabled by default. NOTE: Rules
  should not expect specific non-UTF-8 or UTF-8 encoding in the body.
  Matching is done against the raw body, which may vary depending on
  normalize_charset setting and whether UTF-8 decoding was successful.

* Mail::SPF is now the only supported module used by the SPF plugin. 

* Mail::SPF::Query use is deprecated, along with settings
  do_not_use_mail_spf, do_not_use_mail_spf_query.

 * SPF lookups are not done asynchronously and you may consider using
   an SPF filter at the MTA level (pypolicyd-spf / spf-engine / etc)
   which generates a Received-SPF header that can be parsed by
   SpamAssassin.

 * The default sa-update ruleset doesn't make ASN lookups or header
   additions anymore.  Configure desired methods (asn_use_geodb /
   asn_use_dns) and add_header clauses manually, as described in
   documentation for the Mail::SpamAssassin::Plugin::ASN.


New configuration options
-------------------------

All rules, functions, command line options and modules that contain
"whitelist" or "blacklist" have been renamed to "welcomelist" and
"blocklist" terms

Old options will continue to work for backwards compatibility until at
least the Apache SpamAssassin version 4.1.0 release

New tflag "nolog" added to hide info coming from rules in SpamAssassin
reports

New dns_options "nov4" and "nov6" added.
IMPORTANT:; You must set nov6 if your DNS resolver is filtering IPv6
AAAA replies.

Razor2 razor_fork option added. It will fork separate Razor2 process
and read in the results later asynchronously, increasing
throughput. When this is used, rule priorities are automatically
adjusted to -100.

Pyzor pyzor_fork option added. It will fork separate Pyzor process and
read in the results later asynchronously, increasing throughput. When
this is used, rule priorities are automatically adjusted to -100

urirhsbl and urirhssub rules now support "notrim" tflag, which forces
querying the full hostname, instead of trimmed domain

report_charset now defaults to UTF-8 which may change the rendering of
SpamAssassin reports

Notable Internal changes
------------------------

Meta rules no longer use priority values, they are evaluated
dynamically when the rules they depend on are finished

DNS and other asynchronous lookups like DCC or Razor2 plugins are now
launched when priority -100 is reached. This allows short circuiting
at lower priority without sending unneeded DNS queries

New internal Mail::SpamAssassin::GeoDB module supporting RelayCountry
and URILocalBL plugins provides a unified interface to Geographic IP
modules. These include:
    MaxMind::DB::Reader (GeoIP2)
    Geo::IP
    IP::Country::DB_File
    IP::Country::Fast.

Bayes and TxRep Message-ID tracking now uses a different hashing
method


Other updates
-------------

None noted.


Optimizations
-------------

Apache SpamAssassin 4.0.0 represents years of work by the project with
numerous improvements, new rule types, and internal native handling of
messages in international languages. These three key optimizations
will improve the efficiency of SpamAssassin:

    DNS queries are now done asynchronously for overall speed
    improvements

    DCC checks can now use dccifd asynchronously for improved throughput

    Pyzor and Razor fork use separate processes done asynchronously
    for increased throughput


Downloading and availability
----------------------------

Downloads are available from:

https://spamassassin.apache.org/downloads.cgi

[INSERT SHA256 and SHA512 HASHES OF FILES HERE AFTER RELEASE IS PREPARED]

Note that the Rules files, aka *-rules-*.tgz, are only necessary if
you cannot, or do not wish to, run "sa-update" after
installation. Using sa-update will download the latest rules

See the INSTALL and UPGRADE files in the distribution for important
installation notes


GPG Verification Procedure
--------------------------
The release files also have a .asc accompanying them.  The file serves
as an external GPG signature for the given release file.  The signing
key is available via the keys.gnupg.net or keys.openpgp.org key
servers, as well as https://www.apache.org/dist/spamassassin/KEYS


The following key is used to sign SA releases 3.3.0 and later:

pub   4096R/F7D39814 2009-12-02
      Key fingerprint = D809 9BC7 9E17 D7E4 9BC2  1E31 FDE5 2F40 F7D3 9814
uid                  SpamAssassin Project Management Committee <private@spamassassin.apache.org>
uid                  SpamAssassin Signing Key (Code Signing Key, replacement for 1024D/265FA05B) <dev@spamassassin.apache.org>
sub   4096R/7B3265A5 2009-12-02

The following key is used to sign rule updates:

pub   4096R/5244EC45 2005-12-20
      Key fingerprint = 5E54 1DC9 59CB 8BAC 7C78  DFDC 4056 A61A 5244 EC45
uid                  updates.spamassassin.org Signing Key <release@spamassassin.org>
sub   4096R/24F434CE 2005-12-20

To verify a release file, download the file with the accompanying .asc
file and run the following commands:

  gpg --verbose --keyserver keys.openpgp.org --recv-key FDE52F40F7D39814
  gpg --verify Mail-SpamAssassin-3.4.6.tar.bz2.asc
  gpg --fingerprint FDE52F40F7D39814

Then verify that the key matches the signature.

Note that older versions of gnupg may not be able to complete the
steps above. Specifically, GnuPG v1.0.6, 1.0.7 & 1.2.6 failed while
v1.4.11 worked flawlessly

See https://www.apache.org/info/verification.html for more information
on verifying Apache releases


About Apache SpamAssassin
-------------------------

Apache SpamAssassin is a mature, widely-deployed open source project
that provides filtering to classify email to block spam, malware, and
phishes.

Apache SpamAssassin uses a variety of mechanisms including mail header
and text analysis, Bayesian filtering, DNS blocklists, collaborative
filtering databases, and meta concepts to lower incorrect
classification.

Apache SpamAssassin uses a highly modular architecture that allows
other technologies to be quickly incorporated as plugins to easily add
or replace existing methods.

Apache SpamAssassin typically runs on a server using either command
line utilities or an API to classify email so a mail system can use
the results before the message reaches mailboxes.

Most of the Apache SpamAssassin is written in Perl natively supporting
Unix, Linux, and macOS platforms and Microsoft Windows using
Strawberry Perl.

For more information, visit https://spamassassin.apache.org/


About The Apache Software Foundation
------------------------------------

Established in 1999, The Apache Software Foundation provides
organizational, legal, and financial support for more than 100
freely-available, collaboratively-developed Open Source projects. The
pragmatic Apache License enables individual and commercial users to
easily deploy Apache software; the Foundation's intellectual property
framework limits the legal exposure of its 2,500+ contributors.

For more information, visit https://www.apache.org/
