Use dehydrated on FreeBSD to automatically renew SSL certificates using DNS-01 verification method with bind on a remote host.
I will show here an example for the domain fechner.net
using a wildcard certificate one and two levels deep (*.fechner.net
, *.idefix.fechner.net
).
Installation
Install it:
To enable automatic renewal:
echo weekly_dehydrated_enable=\"YES\" >> /etc/periodic.conf
Configuration
Bind
We do now the configuration on the external server bind is running.
For this we will use for each domain an extra key and an isolated zone file, to archive this, we delegate the acme related parts to an extra zone file.
Let’s create the key. I use as domain fechner.net
, so replace the value with your domain name:
tsig-keygen -a sha512 acme_fechner.net >> /usr/local/etc/namedb/keys.conf
chown bind:bind /usr/local/etc/namedb/keys.conf
chmod 640 /usr/local/etc/namedb/keys.conf
Make sure the keys.conf
is loaded in /usr/local/etc/namedb/named.conf
:
/usr/local/etc/namedb/named.conf
...
include "/usr/local/etc/namedb/keys.conf";
Now we create a new zone file for the acme related zone updates.
I store my zone files in /usr/local/etc/namedb/master/fechner.net/
, so create there a new file _acme-challenge.fechner.net
:
/usr/local/etc/namedb/master/fechner.net/_acme-challenge.fechner.net
$TTL 2m ; default TTL for zone
@ IN SOA ns.fechner.net. hostmaster.fechner.net. (
1 ; serial number
2m ; refresh
2m ; retry
2m ; expire
2m ; minimum
)
@ IN NS ns.fechner.net.
If you want to add a wildcard one level deeper, e.g. *.idefix.fechner.net, create also a file _acme-challenge.idefix.fechner.net
.
Now we load this new zone. My master zones are defined in /usr/local/etc/namedb/named.zones.master
:
/usr/local/etc/namedb/named.zones.master
zone "_acme-challenge.fechner.net" {
type master;
file "/usr/local/etc/namedb/master/fechner.net/_acme-challenge.fechner.net";
masterfile-format text;
allow-update { key acme_fechner.net; };
};
Make sure permissions are correct:
chown bind:bind /usr/local/etc/namedb/keys.conf
chmod 640 /usr/local/etc/namedb/keys.conf
chown bind:bind /usr/local/etc/namedb/master/fechner.net/_acme-challenge*fechner.net
chmod 644 /usr/local/etc/namedb/master/fechner.net/_acme-challenge*fechner.net
Restart bind and verify the zone is correctly loaded:
service named restart
dig _acme-challenge.fechner.net.
You should see something like this (the SOA record):
; <<>> DiG 9.18.24 <<>> _acme-challenge.fechner.net.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1852
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: dc08ffe2f65683be0100000065d99f05e8c78dde55084b9d (good)
;; QUESTION SECTION:
;_acme-challenge.fechner.net. IN A
;; AUTHORITY SECTION:
_acme-challenge.fechner.net. 120 IN SOA ns.fechner.net. hostmaster.fechner.net. 1 120 120 120 120
;; Query time: 11 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP)
;; WHEN: Sat Feb 24 08:47:17 CET 2024
;; MSG SIZE rcvd: 134
Now we add a delegation in the domain fechner.net
by adding the following line:
/usr/local/etc/namedb/master/fechner.net
;-- DNS delegation for acme validation
_acme-challenge IN NS fechner.net.
; if you use a two level subdomain like *.idefix.fechner.net
; _acme-challenge.idefix IN NS fechner.net.
Make sure you reload the zone file and it is valid.
Dehydrated
Now we can start to finalize the configuration for dehydrated.
Edit /usr/local/etc/dehydrated/config
.
For testing we set the CA to letsencrypt-test
make sure you use letsencrypt
if everything is working as expected!
/usr/local/etc/dehydrated/config
CA="letsencrypt-test"
CHALLENGETYPE="dns-01"
CONTACT_EMAIL="_your-email_"
HOOK="/usr/local/etc/dehydrated/hook.sh"
OCSP_FETCH="yes"
Now edit /usr/local/etc/dehydrated/domains.txt
(here fechner.net is not part of the certificate, maybe you want to for your certificate!):
/usr/local/etc/dehydrated/domains.txt
*.fechner.net *.idefix.fechner.net > star_fechner_net_rsa
*.fechner.net *.idefix.fechner.net > star_fechner_net_ecdsa
Now we configure the keys with (if you do not want to have a RSA key, you can skip this and remove the rsa line in domains.txt):
mkdir -p /usr/local/etc/dehydrated/certs/star_fechner_net_rsa
echo KEY_ALGO=\"rsa\" > /usr/local/etc/dehydrated/certs/star_fechner_net_rsa/config
chmod 700 /usr/local/etc/dehydrated/certs/star_fechner_net_rsa
Now we must store the acme_fechner.net
key file we create with the tsig-keygen
command on the bind server:
mkdir -p /usr/local/etc/dehydrated/tsig_keys
Make sure you paste the content from tsig genarete key from /usr/local/etc/namedb/keys.conf
to the file /usr/local/etc/dehydrated/tsig_keys/fechner.net.key
Secure it with:
chown root:wheel /usr/local/etc/dehydrated/tsig_keys/fechner.net.key
chmod 600 /usr/local/etc/dehydrated/tsig_keys/fechner.net.key
# if you use *.idefix.fechner.net
# ln -s /usr/local/etc/dehydrated/tsig_keys/fechner.net.key /usr/local/etc/dehydrated/tsig_keys/idefix.fechner.net.key
Now edit /usr/local/etc/dehydrated/hook.sh
To create the required DNS entries:
/usr/local/etc/dehydrated/hook.sh
#!/usr/local/bin/bash
DNSSERVER="fechner.net"
declare -A alg2ext=( ["rsaEncryption"]="rsa" ["id-ecPublicKey"]="ecdsa" )
deploy_challenge() {
local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
local NSUPDATE="nsupdate -k /usr/local/etc/dehydrated/tsig_keys/${DOMAIN}.key"
# This hook is called once for every domain that needs to be
# validated, including any alternative names you may have listed.
#
# Parameters:
# - DOMAIN
# The domain name (CN or subject alternative name) being
# validated.
# - TOKEN_FILENAME
# The name of the file containing the token to be served for HTTP
# validation. Should be served by your web server as
# /.well-known/acme-challenge/${TOKEN_FILENAME}.
# - TOKEN_VALUE
# The token value that needs to be served for validation. For DNS
# validation, this is what you want to put in the _acme-challenge
# TXT record. For HTTP validation it is the value that is expected
# be found in the $TOKEN_FILENAME file.
printf 'server %s\nupdate add _acme-challenge.%s 300 IN TXT "%s"\nsend\n' "${DNSSERVER}" "${DOMAIN}" "${TOKEN_VALUE}" | ${NSUPDATE}
}
The remove the required DNS entries again (edit /usr/local/etc/dehydrated/hook.sh
):
/usr/local/etc/dehydrated/hook.sh
clean_challenge() {
local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
local NSUPDATE="nsupdate -k /usr/local/etc/dehydrated/tsig_keys/${DOMAIN}.key"
# This hook is called after attempting to validate each domain,
# whether or not validation was successful. Here you can delete
# files or DNS records that are no longer needed.
#
# The parameters are the same as for deploy_challenge.
printf 'server %s\nupdate delete _acme-challenge.%s TXT "%s"\nsend\n' "${DNSSERVER}" "${DOMAIN}" "${TOKEN_VALUE}" | ${NSUPDATE}
}
To automatically copy the created certificates to the destination your services are expecting it (edit /usr/local/etc/dehydrated/hook.sh
):
/usr/local/etc/dehydrated/hook.sh
deploy_cert() {
local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" CHAINFILE="${5}" TIMESTAMP="${6}"
local SRC=$(dirname ${KEYFILE})
local DST=/usr/local/etc/haproxy/certs
local ALG=$(openssl x509 -in ${SRC}/cert.pem -noout -text | awk -F':' '/Public Key Algorithm/ {print $2}' | tr -d ' ')
local EXT=${alg2ext[${ALG}]}
# This hook is called once for each certificate that has been
# produced. Here you might, for instance, copy your new certificates
# to service-specific locations and reload the service.
#
# Parameters:
# - DOMAIN
# The primary domain name, i.e. the certificate common
# name (CN).
# - KEYFILE
# The path of the file containing the private key.
# - CERTFILE
# The path of the file containing the signed certificate.
# - FULLCHAINFILE
# The path of the file containing the full certificate chain.
# - CHAINFILE
# The path of the file containing the intermediate certificate(s).
# - TIMESTAMP
# Timestamp when the specified certificate was created.
# dovecot
service dovecot restart
#postfix
service postfix restart
# haproxy
ln -sf ${FULLCHAINFILE} ${DST}/${DOMAIN}.${EXT}
ln -sf ${KEYFILE} ${DST}/${DOMAIN}.${EXT}.key
service haproxy restart
}
For the OCSP information to be deployed to haproxy:
/usr/local/etc/dehydrated/hook.sh
deploy_ocsp() {
local DOMAIN="${1}" OCSPFILE="${2}" TIMESTAMP="${3}"
local SRC=$(dirname ${OCSPFILE})
local DST=/usr/local/etc/haproxy/certs
local ALG=$(openssl x509 -in ${SRC}/cert.pem -noout -text | awk -F':' '/Public Key Algorithm/ {print $2}' | tr -d ' ')
local EXT=${alg2ext[${ALG}]}
# This hook is called once for each updated ocsp stapling file that has
# been produced. Here you might, for instance, copy your new ocsp stapling
# files to service-specific locations and reload the service.
#
# Parameters:
# - DOMAIN
# The primary domain name, i.e. the certificate common
# name (CN).
# - OCSPFILE
# The path of the ocsp stapling file
# - TIMESTAMP
# Timestamp when the specified ocsp stapling file was created.
ln -sf ${OCSPFILE} ${DST}/${DOMAIN}.${EXT}.ocsp
service haproxy restart
}
To get errors by email if something fails:
/usr/local/etc/dehydrated/hook.sh
invalid_challenge() {
local DOMAIN="${1}" RESPONSE="${2}"
# This hook is called if the challenge response has failed, so domain
# owners can be aware and act accordingly.
#
# Parameters:
# - DOMAIN
# The primary domain name, i.e. the certificate common
# name (CN).
# - RESPONSE
# The response that the verification server returned
printf "Subject: Validation of ${DOMAIN} failed!\n\nOh noez!" | sendmail root
}
request_failure() {
local STATUSCODE="${1}" REASON="${2}" REQTYPE="${3}" HEADERS="${4}"
# This hook is called when an HTTP request fails (e.g., when the ACME
# server is busy, returns an error, etc). It will be called upon any
# response code that does not start with '2'. Useful to alert admins
# about problems with requests.
#
# Parameters:
# - STATUSCODE
# The HTML status code that originated the error.
# - REASON
# The specified reason for the error.
# - REQTYPE
# The kind of request that was made (GET, POST...)
# - HEADERS
# HTTP headers returned by the CA
printf "Subject: HTTP request failed failed!\n\nA http request failed with status ${STATUSCODE}!" | sendmail root
}
The rest of the file you can leave untouched.
Make the hook executable:
chmod +x /usr/local/etc/dehydrated/hook.sh
Haproxy
Configuration part for haproxy:
/usr/local/etc/haproxy.conf
...
frontend www-https
bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/certs/ alpn h2,http/1.1
bind :::443 ssl crt /usr/local/etc/haproxy/certs/ alpn h2,http/1.1
...
Postfix
Configuration part for postfix:
/usr/local/etc/postfix/main.cf
smtpd_tls_chain_files =
/usr/local/etc/dehydrated/certs/star_fechner_net_ecdsa/privkey.pem
/usr/local/etc/dehydrated/certs/star_fechner_net_ecdsa/fullchain.pem
/usr/local/etc/dehydrated/certs/star_fechner_net_rsa/privkey.pem
/usr/local/etc/dehydrated/certs/star_fechner_net_rsa/fullchain.pem
Dovecot
Configuration part for dovecot:
/usr/local/etc/dovecot/local.conf
ssl_cert = </usr/local/etc/dehydrated/certs/star_fechner_net_ecdsa/fullchain.pem
ssl_key = </usr/local/etc/dehydrated/certs/star_fechner_net_ecdsa/privkey.pem
ssl_alt_cert = </usr/local/etc/dehydrated/certs/star_fechner_net_rsa/fullchain.pem
ssl_alt_key = </usr/local/etc/dehydrated/certs/star_fechner_net_rsa/privkey.pem
Test
We make the tests against to test environment of letsencrypt, make sure you have CA="letsencrypt-test"
.
At first register at the CA and accept their terms:
dehydrated --register --accept-terms
To test it (make sure you use the test CA):
dehydrated -c --force --force-validation
Go Live
If everything succeeded you must switch to the letsencrypt production environment.
Change in config
:
/usr/local/etc/dehydrated/config
Remove the test certificates:
rm -R /usr/local/etc/dehydrated/certs
Now get the certificates with:
dehydrated --register --accept-terms
dehydrated -c
You maybe want to monitor now your certificates and the OCSP information that the refresh is working as expected.
Reference:
We use the following folder structure:
/usr/local/vmail/%d/%u/mail/
^ ^ ^
| | |- Directory were emails are stored in maildir format
| |- the username part of the email address
|- the domain
/usr/local/vmail/%d/%u/sieve/
^- Folder to store sieve filters
/usr/local/etc/dovecot
^- Hold configuration files for dovecot
dovecot/sieve
^- Directory for global sieve scripts for all users
/usr/local/etc/postfix
^- Hold configuration files for postfix
We will use Postfix, Dovecot2 with virtual domains managed by ViMbAdmin and everything stored on a mysql database. As password scheme BLF-CRYPT is used, see http://wiki2.dovecot.org/Authentication/PasswordSchemes
.
Base System
I will start from a plain installation.
Make sure your system is up-to-date:
pkg update
freebsd-update fetch
freebsd-update install
reboot
Install MYSQL
pkg install mariadb1011-server
echo 'mysql_enable="YES"' >> /etc/rc.conf
service mysql-server start
mysql_secure_installation
Install dcc-dccd
pkg install dcc-dccd
echo "DCCM_LOG_AT=NEVER" >> /usr/local/dcc/dcc_conf
echo "DCCM_REJECT_AT=MANY" >> /usr/local/dcc/dcc_conf
echo "DCCIFD_ENABLE=on" >> /usr/local/dcc/dcc_conf
echo "0 2 * * * root /usr/bin/find /usr/local/dcc/log/ -not -newermt '1 days ago' -delete" >> /etc/crontab
sysrc dccifd_enable="YES"
service dccifd start
Install clamav and clamav-unofficial-sigs
pkg install clamav clamav-unofficial-sigs
sysrc clamav_freshclam_enable="YES"
sysrc clamav_clamd_enable="YES"
echo 'user_configuration_complete="yes"' >> /usr/local/etc/clamav-unofficial-sigs/user.conf
/usr/local/bin/clamav-unofficial-sigs.sh
# Seems not to work
# clamav-unofficial-sigs.sh --install-cron
echo "0 1 * * * root /usr/local/bin/clamav-unofficial-sigs.sh" >> /etc/crontab
service clamav-freshclam restart
service clamav-clamd restart
Install rspamd
pkg install -qy rspamd redis
sysrc rspamd_enable="YES"
sysrc redis_enable="YES"
Install mailman
cd /usr/ports/mail/mailman/
make install clean
(select DOCS, NLS, POSTFIX)
Install Sympa
pkg install sympa spawn-fcgi
Install PHP
# Make sure following PHP modules are available: MCRYPT, MYSQL, MYSQLI, PDO_MYSQL, IMAP, GETTEXT, JSON
pkg install -qy php81 php81-extensions php81-composer2
sysrc php_fpm_enable="YES"
cp -f /usr/local/etc/php.ini-production /usr/local/etc/php.ini
sed -i '' -e 's/;date.timezone =/date.timezone = "Europe\/Berlin"/g' /usr/local/etc/php.ini
service php-fpm restart
Install NGINX
pkg install -qy nginx
sysrc nginx_enable="YES"
cd /usr/local/etc/nginx
git clone https://gitlab.fechner.net/mfechner/nginx_config.git snipets
mkdir -p /usr/local/etc/nginx/sites
mkdir -p /usr/local/etc/nginx/conf.d
mkdir -p /usr/home/http/webmail/logs
chown www /usr/home/http/webmail/logs
sed -i '' -e "s/ listen 127.0.0.1:8082 proxy_protocol;/ listen *:8082;/g" /usr/local/etc/nginx/snipets/listen.conf
sed -i '' -e "s/.*fastcgi_param HTTPS on;/ fastcgi_param HTTPS off;/g" /usr/local/etc/nginx/snipets/vimbadmin.conf
echo "load_module /usr/local/libexec/nginx/ngx_http_brotli_filter_module.so;" > /usr/local/etc/nginx/nginx.conf
echo "load_module /usr/local/libexec/nginx/ngx_http_brotli_static_module.so;" >> /usr/local/etc/nginx/nginx.conf
echo "worker_processes 4;" >> /usr/local/etc/nginx/nginx.conf
echo "events {" >> /usr/local/etc/nginx/nginx.conf
echo " worker_connections 1024;" >> /usr/local/etc/nginx/nginx.conf
echo "}" >> /usr/local/etc/nginx/nginx.conf
echo "http {" >> /usr/local/etc/nginx/nginx.conf
echo " include mime.types;" >> /usr/local/etc/nginx/nginx.conf
echo " default_type application/octet-stream;" >> /usr/local/etc/nginx/nginx.conf
echo " sendfile on;" >> /usr/local/etc/nginx/nginx.conf
echo " keepalive_timeout 65;" >> /usr/local/etc/nginx/nginx.conf
echo " index index.php index.html;" >> /usr/local/etc/nginx/nginx.conf
echo " include conf.d/*.conf;" >> /usr/local/etc/nginx/nginx.conf
echo " include sites/*.conf;" >> /usr/local/etc/nginx/nginx.conf
echo "}" >> /usr/local/etc/nginx/nginx.conf
echo "map $scheme $php_https { default off; https on; }" > /usr/local/etc/nginx/conf.d/php.conf
echo "" >> /usr/local/etc/nginx/conf.d/php.conf
echo "# Relaxe the timeouts" >> /usr/local/etc/nginx/conf.d/php.conf
echo "client_header_timeout 3000;" >> /usr/local/etc/nginx/conf.d/php.conf
echo "client_body_timeout 3000;" >> /usr/local/etc/nginx/conf.d/php.conf
echo "fastcgi_read_timeout 3000;" >> /usr/local/etc/nginx/conf.d/php.conf
echo "" >> /usr/local/etc/nginx/conf.d/php.conf
echo "upstream php-handler {" >> /usr/local/etc/nginx/conf.d/php.conf
echo " server 127.0.0.1:9000;" >> /usr/local/etc/nginx/conf.d/php.conf
echo "}" >> /usr/local/etc/nginx/conf.d/php.conf
echo "server {" > /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " server_name _ ${HOSTNAME};" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " root /usr/local/www/roundcube;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo "" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " access_log /usr/home/http/webmail/logs/access.log;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " error_log /usr/home/http/webmail/logs/error.log;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo "" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " include snipets/vimbadmin.conf;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " include snipets/rspamd.conf;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo "" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " location ~ \.php(?:$|/) {" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " include fastcgi_params;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " fastcgi_pass php-handler;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " }" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo "" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo " include snipets/virtualhost.conf;" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
echo "}" >> /usr/local/etc/nginx/sites/${HOSTNAME}.conf
service nginx restart
Install Dovecot
cd /usr/ports/mail/dovecot
#(select MYSQL)
make install clean
sysrc dovecot_enable="YES"
Copy standard config files:
cp -a /usr/local/etc/dovecot/example-config/ /usr/local/etc/dovecot/
pkg install dovecot-pigeonhole
Install Postfix
cd /usr/ports/mail/postfix
#(select MYSQL, SPF, TLS, DOVECOT2)
make install clean
sysrc sendmail_enable="NO"
sysrc sendmail_submit_enable="NO"
sysrc sendmail_outbound_enable="NO"
sysrc sendmail_msp_queue_enable="NO"
sysrc postfix_enable="YES"
sysrc -f /etc/periodic.conf daily_clean_hoststat_enable="NO"
sysrc -f /etc/periodic.conf daily_status_mail_rejects_enable="NO"
sysrc -f /etc/periodic.conf daily_status_include_submit_mailq="NO"
sysrc -f /etc/periodic.conf daily_submit_queuerun="NO"
Installing Postfix SPF
Not used anymore, all handled by rspamd
pkg install postfix-policyd-spf-perl
Install ViMbAdmin
Create several accounts in the mysql database, we give the users only the rights they require, e.g. for dovecot and postfix user select permissions are enough. The account vimbadmin needs more rights to edit data (make sure to replace the password! pwgen -s 20 is maybe a good start). Make sure that every user has an own password!
mysql -u root -p
create database vimbadmin;
grant all privileges on vimbadmin.* to 'vimbadmin'@'localhost' identified by 'password';
grant select on vimbadmin.* to 'dovecot'@'localhost' identified by 'password';
grant select on vimbadmin.* to 'postfix'@'localhost' identified by 'password';
exit
Install ViMbAdmin (follow the instruction https://github.com/opensolutions/ViMbAdmin/wiki/Installation)
:
mkdir -p /usr/local/www
cd /usr/local/www
# git clone https://github.com/mfechner/ViMbAdmin.git
git clone https://github.com/opensolutions/ViMbAdmin.git
cd ViMbAdmin
composer install --dev
chown -R www var/
cd public
cp .htaccess.dist .htaccess
cd ..
Make sure you change the following options (replace values with correct values for your domains):
/usr/local/www/ViMbAdmin/application/configs/application.ini
resources.doctrine2.connection.options.password = 'password'
defaults.mailbox.uid = 5000
defaults.mailbox.gid = 5000
defaults.mailbox.maildir = "maildir:/usr/local/vmail/%d/%u/mail:LAYOUT=fs"
defaults.mailbox.homedir = "/usr/local/vmail/%d/%u"
defaults.mailbox.min_password_length = 20
defaults.mailbox.password_scheme = "dovecot:BLF-CRYPT"
defaults.mailbox.dovecot_pw_binary = "/usr/local/bin/doveadm pw"
server.smtp.host = "smtp-host-name"
server.pop3.host = "pop3-hostname"
server.imap.host = "imap-hostname"
server.webmail.host = "https://webmail-hostname"
identity.orgname = "Lostinspace"
identity.name = "Lostinspace Support Team"
identity.email = "admins@hostname"
identity.autobot.name = "ViMbAdmin Autobot"
identity.autobot.email = "autobot@hostname"
identity.mailer.name = "ViMbAdmin Autobot"
identity.mailer.email = "do-not-reply@hostname"
identity.siteurl = "https://link-to-vimbadmin-website/vimbadmin/"
server.email.name = "ViMbAdmin Administrator"
server.email.address = "support@example.com"
; If you have to authenticate on your mailserver to send email you want to set:
resources.mailer.smtphost = "localhost"
resources.mailer.username = "<user>"
resources.mailer.password = "<password>"
resources.mailer.auth = "login"
resources.mailer.ssl = "tls"
resources.mailer.port = "587"
./bin/doctrine2-cli.php orm:schema-tool:create
Now access the website:
https://hostname/vimbadmin/
and follow the instructions there.
Create user and group that store the emails:
pw groupadd vmail -g 5000
pw useradd vmail -u 5000 -g vmail -s /usr/sbin/nologin -d /nonexistent -c "Virtual Mail Owner"
mkdir -p /usr/local/vmail
chown vmail /usr/local/vmail
chgrp vmail /usr/local/vmail
chmod 770 /usr/local/vmail
Create a random password and hash it for rspamd:
pwgen 20 1
rspamadm pw -p ${RSPAMD_PW}
Create config files:
# maybe set in /usr/local/etc/redis.conf
# echo "maxmemory 512mb" >> /usr/local/etc/redis.conf
# echo "maxmemory-policy volatile-lru" >> /usr/local/etc/redis.conf
mkdir -p /usr/local/etc/rspamd/local.d
/usr/local/etc/rspamd/local.d/antivirus.conf
clamav {
symbol = "CLAM_VIRUS";
type = "clamav";
servers = "/var/run/clamav/clamd.sock";
patterns {
JUST_EICAR = '^Eicar-Test-Signature$';
}
action = "reject";
whitelist = "/usr/local/etc/rspamd/antivirus.wl";
}
/usr/local/etc/rspamd/local.d/worker-controller.inc
password = "${PASSWORD_HASH}";
# dovecot will use this socket to communicate with rspamd
bind_socket = "/var/run/rspamd/rspamd.sock mode=0666";
# you can comment this out if you don't need the web interface
bind_socket = "127.0.0.1:11334";
/usr/local/etc/rspamd/local.d/worker-normal.inc
# we're not running rspamd in a distributed setup, so this can be disabled
# the proxy worker will handle all the spam filtering
enabled = false;
/usr/local/etc/rspamd/local.d/worker-proxy.inc
# this worker will be used as postfix milter
milter = yes;
# note to self - tighten up these permissions
bind_socket = "/var/run/rspamd/milter.sock mode=0666";
# the following specifies self-scan mode, for when rspamd is on the same
# machine as postfix
timeout = 120s;
upstream "local" {
default = yes;
self_scan = yes;
}
/usr/local/etc/rspamd/local.d/redis.conf
# just specifying a server enables redis for all modules that can use it
servers = "127.0.0.1";
/usr/local/etc/rspamd/local.d/classifier-bayes.conf
autolearn = true;
backend = "redis";
/usr/local/etc/rspamd/local.d/dcc.conf
# path to dcc socket
host = "/usr/local/dcc/dccifd";
timeout = 5.0;
/usr/local/etc/rspamd/local.d/dkim_signing.conf
# enable dkim signing - we will set this up in the DKIM section later
path = "/var/db/rspamd/dkim/$domain.$selector.private";
selector = "mail";
check_pubkey=true;
/usr/local/etc/rspamd/local.d/mx_check.conf
# checks if sender's domain has at least one connectable MX record
enabled = true;
/usr/local/etc/rspamd/local.d/phishing.conf
# check messages against some anti-phishing databases
openphish_enabled = true;
phishtank_enabled = true;
/usr/local/etc/rspamd/local.d/replies.conf
# whitelist messages from threads that have been replied to
action = "no action";
/usr/local/etc/rspamd/local.d/surbl.conf
# follow redirects when checking URLs in emails for spaminess
redirector_hosts_map = "/usr/local/etc/rspamd/redirectors.inc";
/usr/local/etc/rspamd/local.d/url_reputation.conf
# check URLs within messages for spaminess
enabled = true;
/usr/local/etc/rspamd/local.d/url_tags.conf
# cache some URL tags in redis
enabled = true;
Create the dkim signing keys (this needs to be done for every domain; I use here domain fechner.net):
mkdir -p /var/db/rspamd/dkim
chown rspamd:rspamd /var/db/rspamd/dkim
chmod 750 /var/db/rspamd/dkim
rspamadm dkim_keygen -b 2048 -d fechner.net -k /var/db/rspamd/dkim/fechner.net.mail.private > fechner.net.dkim.txt
Now make sure you include the public DNS key to your zone by copy the fechner.net.dkim.txt
to the directory where the zone files are located:
$ORIGIN fechner.net.
...
$INCLUDE fechner.net.dkim.txt
...
sysrc rspamd_enable="YES"
sysrc redis_enable="YES"
service redis start
service rspamd start
Create dh.pem:
mkdir -p /usr/local/etc/ssl
cd /usr/local/etc/ssl
openssl genpkey -genparam -algorithm DH -out dh_512.pem -pkeyopt dh_paramgen_prime_len:512
openssl genpkey -genparam -algorithm DH -out dh_1024.pem -pkeyopt dh_paramgen_prime_len:1024
openssl genpkey -genparam -algorithm DH -out dh_2048.pem -pkeyopt dh_paramgen_prime_len:2048
openssl genpkey -genparam -algorithm DH -out dh_4096.pem -pkeyopt dh_paramgen_prime_len:4096
Now we configure dovecot, set the config files based on this diff.:
diff -ur /usr/local/share/doc/dovecot/example-config/conf.d/10-auth.conf ./conf.d/10-auth.conf
--- /usr/local/share/doc/dovecot/example-config/conf.d/10-auth.conf 2014-08-19 20:38:20.043506000 +0200
+++ ./conf.d/10-auth.conf 2014-08-19 20:06:07.528052364 +0200
@@ -119,7 +119,7 @@
#!include auth-deny.conf.ext
#!include auth-master.conf.ext
-!include auth-system.conf.ext
+#!include auth-system.conf.ext
#!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
diff -ur /usr/local/share/doc/dovecot/example-config/conf.d/10-ssl.conf ./conf.d/10-ssl.conf
--- /usr/local/share/doc/dovecot/example-config/conf.d/10-ssl.conf 2014-08-19 20:38:20.044506000 +0200
+++ ./conf.d/10-ssl.conf 2014-08-19 22:27:15.827087484 +0200
@@ -9,8 +9,8 @@
# dropping root privileges, so keep the key file unreadable by anyone but
# root. Included doc/mkcert.sh can be used to easily generate self-signed
# certificate, just make sure to update the domains in dovecot-openssl.cnf
-ssl_cert = </etc/ssl/certs/dovecot.pem
-ssl_key = </etc/ssl/private/dovecot.pem
+#ssl_cert = </etc/ssl/certs/dovecot.pem
+#ssl_key = </etc/ssl/private/dovecot.pem
# If key file is password protected, give the password here. Alternatively
# give it when starting dovecot with -p parameter. Since this file is often
diff -ur /usr/local/share/doc/dovecot/example-config/dovecot-sql.conf.ext ./dovecot-sql.conf.ext
--- /usr/local/share/doc/dovecot/example-config/dovecot-sql.conf.ext 2014-08-19 20:38:20.064506000 +0200
+++ ./dovecot-sql.conf.ext 2014-08-19 22:33:01.703040984 +0200
@@ -29,7 +29,7 @@
# );
# Database driver: mysql, pgsql, sqlite
-#driver =
+driver = mysql
# Database connection string. This is driver-specific setting.
#
@@ -68,14 +68,14 @@
# connect = host=sql.example.com dbname=virtual user=virtual password=blarg
# connect = /etc/dovecot/authdb.sqlite
#
-#connect =
+connect = host=localhost dbname=vimbadmin user=dovecot password=<password>
# Default password scheme.
#
# List of supported schemes is in
# http://wiki2.dovecot.org/Authentication/PasswordSchemes
#
-#default_pass_scheme = MD5
+default_pass_scheme = BLF-CRYPT
# passdb query to retrieve the password. It can return fields:
# password - The user's password. This field must be returned.
@@ -137,5 +137,12 @@
# home AS userdb_home, uid AS userdb_uid, gid AS userdb_gid \
# FROM users WHERE userid = '%u'
+password_query = SELECT username as user, password as password, \
+ homedir AS userdb_home, maildir AS userdb_mail, \
+ concat('*:bytes=', quota) as userdb_quota_rule, uid as userdb_uid, gid as userdb_gid \
+ FROM mailbox \
+ WHERE username = '%Lu' AND active = '1' \
+ AND ( access_restriction = 'ALL' OR LOCATE( '%Us', access_restriction ) > 0 )
+
+user_query = SELECT homedir AS home, maildir AS mail, \
+ concat('*:bytes=', quota) as quota_rule, uid, gid \
+ FROM mailbox WHERE username = '%u'
# Query to get a list of all usernames.
- #iterate_query = SELECT username AS user FROM users
+ iterate_query = SELECT username AS user FROM mailbox
Now create a new config file that hold all settings:
/usr/local/etc/dovecot/local.conf
service auth {
unix_listener auth-userdb {
mode = 0666
user = vmail
group = vmail
}
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix
}
# Auth process is run as this user.
#user = $default_internal_user
user=root
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0660
group = postfix
user = postfix
}
user = vmail
}
# ***** Configure location for mailbox
mail_location = maildir:/usr/local/vmail/%d/%u
# ***** Authenticate against sql database *****
auth_mechanisms = plain login
passdb {
driver = sql
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}
userdb {
driver = prefetch
}
userdb {
driver = sql
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}
# ***** use uid and gid for vmail
mail_uid = 5000
mail_gid = 5000
mail_privileged_group = 5000
mail_access_groups = 5000
first_valid_uid = 5000
last_valid_uid = 5000
first_valid_gid = 5000
last_valid_gid = 5000
maildir_copy_with_hardlinks = yes
# ***** Modules we use *****
mail_plugins = $mail_plugins
# **** SSL config *****
ssl = yes
ssl_cert = </var/db/acme/certs/<domain>/fullchain.cer
ssl_key = </var/db/acme/certs/<domain>/<domain>.key
#ssl_alt_cert = </
#ssl_alt_key = </
ssl_require_crl = no
ssl_prefer_server_ciphers = yes
ssl_dh=</usr/local/etc/ssl/dh_4096.pem
ssl_min_protocol = TLSv1.2
# ***** Configure POP3 *****
protocol pop3 {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins quota
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
# **** Configure IMAP *****
protocol imap {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins quota imap_quota imap_sieve
}
# ***** LDA Config *****
postmaster_address = postmaster@%d
hostname = <fqdn>
quota_full_tempfail = yes
recipient_delimiter = +
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
protocol lda {
mail_plugins = $mail_plugins sieve quota
}
# ***** LMTP Config *****
protocol lmtp {
postmaster_address = postmaster@%d
mail_plugins = quota sieve
}
# ***** Plugin Configuration *****
plugin {
# autocreate plugin
# This plugin allows administrator to specify mailboxes that must always
# exist for all users. They can optionally also be subscribed. The
# mailboxes are created and subscribed always after user logs in.
# Namespaces are fully supported, so namespace prefixes need to be used
# where necessary.
autocreate = Sent
autocreate2 = Drafts
autocreate3 = Junk
autocreate4 = Trash
#autocreate5 = ..etc..
autosubscribe = Sent
autosubscribe2 = Drafts
autosubscribe3 = Junk
autosubscribe4 = Trash
#autosubscribe5 = ..etc
sieve = ~/sieve/dovecot.sieve
sieve_dir = ~/sieve
sieve_extensions = +notify +imapflags +spamtest +spamtestplus +relational +comparator-i;ascii-numeric
sieve_before = /usr/local/etc/dovecot/sieve/
# ***** Quota Configuration *****
quota = maildir:User quota
sieve_plugins = sieve_imapsieve sieve_extprograms
# From elsewhere to Junk folder
imapsieve_mailbox1_name = Junk
imapsieve_mailbox1_causes = COPY FLAG
imapsieve_mailbox1_before = file:/usr/local/etc/dovecot/sieve/report-spam.sieve
# From Spam folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/usr/local/etc/dovecot/sieve/report-ham.sieve
sieve_pipe_bin_dir = /usr/local/etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
}
# ***** Quota Configuration *****
plugin {
# quota = maildir:User quota
}
# ***** Configure Sieve *****
protocols = $protocols sieve
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service managesieve {
}
protocol sieve {
}
##
## Mailbox definitions
##
# NOTE: Assumes "namespace inbox" has been defined in 10-mail.conf.
namespace inbox {
#mailbox name {
# auto=create will automatically create this mailbox.
# auto=subscribe will both create and subscribe to the mailbox.
#auto = no
# Space separated list of IMAP SPECIAL-USE attributes as specified by
# RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash
#special_use =
#}
# These mailboxes are widely used and could perhaps be created automatically:
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Trash {
special_use = \Trash
}
# For \Sent mailboxes there are two widely used names. We'll mark both of
# them as \Sent. User typically deletes one of them if duplicates are created.
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
# If you have a virtual "All messages" mailbox:
#mailbox virtual/All {
# special_use = \All
#}
# If you have a virtual "Flagged" mailbox:
#mailbox virtual/Flagged {
# special_use = \Flagged
#}
}
# ***** Logging *****
auth_verbose = no
auth_debug_passwords = no
mail_debug = no
To use the sieve plugin in Thunderbird use this here: https://github.com/thsmi/sieve/blob/master/nightly/README.md
As we configured the folder /usr/local/etc/dovecot/sieve to hold standard scripts for all users:
mkdir -p /usr/local/etc/dovecot/sieve/global
chown -R vmail:vmail /usr/local/etc/dovecot/sieve
Now create a new file with content:
/usr/local/dovecot/etc/dovecot/sieve/global/move-spam.sieve
require ["fileinto","mailbox"];
if anyof (header :contains ["X-Spam-Flag"] "YES",
header :contains ["X-Spam"] "YES",
header :contains ["Subject"] "*** SPAM ***"
)
{
fileinto :create "Junk";
}
/* Other messages get filed into INBOX */
/usr/local/dovecot/etc/dovecot/sieve/report-ham.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.mailbox" "*" {
set "mailbox" "${1}";
}
if string "${mailbox}" "Trash" {
stop;
}
if environment :matches "imap.email" "*" {
set "email" "${1}";
}
pipe :copy "train-ham.sh" [ "${email}" ];
/usr/local/dovecot/etc/dovecot/sieve/report-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.email" "*" {
set "email" "${1}";
}
pipe :copy "train-spam.sh" [ "${email}" ];
Compile all rules:
cd /usr/local/etc/dovecot/sieve
sievec report-ham.sieve
sievec report-spam.sieve
/usr/local/etc/dovecot/sieve/train-ham.sh
#!/bin/sh
exec /usr/local/bin/rspamc -h /var/run/rspamd/rspamd.sock learn_ham
/usr/local/etc/dovecot/sieve/train-spam.sh
#!/bin/sh
exec /usr/local/bin/rspamc -h /var/run/rspamd/rspamd.sock learn_spam
chown vmail .
chown vmail *
chgrp vmail .
chgrp vmail *
chmod +x *.sh
service dovecot restart
Migrate mbox to Maildir
We have to migrate a mbox to Maildir.
The inbox is on: /var/mail/.
The other folders are /usr/home//mail.
Make sure the dovecot user can read/write to the folders (make sure you remember the permission to undo it):
chgrp vmail /var/mail
chmod g+w /var/mail
chgrp vmail /var/mail/<user>
chgrp -R vmail /usr/home/<user>/mail
Now we convert it:
dsync -v -u <newdovecotuser> mirror mbox:/usr/home/<user>/mail/:INBOX=/var/mail/<user>
Restore permissions on the old folder/files or remove them if migration was successfully finished.
/usr/local/mailman/Mailman/mm_cfg.py
MTA = 'Postfix'
POSTFIX_STYLE_VIRTUAL_DOMAINS = ['domain1.de', 'domain2.org' ]
SMTPHOST = 'full-smtp-host-name-to-connect'
Create required files:
cd /usr/local/mailman
bin/genaliases
chown mailman:mailman data/aliases*
chmod g+w data/aliases*
chown mailman:mailman data/virtual-mailman*
chmod g+w data/virtual-mailman*
Setup database for Sympa
mysql -u root -p
create database sympa CHARACTER SET utf8mb4;
grant all privileges on sympa.* to 'sympa'@'localhost' identified by '_PW_';
quit
Setup logging for Sympa
touch /var/log/sympa.log
chmod 640 /var/log/sympa.log
mkdir -p /usr/local/etc/syslog.d
Create file /usr/local/etc/syslog.d/sympa.conf:
local1.* -/var/log/sympa.log
Restart syslog:
Before we start we need to update the file: /usr/local/etc/sympa/sympa.conf
########################################################################
# Initial configuration
# See https://sympa-community.github.io/manual/install/generate-initial-configuration.html
########################################################################
domain fechner.net
listmaster spam@fechner.net
#lang en-US
########################################################################
# Setup database
# See https://sympa-community.github.io/manual/install/setup-database.html
########################################################################
db_type MySQL
db_name sympa
db_host localhost
#db_port
db_user sympa
db_passwd _PW_
#db_env
########################################################################
# Configure system log
# See https://sympa-community.github.io/manual/install/configure-system-log.html
########################################################################
syslog LOCAL1
log_socket_type unix
########################################################################
# Configure mail server
# See https://sympa-community.github.io/manual/install/configure-mail-server.html
########################################################################
sendmail_aliases /usr/local/etc/sympa/sympa_transport
aliases_program postmap
aliases_db_type hash
sendmail /usr/local/sbin/sendmail
#sendmail_args (if you use sendmail(1), this need not change)
########################################################################
# Configure HTTP server
# See https://sympa-community.github.io/manual/install/configure-http-server.html
########################################################################
mhonarc /usr/local/bin/mhonarc
#log_facility LOCAL1
# If you chose single domain setting, you may have to define following
# parameters in this sympa.conf file. Otherwise, if you chose virtual
# domain setting (recommended), you should define them in robot.conf by
# each domain.
#wwsympa_url (You must define this parameter to enable web interface)
########################################################################
# Customizing Sympa
# You can customize Sympa, its web interface and/or SOAP/HTTP service
# defining more parameters in this file sympa.conf or robot.conf by each
# domain.
# For more details see https://sympa-community.github.io/manual/customize.html
########################################################################
#log_level 1024
max_size 20971520
Fix permissions:
chgrp sympa /usr/local/etc/sympa/sympa.conf
chmod g+w /usr/local/etc/sympa
Create database structure with:
Tests
Test logging with:
/usr/local/libexec/sympa/testlogs.pl
Create file /usr/local/etc/sympa/list_aliases.tt2:
#--- [% list.name %]@[% list.domain %]: list transport map created at [% date %]
[% list.name %]@[% list.domain %] sympa:[% list.name %]@[% list.domain %]
[% list.name %]-request@[% list.domain %] sympa:[% list.name %]-request@[% list.domain %]
[% list.name %]-editor@[% list.domain %] sympa:[% list.name %]-editor@[% list.domain %]
[% list.name %]-subscribe@[% list.domain %] sympa:[% list.name %]-subscribe@[%list.domain %]
[% list.name %]-unsubscribe@[% list.domain %] sympa:[% list.name %]-unsubscribe@[% list.domain %]
[% list.name %][% return_path_suffix %]@[% list.domain %] sympabounce:[% list.name %]@[% list.domain %]
Create some files:
touch /usr/local/etc/sympa/transport.sympa
touch /usr/local/etc/sympa/virtual.sympa
touch /usr/local/etc/sympa/sympa_transport
chmod 660 /usr/local/etc/sympa/sympa_transport
chown root:sympa /usr/local/etc/sympa/sympa_transport
postmap hash:/usr/local/etc/sympa/transport.sympa
postmap hash:/usr/local/etc/sympa/virtual.sympa
chmod g+w /usr/local/etc/sympa/sympa_transport*
/usr/local/libexec/sympa/sympa_newaliases.pl
Add to /usr/local/etc/postfix/master.cf
sympa unix - n n - - pipe
flags=hqRu null_sender= user=sympa argv=/usr/local/libexec/sympa/queue ${nexthop}
sympabounce unix - n n - - pipe
flags=hqRu null_sender= user=sympa argv=/usr/local/libexec/sympa/bouncequeue ${nexthop}
Add to /usr/local/etc/postfix/main.cf
virtual_mailbox_domains = hash:/usr/local/etc/sympa/transport.sympa
virtual_mailbox_maps = hash:/usr/local/etc/sympa/transport.sympa,
hash:/usr/local/etc/sympa/sympa_transport,
hash:/usr/local/etc/sympa/virtual.sympa
virtual_alias_maps = hash:/usr/local/etc/sympa/virtual.sympa
transport_maps = hash:/usr/local/etc/sympa/transport.sympa,
hash:/usr/local/etc/sympa/sympa_transport
recipient_delimiter = +
Adding new domain fechner.net with:
mkdir -m 755 /usr/local/etc/sympa/fechner.net
touch /usr/local/etc/sympa/fechner.net/robot.conf
chown -R sympa:sympa /usr/local/etc/sympa/fechner.net
mkdir -m 750 /usr/local/share/sympa/list_data/fechner.net
chown sympa:sympa /usr/local/share/sympa/list_data/fechner.net
Modify /usr/local/etc/sympa/fechner.net/robot.conf:
wwsympa_url https://fechner.net/sympa
listmaster idefix@fechner.net
Edit /usr/local/etc/sympa/transport.sympa
fechner.net error:User unknown in recipient table
sympa@fechner.net sympa:sympa@fechner.net
listmaster@fechner.net sympa:listmaster@fechner.net
bounce@fechner.net sympabounce:sympa@fechner.net
abuse-feedback-report@fechner.net sympabounce:sympa@fechner.net
Edit /usr/local/etc/sympa/virtual.sympa
sympa-request@fechner.net postmaster@localhost
sympa-owner@fechner.net postmaster@localhost
Recreate the DB files:
postmap hash:/usr/local/etc/sympa/transport.sympa
postmap hash:/usr/local/etc/sympa/virtual.sympa
chmod g+w /usr/local/etc/sympa/sympa_transport*
Enable sympa and start it:
sysrc sympa_enable="YES"
service sympa start
sysrc spawn_fcgi_enable="YES"
sysrc spawn_fcgi_app="/usr/local/libexec/sympa/wwsympa.fcgi"
sysrc spawn_fcgi_bindsocket="/var/run/sympa/wwsympa.socket"
sysrc spawn_fcgi_bindsocket_mode="0777"
sysrc spawn_fcgi_username="sympa"
sysrc spawn_fcgi_groupname="sympa"
service spawn-fcgi start
Copy standard configuration files and modify them:
cd /usr/local/etc/
cp clamd.conf.sample clamd.conf
cp freshclam.conf.sample freshclam.conf
# not used anymore, is handeled by rspamd
# cp clamsmtpd.conf.sample clamsmtpd.conf
/usr/local/etc/freshclam.conf
--- freshclam.conf.sample 2016-03-19 10:55:28.000000000 +0100
+++ freshclam.conf 2016-03-19 11:27:09.857817239 +0100
@@ -71,6 +71,7 @@
# code. See http://www.iana.org/cctld/cctld-whois.htm for the full list.
# You can use db.XY.ipv6.clamav.net for IPv6 connections.
#DatabaseMirror db.XY.clamav.net
+DatabaseMirror db.de.clamav.net
/usr/local/etc/clamsmtpd.conf
Not used anymore, handled by rspamd
--- clamsmtpd.conf.sample 2016-04-02 04:13:28.000000000 +0200
+++ clamsmtpd.conf 2016-04-02 12:46:37.399587985 +0200
@@ -8,7 +8,7 @@
# The address to send scanned mail to.
# This option is required unless TransparentProxy is enabled
-OutAddress: 10026
+OutAddress: 10029
@@ -26,13 +26,13 @@
#XClient: off
# Address to listen on (defaults to all local addresses on port 10025)
-#Listen: 0.0.0.0:10025
+Listen: 127.0.0.1:10028
# The address clamd is listening on
-#ClamAddress: /var/run/clamav/clamd.sock
+ClamAddress: /var/run/clamav/clamd.sock
# A header to add to all scanned email
-#Header: X-Virus-Scanned: ClamAV using ClamSMTP
+Header: X-Virus-Scanned: ClamAV using ClamSMTP
# Directory for temporary files
#TempDirectory: /tmp
@@ -47,7 +47,7 @@
#TransparentProxy: off
# User to switch to
-#User: clamav
+User: clamav
# Virus actions: There's an option to run a script every time a virus is found.
# !IMPORTANT! This can open a hole in your server's security big enough to drive
Configure Postfix
Add the following lines to main.cf
:
/usr/local/etc/postfix/main.cf
# enable TLS
tls_append_default_CA = yes
smtpd_tls_received_header = yes
#smtpd_tls_key_file = /etc/mail/certs/req.pem
#smtpd_tls_cert_file = /etc/mail/certs/newcert.pem
#smtpd_tls_key_file = /usr/local/etc/letsencrypt/live/${DOMAIN}/privkey.pem
#smtpd_tls_cert_file = /usr/local/etc/letsencrypt/live/${DOMAIN}/fullchain.pem
smtpd_tls_chain_files =
/var/db/acme/certs/${DOMAIN}/${DOMAIN}.key
/var/db/acme/certs/${DOMAIN}/fullchain.cer
smtpd_tls_loglevel = 1
# enable smtp auth as Server
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions =
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_unauth_pipelining,
permit_mynetworks,
permit_sasl_authenticated,
reject_invalid_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unauth_destination,
reject_unknown_reverse_client_hostname,
reject_unknown_client,
reject_unknown_hostname,
check_client_access hash:/usr/local/etc/postfix/client_checks,
check_sender_access hash:/usr/local/etc/postfix/sender_checks,
reject_non_fqdn_hostname,
check_policy_service unix:private/spf-policy
smtpd_helo_restrictions =
permit_mynetworks,
# check_helo_access hash:/etc/postfix/ehlo_whitelist,
reject_non_fqdn_hostname,
reject_invalid_hostname
#mua_client_restrictions =
mua_helo_restrictions = permit_sasl_authenticated,reject
#mua_sender_restrictions =
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
broken_sasl_auth_clients = yes
smtpd_helo_required = yes
strict_rfc821_envelopes = yes
disable_vrfy_command = yes
smtpd_delay_reject = yes
smtpd_sender_restrictions =
permit_mynetworks,
reject_unknown_sender_domain
# check_sender_access hash:/etc/postfix/sender_access,
smtpd_data_restrictions =
reject_unauth_pipelining
smtpd_client_restrictions =
permit_sasl_authenticated,
# check_client_access hash:/etc/postfix/client_access,
# enable ipv6 and ipv4
# inet_protocols = all
# limit message size to 100MB
message_size_limit = 104857600
mailbox_size_limit = 512000000
virtual_mailbox_limit = 512000000
# increase timeouts to prevent queue write file errors
#smtpd_timeout=600s
smtpd_proxy_timeout=600s
#smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
# Virtual Domain Configuration
virtual_alias_maps = mysql:/usr/local/etc/postfix/mysql/virtual_alias_maps.cf
#, hash:/usr/local/mailman/data/virtual-mailman
virtual_gid_maps = static:5000
virtual_mailbox_base = /usr/local/vmail
virtual_mailbox_domains = mysql:/usr/local/etc/postfix/mysql/virtual_domains_maps.cf
virtual_mailbox_maps = mysql:/usr/local/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 5000
virtual_uid_maps = static:5000
#dovecot_destination_recipient_limit = 1
virtual_transport = lmtp:unix:private/dovecot-lmtp
home_mailbox = Maildir/
smtpd_sasl_authenticated_header = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
# Mailman
alias_maps = hash:/etc/mail/aliases,
hash:/etc/mail/aliases.own
#, hash:/usr/local/mailman/data/aliases
# SPF
spf-policy_time_limit = 3600
# optimize SSL configuration
smtpd_tls_security_level = may
smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3
smtpd_tls_protocols = !SSLv2 !SSLv3
smtpd_tls_dh512_param_file = /usr/local/etc/ssl/dh_512.pem
tls_preempt_cipherlist = yes
smtpd_tls_loglevel = 1
smtp_dns_support_level = dnssec
smtp_tls_security_level=dane
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3
smtp_tls_protocols = !SSLv2, !SSLv3
smtp_tls_mandatory_ciphers = high
smtp_tls_loglevel = 1
# Sender Rewriting
sender_canonical_maps = tcp:127.0.0.1:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:127.0.0.1:10002
recipient_canonical_classes= envelope_recipient
## Postscreen setup
postscreen_access_list = permit_mynetworks,cidr:/usr/local/etc/postfix/postscreen_access.cidr
postscreen_blacklist_action = drop
# DNS Blackhole Lists
postscreen_dnsbl_threshold = 8
postscreen_dnsbl_sites =
b.barracudacentral.org=127.0.0.2*7
dnsbl.inps.de=127.0.0.2*7
bl.mailspike.net=127.0.0.2*5
bl.mailspike.net=127.0.0.[10;11;12]*4
dnsbl.sorbs.net=127.0.0.10*8
dnsbl.sorbs.net=127.0.0.5*6
dnsbl.sorbs.net=127.0.0.7*3
dnsbl.sorbs.net=127.0.0.8*2
dnsbl.sorbs.net=127.0.0.6*2
dnsbl.sorbs.net=127.0.0.9*2
zen.spamhaus.org=127.0.0.[10..11]*8
zen.spamhaus.org=127.0.0.[4..7]*6
zen.spamhaus.org=127.0.0.3*4
zen.spamhaus.org=127.0.0.2*3
bl.spamcop.net*2
hostkarma.junkemailfilter.com=127.0.0.2*3
hostkarma.junkemailfilter.com=127.0.0.4*1
hostkarma.junkemailfilter.com=127.0.1.2*1
dnsbl-1.uceprotect.net*2
dnsbl-2.uceprotect.net*2
dnsbl-3.uceprotect.net*3
wl.mailspike.net=127.0.0.[18;19;20]*-2
list.dnswl.org=127.0.[0..255].0*-3
list.dnswl.org=127.0.[0..255].1*-4
list.dnswl.org=127.0.[0..255].[2..255]*-6
hostkarma.junkemailfilter.com=127.0.0.1*-2
postscreen_dnsbl_action = enforce
# Pregreeting
postscreen_greet_action = enforce
# Additional Postscreen Tests
postscreen_pipelining_enable = no
postscreen_non_smtp_command_enable = no
postscreen_non_smtp_command_action = drop
postscreen_bare_newline_enable = no
# OpenDKIM (port 8891), OpenDMARC (port 8893)
#milter_default_action = accept
#smtpd_milters = inet:localhost:8891
#non_smtpd_milters = inet:localhost:8891
compatibility_level = 2
# Milter configuration used for rspamd
# milter_default_action = accept
smtpd_milters = unix:/var/run/rspamd/milter.sock
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
Edit master.cf
to have this:
/usr/local/etc/postfix/master.cf
#
# Postfix master process configuration file. For details on the format
# of the file, see the master(5) manual page (command: "man 5 master" or
# on-line: http://www.postfix.org/master.5.html).
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
#smtp inet n - n - - smtpd
smtp inet n - n - 1 postscreen
smtpd pass - - n - - smtpd
dnsblog unix - - n - 0 dnsblog
tlsproxy unix - - n - 0 tlsproxy
submission inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_reject_unlisted_recipient=no
# -o smtpd_client_restrictions=$mua_client_restrictions
-o smtpd_helo_restrictions=$mua_helo_restrictions
# -o smtpd_sender_restrictions=$mua_sender_restrictions
-o smtpd_recipient_restrictions=
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATING
smtps inet n - n - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_reject_unlisted_recipient=no
# -o smtpd_client_restrictions=$mua_client_restrictions
-o smtpd_helo_restrictions=$mua_helo_restrictions
# -o smtpd_sender_restrictions=$mua_sender_restrictions
-o smtpd_recipient_restrictions=
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATING
#628 inet n - n - - qmqpd
pickup unix n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr unix n - n 300 1 qmgr
#qmgr unix n - n 300 1 oqmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent. See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
#maildrop unix - n n - - pipe
# flags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
# lmtp cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
# mailbox_transport = lmtp:inet:localhost
# virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus unix - n n - - pipe
# user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
#
# Old example of delivery via Cyrus.
#
#old-cyrus unix - n n - - pipe
# flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
#uucp unix - n n - - pipe
# flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# ====================================================================
#
# Other external delivery methods.
#
#ifmail unix - n n - - pipe
# flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
#
#bsmtp unix - n n - - pipe
# flags=Fq. user=bsmtp argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
#
#scalemail-backend unix - n n - 2 pipe
# flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
# ${nexthop} ${user} ${extension}
#
#mailman unix - n n - - pipe
# flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
# ${nexthop} ${user}
# SPF check
spf-policy unix - n n - 0 spawn
user=spfcheck argv=/usr/local/libexec/postfix-policyd-spf-perl
Create SQL related configuration:
mkdir -p /usr/local/etc/postfix/mysql
cd !$
Create the following files:
/usr/local/etc/postfix/mysql/virtual_alias_maps.cf
user = postfix
password = <password>
hosts = 127.0.0.1
dbname = vimbadmin
query = SELECT goto FROM alias WHERE address = '%s' AND active = '1'
/usr/local/etc/postfix/mysql/virtual_domains_maps.cf
user = postfix
password = <password>
hosts = 127.0.0.1
dbname = vimbadmin
query = SELECT domain FROM domain WHERE domain = '%s' AND backupmx = '0' AND active = '1'
/usr/local/etc/postfix/mysql/virtual_mailbox_maps.cf
user = postfix
password = <password>
hosts = 127.0.0.1
dbname = vimbadmin
table = mailbox
select_field = maildir
where_field = username
/usr/local/etc/postfix/mysql/virtual_transport_maps.cf
user = postfix
password = <password>
hosts = 127.0.0.1
dbname = vimbadmin
table = domain
select_field = transport
where_field = domain
additional_conditions = and backupmx = '0' and active = '1'
Secure files as we have passwords in them:
chown root:postfix *.cf
chmod 640 *.cf
If you would like to block a domain add the following lines:
/usr/local/etc/postfix/client_checks
# IP/HOSTNAME REJECT User unkown
To block senders:
/usr/local/etc/postfix/sender_checks
# email REJECT User unkown
Create the map with:
cd /usr/local/etc/postfix
postconf -e inet_protocols=all
Make sure you build your required database files
cd /etc/mail
touch aliases.own
postalias aliases
postalias aliases.own
cd /usr/local/etc/postfix
touch client_checks
touch sender_checks
postmap client_checks
postmap sender_checks
SRS is sender Rewriting Scheme daemon which is required if you forward mails and to not break SPF.
pkg install postsrsd
sysrc postsrsd_enable=YES
sysrc postsrsd_flags=" -4"
Add to main.cf
file from postfix:
/usr/local/etc/postfix/main.cf
# Sender Rewriting
sender_canonical_maps = tcp:127.0.0.1:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:127.0.0.1:10002
recipient_canonical_classes= envelope_recipient
Start it with:
/usr/local/etc/rc.d/postsrsd restart
/usr/local/etc/rc.d/postfix restart
WARNING! Make sure you include all domains you host to the ignore list for postsrsd. If not that could break SPF setup and will in worst case cause that all emails are bounced and not accepted by remote mail servers!
postsrsd_exclude_domains="domain1.de,domain2.net,domain3.de"
Maybe you have to apply this patch to fix a bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199797
Postscreen
Reference:
Postscreen is used to block spammers as early as possible using some checks including rbl lists.
Create the file:
/usr/local/etc/postfix/postscreen_access.cidr
# Rules are evaluated in the order as specified.
127.0.0.1 permit
::1 permit
You must restart postfix if you have changed the postscreen_access.cidr
!
OpenDKIM (WIP)
References:
cd /usr/ports/mail/opendkim
make install clean
sysrc milteropendkim_enable="YES"
mkdir -p /var/db/opendkim
chown mailnull /var/db/opendkim/
Generate key for domain fechner.net
pkg install opendkim
opendkim-genkey -r -D /var/db/opendkim -d fechner.net
mv default.private fechner.net.dkim.private
mv default.txt fechner.net.dkim.txt
Copy the public key to your DNS server:
cp fechner.net.dkim.txt /usr/local/etc/namedb/master/fechner.net/
chown bind /usr/local/etc/namedb/master/fechner.net/fechner.net.dkim.txt
chmod 644 /usr/local/etc/namedb/master/fechner.net/fechner.net.dkim.txt
Make sure your DNS zone includes:
; include dkim public key
$INCLUDE /usr/local/etc/namedb/master/fechner.net/fechner.net.dkim.txt
_adsp._domainkey IN TXT "dkim=unknown"
Increase your serial and reload the zone.
Make sure your zone is correct with:
host -t TXT default._domainkey.fechner.net
dig +norec @localhost -t TXT default._domainkey.fechner.net
Now we configure what key is used for what domain:
/usr/local/etc/mail/DkimSigningTable
# format:
# $pattern $keyname
*@fechner.net fechner.net
/usr/local/etc/mail/DkimKeyTable
# format
# $keyname $domain:$selector:$keypath
fechner.net fechner.net:default:/var/db/opendkim/fechner.net.dkim.private
The last part is the configuration file:
/usr/local/etc/mail/opendkim.conf
--- opendkim.conf.sample 2015-04-29 12:06:58.290800000 +0200
+++ opendkim.conf 2015-04-29 15:56:05.735861987 +0200
@@ -116,7 +116,7 @@
## operation. Thus, cores will be dumped here and configuration files
## are read relative to this location.
-# BaseDirectory /var/run/opendkim
+BaseDirectory /var/db/opendkim
## BodyLengthDB dataset
## default (none)
@@ -175,7 +175,7 @@
## Specify for which domain(s) signing should be done. No default; must
## be specified for signing.
-Domain example.com
+Domain anny.lostinspace.de
## DomainKeysCompat { yes | no }
## default "no"
@@ -261,7 +261,7 @@
## a base64-encoded DER format private key, or a path to a file containing
## one of those.
-# KeyTable dataset
+KeyTable /usr/local/etc/mail/DkimKeyTable
## LocalADSP dataset
## default (none)
@@ -290,7 +290,7 @@
## in the amount of log data generated for each message, so it should be
## limited to debugging use and not enabled for general operation.
-# LogWhy no
+LogWhy Yes
## MacroList macro[=value][,...]
##
@@ -659,7 +659,7 @@
## is set, all possible lookup keys will be attempted which may result
## in multiple signatures being applied.
-# SigningTable filename
+SigningTable refile:/usr/local/etc/mail/DkimSigningTable
## SingleAuthResult { yes | no}
## default "no"
@@ -687,7 +687,7 @@
## inet:port to listen on all interfaces
## local:/path/to/socket to listen on a UNIX domain socket
-Socket inet:port@localhost
+Socket inet:8891@localhost
## SoftwareHeader { yes | no }
## default "no"
@@ -746,7 +746,7 @@
##
## Log success activity to syslog?
-# SyslogSuccess No
+SyslogSuccess Yes
## TemporaryDirectory path
## default /tmp
Now start the milter with:
sysrc milteropendkim_enable=YES
service milter-opendkim start
Check the /var/log/maillog for possible error messages, if you found no error message we can continue setting up postfix:
/usr/local/etc/postfix/main.cf
# OpenDKIM
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891
To test it write an email to check-auth@verifier.port25.com
OpenDmarc (WIP)
A first comment: I do NOT recommend to use DMARC. It breaks most of the existing mailing lists, and you will get problem from a lot of mail servers, that are not configured 100% correctly.
DMARC will cause a lot of false-positives. I decided to not use it for any domain I manage.
Depending on your needs, this could be different.
References:
cd /usr/ports/mail/opendmarc
make install clean
Configuration:
cp /usr/local/etc/mail/opendmarc.conf.sample opendmarc.conf
Modify the configuration like this:
/usr/local/etc/mail/opendmarc.conf
--- opendmarc.conf.sample 2015-04-29 11:17:12.018006000 +0200
+++ opendmarc.conf 2015-04-30 05:35:34.395463225 +0200
@@ -90,7 +90,7 @@
## Requests addition of the specified email address to the envelope of
## any message that fails the DMARC evaluation.
#
-# CopyFailuresTo postmaster@localhost
+CopyFailuresTo postmaster@fechner.net
## DNSTimeout (integer)
## default 5
@@ -118,7 +118,7 @@
## purported sender of the message has requested such reports. Reports are
## formatted per RFC6591.
#
-# FailureReports false
+FailureReports true
## FailureReportsBcc (string)
## default (none)
@@ -129,7 +129,7 @@
## If no request is made, they address(es) are used in a To: field. There
## is no default.
#
-# FailureReportsBcc postmaster@example.coom
+FailureReportsBcc postmaster@fechner.net
## FailureReportsOnNone { true | false }
## default "false"
@@ -273,7 +273,7 @@
## either in the configuration file or on the command line. If an IP
## address is used, it must be enclosed in square brackets.
#
-# Socket inet:8893@localhost
+Socket inet:8893@localhost
## SoftwareHeader { true | false }
## default "false"
@@ -283,7 +283,7 @@
## delivery. The product's name, version, and the job ID are included in
## the header field's contents.
#
-# SoftwareHeader false
+SoftwareHeader true
## SPFIgnoreResults { true | false }
## default "false"
@@ -312,7 +312,7 @@
##
## Log via calls to syslog(3) any interesting activity.
#
-# Syslog false
+Syslog true
## SyslogFacility facility-name
## default "mail"
@@ -343,7 +343,7 @@
## specific file mode on creation regardless of the process umask. See
## umask(2) for more information.
#
-# UMask 077
+UMask 0002
## UserID user[:group]
## default (none)
sysrc opendmarc_enable="YES"
touch /var/run/opendmarc.pid
chown mailnull:mailnull /var/run/opendmarc.pid
/usr/local/etc/rc.d/opendmarc start
In postfix add:
/usr/local/etc/postfix/main.cf
# OpenDKIM (port 8891), OpenDMARC (port 8893)
milter_default_action = accept
smtpd_milters = inet:localhost:8891, inet:localhost:8893
non_smtpd_milters = inet:localhost:8891, inet:localhost:8893
Restart postfix:
The last step would be to add a dmarc TXT record to your DNS zone. You can use therefore:
A record could look like:
_dmarc IN TXT "v=DMARC1; p=none; sp=none; rua=mailto:postmaster@fechner.net; ruf=mailto:postmaster@fechner.net; rf=afrf; pct=100; ri=86400"
To test the setup you can send an email to mentioned addresses here: http://dmarc.org/resources/deployment-tools/
Antispam Plugin for Dovecot
cd /usr/ports/mail/dovecot2-antispam-plugin
make install clean
/usr/local/etc/dovecot/local.conf
# antispam plugin
protocol imap {
mail_plugins = $mail_plugins antispam
}
SOLR integration in dovecot
This needs rework for new solr version
Make sure dovecot is compiled with solr support.
Make sure solr is running.
Create a new core for solr:
su -m solr -c "/usr/local/solr/bin/solr create_core -c dovecot"
Make sure we switch result from json to xml by editing:
/var/db/solr/dovecot/conf/solrconfig.xml
<!-- The following response writers are implicitly configured unless
overridden...
-->
<queryResponseWriter name="xml"
default="true"
class="solr.XMLResponseWriter" />
<!--
Copy the dovecot schema.xml configuration:
rm /var/db/solr/dovecot/conf/managed-schema
Create the schema file:
/var/db/solr/dovecot/conf/schema.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!--
For fts-solr:
This is the Solr schema file, place it into solr/conf/schema.xml. You may
want to modify the tokenizers and filters.
-->
<schema name="dovecot" version="1.5">
<!-- IMAP has 32bit unsigned ints but java ints are signed, so use longs -->
<fieldType name="string" class="solr.StrField" />
<fieldType name="long" class="solr.TrieLongField" />
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPossessiveFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.EnglishMinimalStemFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPossessiveFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.EnglishMinimalStemFilterFactory"/>
</analyzer>
</fieldType>
<!-- boolean type: "true" or "false" -->
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true"/>
<fieldType name="booleans" class="solr.BoolField" sortMissingLast="true" multiValued="true"/>
<!--
Numeric field types that index values using KD-trees.
Point fields don't support FieldCache, so they must have docValues="true" if needed for sorting, faceting, functions, etc.
-->
<fieldType name="pint" class="solr.IntPointField" docValues="true"/>
<fieldType name="pfloat" class="solr.FloatPointField" docValues="true"/>
<fieldType name="plong" class="solr.LongPointField" docValues="true"/>
<fieldType name="pdouble" class="solr.DoublePointField" docValues="true"/>
<fieldType name="pints" class="solr.IntPointField" docValues="true" multiValued="true"/>
<fieldType name="pfloats" class="solr.FloatPointField" docValues="true" multiValued="true"/>
<fieldType name="plongs" class="solr.LongPointField" docValues="true" multiValued="true"/>
<fieldType name="pdoubles" class="solr.DoublePointField" docValues="true" multiValued="true"/>
<!-- The format for this date field is of the form 1995-12-31T23:59:59Z, and
is a more restricted form of the canonical representation of dateTime
http://www.w3.org/TR/xmlschema-2/#dateTime
The trailing "Z" designates UTC time and is mandatory.
Optional fractional seconds are allowed: 1995-12-31T23:59:59.999Z
All other components are mandatory.
Expressions can also be used to denote calculations that should be
performed relative to "NOW" to determine the value, ie...
NOW/HOUR
... Round to the start of the current hour
NOW-1DAY
... Exactly 1 day prior to now
NOW/DAY+6MONTHS+3DAYS
... 6 months and 3 days in the future from the start of
the current day
-->
<!-- KD-tree versions of date fields -->
<fieldType name="pdate" class="solr.DatePointField" docValues="true"/>
<fieldType name="pdates" class="solr.DatePointField" docValues="true" multiValued="true"/>
<!--Binary data type. The data should be sent/retrieved in as Base64 encoded Strings -->
<fieldType name="binary" class="solr.BinaryField"/>
<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100" multiValued="true">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
<!-- in this example, we will only use synonyms at query time
<filter class="solr.SynonymGraphFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
<filter class="solr.FlattenGraphFilterFactory"/>
-->
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
<filter class="solr.SynonymGraphFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<field name="id" type="string" indexed="true" stored="true" required="true" />
<field name="uid" type="long" indexed="true" stored="true" required="true" />
<field name="box" type="string" indexed="true" stored="true" required="true" />
<field name="_text_" type="text_general" indexed="true" stored="false" multiValued="true"/>
<field name="user" type="string" indexed="true" stored="true" required="true" />
<field name="hdr" type="text" indexed="true" stored="false" />
<field name="body" type="text" indexed="true" stored="false" />
<field name="from" type="text" indexed="true" stored="false" />
<field name="to" type="text" indexed="true" stored="false" />
<field name="cc" type="text" indexed="true" stored="false" />
<field name="bcc" type="text" indexed="true" stored="false" />
<field name="subject" type="text" indexed="true" stored="false" />
<!-- Used by Solr internally: -->
<field name="_version_" type="long" indexed="true" stored="true"/>
<uniqueKey>id</uniqueKey>
</schema>
Now restart solr and check the logfile:
tail -F /var/log/solr/solr.log
You should not see an error message but something like this:
2017-10-12 15:57:07.584 INFO (searcherExecutor-7-thread-1-processing-x:dovecot) [ x:dovecot] o.a.s.c.SolrCore [dovecot] Registered new searcher Searcher@2ed6c198[dovecot] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
Now we can configure dovecot.
/usr/local/etc/dovecot/conf.d/10-mail.conf
mail_plugins = $mail_plugins fts fts_solr
/usr/local/etc/dovecot/conf.d/90-plugin.conf
plugin {
fts_autoindex = yes
fts = solr
fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
}
Restart dovecot:
Keep the tail on the solr log file running and execute:
doveadm index -u idefix inbox
Now after some seconds you should see that solr is indexing emails.
We would like to add some maintenance tasks for solr:
# Optimize solr dovecot storage
2 2 * * * root curl "http://127.0.0.1:8983/solr/dovecot/update?optimize=true"
5 */1 * * * root curl "http://127.0.0.1:8983/solr/dovecot/update?commit=true"
That’s it, have fun ;)