188.8.131.52. Filtering Based on the Message Contents The validation and restriction system would not be complete without a way to apply checks to the message contents. Postfix differentiates the checks applying to the email headers from those applying to the email body. Пример 11.11. Enabling content-based filters header_checks = regexp:/etc/postfix/header_checks body_checks = regexp:/etc/postfix/body_checks Both files contain a list of regular expressions (commonly known as regexps or regexes) and associated actions to be triggered when the email headers (or body) match the expression. QUICK LOOK Regexp tables The /usr/share/doc/postfix-doc/examples/header_checks.gz file contains many explanatory comments and can be used as a starting point for creating the /etc/postfix/header_checks and /etc/postfix/body_checks files. Пример 11.12. Example /etc/postfix/header_checks file
/^X-Mailer: GOTO Sarbacane/ REJECT I fight spam (GOTO Sarbacane) /^Subject: *Your email contains VIRUSES/ DISCARD virus notification BACK TO BASICS Regular expression The regular expression term (shortened to regexp or regex) references a generic notation for expressing a description of the contents and/or structure of a string of characters. Certain special characters allow defining alternatives (for instance, foo|bar matches either “foo” or “bar”), sets of allowed characters (for instance, [0-9] means any digit, and — a dot — means any character), quantifications ( s? matches either s or the empty string, in other words 0 or 1 occurrence of s ; s+ matches one or more consecutive s characters; and so on). Parentheses allow grouping search results. The precise syntax of these expressions varies across the tools using them, but the basic features are similar. → http://en.wikipedia.org/wiki/Regular_expression The first one checks the header mentioning the email software; if GOTO Sarbacane (a bulk email software) is found, the message is rejected. The second expression controls the message subject; if it mentions a virus notification, we can decide not to reject the message but to discard it immediately instead. Using these filters is a double-edged sword, because it is easy to make the rules too generic and to lose legitimate emails as a consequence. In these cases, not only the messages will be lost, but their senders will get unwanted (and annoying) error messages. 11.1.4. Setting Up greylisting “Greylisting” is a filtering technique according to which a message is initially rejected with a temporary error code, and only accepted on a further try after some delay. This filtering is particularly efficient against spam sent by the many machines infected by worms and viruses, since this software rarely acts as a full SMTP agent (by checking the error code and retrying failed messages later), especially since many of the harvested addresses are really invalid and retrying would only mean losing time. Postfix doesn't provide greylisting natively, but there is a feature by which the decision to accept or reject a given message can be delegated to an external program. The postgrey package contains just such a program, designed to interface with this access policy delegation service. Once postgrey is installed, it runs as a daemon and listens on port 10023. Postfix can then be configured to use it, by adding the check_policy_service parameter as an extra restriction: smtpd_recipient_restrictions = permit_mynetworks, [...] check_policy_service inet:127.0.0.1:10023 Each time Postfix reaches this rule in the ruleset, it will connect to the postgrey daemon and send it information concerning the relevant message. On its side, Postgrey considers the IP address/sender/recipient triplet and checks in its database whether that same triplet has been seen recently. If so, Postgrey replies that the message should be accepted; if not, the reply indicates that the message should be temporarily rejected, and the triplet gets recorded in the
database. The main disadvantage of greylisting is that legitimate messages get delayed, which is not always acceptable. It also increases the burden on servers that send many legitimate emails. IN PRACTICE Shortcomings of greylisting Theoretically, greylisting should only delay the first mail from a given sender to a given recipient, and the typical delay is in the order of minutes. Reality, however, can differ slightly. Some large ISPs use clusters of SMTP servers, and when a message is initially rejected, the server that retries the transmission may not be the same as the initial one. When that happens, the second server gets a temporary error message due to greylisting too, and so on; it may take several hours until transmission is attempted by a server that has already been involved, since SMTP servers usually increase the delay between retries at each failure. As a consequence, the incoming IP address may vary in time even for a single sender. But it goes further: even the sender address can change. For instance, many mailing-list servers encode extra information in the sender address so as to be able to handle error messages (known as bounces). Each new message sent to a mailing-list may then need to go through greylisting, which means it has to be stored (temporarily) on the sender's server. For very large mailing-lists (with tens of thousands of subscribers), this can soon become a problem. To mitigate these drawbacks, Postgrey manages a whitelist of such sites, and messages emanating from them are immediately accepted without going through greylisting. This list can easily be adapted to local needs, since it is stored in the /etc/postgrey/whitelist_clients file. GOING FURTHER Selective greylisting with milter-greylist The drawbacks of greylisting can be mitigated by only using greylisting on the subset of clients that are already considered as probable sources of spam (because they are listed in a DNS blacklist). This is not possible with postgrey but milter-greylist can be used in such a way. In that scenario, since DNS blacklists never triggers a definitive rejection, it becomes reasonable to use aggressive blacklists, including those listing all dynamic IP addresses from ISP clients (such as pbl.spamhaus.org or dul.dnsbl.sorbs.net ). Since milter-greylist uses Sendmail's milter interface, the postfix side of its configuration is limited to “ smtpd_milters = unix:/var/run/milter-greylist/milter-greylist.sock ”. The greylist.conf(5) manual page documents /etc/milter- greylist/greylist.conf and the numerous ways to configure milter-greylist. You will also have to edit /etc/default/milter-greylist to actually enable the service. 11.1.5. Customizing Filters Based On the Recipient Раздел 11.1.3, «Restrictions for Receiving and Sending» and Раздел 11.1.4, «Setting Up greylisting» reviewed many of the possible restrictions. They all have their use in limiting the amount of received spam, but they also all have their drawbacks. It is therefore more and more common to customize the set of filters depending on the recipient. At Falcot Corp, greylisting is interesting for most users, but it hinders the work of some users who need low latency in their emails (such as the technical support service). Similarly, the commercial service sometimes has problems receiving emails from some Asian providers who may be listed in blacklists; this service asked for a non-filtered address so as to be able to correspond. Postfix provides such a customization of filters with a “restriction class” concept. The classes are declared in the smtpd_restriction_classes parameter, and defined the same way as smtpd_recipient_restrictions . The check_recipient_access directive then defines a table mapping a given recipient to the appropriate set of restrictions.
Пример 11.13. Defining restriction classes in main.cf smtpd_restriction_classes = greylisting, aggressive, permissive greylisting = check_policy_service inet:127.0.0.1:10023 aggressive = reject_rbl_client sbl-xbl.spamhaus.org, check_policy_service inet:127.0.0.1:10023 permissive = permit smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, check_recipient_access hash:/etc/postfix/recipient_access Пример 11.14. The /etc/postfix/recipient_access file # Unfiltered addresses firstname.lastname@example.org permissive email@example.com permissive firstname.lastname@example.org permissive # Aggressive filtering for some privileged users email@example.com aggressive # Special rule for the mailing-list manager firstname.lastname@example.org reject_unverified_sender # Greylisting by default falcot.com greylisting 11.1.6. Integrating an Antivirus The many viruses circulating as attachments to emails make it important to set up an antivirus at the entry point of the company network, since despite an awareness campaign, some users will still open attachments from obviously shady messages. The Falcot administrators selected clamav for their free antivirus. The main package is clamav, but they also installed a few extra packages such as arj, unzoo, unrar and lha, since they are required for the antivirus to analyze attachments archived in one of these formats. The task of interfacing between antivirus and the email server goes to clamav-milter. A milter (short for mail filter) is a filtering program specially designed to interface with email servers. A milter uses a standard application programming interface (API) that provides much better performance than filters external to the email servers. Milters were initially introduced by Sendmail, but Postfix soon followed suit. QUICK LOOK A milter for Spamassassin The spamass-milter package provides a milter based on SpamAssassin, the famous unsolicited email detector. It can be used to flag messages as probable spams (by adding an extra header) and/or to reject the messages altogether if their “spamminess” score goes beyond a given threshold.
Once the clamav-milter package is installed, the milter should be reconfigured to run on a TCP port rather than on the default named socket. This can be achieved with dpkg-reconfigure clamav-milter. When prompted for the “Communication interface with Sendmail”, answer “ inet:email@example.com ”. NOTE Real TCP port vs named socket The reason why we use a real TCP port rather than the named socket is that the postfix daemons often run chrooted and do not have access to the directory hosting the named socket. You could also decide to keep using a named socket and pick a location within the chroot ( /var/spool/postfix/ ). The standard ClamAV configuration fits most situations, but some important parameters can still be customized with dpkg-reconfigure clamav-base. The last step involves telling Postfix to use the recently-configured filter. This is a simple matter of adding the following directive to /etc/postfix/main.cf : # Virus check with clamav-milter smtpd_milters = inet:[127.0.0.1]:10002 If the antivirus causes problems, this line can be commented out, and service postfix reload should be run so that this change is taken into account. IN PRACTICE Testing the antivirus Once the antivirus is set up, its correct behavior should be tested. The simplest way to do that is to send a test email with an attachment containing the eicar.com (or eicar.com.zip ) file, which can be downloaded online: → http://www.eicar.org/86-0-Intended-use.html This file is not a true virus, but a test file that all antivirus software on the market diagnose as a virus to allow checking installations. All messages handled by Postfix now go through the antivirus filter. 11.1.7. Authenticated SMTP Being able to send emails requires an SMTP server to be reachable; it also requires said SMTP server to send emails through it. For roaming users, this may need regularly changing the configuration of the SMTP client, since Falcot's SMTP server rejects messages coming from IP addresses apparently not belonging to the company. Two solutions exist: either the roaming user installs an SMTP server on their computer, or they still use the company server with some means of authenticating as an employee. The former solution is not recommended since the computer won't be permanently connected, and it won't be able to retry sending messages in case of problems; we will focus on the latter solution. SMTP authentication in Postfix relies on SASL (Simple Authentication and Security Layer). It requires installing the libsasl2-modules and sasl2-bin packages, then registering a password in the SASL database for each user that needs authenticating on the SMTP server. This is done with the saslpasswd2 command, which takes several parameters. The -u option defines the
authentication domain, which must match the smtpd_sasl_local_domain parameter in the Postfix configuration. The -c option allows creating a user, and -f allows specifying the file to use if the SASL database needs to be stored at a different location than the default ( /etc/sasldb2 ). # saslpasswd2 -u `postconf -h myhostname` -f /var/spool/postfix/etc/sasldb2 -c jean [... type jean's password twice ...] Note that the SASL database was created in Postfix's directory. In order to ensure consistency, we also turn /etc/sasldb2 into a symbolic link pointing at the database used by Postfix, with the ln -sf /var/spool/postfix/etc/sasldb2 /etc/sasldb2 command. Now we need to configure Postfix to use SASL. First the postfix user needs to be added to the sasl group, so that it can access the SASL account database. A few new parameters are also needed to enable SASL, and the smtpd_recipient_restrictions parameter needs to be configured to allow SASL-authenticated clients to send emails freely. Пример 11.15. Enabling SASL in /etc/postfix/main.cf # Enable SASL authentication smtpd_sasl_auth_enable = yes # Define the SASL authentication domain to use smtpd_sasl_local_domain = $myhostname [...] # Adding permit_sasl_authenticated before reject_unauth_destination # allows relaying mail sent by SASL-authenticated users smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, [...] EXTRA Authenticated SMTP client Most email clients are able to authenticate to an SMTP server before sending outgoing messages, and using that feature is a simple matter of configuring the appropriate parameters. If the client in use does not provide that feature, the workaround is to use a local Postfix server and configure it to relay email via the remote SMTP server. In this case, the local Postfix itself will be the client that authenticates with SASL. Here are the required parameters: smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd relay_host = [mail.falcot.com] The /etc/postfix/sasl_passwd file needs to contain the username and password to use for authenticating on the mail.falcot.com server. Here is an example: [mail.falcot.com] joe:LyinIsji As for all Postfix maps, this file must be turned into /etc/postfix/sasl_passwd.db with the postmap command.
11.2. Web Server (HTTP) The Falcot Corp administrators decided to use the Apache HTTP server, included in Debian Jessie at version 2.4.10. ALTERNATIVE Other web servers Apache is merely the most widely-known (and widely-used) web server, but there are others; they can offer better performance under certain workloads, but this has its counterpart in the smaller number of available features and modules. However, when the prospective web server is built to serve static files or to act as a proxy, the alternatives, such as nginx and lighttpd, are worth investigating. 11.2.1. Installing Apache Installing the apache2 package is all that is needed. It contains all the modules, including the Multi-Processing Modules (MPMs) that affect how Apache handles parallel processing of many requests (those used to be provided in separate apache2-mpm-* packages). It will also pull apache2-utils containing the command line utilities that we will discover later. The MPM in use affects significantly the way Apache will handle concurrent requests. With the worker MPM, it uses threads (lightweight processes), whereas with the prefork MPM it uses a pool of processes created in advance. With the event MPM it also uses threads, but the inactive connections (notably those kept open by the HTTP keep-alive feature) are handed back to a dedicated management thread. The Falcot administrators also install libapache2-mod-php5 so as to include the PHP support in Apache. This causes the default event MPM to be disabled, and prefork to be used instead, since PHP only works under that particular MPM. SECURITY Execution under the www-data user By default, Apache handles incoming requests under the identity of the www-data user. This means that a security vulnerability in a CGI script executed by Apache (for a dynamic page) won't compromise the whole system, but only the files owned by this particular user. Using the suexec modules allows bypassing this rule so that some CGI scripts are executed under the identity of another user. This is configured with a SuexecUserGroup usergroup directive in the Apache configuration. Another possibility is to use a dedicated MPM, such as the one provided by libapache2-mpm-itk. This particular one has a slightly different behavior: it allows “isolating” virtual hosts (actually, sets of pages) so that they each run as a different user. A vulnerability in one website therefore cannot compromise files belonging to the owner of another website. QUICK LOOK List of modules The full list of Apache standard modules can be found online. → http://httpd.apache.org/docs/2.4/mod/index.html Apache is a modular server, and many features are implemented by external modules that the
main program loads during its initialization. The default configuration only enables the most common modules, but enabling new modules is a simple matter of running a2enmod module ; to disable a module, the command is a2dismod module . These programs actually only create (or delete) symbolic links in /etc/apache2/mods-enabled/ , pointing at the actual files (stored in /etc/apache2/mods-available/ ). With its default configuration, the web server listens on port 80 (as configured in /etc/apache2/ports.conf ), and serves pages from the /var/www/html/ directory (as configured in /etc/apache2/sites-enabled/000-default.conf ). GOING FURTHER Adding support for SSL Apache 2.4 includes the SSL module required for secure HTTP (HTTPS) out of the box. It just needs to be enabled with a2enmod ssl, then the required directives have to be added to the configuration files. A configuration example is provided in /etc/apache2/sites-available/default-ssl.conf → http://httpd.apache.org/docs/2.4/mod/mod_ssl.html Some extra care must be taken if you want to favor SSL connections with Perfect Forward Secrecy (those connections use ephemeral session keys ensuring that a compromission of the server's secret key does not result in the compromission of old encrypted traffic that could have been stored while sniffing on the network). Have a look at Mozilla's recommandations in particular: → https://wiki.mozilla.org/Security/Server_Side_TLS#Apache 11.2.2. Configuring Virtual Hosts A virtual host is an extra identity for the web server. Apache considers two different kinds of virtual hosts: those that are based on the IP address (or the port), and those that rely on the domain name of the web server. The first method requires allocating a different IP address (or port) for each site, whereas the second one can work on a single IP address (and port), and the sites are differentiated by the hostname sent by the HTTP client (which only works in version 1.1 of the HTTP protocol — fortunately that version is old enough that all clients use it already). The (increasing) scarcity of IPv4 addresses usually favors the second method; however, it is made more complex if the virtual hosts need to provide HTTPS too, since the SSL protocol hasn't always provided for name-based virtual hosting; the SNI extension (Server Name Indication) that allows such a combination is not handled by all browsers. When several HTTPS sites need to run on the same server, they will usually be differentiated either by running on a different port or on a different IP address (IPv6 can help there). The default configuration for Apache 2 enables name-based virtual hosts. In addition, a default virtual host is defined in the /etc/apache2/sites-enabled/000-default.conf file; this virtual host will be used if no host matching the request sent by the client is found. CAUTIONFirst virtual host Requests concerning unknown virtual hosts will always be served by the first defined virtual host, which is why we defined www.falcot.com first here.
QUICK LOOK Apache supports SNI The Apache server supports an SSL protocol extension called Server Name Indication (SNI). This extension allows the browser to send the hostname of the web server during the establishment of the SSL connection, much earlier than the HTTP request itself, which was previously used to identify the requested virtual host among those hosted on the same server (with the same IP address and port). This allows Apache to select the most appropriate SSL certificate for the transaction to proceed. Before SNI, Apache would always use the certificate defined in the default virtual host. Clients trying to access another virtual host would then display warnings, since the certificate they received didn't match the website they were trying to access. Fortunately, most browsers now work with SNI; this includes Microsoft Internet Explorer starting with version 7.0 (starting on Vista), Mozilla Firefox starting with version 2.0, Apple Safari since version 3.2.1, and all versions of Google Chrome. The Apache package provided in Debian is built with support for SNI; no particular configuration is therefore needed. Care should also be taken to ensure that the configuration for the first virtual host (the one used by default) does enable TLSv1, since Apache uses the parameters of this first virtual host to establish secure connections, and they had better allow them! Each extra virtual host is then described by a file stored in /etc/apache2/sites-available/ Setting up a website for the falcot.org domain is therefore a simple matter of creating the following file, then enabling the virtual host with a2ensite www.falcot.org. Пример 11.16. The /etc/apache2/sites-available/www.falcot.org.conf file ServerName www.falcot.org ServerAlias falcot.org DocumentRoot /srv/www/www.falcot.org
The Apache server, as configured so far, uses the same log files for all virtual hosts (although this could be changed by adding CustomLog directives in the definitions of the virtual hosts). It therefore makes good sense to customize the format of this log file to have it include the name of the virtual host. This can be done by creating a /etc/apache2/conf- available/customlog.conf file that defines a new format for all log files (with the LogFormat directive) and by enabling it with