Issues when Upgrading to Shorewall 4.4 (Upgrading from Debian Lenny to Squeeze)

Tom Eastep

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License.

2023/02/18


Table of Contents

Introduction
Packaging Differences
Issues Most Likely to Cause Problems or Concerns
shorewall.conf
/etc/shorewall/zones
/etc/shorewall/ipsec
/etc/shorewall/interfaces
/etc/shorewall/hosts
/etc/shorewall/policy
/etc/shorewall/masq
/etc/shorewall/rules
/etc/shorewall/routestopped
/etc/shorewall/tos
Extension Scripts
Ipsets
Simple Traffic Shaping
Additional Sources of Information

Introduction

Debian Lenny includes Shorewall version 4.0.15 while Squeeze includes Shorewall 4.4. Because there are significant differences between the two product versions, some users may experience upgrade issues. This article outlines those issues and offers advice for dealing with them.

Note

Although this article is targeted specifically at Lenny -> Squeeze upgrades, it should be useful to any Shorewall-shell user upgrading to Shorewall 4.4.x. Footnotes are used to flag areas where non-Debian users may experience different results.

Packaging Differences

The first key difference between Shorewall 4.0 and Shorewall 4.4 is in the packaging[1]. In Lenny, there are six Shorewall packages:

  1. shorewall-common — Contains the basic components needed to create an IPv4 firewall.

  2. shorewall-shell — The legacy Shorewall configuration compiler written in Bourne shell.

  3. shorewall — A transitional package that depends on shorewall-common and shorewall-shell. Installing this package installs both shorewall-common and shorewall-shell.

  4. shorewall-perl — A re-implementation of the Shorewall configuration compiler in Perl. This compiler has many advantages over the shell-based compiler:

    • The compiler is much faster

    • The compiler does a much better job of validating the configuration, thus avoiding run-time errors.

    • The compiler produces better and more consistent diagnostic messages.

    • The compiler produces a script that runs much faster and that does not reject/drop connections during start/restart.

  5. shorewall-lite — A small package that can run scripts generated by shorewall-shell or shorewall-perl. Allows centralized firewall administration.

  6. shorewall-doc — Documentation.

In Squeeze, there are six slightly different packages:

  1. shorewall — Contains everything needed to create an IPv4 firewall. It combines the former shorewall-common and shorewall-perl packages.

  2. shorewall6 — Depends on shorewall. Adds those components needed to create an IPv6 firewall.

  3. shorewall-lite — Same as in Lenny; only runs IPv4 firewall scripts.

  4. shorewall6-lite — Similar to shorewall-lite, except that it only runs IPv6 firewall scripts.

  5. shorewall-init — Allows the firewall to be closed before interfaces are brought up and also allows the firewall to react to interfaces coming up and going down.

  6. shorewall-doc — Documentation.

Warning

Do not purge the old packages (shorewall-common, shorewall-shell and shorewall-perl) until after the new shorewall package has been installed.

The key change in Squeeze that may produce upgrade issues is that Squeeze does not include the shell-based configuration compiler. As a consequence, unless you are already using Shorewall-perl on Lenny, an upgrade from Lenny to Squeeze will mean that you will be switching from the old shell-based compiler to the new Perl-based compiler[2]. While the two compilers are highly compatible, there are some differences. Those differences are detailed in the following sections.

Issues Most Likely to Cause Problems or Concerns

shorewall.conf

As always, when upgrading from one major release of Shorewall to another, the installer will prompt you about replacing your existing shorewall.conf with the updated one from the package. Shorewall is designed with the assumption that users will never replace shorewall.conf and retaining your existing file will always produce upward-compatible behavior.

That having been said, there are a few settings that you may have in your shorewall.conf that will cause compilation warning or error messages after the upgrade.

BLACKLISTNEWONLY

If you have BLACKLISTNEWONLY=No together with FASTACCEPT=Yes, you will receive this error:

ERROR: BLACKLISTNEWONLY=No may not be specified with FASTACCEPT=Yes

To eliminate the error, reverse the setting of one of the options.

Note

This combination never worked correctly in earlier versions -- to duplicate the earlier behavior, you will want to set BLACKLISTNEWONLY=Yes.

BRIDGING

If you have set this option to Yes, you will receive the following error:

ERROR: BRIDGING=Yes is not supported by Shorewall 4.4.x

You should not be receiving this error if you are upgrading from Lenny since BRIDGING=Yes did not work in that release either[3]. If you have a bridge configuration where you want to control connections through the bridge, you will want to visit https://shorewall.org/bridge-Shorewall-perl.html[4].

DELAYBLACKLISTLOAD

If you have set this option to Yes, you will receive the following warning:

WARNING: DELAYBLACKLIST=Yes is not supported by Shorewall 4.4.x

To eliminate the warning, set DELAYBLACKLISTLOAD=No or remove the setting altogether.

DYNAMIC_ZONES

If you have set this option to Yes, you will receive the following warning:

WARNING: DYNAMIC_ZONES=Yes is not supported by Shorewall 4.4.x

To eliminate the warning, set DYNAMIC_ZONES=No or remove the setting altogether. See this article to learn how to set up Dynamic Zones under Shorewall 4.4.

FW

If a setting for FW appears in your shorewall.conf file, you will receive this warning:

WARNING: Unknown configuration option (FW) ignored.

Remove the setting from the file and modify your /etc/shorewall/zones file as described below.

IPSECFILE

If you have specified IPSECFILE=ipsec or IPSECFILE= or if you do not have a setting for IPSECFILE, then you will receive the following error:

ERROR: IPSECFILE=ipsec is not supported by Shorewall 4.4.x

To eliminate the warning, you will need to:

  1. Set IPSECFILE=zones

  2. Modify your /etc/shorewall/zones file as described below.

PKTTYPE

The PKTTYPE option is ignored by Shorewall-perl. Shorewall-perl will use Address type match if it is available; otherwise, it will behave as if PKTTYPE=No had been specified.

RFC1918_LOG_LEVEL

If you have specified any setting for this option, you will receive the following warning:

WARNING: RFC1918_LOG_LEVEL=value ignored. The 'norfc1918' interface/host option is no longer supported.

To eliminate the warning, set RFC1918_LOG_LEVEL= or simply remove the setting altogether.

RFC1918_STRICT

If you have set this option to Yes, you will receive the following warning:

WARNING: RFC1918_STRICT=Yes is not supported by Shorewall 4.4.x

To eliminate the warning, set RFC1918_STRICT=No or remove the setting altogether.

SAVE_IPSETS

Shorewall 4.4.0-4.4.5 will issue a warning if you set SAVE_IPSETS=Yes in shorewall.conf:

WARNING SAVE_IPSETS=Yes is not supported by Shorewall 4.4.x

To eliminate this message, you will need to set SAVE_IPSETS=No or remove the setting altogether.

See below for additional information regarding ipsets in Shorewall 4.4.

SHOREWALL_COMPILER

If you have specified SHOREWALL_COMPILER=shell, you will receive the following warning message:

WARNING: SHOREWALL_COMPILER=shell ignored. Shorewall-shell support has been removed in this release

To eliminate the warning, set SHOREWALL_COMPILER=perl or simply remove the setting altogether.

USE_ACTIONS

If you have set this option to No, you will receive the following warning:

WARNING: USE_ACTIONS=No is not supported by Shorewall 4.4.x

To eliminate the warning, set USE_ACTIONS=Yes or remove the setting altogether.

/etc/shorewall/zones

If the column headings in your /etc/shorewall/zones file look like this:

#ZONE   DISPLAY         COMMENTS
net     Net             The big bad net
loc     Local           The local LAN

then you are using the original zones file format that has been deprecated since Shorewall 3.0.

You will need to convert to the new file format which has the following headings:

#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS

You will need to add an entry for your firewall zone. The default name for the firewall zone is 'fw' but may have been overriden in your old configuration using the FW option in shorewall.conf.

#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall

The remainder of your zones will have type 'ipv4' unless they are mentioned in your /etc/shorewall/ipsec file (see below).

#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall
net     ipv4            # The big bad net
loc     ipv4            # The local LAN

/etc/shorewall/ipsec

This file is no longer used -- its specifications are now included in /etc/shorewall/zones.

Take this example:

#ZONE   IPSEC    OPTIONS         IN              OUT
#       ONLY                     OPTIONS         OPTIONS
ipsec1  Yes
ipsec2  No

This would translate to the following entries in /etc/shorewall/zones:

#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS
ipsec1  ipsec4
ipsec2  ipv4

Any OPTIONS, IN OPTIONS and OUT OPTIONS should simply be copied from /etc/shorewall/ipsec to /etc/shorewall/zones.

/etc/shorewall/interfaces

The BROADCAST column is essentially unused in Squeeze. If it contains anything except 'detect' or '-', then you will receive this warning[5]:

WARNING: Shorewall no longer uses broadcast addresses in rule generation when Address Type Match is available

To eliminate the warning, replace the contents of the BROADCAST column with '-' or 'detect'.

The 'norfc1918' option has been removed. If you specify the option, you will receive the following warning:

WARNING: Support for the norfc1918 interface option has been removed from Shorewall

To eliminate the warning, simply remove the 'norfc1918' option from the OPTIONS list. You may wish to consider NULL_ROUTE_RFC1918=Yes as a replacement (see shorewall.conf (5)).

/etc/shorewall/hosts

The 'norfc1918' option has been removed. If you specify the option, you will receive the following warning:

WARNING: The 'norfc1918' option is no longer supported

To eliminate the warning, simply remove the 'norfc1918' option from the OPTIONS list. You may wish to consider NULL_ROUTE_RFC1918=Yes as a replacement (see shorewall.conf (5)).

/etc/shorewall/policy

Shorewall 4.4 detects dead policy file entries that result when an entry is masked by an earlier more general entry.

Example:

#SOURCE DEST     POLICY    LOG LEVEL
all     all      REJECT    info
loc     net      ACCEPT

Shorewall-shell silently accepted the above even though the loc->net policy is useless. Shorewall-perl generates a fatal compilation error:

ERROR: Policy "loc net ACCEPT" duplicates earlier policy "all all REJECT"

/etc/shorewall/masq

There is a long tradition of specifying an interface name in the SOURCE column of this file.

Masquerading/SNAT occurs in the Netfilter POSTROUTING chain where an incoming interface may not be specified in iptables rules. Consequently, while processing the shorewall start and shorewall restart commands, the generated script must examine the firewall's main routing table to determine those networks that are routed out of the interface; the script then adds a MASQUERADE/SNAT rule for connections from each of those networks. This additional processing requires the named interface to be up and configured when Shorewall starts or restarts.

Users often complain that Shorewall fails to start at boot time because a VPN interface that is named as a masq SOURCE isn't up and configured during boot.

To emphasize this restriction, if an interface is named in the SOURCE column of one or more entries, a single warning is issued as follows:

WARNING: Using an interface as the masq SOURCE requires the interface to be up and configured when Shorewall starts/restarts

To suppress this warning, replace the interface name with the list of networks that are routed out of the interface.

Example.

Existing entry:

#INTERFACE              SOURCE          ADDRESS         PROTO   PORT(S) IPSEC   MARK    USER/
#                                                                                       GROUP
eth0                    eth1

Current routing configuration:

gateway:~# ip route ls dev eth1
172.20.1.0/24  proto kernel  scope link  src 172.20.1.254 
224.0.0.0/4  scope link 
gateway:~# 

Replacement entry:

#INTERFACE              SOURCE          ADDRESS         PROTO   PORT(S) IPSEC   MARK    USER/
#                                                                                       GROUP
eth0                    172.20.1.0/24

Note that no entry is included for 224.0.0.0/4 since that is the multicast IP range and there should never be any packets with a SOURCE IP address in that network.

/etc/shorewall/rules

If you include a destination zone in a 'nonat' rule, Shorewall issues the following warning:

WARNING: Destination zone (zonename) ignored.

Nonat rules include:

DNAT-
REDIRECT-
NONAT

To eliminate the warning, remove the DEST zone.

Example.

Before:

#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME
#                                                       PORT(S) PORT(S)         DEST            LIMIT           GROUP
NONAT           loc             net             tcp     80

After:

#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME
#                                                       PORT(S) PORT(S)         DEST            LIMIT           GROUP
NONAT           loc             -               tcp     80

Shorewall 4.4 versions prior to 4.4.19 do not support icmp type lists in the DEST PORT(S) column. Only a single ICMP type may be listed. If you have a shell variable with a list of ICMP types that you use in a rule, you can work around this limitation as follows. Replace this rule:

#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME
#                                                       PORT(S) PORT(S)         DEST            LIMIT           GROUP
ACCEPT          z1              z2              icmp    $ITYPES

with:

#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME
#                                                       PORT(S) PORT(S)         DEST            LIMIT           GROUP

BEGIN SHELL
for type in $ITYPES; do
ACCEPT          z1              z2              icmp    $type
done
END SHELL

/etc/shorewall/routestopped

The 'critical' option is no longer needed and hence is no longer supported. If you have critical hosts defined, you will receive this warning:

WARNING: The 'critical' option is no longer supported (or needed)

To suppress the warning, simply remove the option.

Shorewall 4.4 also treats the routestopped file differently from earlier releases. Previously, the routestopped file was parsed during shorewall stop processing so that changes made to the file while Shorewall was running would be applied at the next stop. This is no longer the case -- the routestopped file is processed during compilation just like the rest of the configuration files so that when shorewall stop is issued, the firewall will pass traffic based on the contents of the routestopped file at the last start or restart.

If you change the routestopped file and now want to stop the firewall, you can run this sequence of commands:

shorewall compile
shorewall stop

/etc/shorewall/tos

The /etc/shorewall/tos file now has zone-independent SOURCE and DEST columns as do all other files except the rules and policy files.

The SOURCE column may be one of the following:

[all:]<address>[,...]
[all:]<interface>[:<address>[,...]]
$FW[:<address>[,...]]

The DEST column may be one of the following:

[all:]<address>[,...]
[all:]<interface>[:<address>[,...]]

This is a permanent change. The old zone-based rules have never worked right and this is a good time to replace them. We have tried to make the new syntax cover the most common cases without requiring change to existing files. In particular, it will handle the tos file released with Shorewall 1.4 and earlier.

Extension Scripts

With the shell-based compiler, all extension scripts were copied into the compiled script and executed at run-time. In some cases, this approach doesn't work with Shorewall Perl because (almost) the entire rule set is built by the compiler. As a result, Shorewall-perl runs some extension scripts at compile-time rather than at run-time. Because the compiler is written in Perl, these extension scripts from earlier versions will no longer work.

The following table summarizes when the various extension scripts are run:

Compile-time (Must be written in Perl)Run-timeEliminated
initdoneclearcontinue
macloginit 
Per-chain (including those associated with actions)start 
 started 
 stop 
 stopped 
 tcclear 

Compile-time extension scripts are executed using the Perl 'eval `cat <file>`' mechanism. Be sure that each script returns a 'true' value; otherwise, the Shorewall-perl compiler will assume that the script failed and will abort the compilation.

When a script is invoked, the $chainref scalar variable will usually hold a reference to a chain table entry.

$chainref->{name} contains the name of the chain
$chainref->{table} holds the table name

To add a rule to the chain:

add_rule $chainref, the-rule

Where

the rule is a scalar argument holding the rule text. Do not include "-A chain-name"

Example:

add_rule $chainref, '-j ACCEPT';

To insert a rule into the chain:

insert_rule $chainref, rulenum, the-rule

The log_rule_limit function works like it does in the shell compiler with three exceptions:

  • You pass the chain reference rather than the name of the chain.

  • The commands are 'add' and 'insert' rather than '-A' and '-I'.

  • There is only a single "pass as-is to iptables" argument (so you must quote that part

Example:

    log_rule_limit
              'info' , 
              $chainref , 
              $chainref->{name},
              'DROP' , 
              '',    #Limit
              '' ,   #Log tag
              'add'
              '-p tcp ';         

Here is an example of an actual initdone script used with Shorewall 3.4:

run_iptables -t mangle -I PREROUTING -p esp -j MARK --set-mark 0x50
run_iptables -t filter -I INPUT -p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT
run_iptables -t filter -I OUTPUT -p udp --sport 1701 -j ACCEPT

Here is the corresponding script used with Shorewall 4.4:

use Shorewall::Chains;

insert_rule $mangle_table->{PREROUTING}, 1, "-p esp -j MARK --set-mark 0x50";
insert_rule $filter_table->{INPUT},      1, "-p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT";
insert_rule $filter_table->{OUTPUT},     1, "-p udp --sport 1701 -j ACCEPT";

1;

The initdone script is unique because the $chainref variable is not set before the script is called. The above script illustrates how the $mangle_table, $filter_table, and $nat_table references can be used to add or insert rules in arbitrary chains.

Ipsets

Shorewall 4.4 insists that ipset names begin with a letter and be composed of alphanumeric characters, underscores (_) and dashes (-). When used in a Shorewall configuration file, the name must be preceded by a plus sign (+) as with the shell-based compiler.

Shorewall 4.4.6 re-introduced SAVE_IPSETS=Yes with slightly different semantics:

  • The contents of the ipsets are saved during processing of the stop command in addition to during processing of the save command.

  • The contents of the ipsets are restored during processing of the start command in addition to during processing of the restore command. When restore is being run when Shorewall is not in the stopped state (such as when it is run to recover from a failed start, restart or refresh) ipsets are not restored.

  • Specifying an ipset in shorewall-routestopped (5) is prohibited when SAVE_IPSETS=Yes.

Simple Traffic Shaping

If you find that output bandwidth is extremely limited, it is likely due to TCP Segmentation Offload (TSO) and/or Generic Segmentation Offload (GSO) being enabled in the network adapter. To verify that, install the ethtool package and use the -k command:

root@gateway:~# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off
root@gateway:~#

If that is the case, you can correct the problem by adjusting the minburst setting in /etc/shorewall/tcinterfaces. We suggest starting at 10-12kb and adjust as necessary. Example:

#INTERFACE	TYPE		IN-BANDWIDTH			OUT-BANDWIDTH
eth0		External	50mbit:200kb			5.0mbit:100kb:200ms:100mbit:10kb

Alternatively, you can turn off TSO and GSO using this command in /etc/shorewall/init:

ethtool -k ethN tso off gso off

Additional Sources of Information

The following articles provide additional information.



[1] Most distributions use a similar packaging structure. Note, however, that the 'shorewall' package in Simon Mater's RPMs for RedHat/Fedora/CentOS is like the Lenny shorewall-common package.

[2] Note that Perl is a required package on Debian. If you are running an embedded distribution which does not include Perl and it is not feasible to install Perl on your firewall, then you should consider installing Shorewall on another system in your network (may be a Windows™ system running Cygwin™ or an AppleMacIntosh™ running OS X) and installing Shorewall-lite on your firewall.

[3] If you are upgrading from a release using a kernel earlier than 2.6.20, then BRIDGING=Yes did work correctly with Shorewall-shell.

[4] Kernel 2.6.20 or later is required.

[5] Users whose kernel and/or iptables do not include Address Type Match Support can continue to list broadcast addresses in this column; no warning will be issued.