mjhjmtu

I have an excellent book called "Linux Firewalls: Attack Detection and Response" by Michael Rash. I have a few questions before I begin.

I want to make an enterprise grade iptables Firewall and was wondering if I will need to do my own kernel compilation like it says in the book or nowadays is it ok to just download the Debian/linux OS server and plainly install Iptables onto it and start configuring?

I was wondering since nftables is a newer improved version of iptables does it go about the same way of installation? (did not find research material on nftables)

As for firewalls, I would be worried where they are placed, your Internet speeds, and how much rules you need on them. They can pretty much dictate the kind of hardware you will need. Be aware for more performance/higher speeds, you may need better NIC cards. In the past, I used top tier Intel Pro cards.

About router/firewalls in ISP settings, I used to have at the ISP I was running, a Linux router with IPtables for firewalling/accounting. In time, I replaced it with a Cisco ISP grade router, created access lists to block the few ports I needed to cut (mostly Windows default ports, SQLSERVER and not much more) and started sending netflow to a Linux server to do the customers data accounting when our capacity started growing.

Beware that if you are a cable plant outfit, layer 2/3 firewall rules can de added to the DOCSIS modem configurations. You can save a significant upstream bandwidth that way.

As for an open source firewall, I do recommend pfSense. I used it in the past to protect the corporate network of the ISP, and nowadays using them to provide native client VPNs to OS/X, Linux and Windows 7-10. They also support full fail-over, where if the master fails, the slave maintains the state of the connections over time, and pick ups everything. pfSense runs on top of FreeBSD, and has a graphical management interface that is very flexible.

https://www.pfsense.org

Concerning iptables/VPN in Linux, I am using a Debian also as a firewall and VPN (with strongswan) to secure a special network, and it is not necessary to mess with kernel compilations.

As for layer-7 traffic shapping, we tried to do it for a while with Linux, but it was not very efficient, and it was a time-consuming process. We ended up going for a NetEnforcer traffic shapper.

nftables are currently under development to replace iptables, and while they don't say as much, I would consider it "beta" for now. I don't have any insight into their timeline, but you can read more here:
http://netfilter.org/projects/nftables/

Many linux distributions already have iptables enabled by default. Either it's compiled in, or they load the module on boot (most common). The easiest way to tell would be to run:

lsmod | grep ip_tables

You should see a line that says ip_tables if the module is loaded. You can also try:

iptables -L

to see if you get anything back. Most boxes by default will have empty "chains" which basically means allow everything (default to allow is configured by default).

I'd use shorewall in preference to writing iptables rules directly. There are also alternatives such as firewalld.

With regard to kernel compilation it really depends whether or not the features that you need are available either in the stock kernel or as a modular add-in. If they are not, then you're going to need to roll your own. However, that's not really very enterprisey as it means more work each time there's a kernel package upgrade.

In the comments you asked about ISP grade packet filtering. I think that you would be best using the ipset extension to iptables for this sort of work. In terms of protection, it allows you to build sets of thousands (if not tens of thousands) of similar rules that can run without significantly slowing the traffic flowing through your rulesets.

I assume you'd also be looking at full scale IDS/IPS.