Table of Contents
Introduction and requirements
NiceShaper works to give you optimal bandwidth utilization and simplify of configure traffic shaping in Linux Operating System which can be really complex. I always seek when creating solutions to makes them flexible, intuitive and clear. To makes them be tool for simplify what is to much complex and at once gives ability to configure everything what is worth to have control on.
It's possible to create an entry configuration without reading this documentation. It's thanks to example configuration files included in program package. Solid knowledge of networks is prerequisite. Keep in mind, when you get first look at included example configuration files next it is worth to look on this documentation unless you don't want to create optimal traffic shaping. NiceShaper really seeks but can't outguess all your needs so knowledge of IP network and Linux System is helpful. Still don't be fear when you do not understand everything here, keep in mind that this documentation is far beyond your needs for the first time. After all NiceShaper goal is simplify what in HTB is hard and at once to gives you additional features.
In minimal configuration you need router with your favorite Linux System, with installed iptables and compiled in kernel HTB and SFQ packet scheduling algorithms and U32 kernel filter.
Hereinafter HTB class means HTB packet scheduling class and NiceShaper class is in NiceShaper level. Queuing algorithms means SFQ and others classless queueing disciplines. Kernel filters means U32 and FW and it is not the same as NiceShaper filters which are on NiceShaper level of configuration.
In most popular Linux Distributions HTB, SFQ, U32 and FW filter are compiled in kernel or as modules by default. Iptables program too but apart from IMQ.
For purpose of this documentation we assume that we have a router equipped with two interfaces. WAN uplink is connected to interface eth0 and local LAN to eth1. Public address of the router is 198.51.100.100 and LAN network is 192.168.0.0/24.
As we discount poor ingress qdisc, it's fundamental that egress shaping occurs on outgoing interface, so, shaping of download traffic has to be placed on eth1 interface. On the other hand, shaping of upload traffic has to be placed on eth0. This is because the downloaded packets incomes via eth0 interface and outgoes to the local network via eth1 interface (for above assumptions!). Respectively, the uploaded packets incomes from local network via eth1 and outgoes to the internet via eth0 interface. It's all valid for above assumptions and may be required to be adjusted in configuration prepared for the other environment!
Proper choose of interface is crucial to define hosts, classes, and some of configuration directives. For example, if LAN network is hidden behind NAT and we need to shape upload by packet marking method then we need to put it on eth0 by mark-on-iface eth0 directive.
I would like to add that upload shaping of NATed network is also possible with IMQ interfaces. Both, packets marking and IMQ interfaces are documented in hereafter.
Remember as these examples above are valid for previous assumes not necessarily for your router.
For the end of the introduction. Iptables and Iproute do not understand concept of interface aliases, so in NiceShaper configurations you should omit colon and alias number from interface. It's safe. On the other side, VLANs written as ethX.vid are fully supported.
Complementing about the VLANs in Linux. If you create the tagged sub-interface and run traffic shaping on it, you should completely abandon the untagged interface. Otherwise, you may find that the kernel queues tagged data frames twice. Once on tagged sub-interface and second time on physical one.
NiceShaper has good scalability on the routers used by no more than half thousand of hosts - although there are notices about the NiceShaper usage in bigger networks. In such bigger networks there is rather a hash table requirement. Hash tables are unsupported so far, because of being too inflexible for NiceShaper needs.
Please be aware that NiceShaper has been written for polish Network Administrators since start of the project, so there is a forum on the web page but it is useless for you because of polish language. Also you should realise that this documentation can be strongly language incorrect and even incomplete. If you have corrections and share with me i will be appreciate.
Installation
Compilation and installation dependencies are: c++ compiler from gcc package, C and C++ standard libraries, and make utility. Unpacked package have to be compiled with omitted ./configure, because NiceShaper is Linux only software.
$ bunzip2 niceshaper-%{wersja}.tar.bz2
$ tar xf niceshaper-%{wersja}.tar
$ cd niceshaper-%{wersja}
$ make
$ su
# make install
Make install command copies the compiled binary to the /usr/local/bin directory. The binary path may be changed using BINDIR environment variable. Make install command creates important directories: /etc/niceshaper, /var/lib/niceshaper, and /usr/share/doc/niceshaper. Example configuration files are copied to the /etc/niceshaper directory. If configuration files are found there then the actual files are copied with "-dist" postfix. The documentation and the rest of package content is copied to /usr/share/doc/niceshaper.
Configuration
Configuration files syntax.
Common and required configuration files are /etc/niceshaper/config.conf and /etc/niceshaper/class.conf.
include file path - used in these files can include next ones. This directive accepts absolute and relative paths (by default to /etc/niceshaper/).
There are a few different syntax types for directives:
Syntax type #1) directive value - It is directive with one value only, e.g.:
rate 128kB/s
Syntax type #2) directive value [value] - It is directive with one or number of values separated with whitespaces, e.g.:
mark-on-ifaces eth0 eth1
Syntax type #3) directive parameter value [parameter value] - It is directive with one or number of parameter-value pairs, e.g.:
status file /var/www/niceshaper/status.txt unit kB/s file-mode 644
Order of above directives in a scope does not matter.
When NiceShaper is starting, configuration parser splits type #2 and #3 directives into separated lines, so if you prefer you can write these directives also like that:
- mark-on-ifaces eth0
- mark-on-ifaces eth1
- status file /var/www/niceshaper/status.txt
- status unit kB/s
- status file-mode 644
Syntax type #4) match directive looks and works like type 3, but it's additional type. This is because you can not split it as it may change logic. Only order changing is permitted.
This is example of filter which classify packets with source address 10.10.10.5 and destination in network 192.168.0.0/29.
match srcip 10.10.10.5 dstip 192.168.0.0/29
After split, gives 2 autonomous filters:
- match srcip 10.10.10.5
- match dstip 192.168.0.0/29
These two filter are properly in the mining of NiceShaper configuration syntax, but they are not identical with source filter.
Syntax type #5) Directives class is the most inflexible one, it can not be split and order changes at all, e.g.:
class dl eth1 pc55
In addition to differences in syntax, directives are also divided according to the scope of operation, for directives of global section, directives of functional sections and directives of classes. When it comes to the directives of classes, all except the class header and filters can be placed in the functional sections configurations, providing default values for all classes included in the section. In the rest of the documentation, the three groups are clearly separated in groups.
The basic units of capacity is b/s (bits per second), and B/s (Bytes per second). The prefixes are k (kilo) and M (mega). The suffix '/s' is not compulsory and the distinction between rate and the amount of transferred data is based on the context of use. Default unit of bandwidth is b/s (bits per second).
NiceShaper gives you some special chars: Char hash "#" is using for comment out the rest of line. Comment block is "<# comment #>" and is using to affect some piece of configuration line. Commented configuration is useless. Char ";" means end of line, it gives you way to minimize length of config files by write a lot of semicolon separated directives in one line.
Each change in configuration require to restart NiceShaper.
Remember, all examples are only examples and might not be optimal for you, although they are good point to start.
Main configuration file - with example
Main configuration file /etc/niceshaper/config.conf is divided into sections. One called global, and minimum one, and in practice two or even more functional sections. Functional section is NiceShaper classes container configured for grouping classes and shape packages on appointed interfaces. Functional section reflect one direction of traffic so each the internet access uplink needs 2 functional sections to be fully controlled. Each functional section has got individual configuration, contained classes and defaults for these classes. Section can contain classes that works on various interfaces. Any number of functional sections may be placed on an interface.
An example of main configuration file:
- <global>
-
- run dl ul
- mark-on-ifaces eth0
- status unit kB/s
- status file /var/www/niceshaper/status.txt
- status file-owner root file-group root file-mode 644
- log syslog yes
- log terminal yes
- log file no
- local-subnets 192.168.0.0/24
- <dl>
-
- section speed 20Mb/s
- section shape 18Mb/s
- mode download
- reload 2s
- low 256kb/s
- ceil 5Mb/s
- strict 70
- <ul>
-
- section speed 2Mb/s shape 1800kb/s
- mode upload
- reload 1s
- low 64kb/s
- ceil 256kb/s
- strict 70
Global section directives:
- run - list of configured sections which you want to run.
- mark-on-ifaces - List of interfaces on which you want to turn on packet marking functionality. It means that iptables rules in mangle table will be created and FW kernel filters will be created in place of U32 filters. This option is required if you want to control upload from hosts located behind a NAT or to use iptables filters instead of U32 kernel filter to get additional filtering abilities.
- lang en|pl - specifies messages language. By default this is taken from LANG environment variable. Apart from default English language you can use pl_PL.UTF-8 in LANG environment variable or just use lang directive with pl value.
- local-subnets - list of controlled local subnets. In iptables space, packets routed to specified subnets go to the chain ns_dwload and packets originated from them to the chain ns_upload.
- iface-<dev> {speed|do-not-shape-method|unclassified-method|fallback-rate|mode} - configuration of network interfaces. Name of the directive includes a name of an interface (but without an alias part after colon), such as, iface-eth0, iface-imq1 etc.
-
- speed - defines the physical interface bandwidth, e.g. for ethernet it will be 100Mb/s or 1000Mb/s. This parameter is required for all interfaces on which you define a wrapper type class or do-not-shape type class with the interface parameter do-not-shape-method safe.
- do-not-shape-method safe|full-throttle - configure the way of handle NiceShaper do-not-shape classes (default: safe).
-
- safe - kernel filters of do-not-shape classes, flows traffic to a common shared HTB class described in "cooperation with HTB". This solution does not provide full interface speed for transfers from the router to a local network, but protects traffic handled by the standard classes.
- full-throttle - traffic from do-not-shape type classes, will be dequeued at hardware speed. It bypass a common shared class described in safe method. Please note that this method can results in fully saturation of the network interface, so prevents from proper shaping traffic handled by the standard class.
- unclassified-method fallback-class|do-not-control - configure the way of handle traffic unclassified by defined NiceShaper filters (default: fallback-class). WARNING: It is serious problem when you have unclassified traffic outgoing from controlled interface!
-
- fallback-class - unclassified traffic flows to fallback class then traffic is almost blocked to minimize impact on internet throughput usage.
- do-not-control - bring back behaviour from version 1.0pre3 and earlier. It means that unclassified packets are dequeued at interface speed. It is highly incorrect and this option is compromise.
- fallback-rate - throughput assigned to a HTB class which will handle unclassified packets. This is nothing but class in HTB designated as the default. To allow link sharing to work correctly nothing should flow into this HTB class (default: 100 kb/s).
- mode download|upload - The wrapper and do-not-shape classes, if work alone on an interface (it means, not together with standard classes), require to set the flow direction of traffic controlled on this interface.
- status {unit|classes|sum|do-not-shape|file|file-owner|file-group|file-mode|file-rewrite} - how to display work statistics. Four last parameters only apply if automatic dump statistics to file is enabled.
-
- unit - bandwidth units for statistics (default: kB/s).
- classes all|active|working|no - define which classes should be displayed in statistics (default: working).
-
- all - display all configured classes.
- active - display active classes only.
- working - display classes with activity for no longer than 'hold' parameter defined.
- sum top|bottom|no - place to display summary of section utilize.
- do-not-shape yes|no - turns on accounting and displaying do-not-shape classes in statistics. Warning: This option turns on usage of iptables rules for all classes which belongs to section which have got at least one do-not-shape class. After that it affect all sections with the same mode value (default: no).
- file filepath|no - automatic dump statistics to specified file, i.e. to be web accessible for clients (default: no).
- file-owner - status file owner (default: root).
- file-group - status file group (default: root).
- file-mode - chmod for status file (default: 644).
-
- file-rewrite - sets the frequency of the automatic dump of statistics to a file in seconds. Ranges from 1s to 3600s. Along with lowering the value of this parameter increases the frequency of disk operations. default: 30s.
- listen {address|password} - a working NiceShaper process after you run niceshaper status or niceshaper show command provides demanded information using TCP/IP protocol. Thanks to this directive you can enable remote access for e.g. from Administrator workstation.
-
- address ip[:port] - parameter replaces a listening IP address and an optionally port, enabling remote connections. By default from point of security NiceShaper listens on 127.0.0.1:6423/TCP, so without the ability to connect from remote. You can connect to working NiceShaper thanks to niceshaper status and show commands with runtime parameter --remote.
- password - by default password for connection authorization is randomly generated each time you start NiceShaper. Locally triggered connections reads password (and the IP address and listening port) from /var/lib/niceshaper/supervisor.info. For remote calling it is required to set your own password. You can connect using password thanks to runtime parameter --password.
- log {syslog|terminal|file} - infos, warnings and errors logging.
-
- syslog yes/no - log to syslog, (default: yes).
- terminal yes/no - (default: yes), after proper initialization automatically turned to no.
- file file|no - log to full path specified file (default: no).
- iptables {download-hook|upload-hook|imq-autoredirect} - directive for iptables configuration.
-
- download-hook PREROUTING|POSTROUTING - change default iptables build-in chain for mode download (default is POSTROUTING). Changing of this parameter only in justified cases, it's not recommended.
- upload-hook PREROUTING|POSTROUTING - change default iptables build-in chain for mode upload (default is POSTROUTING. In versions older than 1.2pre1 it was PREROUTING). Changing of this parameter only in justified cases, it's not recommended
- target ACCEPT|RETURN - target for all of NiceShaper created filters (default is ACCEPT).
- imq-autoredirect yes|no - automatic redirect on IMQ. It makes -j IMQ --todev rules in iptables (default: yes).
- fallback {iptables} - in case of problems with some system components allows you to launch in emergency by other less sophisticated methods.
-
- iptables - if iptables rules are generated starting NiceShaper with many classes is time-consuming. Version 1.0pre2 provided iptables-save and iptables-restore tool usage to speeds up start. This option allows you to return to pure and slow iptables command.
- debug {iptables} - prints system commands before execute it.
-
- iptables - with fallback iptables prints iptables commands before send them to system. Without this option leaves in /var/lib/niceshaper an iptables-restore script.
Functional section directives:
- section {speed|shape|htb-burst|htb-cburst} - parameters of section (in most cases bandwidth of your internet access).
-
- speed - bandwidth of your internet access.
- shape - section traffic you expect to aim. It should be in range of 80% to 95% of a link performance. When set too high, interactive connections works in uncomfortable conditions. Common mistake is to set this so high, so it is newer reached by your section (classes in section). In this case NiceShaper will not work properly, because NiceShaper starts to work when you have overload above section shape. Remember: trigger to cut classes is passed section shape level by summary section traffic.
- htb-burst - burst for a HTB root class created for the section as a parent to ordinary classes. Burst is discussed in detail in the description of the classes HTB burst parameter. By default, automatically calculated, but as high as that of any of it children if where it is found.
- htb-cburst - as htb-burst but for ceil.
- reload - how frequency renew HTB configuration. Vales within range 1s to 5s are safe, effective and do not generate CPU load. If you have idle processor capacity let's try to reduce this value. It will boost interaction with users caprice. If you have a lot of classes (hosts) and you CPU is heavy loaded let's try to bump up. NiceShaper to help you find the best value generates and logs load report for each running section. Report are generated every 1 hour. Allowable value are from 0.1s to 60s and step is 0.1s. High values modifies dynamic traffic shaping into static traffic shaping so values above 5s are not recommended. NiceShaper with high reload value can't react efficiently to frequent traffic changes.
- mode download|upload - it is key directive to ensure proper cooperation with iptables. For all sections working in download mode a common chain named ns_dwload is created which is targeted from built-in POSTROUTING chain. Similarly, for the section mode of upload a chain called ns_upload is created also with a redirection from the chain POSTROUTING (in version 1.2pre1 and earlier it was a chain PREROUTING). Changing a built-in chain, can be achieved using iptables directive with download-hook and upload-hook options.
You can place any classes parameters in functional section configuration, these becomes to be defaults for whole classes in a section.
Hosts and classes in NiceShaper
File /etc/niceshaper/class.conf holds list of hosts definitions and/or classes with their configuration. We can assume that NiceShaper class is an extended HTB class. Each NiceShaper class is built by a header, at least one filter and eventually additional options. Corresponding HTB class is created when NiceShaper notices activity and removed after a set period of inactivity. This gives optimal value of the rate parameter as only active HTB classes exists at the same time. Each packet leaving the interface is classified using filters to the first matching class.
It is vitally important to define classes for whole traffic potentially observed on a link. If not there will appear traffic impossible to control, e.g, if not all hosts in local network will be assigned to classes, these will steal bandwidth from known classes.
Network host:
NiceShaper classes are advanced and flexible tool but simple directive named 'host' is in most cases sufficient and clear way to create fairly traffic shaping for all or most of common hosts in local network. It's the one of element thanks that NiceShaper offers such comfort. Host directive can give you a way to configure traffic shaping in NiceShaper without get to know classes.
To define host you need: list of section in which host must be placed paired with interface on which traffic shaping occurred, ip address and assigned name. Host directive may be placed into one or more of running sections. NiceShaper translate host directive to classes automatically for you.
We can assume that host directive is some kind of NiceShaper macro.
NiceShaper host is placed in one line with syntax given below:
host section interface [section interface] ip name
For example two host:
- host dl eth1 ul eth0 192.168.0.10 pc10
- host dl eth1 192.168.0.11 pc11
These host directives will be translate, by NiceShaper configuration parser, to these classes:
- class dl eth1 pc10
- match dstip 192.168.0.10
- class ul eth0 pc10
- match srcip 192.168.0.10
- class dl eth1 pc11
- match dstip 192.168.0.11
NiceShaper inserts filter test dstip or srcip depending on section mode.
Host are pointed out only by ip address. When you need another filtering tool or overwrite default parameters use classes instead of hosts.
Be carefully as above example of host pc11, which is included only in one section, is only example of how host directive syntax works. You should not leave any host traffic without control especially on upload.
Class structure
In the current version NiceShaper provides 4 types of classes. These are the standard class and a classes for special purposes: wrapper, do-not-shape, and virtual.
- class - Base class. Class of this type operates within the section to which it belongs and is subject to dynamic shaping.
- class-wrapper - Class of this type has a fixed bandwidth, doesn't belong to any section, observed traffic isn't taken into any section traffic summary, isn't accounted by shaping algorithm. Class wrapper requires rate parameter. This class is useful when you want to restrict traffic that does not affect uplink bandwidth, e.g. transfer from a local file server to a lan network, when the only reason for controlling is to not overload the local wireless network.
- class-do-not-shape - Class of this type doesn't receive the HTB class, only kernel filters are created for bypassing a section root HTB class. Thanks that traffic is not restricted like in other classes. Class of this type, identically as type wrapper, doesn't belong to any section, observed traffic isn't taken into any section traffic summary, isn't accounted by shaping algorithm. Depending on status do-not-shape value NiceShaper may create rules in iptables so we can accounting and show bandwidth usage in statistics, but be carefully as it adds additional workload to operating system. At the end let say that wrapper class is a safer option for local traffic.
- class-virtual - Class of this type has entries only in iptables and not in HTB tree. Virtual class can't shape traffic and is useful for measuring and reporting potential traffic. Class of this type belongs to a section, but observed traffic isn't taken into any section traffic summary and isn't accounted by shaping algorithm. Packets matched to virtual class stay in filters list matching procedure so virtual class needs to be followed by another standard class which catch this traffic.
Class definition starts with a header and ends with a next class header or end of file.
Header of standard class and virtual class is as follows:
class|class-virtual section interface name
Wrapper class and do-not-shape class doesn't belong to any section so their header is as follows:
class-wrapper|class-do-not-shape interface name
Where:
section - section to whose a class belongs.
interface - network interface on which a HTB class resides.
name - class name.
Each class must contains one or more filters:
match test <test> [test <test>]
Filters classifies packets to classes. This applies to packets outgoing from class interface only (egress queue). Kernel filters are created when NiceShaper starts and, as you know after read of "Cooperation with iptables", iptables rules if needed. More than one test can be joined in one filter to be more specific on assign traffic to classes. Classes can contains many filters to aggregate more traffic.
An example of the simplest class is as follows:
- class dl eth1 pc55
- match dstip 192.168.0.55
Everywhere in configuration you can use semi-colon instead of new line:
- class dl eth1 pc55; match dstip 192.168.0.55
Class parameters:
- low - minimal rate for class . The class with traffic lower than that value is not participate in section overload. default: 8b/s.
- ceil - maximal rate for class (default is equal section shape).
- rate - static rate. When used, low and ceil are set equal this value and class shaping method becomes static instead of dynamic.
- strict 0 to 100 - it is a key parameter of the dynamic traffic shaping algorithm. This parameter is very useful in case you want to prioritize the classes between each other. Strict parameter indicates to the class, by percentage in the range from low to ceil, greater responsibility threshold for the existing overload. Default value is 70. For example, if low is 100kB/s and ceil is 200kB/s then for strict equal 50 higher responsibility level is above 150kB/s, and it is 170kB/s for strict equal 70. Overload is a condition where the section traffic exceeds the section shape parameter value. If a value of the strict parameter is lower, in relation to the other classes, then the class is treats more restrictive. Similarly, higher value increases tolerance to the class traffic. More specifically, the designated class responsibility for the overload is a function of the strict parameter and traffic generated by this class. This function is a linear increasing function, consists of a few straight lines. The traffic lower or equal to the low value gives no responsibility. Traffic within the range between the low and number indicated from strict, gives a low degree of responsibility. When the load is equal indicated from the strict, then there is exactly the average responsibility. When the load exceeds it, responsibility becomes high. The maximum responsibility occurs when a load is equal to the ceil value.
- strict 0 to 100 - it is a key parameter of dynamic traffic shaping algorithm. This parameter is very useful in case you want to prioritize classes. Default value is 70. Strict is percent of range from low to ceil, that if crossed by a class gives higher level of responsibility for section overload. For example: low is 100kB/s, and ceil is 200kB/s, then for strict equal 50, higher responsibility level is above 150kB/s, and it is 170kB/s for strict equal 70 and so on.
- hold - Inactive time after that class is unloaded from HTB (default: 30s).
- set-mark - parameter applies in a situation where packets marking on class interface is enabled by mark-on-ifaces directive. Then, use it to set the value of the mark. By default, each class is automatically assigned for proper mark value. More in "Packets marking".
- htb {prio|scheduler|burst|cburst} - HTB class parameters.
-
- prio - HTB class priority, accepts 0 to 7 where lower value is greater prio (default: 5).
- scheduler sfq|esfq|no - queuing algorithm for a HTB class (default: sfq).
- burst - Determines how large amount of data can be sent at full interface speed, without going to handle the traffic of another classes. It can have a positive impact for the classes that supports web traffic or another else of a similar nature. By default, the burst is calculated by the algorithm obtained from the iproute package, but sometimes the value thus calculated is too low (this also applies to the tc), being able to cause difficulties in take advantage of high capacity of tens of megabits of a large number of simultaneously working classes. This problem mostly concerns not the ordinary classes but HTB root class created automatically for each section, as a parent for ordinary classes. In case of problems with the saturation of large links, you can try to increase the value of starting burst and cburst for sections, see section HTB-burst and section HTB-cburst.
- cburst - as burst but for ceil.
- sfq {perturb} - SFQ queuing algorithm parameters if used in classes.
-
- perturb - SFQ perturb (default: 10).
- esfq {perturb|hash} - ESFQ scheduler parameters if used in classes.
-
- perturb - ESFQ perturb (default: 10)
- hash classic|src|dst - ESFQ hash (default: classic).
Each option of classes can be placed in a functional section configuration, and becomes to be defaults to not repeat the same values for each class. Next if you wish these defaults can be override in classes with individual values.
Basics filters tests:
- proto tcp|udp|icmp - protocol.
- srcip - source IP address.
- dstip - destination IP address.
- srcport|sport - source port. Needs to choose tcp or udp protocol.
- dstport|dport - destination port. Needs to choose tcp or udp protocol.
- from-local - Allows controlling the local machine outgoing traffic (local machine means router on which this instance of NiceShaper works). Supersedes srcip in filter. Given IP address must be properly configured on one of router interfaces. If iptables is in use and hook is changed to PREROUTING or interface of class works in upload mode then filter duplicate is created in POSTROUTING chain and the target is set to section chain. It's important to ensure that outgoing packets are redirected to section chain.
- to-local - Works analogously to from-local. Allows controlling the local machine incoming traffic. Supersedes dstip in filter. If iptables is in use and hook is POSTROUTING (default) or interface of class works in download mode then filter duplicate in PREROUTING chain is created.
- out-iface - the specified interface is added to iptables filter as the outgoing interface (-o). It makes sense only if the interface of the class is IMQ interface then it points out the physical interface.
- in-iface - the specified interface is added to iptables filter as the incoming interface (-i). Analogously, it makes sense only if the interface of the class is IMQ interface then it points out the physical interface
srcip and dstip can match single ip address or network with subnet mask. In second case allows bits counter notation e.g. /24 or doted notation e.g. 255.255.255.0.
Mask can be non-continuous (e.g. 255.255.128.255) but in this case needs mark-on-ifaces because of it needs fw filter instead of u32.
Example filters:
- match srcip 192.168.0.77 - packets with source ip address 192.168.0.77.
- match srcip 217.74.65.69 srcport 110 dstip 192.168.0.0/29 proto tcp - mail retrieved by POP3 from 217.74.65.69 to 192.168.0.0/29 subnet.
Tests requiring packets marking on a class interface:
Iptables is equipped with a huge number of filters that are unfortunately not feasible using u32 filter. Therefore, these filters require packet marking by mark-on-ifaces directive. Each packet captured and marked by iptables can now easily be enqueued to the appropriate HTB class thanks to received mark value.
CAUTION! Some of these filters may need additional kernel and iptables features compiled in.
- not-srcip - source addresses other than specified.
- not-dstip - destination addresses other than specified.
- not-srcport|not-sport - source ports other than specified (specify tcp or udp protocol needed).
- not-dstport|not-dport - destination ports different than specified (specify tcp or udp protocol needed).
- length - packet length in bytes, e.g. 500, :500 (lower and equal 500), 500: (500 and greater), 128:500 (128 to 500).
- state new|established|related|invalid|untracked - packet state:
-
- new - packets that starts a new connection.
- established - packets belongs to already established connection.
- related - packets related to existing connection.
- invalid - packets impossible to be recognized.
- untracked - packets which do not belongs to tracked connection.
- tos - TOS field value.
- ttl - TTL equal to the specified value.
- ttl-lower - TTL smaller than a given value.
- ttl-greater - TTL greater than the specified value.
- mark - Packets associated with given mark. This mark value will be protected so NiceShaper will not assign automatically your values to any other class. Warning: be carefully to rely on this value after packet leave a NiceShaper iptables chain, because your value of packet will be replaced with automatically generated new one. You can still avoid this thanks to set-mark class parameter.
Macros of the class.conf file
Macros in a classes file are introduced to improve create of a lot of classes if they are similar. Macro iterates in a loop to duplicate specified area. Macro is built with a header, content and ending mark. Header is built by curly brackets that contains macro type and parameters. Content is ordinary class file area. In content there you may put some special chars, like $ (dollar) and % (percent). These chars are replaced by expected number or string. Macro ending mark is curly brackets that contains slash.
In actual NiceShape version are implemented 3 macro types - sequence, foreach-elem, foreach-pair.
sequence macro:
{sequence from to} content {/} - sequence macro generates ascending numeric values in specified range. Values "from" and "to" must be positive integers, in range from 0 to 65535. Values are inserted in place of dollar special character.
Example of sequence usage:
- {sequence 10 12}
- class dl eth1 pc$
- match dstip 192.168.0.$
- {/}
Macro generates:
- class dl eth1 pc10
- match dstip 192.168.0.10
- class dl eth1 pc11
- match dstip 192.168.0.11
- class dl eth1 pc12
- match dstip 192.168.0.12
foreach-elem macro:
{foreach-elem list} content {/} - foreach-elem macro gets each element from given list and inserts it in place of dollar special char. List must contain numeric or text values separated by whitespaces.
Example of foreach-elem usage:
- {foreach-elem 10 12 15}
- class dl eth1 pc$
- match dstip 192.168.0.$
- {/}
Macro generates:
- class dl eth1 pc10
- match dstip 192.168.0.10
- class dl eth1 pc12
- match dstip 192.168.0.12
- class dl eth1 pc15
- match dstip 192.168.0.15
foreach-pair macro:
{foreach-pair list_of_pairs} content {/} - foreach-pair macro gets pairs of values from list. List must contain numeric or text values. Pairs are separated by commas, elements in pair are separated by whitespace. First element in pair is named a key and is inserted in place of percent char. Second element in pair is inserted in place of dollar.
Example of foreach-pair usage:
- {foreach-pair accounting1 11, accounting2 12, marketing1 21}
- class dl eth1 %
- match dstip 192.168.0.$
- {/}
Macro generates:
- class dl eth1 accounting1
- match dstip 192.168.0.11
- class dl eth1 accounting2
- match dstip 192.168.0.12
- class dl eth1 marketing1
- match dstip 192.168.0.21
Packet marking
Packet marking is a procedure to assign a virtual tag to a packet. Thanks to that it is still possible to identify packet sender after change the source address by SNAT (IP masquerade). The value is assigned by iptables, maintained by the Linux kernel and recognizable by iptables and the kernel FW filters. FW kernel filters classifies packets into a appropriate HTB class using test based on a virtual mark instead of a packet header.
Marking packages mostly do not require patching iptables, iproute or kernel. NiceShaper manages all completely automatically so it's very easy to use this mechanism. Therefore, packet marking is a good way to avoid use of IMQ interfaces, which are comfortable but required patching the operating system important components. Marking allows you to comfortable control upload of your masqueraded computers.
So if packet marking on interface is turned on by mark-on-ifaces directive then NiceShaper for each class working on this interface will assign a unique mark value and U32 kernel filter on that interface will be replaced with FW. All this is done completely automatically. However if necessary it is possible to make intervention in this process thanks to a set-mark parameter. The value of set-mark must be unique for each class.
Iptables so NiceShaper too accepts mark values from 0 to 4294967295 (32bit unsigned value) written in decimal or hex started with 0x.
Usage of IMQ interfaces
NiceShaper supports IMQ compiled in AB mode (after NAT in PREROUTING chain and before NAT in POSTROUTING chain).
From the configuration of NiceShaper IMQ interfaces are the same way as physical ones. You can forget about their virtual, mostly, with complete freedom to mix classes on physical and IMQ interfaces.
Niceshaper filters require to indicate a physical interface by out-iface test (or in-iface if iptables hook is changed to PREROUTING).
Example snip from class.conf file:
- class ul imq5 pc5.82
- match srcip 10.10.5.82 out-iface eth0
- class ul imq5 pc5.83
- match srcip 10.10.5.83 out-iface eth0
NiceShaper automatically redirects traffic to IMQ interface. This behavior is configurable by iptables imq-autoredirect directive in global section. If you disable the automatic redirect to the IMQ you need to make it your self for NiceShaper by something like iptables ... -j IMQ --todev imqX.
Triggers
Trigger is mechanism that allows you to automatically change the values of some class when a defined case occurs. In the current version NiceShaper two triggers types are implemented, alter and quota.
Controllable parameters of the class:
- low - the minimum bandwidth allocation (default: 8b/s)
- ceil - the maximum bandwidth allocation (default is equal to the section shape).
- rate - fixed bandwidth allocation.
Control parameters of triggers:
- alter {low|ceil|rate} {time-period} - Alter trigger in a certain period of time replaces the defined parameters, for example to increase allocations at night.
-
- {time-period} hh:mm-hh:mm - specifies the time at which the trigger is active, for example, 22:00-05:00. Default: no - trigger is off.
- quota {low|ceil|rate} {day|week|month} [reset-hour] [reset-wday] [reset-mday] - quota trigger is to change the selected class parameter when a class data transfer exceed a limit. This trigger is equipped with three counters: daily, weekly and monthly. Quota trigger is active whenever one or more counters exceed a limit.
-
- day - the daily limit of transferred data. The minimum value is 1MB, supported units are MB, GB and TB. Default: no - daily counter is off.
- week - weekly limit, similarly as daily.
- month - monthly limit, similarly as daily and weekly.
- reset-hour gg:mm - the time at which reset of the daily counter should be performed.
- reset-wday 1 to 7 - similarly as for the daily quotas, specify the day of the week for weekly counter. 1 is Monday, 2 is Tuesday, and so on up to 7 which is Sunday. Reset takes place on the first reload of a section after midnight. Default: 1 - Monday.
- reset-mday 1 do 31 - and again analogous to the daily and weekly quotas. If the value exceeds the last day of a month, such as 31 in the month of 30 days e.g. February, April a reset occurs on the last day of that month. Default: 1.
Triggers are class directives, most convenient way is to define them within the section, such as:
- <dl>
-
- section speed 512kB/s shape 450kB/s
- low 10kB/s
- ceil 100kB/s
- alter ceil 200kB/s time-period 22:00-5:00
- quota ceil 20kB/s day 2GB week 20GB month 500GB
- ...
The quota trigger have higher priority than the alter trigger if both have to be enabled. Triggers works only for standard-class.
Counters of trigger quota between running the program are stored in files section_name.quota in /var/lib/niceshaper directory. These files are updated when NiceShaper is stopping and during his work at 5 minutes intervals. If this file write fails, counters will be irretrievably lost!
Router's self generated traffic shaping
Traffic generated by the router for his own needs is mostly negligibly small and often doesn't need to be shaped. But, in real world, router with Linux often serves file shares to the local network (simpler scenario) or even serves some network services to the internet (harder scenario).
Due to the fact that the traffic shaping takes place at the interface from which packets outgoings a machine, shaping incoming traffic (both on Internet and LAN side) is difficult and requires usage of IMQ interfaces.
To practically deal with this topic, the most convenient is the breakdown of the issues on four separate scenarios.
In all scenarios it's required to add from-local or to-local test to filters. In combination with IMQ interfaces there is additional in-iface test requirement, eventually out-iface for outgoing traffic. It's important to let NiceShaper know physical interface at which packet arrives or leaves and to distinguish WAN side from LAN side traffic.
Keep in mind that in most cases this purpose classes should be on the top of the list of all your classes.
Remind assumption that WAN is connected to eth0 and LAN is connected to eth1. Public address of router is 198.51.100.100, second public address for services is 198.51.100.101, private address is 192.168.0.1, and private network is 192.168.0.0/24.
Traffic between the router and the local network (uplink bandwidth is not involved).
- 1. Router -> Localnet - Router sends traffic to the local network..
- 2. Router <- Localnet - Router gets traffic from the local network.
Traffic between a router and a local network requires to be not shaped or shaping should be minimised. This traffic shouldn't be shaped to the fact that local ethernet connection is mostly fast. Another big mistake would be accounting this traffic as uplink usage with the traffic generated by hosts. Classes in #1 and #2 scenarios (router with local network traffic related) should be wrapper or do-not-shape types, because classes of these types don't belong to any section and aren't taken into process of dynamic traffic shaping.
Traffic between the router and the internet (using uplink bandwidth).
- 3. Router -> Internet - Router sends traffic to the internet..
- 4. Router <- Internet - Router gets traffic from the internet
Putting services on WAN side should be avoided. If SNAT and internet services on WAN side are used, services should use another public IP address. In this situation requirements of two public addresses is forced because there is no way to distinguish local from forwarded (NATed) traffic in the PREROUTING chain of mangle table. This distinguish is possible in INPUT chain (simplifying), but there is no way to redirect traffic to IMQ devices.
1. Router -> Localnet - Router sends traffic to the local network.
Traffic in this scenario is for example, local file or ftp server, local imap or pop3 server, etc.
Using do-not-shape type class - traffic is local so you do not want to shaping and accounting:
- class-do-not-shape eth1 router_to_local
- match from-local 198.51.100.101 dstip 192.168.0.0/24
- match from-local 192.168.0.1 dstip 192.168.0.0/24
Using wrapper type class - traffic is local but we have WiFi bridge on LAN and we are afraid to overload, so we process shaping but with statically allocated bandwidth and still do not account to any section:
- class-wrapper eth1 router_to_local
- match from-local 198.51.100.101 dstip 192.168.0.0/24
- match from-local 192.168.0.1 dstip 192.168.0.0/24
- rate 20Mb/s
2. Router <- Localnet - Router gets traffic from the local network.
Traffic from the LAN to the router. For example, sending e-mail via the local SMTP server, place the files on a local file server, etc.
Using do-not-shape type class - NiceShaper use POSTROUTING chain at iptables and doesn't use ingress shaping at kernel QOS. So, there is no requirement to create any do-not-shape type classes as traffic described in this scenario is uncontrolled in default configuration.
Using wrapper type class - but, if we want to use wrapper type class (analogically to scenario #1), we need to use the IMQ device.
- class-wrapper imq1 router_from_local
- match to-local 198.51.100.101 srcip 192.168.0.0/24 in-iface eth1
- match to-local 192.168.0.1 srcip 192.168.0.0/24 in-iface eth1
- rate 20Mb/s
3. Router -> Internet - Router sends traffic to the internet.
Packets generated locally while sending to the internet e.g., mail service, web pages and other services on the router.
In this examples, two classes are created to separate traffic of two different services.
- class ul eth0 router_www_to_internet
- match from-local 198.51.100.101 proto tcp srcport 80
- ceil 64kB/s
- class ul eth0 router_mx_to_internet
- match from-local 198.51.100.101 proto tcp dstport 25
- ceil 128kB/s
Usage of class of mode upload and filter with a from-local test does not require extensive commentary here. Provide belief that traffic from the router to the internet is properly accounted.
4. Internet => Router - downloading traffic from the internet to the router.
This scenario occurs when a router retrieves traffic from the internet for e.g. the web pages retrieves by a proxy server or system updates that may steal a bandwidth from clients.
- class dl imq0 router_from_internet
- match to-local 198.51.100.101 in-iface eth0
- ceil 1Mb/s
Using a class at download mode section and a filter to-local don't require extensive commentary here too.
By default, the classes that belong to download mode puts their filters in the chain targeted from POSTROUTING where our packets will never arrive, so a to-local test creates a cloned rule in the PREROUTING chain.
Advanced topics
Cooperation with iptables
In some cases NiceShaper uses iptables. In that case it creates rules in mangle table on starting and flushes them on stopping. You should avoid this cases whenever possible because rules for many NiceShaper classes generates additional system load. If at least one NiceShaper class in section required iptables rule, then rules are created for all NiceShaper classes in this section. And furthermore worst, iptables rules are created for each section with the same mode parameter value.
To be precise, iptables rules are required and created for the following cases:
- Packet mangling: It means packets marking or redirecting onto IMQ interfaces. In the case of IMQ redirecting additional load are smaller than in case of packet marking, because in case of packet marking iptables rules counters are checked out on each section reload.
- Checking for classes activity. This occurs when FW kernel filters are set on (look for mark-on-ifaces directive). Iptables rules are required here because FW kernel filters haven't got embedded counters, opposite to U32 kernel filters.
- Checking for activity and accounting traffic of a virtual and a do-not-shape classes. Second one is counting if use with status do-not-shape yes directive value. It occurred because NiceShaper classes of do-not-shape type haven't got HTB class so iptables rule is only way to get statistics about traffic. Sadly generate much system load.
Reading any of the chains not held more than 10 times a second. If a section is reloaded again before this time it will use the previous cached values.
Until version 1.0 NiceShaper used iptables rules as fundamental. Rules was always created and NiceShaper looked for activity and accounted traffic. From version 1.2pre1 iptables stopped to be fundamental.
Cooperation with HTB
NiceShaper creates QOS framework on each controlled interface. This framework looks like on diagram below. It contains HTB classes, queuing algorithms and kernel filters. HTB classes creates tree hierarchy by which traffic is propagated as you see.
NiceShaper doesn't use the tc program. It communicates, using a Netlink protocol, directly with the Linux kernel (the code that provides this functionality is partially based on iproute code). It's the most efficient way to manage QOS objects.
On bottom level of tree there are HTB classes which becomes to be parents for NiceShaper classes defined by you. These HTB classes share full speed of interface but do not borrow from each other.
These special HTB classes act as:
- 1. HTB classes of each of NiceShaper sections that work on a interface.
- 2. HTB class whose is parent for do-not-shape and wrapper type classes.
- 3. HTB fallback class that works with unclassified traffic.
1) Each section which works on an interface gets HTB class with throughput equal to section speed parameter value. This HTB class becomes parent for each child HTB classes within that section. One child HTB class is automatically created as Waiting Room. Traffic classified by filters in first step is directing into Waiting Room. In second step when section is reloading filter are modified to directing traffic to newly created HTB class. After some time of inactivity this newly created HTB class is removed and kernel filter directing traffic to Waiting Room again. This time of inactivity is set by hold parameter which has 30 seconds default. In this solution in one time there are only us much HTB classes as activated NiceShaper classes, thanks for that HTB class rate parameter value is optimal.
2) This HTB class as parent for do-not-shape and wrapper classes is created when on interface works one of or both of specified class types. In case of do-not-shape class it is true only with iface do-not-shape-method safe option. This HTB class throughput is calculated from iface speed minus sections speed minus fallback class.
3) HTB fallback class works with whole traffic outgoing from interface and unclassified by filters. In proper configuration this class should be idle, otherwise you should looking for lack in defined classes and filters.