src-test2/sys/modules/ipfw, branch release/11.3.0

MFC r328988,r328989:

2018-04-11T10:36:20Z

Rework ipfw dynamic states implementation to be lockless on fast path. o added struct ipfw_dyn_info that keeps all needed for ipfw_chk and for dynamic states implementation information; o added DYN_LOOKUP_NEEDED() macro that can be used to determine the need of new lookup of dynamic states; o ipfw_dyn_rule now becomes obsolete. Currently it used to pass information from kernel to userland only. o IPv4 and IPv6 states now described by different structures dyn_ipv4_state and dyn_ipv6_state; o IPv6 scope zones support is added; o ipfw(4) now depends from Concurrency Kit; o states are linked with "entry" field using CK_SLIST. This allows lockless lookup and protected by mutex modifications. o the "expired" SLIST field is used for states expiring. o struct dyn_data is used to keep generic information for both IPv4 and IPv6; o struct dyn_parent is used to keep O_LIMIT_PARENT information; o IPv4 and IPv6 states are stored in different hash tables; o O_LIMIT_PARENT states now are kept separately from O_LIMIT and O_KEEP_STATE states; o per-cpu dyn_hp pointers are used to implement hazard pointers and they prevent freeing states that are locklessly used by lookup threads; o mutexes to protect modification of lists in hash tables now kept in separate arrays. 65535 limit to maximum number of hash buckets now removed. o Separate lookup and install functions added for IPv4 and IPv6 states and for parent states. o By default now is used Jenkinks hash function. Obtained from: Yandex LLC Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D12685

MFC r325960:

2017-11-24T04:42:21Z

Unconditionally enable support for O_IPSEC opcode. IPsec support can be loaded as kernel module, thus do not depend from kernel option IPSEC and always build O_IPSEC opcode implementation as enabled. MFC r325962: Do not invoke IPv4 NAT handler for non IPv4 packets. Libalias expects a packet is IPv4. And in case when it is IPv6, it just translates them as IPv4. This leads to corruption and in some cases to panics. In particular a panic can happen when value of ip6_plen modified to something that leads to IP fragmentation, but actual packet length does not match the IP length. Packets that are not IPv4 will be dropped by NAT rule.

MFC r314651,r318439,r318440:

2017-05-30T04:11:12Z

r314651: sys/modules: normalize .CURDIR-relative paths to SRCTOP This simplifies make output/logic Tested with: `cd sys/modules; make ALL_MODULES=` on amd64 r318439: Normalize .PATH on SRCTOP This will help Jenkins dedupe 9 warnings between the static build and the module build of ipsec(4). Missed in SRCTOP conversion in r314651. MFC with: r314651 r318440: Normalize SYSDIR on SRCTOP instead of .CURDIR This is being done to simplify pathing for CFLAGS and source files.

MFC r304041:

2017-04-03T08:50:54Z

Move logging via BPF support into separate file. * make interface cloner VNET-aware; * simplify cloner code and use if_clone_simple(); * migrate LOGIF_LOCK() to rmlock; * add ipfw_bpf_mtap2() function to pass mbuf to BPF; * introduce new additional ipfwlog0 pseudo interface. It differs from ipfw0 by DLT type used in bpfattach. This interface is intended to used by ipfw modules to dump packets with additional info attached. Currently pflog format is used. ipfw_bpf_mtap2() function uses second argument to determine which interface use for dumping. If dlen is equal to ETHER_HDR_LEN it uses old ipfw0 interface, if dlen is equal to PFLOG_HDRLEN - ipfwlog0 will be used. Obtained from: Yandex LLC Sponsored by: Yandex LLC MFC r304043: Add three helper function to manage tables from external modules. ipfw_objhash_lookup_table_kidx does lookup kernel index of table; ipfw_ref_table/ipfw_unref_table takes and releases reference to table. Obtained from: Yandex LLC Sponsored by: Yandex LLC MFC r304046, 304108: Add ipfw_nat64 module that implements stateless and stateful NAT64. The module works together with ipfw(4) and implemented as its external action module. Stateless NAT64 registers external action with name nat64stl. This keyword should be used to create NAT64 instance and to address this instance in rules. Stateless NAT64 uses two lookup tables with mapped IPv4->IPv6 and IPv6->IPv4 addresses to perform translation. A configuration of instance should looks like this: 1. Create lookup tables: # ipfw table T46 create type addr valtype ipv6 # ipfw table T64 create type addr valtype ipv4 2. Fill T46 and T64 tables. 3. Add rule to allow neighbor solicitation and advertisement: # ipfw add allow icmp6 from any to any icmp6types 135,136 4. Create NAT64 instance: # ipfw nat64stl NAT create table4 T46 table6 T64 5. Add rules that matches the traffic: # ipfw add nat64stl NAT ip from any to table(T46) # ipfw add nat64stl NAT ip from table(T64) to 64:ff9b::/96 6. Configure DNS64 for IPv6 clients and add route to 64:ff9b::/96 via NAT64 host. Stateful NAT64 registers external action with name nat64lsn. The only one option required to create nat64lsn instance - prefix4. It defines the pool of IPv4 addresses used for translation. A configuration of instance should looks like this: 1. Add rule to allow neighbor solicitation and advertisement: # ipfw add allow icmp6 from any to any icmp6types 135,136 2. Create NAT64 instance: # ipfw nat64lsn NAT create prefix4 A.B.C.D/28 3. Add rules that matches the traffic: # ipfw add nat64lsn NAT ip from any to A.B.C.D/28 # ipfw add nat64lsn NAT ip6 from any to 64:ff9b::/96 4. Configure DNS64 for IPv6 clients and add route to 64:ff9b::/96 via NAT64 host. Obtained from: Yandex LLC Relnotes: yes Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D6434 MFC r304048: Replace __noinline with special debug macro NAT64NOINLINE. MFC r304061: Use %ju to print unsigned 64-bit value. MFC r304076: Make statistics nat64lsn, nat64stl an nptv6 output netstat-like: "@value @description" and fix build due to -Wformat errors. MFC r304378 (by bz): Try to fix gcc compilation errors (which are right). nat64_getlasthdr() returns an int, which can be -1 in case of error, storing the result in an uint8_t and then comparing to < 0 is not helpful. Do what is done in the rest of the code and make proto an int here as well. MFC r309187: Fix ICMPv6 Time Exceeded error message translation. MFC r314718: Use new ipfw_lookup_table() in the nat64 too. MFC r315204,315233: Use memset with structure size.

Add External Actions KPI to ipfw(9).

2016-04-14T22:51:23Z

It allows implementing loadable kernel modules with new actions and without needing to modify kernel headers and ipfw(8). The module registers its action handler and keyword string, that will be used as action name. Using generic syntax user can add rules with this action. Also ipfw(8) can be easily modified to extend basic syntax for external actions, that become a part base system. Sample modules will coming soon. Obtained from: Yandex LLC Sponsored by: Yandex LLC

Merge projects/ipfw to HEAD.

2014-10-09T19:32:35Z

Main user-visible changes are related to tables: * Tables are now identified by names, not numbers. There can be up to 65k tables with up to 63-byte long names. * Tables are now set-aware (default off), so you can switch/move them atomically with rules. * More functionality is supported (swap, lock, limits, user-level lookup, batched add/del) by generic table code. * New table types are added (flow) so you can match multiple packet fields at once. * Ability to add different type of lookup algorithms for particular table type has been added. * New table algorithms are added (cidr:hash, iface:array, number:array and flow:hash) to make certain types of lookup more effective. * Table value are now capable of holding multiple data fields for different tablearg users Performance changes: * Main ipfw lock was converted to rmlock * Rule counters were separated from rule itself and made per-cpu. * Radix table entries fits into 128 bytes * struct ip_fw is now more compact so more rules will fit into 64 bytes * interface tables uses array of existing ifindexes for faster match ABI changes: All functionality supported by old ipfw(8) remains functional. Old & new binaries can work together with the following restrictions: * Tables named other than ^\d+$ are shown as table(65535) in ruleset in old binaries Internal changes:. Changing table ids to numbers resulted in format modification for most sockopt codes. Old sopt format was compact, but very hard to extend (no versioning, inability to add more opcodes), so * All relevant opcodes were converted to TLV-based versioned IP_FW3-based codes. * The remaining opcodes were also converted to be able to eliminate all older opcodes at once * All IP_FW3 handlers uses special API instead of calling sooptcopy* directly to ease adding another communication methods * struct ip_fw is now different for kernel and userland * tablearg value has been changed to 0 to ease future extensions * table "values" are now indexes in special value array which holds extended data for given index * Batched add/delete has been added to tables code * Most changes has been done to permit batched rule addition. * interface tracking API has been added (started on demand) to permit effective interface tables operations * O(1) skipto cache, currently turned off by default at compile-time (eats 512K). * Several steps has been made towards making libipfw: * most of new functions were separated into "parse/prepare/show and actuall-do-stuff" pieces (already merged). * there are separate functions for parsing text string into "struct ip_fw" and printing "struct ip_fw" to supplied buffer (already merged). * Probably some more less significant/forgotten features MFC after: 1 month Sponsored by: Yandex LLC

Add pre-alfa version of DXR lookup module.

2014-09-21T18:15:09Z

It does build but (currently) does not work. This change is not intended to be merged along with other ipfw changes.

Add support for multi-field values inside ipfw tables.

2014-08-31T23:51:09Z

This is the last major change in given branch. Kernel changes: * Use 64-bytes structures to hold multi-value variables. * Use shared array to hold values from all tables (assume each table algo is capable of holding 32-byte variables). * Add some placeholders to support per-table value arrays in future. * Use simple eventhandler-style API to ease the process of adding new table items. Currently table addition may required multiple UH drops/ acquires which is quite tricky due to atomic table modificatio/swap support, shared array resize, etc. Deal with it by calling special notifier capable of rolling back state before actually performing swap/resize operations. Original operation then restarts itself after acquiring UH lock. * Bump all objhash users default values to at least 64 * Fix custom hashing inside objhash. Userland changes: * Add support for dumping shared value array via "vlist" internal cmd. * Some small print/fill_flags dixes to support u32 values. * valtype is now bitmask of . New values can hold distinct values for each of this types. * Provide special "legacy" type which assumes all values are the same. * More helpers/docs following.. Some examples: 3:41 [1] zfscurr0# ipfw table mimimi create valtype skipto,limit,ipv4,ipv6 3:41 [1] zfscurr0# ipfw table mimimi info +++ table(mimimi), set(0) +++ kindex: 2, type: addr references: 0, valtype: skipto,limit,ipv4,ipv6 algorithm: addr:radix items: 0, size: 296 3:42 [1] zfscurr0# ipfw table mimimi add 10.0.0.5 3000,10,10.0.0.1,2a02:978:2::1 added: 10.0.0.5/32 3000,10,10.0.0.1,2a02:978:2::1 3:42 [1] zfscurr0# ipfw table mimimi list +++ table(mimimi), set(0) +++ 10.0.0.5/32 3000,0,10.0.0.1,2a02:978:2::1

Move most of the 15 variations on generating opt_inet.h and

2014-08-04T22:37:02Z

opt_inet6.h into kmod.mk by forcing almost everybody to eat the same dogfood. While at it, consolidate the opt_bpf.h and opt_mroute.h targets here too.

* Add generic ipfw interface tracking API

2014-07-28T19:01:25Z

* Rewrite interface tables to use interface indexes Kernel changes: * Add generic interface tracking API: - ipfw_iface_ref (must call unlocked, performs lazy init if needed, allocates state & bumps ref) - ipfw_iface_add_ntfy(UH_WLOCK+WLOCK, links comsumer & runs its callback to update ifindex) - ipfw_iface_del_ntfy(UH_WLOCK+WLOCK, unlinks consumer) - ipfw_iface_unref(unlocked, drops reference) Additionally, consumer callbacks are called in interface withdrawal/departure. * Rewrite interface tables to use iface tracking API. Currently tables are implemented the following way: runtime data is stored as sorted array of {ifidx, val} for existing interfaces full data is stored inside namedobj instance (chained hashed table). * Add IP_FW_XIFLIST opcode to dump status of tracked interfaces * Pass @chain ptr to most non-locked algorithm callbacks: (prepare_add, prepare_del, flush_entry ..). This may be needed for better interaction of given algorithm an other ipfw subsystems * Add optional "change_ti" algorithm handler to permit updating of cached table_info pointer (happens in case of table_max resize) * Fix small bug in ipfw_list_tables() * Add badd (insert into sorted array) and bdel (remove from sorted array) funcs Userland changes: * Add "iflist" cmd to print status of currently tracked interface * Add stringnum_cmp for better interface/table names sorting