4. PF_RING FT API

PF_RING FT library header file.

Defines

FT_API_VERSION
PFRING_FT_ACTION_DEFAULT
PFRING_FT_ACTION_FORWARD
PFRING_FT_ACTION_DISCARD

Discard packet due to filter or shunt

PFRING_FT_ACTION_USER_1
PFRING_FT_ACTION_USER_2
PFRING_FT_ACTION_SLICE

Slice packet headers

PF_RING_FT_FLOW_FLAGS_L7_GUESS

pfring_ft_flow_value.flags: detected L7 protocol is a guess.

PFRING_FT_TABLE_FLAGS_DPI

pfring_ft_create_table() flag: enable nDPI support for L7 protocol detection

PFRING_FT_TABLE_FLAGS_DPI_EXTRA

pfring_ft_create_table() flag: enable nDPI extra dissection (more flow metadata)

PFRING_FT_DECODE_TUNNELS

pfring_ft_create_table() flag: decode tunnels (GTP, L2TP, CAPWAP)

PFRING_FT_IGNORE_HW_HASH

pfring_ft_create_table() flag: ignore hw packet hash (e.g. when it’s asymmetric leading to one flow per direction)

PFRING_FT_IGNORE_VLAN

pfring_ft_create_table() flag: do not include vlan in flow key

PFRING_FT_TABLE_FLAGS_NO_GUESS

pfring_ft_create_table() flag: do not guess the protocol when not detected with DPI

Typedefs

typedef void pfring_ft_table
typedef void pfring_ft_list
typedef void pfring_ft_flow
typedef u_int8_t pfring_ft_action
typedef u_int32_t pfring_ft_in4_addr
typedef void (*pfring_ft_export_list_func)(pfring_ft_list *flows_list, void *user)
typedef void (*pfring_ft_export_flow_func)(pfring_ft_flow *flow, void *user)
typedef void (*pfring_ft_flow_packet_func)(const u_char *data, pfring_ft_packet_metadata *metadata, pfring_ft_flow *flow, void *user)

Enums

enum pfring_ft_direction

Values:

s2d_direction = 0

Source to destination

d2s_direction

Destination to source

PF_RING_FT_FLOW_NUM_DIRECTIONS
enum pfring_ft_flow_status

Values:

PFRING_FT_FLOW_STATUS_ACTIVE = 0

Active flow

PFRING_FT_FLOW_STATUS_IDLE_TIMEOUT

Idle timeout

PFRING_FT_FLOW_STATUS_ACTIVE_TIMEOUT

Terminated after the maximum lifetime

PFRING_FT_FLOW_STATUS_END_DETECTED

Terminated for end of flow (e.g. FIN)

PFRING_FT_FLOW_STATUS_FORCED_END

Terminated for external event (shutdown)

PFRING_FT_FLOW_STATUS_SLICE_TIMEOUT

Flow slice timeout

PFRING_FT_FLOW_STATUS_OVERFLOW

Exported from those with higher inactivity to make room

Functions

pfring_ft_table *pfring_ft_create_table(u_int32_t flags, u_int32_t max_flows, u_int32_t flow_idle_timeout, u_int32_t flow_lifetime_timeout, u_int32_t user_metadata_size)

Create a new flow table.

Return
The flow table on success, NULL on failure.
Parameters
  • flags: Flags to enable selected flow table features.
  • max_flows: Maximum number of concurrent flows the table should be able to handle (use 0 if not sure to use default settings).
  • flow_idle_timeout: Maximum flow idle time (seconds) before expiration (use 0 if not sure to use default: 30s).
  • flow_lifetime_timeout: Maximum flow duration (seconds) before expiration (use 0 if not sure to use default: 2m).
  • user_metadata_size: Size of the user metadata in pfring_ft_flow_value->user

void pfring_ft_destroy_table(pfring_ft_table *table)

Destroy a flow table.

Parameters
  • table: The flow table handle.

void pfring_ft_flow_set_flow_slicing(pfring_ft_table *table, u_int32_t flow_slice_timeout)

Enable flow slicing to peridiocally export flow updates, even when the configured flow_lifetime_timeout is not reached.

Parameters
  • table: The flow table handle.
  • flow_slice_timeout: Maximum flow slice duration (seconds). This should be lower then flow_lifetime_timeout

void pfring_ft_set_new_flow_callback(pfring_ft_table *table, pfring_ft_export_flow_func callback, void *user)

Set the function to be called when a new flow has been created.

Parameters
  • table: The flow table handle.
  • callback: The callback.
  • user: The user data provided to the callback.

void pfring_ft_set_flow_packet_callback(pfring_ft_table *table, pfring_ft_flow_packet_func callback, void *user)

Set the function to be called when a packet and its flow have been processed, for each packet.

Parameters
  • table: The flow table handle.
  • callback: The callback.
  • user: The user data provided to the callback.

void pfring_ft_set_l7_detected_callback(pfring_ft_table *table, pfring_ft_flow_packet_func callback, void *user)

Set the function to be called when a packet and its flow have been processed and the l7 protocol has been just detected.

Parameters
  • table: The flow table handle.
  • callback: The callback (Note: packet/metadata may be NULL).
  • user: The user data provided to the callback.

void pfring_ft_set_flow_export_callback(pfring_ft_table *table, pfring_ft_export_flow_func callback, void *user)

Set the function to be called when a flow expires and needs to be exported. The callback should release the flow calling pfring_ft_flow_free(flow).

Parameters
  • table: The flow table handle.
  • callback: The callback.
  • user: The user data provided to the callback.

void pfring_ft_set_flow_list_export_callback(pfring_ft_table *table, pfring_ft_export_list_func callback, void *user)

Set the function to be called when a some flow expires and need to be exported. This can be used as an optimised alternative to pfring_ft_set_flow_export_callback(). The callback should release all flows in the list calling pfring_ft_flow_free(flow) for each flow. It is possible to iterate all the flows in the list using pfring_ft_list_get_next().

Parameters
  • table: The flow table handle.
  • callback: The callback.
  • user: The user data provided to the callback.

pfring_ft_action pfring_ft_process(pfring_ft_table *table, const u_char *packet, const pfring_ft_pcap_pkthdr *header, const pfring_ft_ext_pkthdr *ext_header)

Provide a raw packet to the flow table for processing. Usually the main capture loop provides all the packets to the hash table calling this function.

Return
The action for the packet, in case filtering rules have been specified.
Parameters
  • table: The flow table handle.
  • packet: The raw packet.
  • header: The packet metadata (including length and timestamp).
  • ext_header: Additional packet metadata not available in the pcap header (including hash).

int pfring_ft_housekeeping(pfring_ft_table *table, u_int32_t epoch)

This should be called when there is no packet to be processed and the main loop is idle, for running housekeeping activities in the flow table.

Return
1 if there is more work to do, 0 if the caller can sleep a bit.
Parameters
  • table: The flow table handle.
  • epoch: The current epoch (sec).

void pfring_ft_flush(pfring_ft_table *table)

Flush all flows (usually called on program termination, before destroying the flow table).

Parameters
  • table: The flow table handle.

pfring_ft_flow *pfring_ft_list_get_next(pfring_ft_list *list)

Pop the next from a flow list.

Return
The flow if the list is not empty, NULL otherwise.
Parameters
  • list: The flow list.

u_int64_t pfring_ft_flow_get_id(pfring_ft_flow *flow)

Get the flow ID.

Return
The flow ID.
Parameters
  • flow: The flow handle.

pfring_ft_flow_key *pfring_ft_flow_get_key(pfring_ft_flow *flow)

Get the flow key.

Return
The flow key.
Parameters
  • flow: The flow handle.

pfring_ft_flow_value *pfring_ft_flow_get_value(pfring_ft_flow *flow)

Get the flow value.

Return
The flow value.
Parameters
  • flow: The flow handle.

struct ndpi_flow_struct *pfring_ft_flow_get_ndpi_handle(pfring_ft_flow *flow)

Get the nDPI flow handle.

Return
The flow value.
Parameters
  • flow: The flow handle.

void pfring_ft_flow_set_action(pfring_ft_flow *flow, pfring_ft_action action)

Set the flow action, to be returned by pfring_ft_process() for all packets for this flow.

Parameters
  • flow: The flow handle.
  • action: The action.

pfring_ft_action pfring_ft_flow_get_action(pfring_ft_flow *flow)

Get the computed/actual flow action, the same returned by pfring_ft_process() for this flow.

Return
The action.
Parameters
  • flow: The flow handle.

int pfring_ft_flow_get_users(pfring_ft_flow *flow)

Return the number of users for the flow (value of the reference counter). This is usually 1, unless slicing is enabled (+1 for each slice not yet released). Calling this on the slice, returns the reference counter of the master flow.

Parameters
  • flow: The flow handle.

void pfring_ft_flow_free(pfring_ft_flow *flow)

Release a flow.

Parameters
  • flow: The flow handle.

void pfring_ft_zmq_export_configure(pfring_ft_table *table, const char *endpoint, const char *server_public_key, u_int8_t probe_mode, u_int8_t disable_compression, u_int8_t use_json)

Configure ZMQ flow export (see pfring_ft_zmq_export_flow)

Parameters
  • table: The flow table handle.
  • endpoint: The ZMQ endpoint.
  • server_public_key: The ZMQ Public encryption key (NULL for clear).
  • probe_mode: Probe mode (connect to the ZMQ collector).
  • disable_compression: Disable message compression.
  • use_json: Use JSON format (Default: TLV).

void pfring_ft_zmq_export_flow(pfring_ft_flow *flow, void *user)

Built-in callback to be provided to pfring_ft_set_flow_export_callback for exporting flows in JSON or TLV format to ZMQ. This implements pfring_ft_export_flow_func. The ZMQ endpoint should be configure with pfring_ft_zmq_export_configure(). The callback also releases the flow calling pfring_ft_flow_free(flow). Usage: pfring_ft_set_flow_export_callback(table, pfring_ft_zmq_export_flow, table);

Parameters
  • flow: The flow to be exported.
  • user: The flow table handle.

void pfring_ft_zmq_export_stats(pfring_ft_table *table, const char *if_name, u_int16_t if_speed, const char *if_ip, const char *management_ip)

Export stats via ZMQ

Parameters
  • table: The flow table handle.
  • if_name: Interface name.
  • if_speed: Interface speed (Mbps).
  • if_ip: Interface IP.
  • management_ip: Management Interface IP.

void pfring_ft_zmq_get_stats(pfring_ft_table *table, pfring_ft_export_stats *stats)

Get ZMQ export stats

Parameters
  • table: The flow table handle.
  • stats: The ZMQ stats (out).

void pfring_ft_set_default_action(pfring_ft_table *table, pfring_ft_action action)

Set the default action for detected L7 protocols with no filtering rule. This can be used to ‘drop all’ traffic, exception made for specific protocols setting the default to PFRING_FT_ACTION_DISCARD and filter actions to PFRING_FT_ACTION_FORWARD Default: PFRING_FT_ACTION_DEFAULT

Parameters

int pfring_ft_load_configuration(pfring_ft_table *table, const char *path)

Load filtering/shunting rules from a configuration file. Please refer to the documentation for the file format.

Return
0 on success, a negative number on failures.
Parameters
  • table: The flow table handle.
  • path: The configuration file path.

int pfring_ft_load_configuration_ext(pfring_ft_table *table, const char *path, pfring_ft_flow_filter *filter)

Load filtering/shunting rules from a configuration file to an external pfring_ft_flow_filter handle. Please refer to the documentation for the file format.

Return
0 on success, a negative number on failures.
Parameters
  • table: The flow table handle.
  • path: The configuration file path.
  • filter: The destination pfring_ft_flow_filter handle.

void pfring_ft_set_shunt_protocol_by_name(pfring_ft_table *table, const char *protocol_name, u_int8_t packets)

Set a shunt rule for a L7 protocol.

Parameters
  • table: The flow table handle.
  • protocol_name: The nDPI protocol name.
  • packets: The number of packets before shunting the flow returning a discard action from pfring_ft_process().

void pfring_ft_set_filter_all_protocols(pfring_ft_table *table, pfring_ft_action action)

Set a default action for all L7 protocols. This is usually used to reset all filtering rules by passing PFRING_FT_ACTION_DEFAULT as action.

Parameters
  • table: The flow table handle.
  • action: The action to set for all protocols.

void pfring_ft_set_filter_protocol_by_name(pfring_ft_table *table, const char *protocol_name, pfring_ft_action action)

Set a filtering rule for a L7 protocol.

Parameters
  • table: The flow table handle.
  • protocol_name: The nDPI protocol name.
  • action: The action returned by pfring_ft_process() for all packets matching the protocol.

char *pfring_ft_l7_protocol_name(pfring_ft_table *table, pfring_ft_ndpi_protocol *protocol, char *buffer, int buffer_len)

Return the L7 protocol name providing the nDPI protocol ID.

Return
The buffer.
Parameters
  • table: The flow table handle.
  • protocol: The nDPI protocol ID.
  • buffer: The output buffer.
  • buffer_len: The output buffer length.

u_int16_t pfring_ft_l7_protocol_id(pfring_ft_table *table, const char *name)

Return the nDPI L7 protocol ID providing the L7 protocol name.

Return
The nDPI protocol ID.
Parameters
  • table: The flow table handle.
  • name: The L7 protocol name.

int pfring_ft_set_ndpi_handle(pfring_ft_table *table, struct ndpi_detection_module_struct *ndpi)

Set the nDPI handle. This is meant to be used for custom nDPI settings only, as FT already creates a nDPI instance internally when using PFRING_FT_TABLE_FLAGS_DPI. FT takes care of releasing the nDPI instance on pfring_ft_destroy_table.

Return
0 on success, a negative number on failures.
Parameters
  • table: The flow table handle.
  • ndpi: The nDPI handle.

struct ndpi_detection_module_struct *pfring_ft_get_ndpi_handle(pfring_ft_table *table)

Return the nDPI handle.

Return
The nDPI handle, NULL if there is no handle.
Parameters
  • table: The flow table handle.

int pfring_ft_load_ndpi_protocols(pfring_ft_table *table, const char *path)

Load custom nDPI protocols from a configuration file. Please refer to the nDPI documentation for the file format. Example: https://github.com/ntop/nDPI/blob/dev/example/protos.txt

Return
0 on success, a negative number on failures.
Parameters
  • table: The flow table handle.
  • path: The configuration file path.

int pfring_ft_load_ndpi_categories(pfring_ft_table *table, const char *path)

Load nDPI categories (defined by hostname) from a configuration file. Please refer to the nDPI documentation for the file format. Example: https://github.com/ntop/nDPI/blob/dev/example/mining_hosts.txt

Return
0 on success, a negative number on failures.
Parameters
  • table: The flow table handle.
  • path: The configuration file path.

int pfring_ft_is_ndpi_available()

Check if nDPI is available.

Return
1 if nDPI is available, 0 otherwise.

pfring_ft_stats *pfring_ft_get_stats(pfring_ft_table *table)

Get flow processing statistics.

Return
The stats struct.
Parameters
  • table: The flow table handle.

void pfring_ft_version(char *version)

Get the PF_RING FT version.

Parameters
  • version: A buffer (32 bytes long) where version is returned. (out)

u_int32_t pfring_ft_api_version()

Get the PF_RING FT API version.

Return
The version number as unsigne inte.

int pfring_ft_license(char *system_id, time_t *license_expiration, time_t *maintenance_expiration)

Get license info.

Return
1 if a valid license is installed, 0 otherwise.
Parameters
  • system_id: A buffer (48 bytes long) where system id is returned. (out)
  • license_expiration: A pointer to a time_t where license expiration is returned. (out)
  • maintenance_expiration: A pointer to a time_t where maintenance expiration is returned. (out)

int pfring_ft_set_license(const char *license_key)

Install a PF_RING FT license key.

Return
1 if the license has been successfully installed, 0 otherwise (e.g. no permissions).
Parameters
  • license_key: The license key.

void pfring_ft_debug(void)

Enable debug mode

struct pfring_ft_flow_filter

Public Members

u_int32_t num_protocols

Number of supported L7 protocols

pfring_ft_action *protocol_to_action

Action per L7 protocol

struct pfring_ft_flow_filter::[anonymous] match
u_int8_t default_npkts

Default number of packets to forward

u_int8_t tcp_npkts

Number of packets to forward in case of TCP

u_int8_t udp_npkts

Number of packets to forward in case of UDP

u_int8_t *protocol_to_npkts

Number of packets to forward per L7 protocol

struct pfring_ft_flow_filter::[anonymous] shunt
struct pfring_ft_pcap_pkthdr

Public Members

struct timeval ts

time stamp

u_int32_t caplen

length of captured portion

u_int32_t len

length original packet (off wire)

struct pfring_ft_ext_pkthdr

Public Members

u_int32_t hash

packet hash

u_int16_t device_id

Source device ID

u_int8_t port_id

Source device port ID

u_int8_t reserved

Padding

struct pfring_ft_packet_metadata

Public Members

pfring_ft_pcap_pkthdr *hdr
pfring_ft_ext_pkthdr *ext_hdr
pfring_ft_direction direction
u_int16_t vlan_id
u_int8_t ip_version
u_int8_t l4_proto
u_int16_t payload_len
u_int16_t reserved
pfring_ft_iphdr *ip4
pfring_ft_ipv6hdr *ip6
union pfring_ft_packet_metadata::[anonymous] l3
pfring_ft_tcphdr *tcp
pfring_ft_udphdr *udp
union pfring_ft_packet_metadata::[anonymous] l4
const u_char *payload
union pfring_ft_ip_address

Public Members

pfring_ft_in4_addr v4
pfring_ft_in6_addr v6
struct pfring_ft_ndpi_protocol

Public Members

u_int16_t master_protocol

e.g. HTTP

u_int16_t app_protocol

e.g. FaceBook

int category
struct pfring_ft_flow_key

Public Members

u_int8_t smac[6]

Source MAC

u_int8_t dmac[6]

Destination MAC

pfring_ft_ip_address saddr

Source IP address (HBO)

pfring_ft_ip_address daddr

Destination IP address (HBO)

u_int8_t ip_version

IP version

u_int8_t protocol

L4 protocol

u_int16_t sport

Source port (HBO)

u_int16_t dport

Destination port (HBO)

u_int16_t vlan_id

VLAN ID (HBO)

struct pfring_ft_flow_dir_value

Public Members

u_int64_t pkts

Number of packets per direction

u_int64_t bytes

Number of bytes per direction

struct timeval first

Time of first packet seen per direction

struct timeval last

Time of last packet seen per direction

u_int8_t tcp_flags

TCP flags per direction

u_int8_t port_id

Port ID (when provided e.g. MetaWatch)

u_int16_t device_id

Device ID (when provided e.g. MetaWatch)

struct pfring_ft_flow_value

Public Members

pfring_ft_flow_dir_value direction[PF_RING_FT_FLOW_NUM_DIRECTIONS]

Metadata per flow direction

pfring_ft_ndpi_protocol l7_protocol

nDPI protocol

u_int32_t tunnel_type

nDPI tunnel type (ndpi_packet_tunnel)

u_int32_t tunnel_id

Tunnel ID (if any)

char *query

DNS query

u_int16_t queryType

DNS query type

u_int16_t replyCode

DNS reply code

struct pfring_ft_flow_value::[anonymous]::[anonymous] dns
char *serverName

TLS Server Name

HTTP Server Name

u_int8_t *sha1_certificate_fingerprint

SHA-1 Certificate Fingerprint (20-bytes)

struct pfring_ft_flow_value::[anonymous]::[anonymous] tls
char *url

HTTP URL

u_int16_t responseCode

HTTP response code

struct pfring_ft_flow_value::[anonymous]::[anonymous] http
u_int8_t type

ICMP Type

u_int8_t code

ICMP Code

struct pfring_ft_flow_value::[anonymous]::[anonymous] icmp
union pfring_ft_flow_value::[anonymous] l7_metadata
pfring_ft_flow_status status
u_int32_t flags

See PFRING_FT_FLOW_STATUS_*

u_char *user

User metadata: this points to the end of the same struct usually. In case of flow slice this points to the original flow’s user data.

struct pfring_ft_stats

Public Members

u_int64_t active_flows

Number of currently active flows

u_int64_t flows

Number of total flows

u_int64_t err_no_room

Flow creation errors due to no room left in the flow table

u_int64_t err_no_mem

Flow creation errors due to memory allocation failures

u_int64_t disc_no_ip

Number of packets not processed because L3 header was missing

u_int64_t max_lookup_depth

Maximum collition list depth during flow lookup

u_int64_t packets

Number of processed packets

u_int64_t bytes

Total number of packet bytes

struct pfring_ft_export_stats

Public Members

u_int32_t num_buffered_records
u_int32_t exported_flows
u_int32_t dropped_flows