4. PF_RING FT API

PF_RING FT library header file.

Defines

FT_API_VERSION

PFRING_FT_ACTION_DEFAULT

PFRING_FT_ACTION_FORWARD

PFRING_FT_ACTION_DISCARD: Discard packet due to filter or shunt

PFRING_FT_ACTION_USER_1

PFRING_FT_ACTION_USER_2

PFRING_FT_ACTION_SLICE: Slice packet headers

PF_RING_FT_FLOW_FLAGS_L7_GUESS: pfring_ft_flow_value.flags: detected L7 protocol is a guess.

PFRING_FT_TABLE_FLAGS_DPI: pfring_ft_create_table() flag: enable nDPI support for L7 protocol detection

PFRING_FT_TABLE_FLAGS_DPI_EXTRA: pfring_ft_create_table() flag: enable nDPI extra dissection (more flow metadata)

PFRING_FT_DECODE_TUNNELS: pfring_ft_create_table() flag: decode tunnels (GTP, L2TP, CAPWAP)

PFRING_FT_IGNORE_HW_HASH: pfring_ft_create_table() flag: ignore hw packet hash (e.g. when it’s asymmetric leading to one flow per direction)

PFRING_FT_IGNORE_VLAN: pfring_ft_create_table() flag: do not include vlan in flow key

PFRING_FT_TABLE_FLAGS_NO_GUESS: pfring_ft_create_table() flag: do not guess the protocol when not detected with DPI

Typedefs

typedef void pfring_ft_table

typedef void pfring_ft_list

typedef void pfring_ft_flow

typedef u_int8_t pfring_ft_action

typedef u_int32_t pfring_ft_in4_addr

typedef void (*pfring_ft_export_list_func)(pfring_ft_list *flows_list, void *user)

typedef void (*pfring_ft_export_flow_func)(pfring_ft_flow *flow, void *user)

typedef void (*pfring_ft_flow_packet_func)(const u_char *data, pfring_ft_packet_metadata *metadata, pfring_ft_flow *flow, void *user)

Enums

enum pfring_ft_direction

Values:

enumerator s2d_direction: Source to destination

enumerator d2s_direction: Destination to source

enumerator PF_RING_FT_FLOW_NUM_DIRECTIONS

enum pfring_ft_flow_status

Values:

enumerator PFRING_FT_FLOW_STATUS_ACTIVE: Active flow

enumerator PFRING_FT_FLOW_STATUS_IDLE_TIMEOUT: Idle timeout

enumerator PFRING_FT_FLOW_STATUS_ACTIVE_TIMEOUT: Terminated after the maximum lifetime

enumerator PFRING_FT_FLOW_STATUS_END_DETECTED: Terminated for end of flow (e.g. FIN)

enumerator PFRING_FT_FLOW_STATUS_FORCED_END: Terminated for external event (shutdown)

enumerator PFRING_FT_FLOW_STATUS_SLICE_TIMEOUT: Flow slice timeout

enumerator PFRING_FT_FLOW_STATUS_OVERFLOW: Exported from those with higher inactivity to make room

Functions

pfring_ft_table *pfring_ft_create_table(u_int32_t flags, u_int32_t max_flows, u_int32_t flow_idle_timeout, u_int32_t flow_lifetime_timeout, u_int32_t user_metadata_size)

Create a new flow table.

Parameters:

flags – Flags to enable selected flow table features.
max_flows – Maximum number of concurrent flows the table should be able to handle (use 0 if not sure to use default settings).
flow_idle_timeout – Maximum flow idle time (seconds) before expiration (use 0 if not sure to use default: 30s).
flow_lifetime_timeout – Maximum flow duration (seconds) before expiration (use 0 if not sure to use default: 2m).
user_metadata_size – Size of the user metadata in pfring_ft_flow_value->user

Returns:

The flow table on success, NULL on failure.

void pfring_ft_destroy_table(pfring_ft_table *table)

Destroy a flow table.

Parameters:: table – The flow table handle.

void pfring_ft_flow_set_flow_slicing(pfring_ft_table *table, u_int32_t flow_slice_timeout)

Enable flow slicing to peridiocally export flow updates, even when the configured flow_lifetime_timeout is not reached.

Parameters:

table – The flow table handle.
flow_slice_timeout – Maximum flow slice duration (seconds). This should be lower then flow_lifetime_timeout

void pfring_ft_set_new_flow_callback(pfring_ft_table *table, pfring_ft_export_flow_func callback, void *user)

Set the function to be called when a new flow has been created.

Parameters:

table – The flow table handle.
callback – The callback.
user – The user data provided to the callback.

void pfring_ft_set_flow_packet_callback(pfring_ft_table *table, pfring_ft_flow_packet_func callback, void *user)

Set the function to be called when a packet and its flow have been processed, for each packet.

Parameters:

table – The flow table handle.
callback – The callback.
user – The user data provided to the callback.

void pfring_ft_set_l7_detected_callback(pfring_ft_table *table, pfring_ft_flow_packet_func callback, void *user)

Set the function to be called when a packet and its flow have been processed and the l7 protocol has been just detected.

Parameters:

table – The flow table handle.
callback – The callback (Note: packet/metadata may be NULL).
user – The user data provided to the callback.

void pfring_ft_set_flow_export_callback(pfring_ft_table *table, pfring_ft_export_flow_func callback, void *user)

Set the function to be called when a flow expires and needs to be exported. The callback should release the flow calling pfring_ft_flow_free(flow).

Parameters:

table – The flow table handle.
callback – The callback.
user – The user data provided to the callback.

void pfring_ft_set_flow_list_export_callback(pfring_ft_table *table, pfring_ft_export_list_func callback, void *user)

Set the function to be called when a some flow expires and need to be exported. This can be used as an optimised alternative to pfring_ft_set_flow_export_callback(). The callback should release all flows in the list calling pfring_ft_flow_free(flow) for each flow. It is possible to iterate all the flows in the list using pfring_ft_list_get_next().

Parameters:

table – The flow table handle.
callback – The callback.
user – The user data provided to the callback.

pfring_ft_action pfring_ft_process(pfring_ft_table *table, const u_char *packet, const pfring_ft_pcap_pkthdr *header, const pfring_ft_ext_pkthdr *ext_header)

Provide a raw packet to the flow table for processing. Usually the main capture loop provides all the packets to the hash table calling this function.

Parameters:

table – The flow table handle.
packet – The raw packet.
header – The packet metadata (including length and timestamp).
ext_header – Additional packet metadata not available in the pcap header (including hash).

Returns:

The action for the packet, in case filtering rules have been specified.

int pfring_ft_housekeeping(pfring_ft_table *table, u_int32_t epoch)

This should be called when there is no packet to be processed and the main loop is idle, for running housekeeping activities in the flow table.

Parameters:

table – The flow table handle.
epoch – The current epoch (sec).

Returns:

1 if there is more work to do, 0 if the caller can sleep a bit.

void pfring_ft_flush(pfring_ft_table *table)

Flush all flows (usually called on program termination, before destroying the flow table).

Parameters:: table – The flow table handle.

pfring_ft_flow *pfring_ft_list_get_next(pfring_ft_list *list)

Pop the next from a flow list.

Parameters:: list – The flow list.
Returns:: The flow if the list is not empty, NULL otherwise.

u_int64_t pfring_ft_flow_get_id(pfring_ft_flow *flow)

Get the flow ID.

Parameters:: flow – The flow handle.
Returns:: The flow ID.

pfring_ft_flow_key *pfring_ft_flow_get_key(pfring_ft_flow *flow)

Get the flow key.

Parameters:: flow – The flow handle.
Returns:: The flow key.

pfring_ft_flow_value *pfring_ft_flow_get_value(pfring_ft_flow *flow)

Get the flow value.

Parameters:: flow – The flow handle.
Returns:: The flow value.

struct ndpi_flow_struct *pfring_ft_flow_get_ndpi_handle(pfring_ft_flow *flow)

Get the nDPI flow handle.

Parameters:: flow – The flow handle.
Returns:: The flow value.

void pfring_ft_flow_set_action(pfring_ft_flow *flow, pfring_ft_action action)

Set the flow action, to be returned by pfring_ft_process() for all packets for this flow.

Parameters:

flow – The flow handle.
action – The action.

pfring_ft_action pfring_ft_flow_get_action(pfring_ft_flow *flow)

Get the computed/actual flow action, the same returned by pfring_ft_process() for this flow.

Parameters:: flow – The flow handle.
Returns:: The action.

int pfring_ft_flow_get_users(pfring_ft_flow *flow)

Return the number of users for the flow (value of the reference counter). This is usually 1, unless slicing is enabled (+1 for each slice not yet released). Calling this on the slice, returns the reference counter of the master flow.

Parameters:: flow – The flow handle.

void pfring_ft_flow_free(pfring_ft_flow *flow)

Release a flow.

Parameters:: flow – The flow handle.

void pfring_ft_zmq_export_configure(pfring_ft_table *table, const char *endpoint, const char *server_public_key, u_int8_t probe_mode, u_int8_t disable_compression, u_int8_t use_json)

Configure ZMQ flow export (see pfring_ft_zmq_export_flow)

Parameters:

table – The flow table handle.
endpoint – The ZMQ endpoint.
server_public_key – The ZMQ Public encryption key (NULL for clear).
probe_mode – Probe mode (connect to the ZMQ collector).
disable_compression – Disable message compression.
use_json – Use JSON format (Default: TLV).

void pfring_ft_zmq_export_flow(pfring_ft_flow *flow, void *user)

Built-in callback to be provided to pfring_ft_set_flow_export_callback for exporting flows in JSON or TLV format to ZMQ. This implements pfring_ft_export_flow_func. The ZMQ endpoint should be configure with pfring_ft_zmq_export_configure(). The callback also releases the flow calling pfring_ft_flow_free(flow). Usage: pfring_ft_set_flow_export_callback(table, pfring_ft_zmq_export_flow, table);

Parameters:

flow – The flow to be exported.
user – The flow table handle.

void pfring_ft_zmq_export_stats(pfring_ft_table *table, const char *if_name, u_int16_t if_speed, const char *if_ip, const char *management_ip)

Export stats via ZMQ

Parameters:

table – The flow table handle.
if_name – Interface name.
if_speed – Interface speed (Mbps).
if_ip – Interface IP.
management_ip – Management Interface IP.

void pfring_ft_zmq_get_stats(pfring_ft_table *table, pfring_ft_export_stats *stats)

Get ZMQ export stats

Parameters:

table – The flow table handle.
stats – The ZMQ stats (out).

void pfring_ft_set_default_action(pfring_ft_table *table, pfring_ft_action action)

Set the default action for detected L7 protocols with no filtering rule. This can be used to ‘drop all’ traffic, exception made for specific protocols setting the default to PFRING_FT_ACTION_DISCARD and filter actions to PFRING_FT_ACTION_FORWARD Default: PFRING_FT_ACTION_DEFAULT

Parameters:

table – The flow table handle.
action – The action returned by pfring_ft_process() by default.

int pfring_ft_load_configuration(pfring_ft_table *table, const char *path)

Load filtering/shunting rules from a configuration file. Please refer to the documentation for the file format.

Parameters:

table – The flow table handle.
path – The configuration file path.

Returns:

0 on success, a negative number on failures.

int pfring_ft_load_configuration_ext(pfring_ft_table *table, const char *path, pfring_ft_flow_filter *filter)

Load filtering/shunting rules from a configuration file to an external pfring_ft_flow_filter handle. Please refer to the documentation for the file format.

Parameters:

table – The flow table handle.
path – The configuration file path.
filter – The destination pfring_ft_flow_filter handle.

Returns:

0 on success, a negative number on failures.

void pfring_ft_set_shunt_protocol_by_name(pfring_ft_table *table, const char *protocol_name, u_int8_t packets)

Set a shunt rule for a L7 protocol.

Parameters:

table – The flow table handle.
protocol_name – The nDPI protocol name.
packets – The number of packets before shunting the flow returning a discard action from pfring_ft_process().

void pfring_ft_set_filter_all_protocols(pfring_ft_table *table, pfring_ft_action action)

Set a default action for all L7 protocols. This is usually used to reset all filtering rules by passing PFRING_FT_ACTION_DEFAULT as action.

Parameters:

table – The flow table handle.
action – The action to set for all protocols.

void pfring_ft_set_filter_protocol_by_name(pfring_ft_table *table, const char *protocol_name, pfring_ft_action action)

Set a filtering rule for a L7 protocol.

Parameters:

table – The flow table handle.
protocol_name – The nDPI protocol name.
action – The action returned by pfring_ft_process() for all packets matching the protocol.

char *pfring_ft_l7_protocol_name(pfring_ft_table *table, pfring_ft_ndpi_protocol *protocol, char *buffer, int buffer_len)

Return the L7 protocol name providing the nDPI protocol ID.

Parameters:

table – The flow table handle.
protocol – The nDPI protocol ID.
buffer – The output buffer.
buffer_len – The output buffer length.

Returns:

The buffer.

u_int16_t pfring_ft_l7_protocol_id(pfring_ft_table *table, const char *name)

Return the nDPI L7 protocol ID providing the L7 protocol name.

Parameters:

table – The flow table handle.
name – The L7 protocol name.

Returns:

The nDPI protocol ID.

int pfring_ft_set_ndpi_handle(pfring_ft_table *table, struct ndpi_detection_module_struct *ndpi)

Set the nDPI handle. This is meant to be used for custom nDPI settings only, as FT already creates a nDPI instance internally when using PFRING_FT_TABLE_FLAGS_DPI. FT takes care of releasing the nDPI instance on pfring_ft_destroy_table.

Parameters:

table – The flow table handle.
ndpi – The nDPI handle.

Returns:

0 on success, a negative number on failures.

struct ndpi_detection_module_struct *pfring_ft_get_ndpi_handle(pfring_ft_table *table)

Return the nDPI handle.

Parameters:: table – The flow table handle.
Returns:: The nDPI handle, NULL if there is no handle.

int pfring_ft_load_ndpi_protocols(pfring_ft_table *table, const char *path)

Load custom nDPI protocols from a configuration file. Please refer to the nDPI documentation for the file format. Example: https://github.com/ntop/nDPI/blob/dev/example/protos.txt

Parameters:

table – The flow table handle.
path – The configuration file path.

Returns:

0 on success, a negative number on failures.

int pfring_ft_load_ndpi_categories(pfring_ft_table *table, const char *path)

Load nDPI categories (defined by hostname) from a configuration file. Please refer to the nDPI documentation for the file format. Example: https://github.com/ntop/nDPI/blob/dev/example/mining_hosts.txt

Parameters:

table – The flow table handle.
path – The configuration file path.

Returns:

0 on success, a negative number on failures.

int pfring_ft_is_ndpi_available()

Check if nDPI is available.

Returns:: 1 if nDPI is available, 0 otherwise.

pfring_ft_stats *pfring_ft_get_stats(pfring_ft_table *table)

Get flow processing statistics.

Parameters:: table – The flow table handle.
Returns:: The stats struct.

void pfring_ft_version(char *version)

Get the PF_RING FT version.

Parameters:: version – A buffer (32 bytes long) where version is returned. (out)

u_int32_t pfring_ft_api_version()

Get the PF_RING FT API version.

Returns:: The version number as unsigne inte.

int pfring_ft_license(char *system_id, time_t *license_expiration, time_t *maintenance_expiration)

Get license info.

Parameters:

system_id – A buffer (48 bytes long) where system id is returned. (out)
license_expiration – A pointer to a time_t where license expiration is returned. (out)
maintenance_expiration – A pointer to a time_t where maintenance expiration is returned. (out)

Returns:

1 if a valid license is installed, 0 otherwise.

int pfring_ft_set_license(const char *license_key)

Install a PF_RING FT license key.

Parameters:: license_key – The license key.
Returns:: 1 if the license has been successfully installed, 0 otherwise (e.g. no permissions).

void pfring_ft_debug(void): Enable debug mode

struct pfring_ft_flow_filter

Public Members

u_int32_t num_protocols: Number of supported L7 protocols

pfring_ft_action *protocol_to_action: Action per L7 protocol

struct pfring_ft_flow_filter::[anonymous] match

u_int8_t default_npkts: Default number of packets to forward

u_int8_t tcp_npkts: Number of packets to forward in case of TCP

u_int8_t udp_npkts: Number of packets to forward in case of UDP

u_int8_t *protocol_to_npkts: Number of packets to forward per L7 protocol

struct pfring_ft_flow_filter::[anonymous] shunt

__attribute__

struct pfring_ft_pcap_pkthdr

Public Members

struct timeval ts: time stamp

u_int32_t caplen: length of captured portion

u_int32_t len: length original packet (off wire)

struct pfring_ft_ext_pkthdr

Public Members

u_int32_t hash: packet hash

u_int16_t device_id: Source device ID

u_int8_t port_id: Source device port ID

u_int8_t reserved: Padding

struct pfring_ft_packet_metadata

Public Members

pfring_ft_pcap_pkthdr *hdr

pfring_ft_ext_pkthdr *ext_hdr

pfring_ft_direction direction

pfring_ft_action action

u_int16_t vlan_id

u_int8_t ip_version

u_int8_t l4_proto

u_int16_t payload_len

u_int16_t reserved

pfring_ft_iphdr *ip4

pfring_ft_ipv6hdr *ip6

union pfring_ft_packet_metadata::[anonymous] l3

pfring_ft_tcphdr *tcp

pfring_ft_udphdr *udp

union pfring_ft_packet_metadata::[anonymous] l4

const u_char *payload

union pfring_ft_ip_address

Public Members

pfring_ft_in4_addr v4

pfring_ft_in6_addr v6

struct pfring_ft_ndpi_protocol

Public Members

u_int16_t master_protocol: e.g. HTTP

u_int16_t app_protocol: e.g. FaceBook

int category

struct pfring_ft_flow_key

Public Members

u_int8_t smac[6]: Source MAC

u_int8_t dmac[6]: Destination MAC

pfring_ft_ip_address saddr: Source IP address (HBO)

pfring_ft_ip_address daddr: Destination IP address (HBO)

u_int8_t ip_version: IP version

u_int8_t protocol: L4 protocol

u_int16_t sport: Source port (HBO)

u_int16_t dport: Destination port (HBO)

u_int16_t vlan_id: VLAN ID (HBO)

struct pfring_ft_flow_dir_value

Public Members

u_int64_t pkts: Number of packets per direction

u_int64_t bytes: Number of bytes per direction

struct timeval first: Time of first packet seen per direction

struct timeval last: Time of last packet seen per direction

u_int8_t tcp_flags: TCP flags per direction

u_int8_t port_id: Port ID (when provided e.g. MetaWatch)

u_int16_t device_id: Device ID (when provided e.g. MetaWatch)

struct pfring_ft_flow_value

Public Members

pfring_ft_flow_dir_value direction[PF_RING_FT_FLOW_NUM_DIRECTIONS]: Metadata per flow direction

pfring_ft_ndpi_protocol l7_protocol: nDPI protocol

u_int32_t tunnel_type: nDPI tunnel type (ndpi_packet_tunnel)

u_int32_t tunnel_id: Tunnel ID (if any)

char *query: DNS query

u_int16_t queryType: DNS query type

u_int16_t replyCode: DNS reply code

struct pfring_ft_flow_value::[anonymous]::[anonymous] dns

char *serverName

TLS Server Name

HTTP Server Name

u_int8_t *sha1_certificate_fingerprint: SHA-1 Certificate Fingerprint (20-bytes)

struct pfring_ft_flow_value::[anonymous]::[anonymous] tls

char *url: HTTP URL

u_int16_t responseCode: HTTP response code

struct pfring_ft_flow_value::[anonymous]::[anonymous] http

u_int8_t type: ICMP Type

u_int8_t code: ICMP Code

struct pfring_ft_flow_value::[anonymous]::[anonymous] icmp

union pfring_ft_flow_value::[anonymous] l7_metadata

pfring_ft_flow_status status

u_int32_t flags: See PFRING_FT_FLOW_STATUS_*

u_char *user: User metadata: this points to the end of the same struct usually. In case of flow slice this points to the original flow’s user data.

struct pfring_ft_stats

Public Members

u_int64_t active_flows: Number of currently active flows

u_int64_t flows: Number of total flows

u_int64_t err_no_room: Flow creation errors due to no room left in the flow table

u_int64_t err_no_mem: Flow creation errors due to memory allocation failures

u_int64_t disc_no_ip: Number of packets not processed because L3 header was missing

u_int64_t max_lookup_depth: Maximum collition list depth during flow lookup

u_int64_t packets: Number of processed packets

u_int64_t bytes: Total number of packet bytes

struct pfring_ft_export_stats

Public Members

u_int32_t num_buffered_records

u_int32_t exported_flows

u_int32_t dropped_flows