nfstream: Flexible Network Data Analysis Framework

CyberPunk Network Tools

Last Release: 04/27/2022 Last Commit: 04/28/2022

nfstream is a python framework for network data analysis. It provides fast, expressive and flexible data structures that are designed to work with offline or online network data. nfstream is a high-level building block with a fundamental structure and can perform real-world and practical analysis of network data using Python.

nfstream: Flexible Network Data Analysis Framework

This framework works as a high-level building block to carry out a flexible analysis of network data. It has expressive, flexible and fast data structures to handle the online or offline network data. nfstream has the abilities and aims to become the most common processing framework to perform network data analysis for researchers and provide reproducibility of data during experiments.

How it works?

Packet observation is the starting point in the monitoring architecture. In this phase, the packet is captured at the Network Interface card (NIC). Some checks like checksum errors are performed after passing.

Packet’s timestamp is based on the reordering process and numerous observation points. With hardware, it can calculate highly accurate timestamping up to 100 nanoseconds.

Flow metering involves the management of the expiration of entry and aggregation of packets into flows. During a metering process, a packet is added into flow entry with a defined key. A table is maintained during flow metering called flow cache. It contains information about the metering process of active flows in the network. A flow key contains IP addresses of source, destination and ports of sender including receiver and protocols.

Features:

Performance: nfstream performs fast as 10 times faster due to pypy3 support. A memory footprint and a small CPU enhance the performing ability of the nfstream.
Layer-7 visibility: deep packet inspection engine for depends on the nDPI. nfstream performs reliable data extraction and identification of encrypted applications.
Flexibility: it provides more flexibility due to NFPlugin.
Machine Learning oriented: Add a trained model as NFPlugin.

Supported Platforms:

Linux, MacOS

Prerequisites:

Check requirements.txt

nfstream Install

Using pip

Install the latest version of nfstream using binary installers:

$ python3 -m pip install nfstream

From source

Linux:

To build from source, run the following commands:

$ sudo apt-get install autoconf automake libtool pkg-config libpcap-dev
$ git clone https://github.com/aouinizied/nfstream.git

Then now go to nfstream directory and run:

$ cd nfstream 
$ python3 -m pip install -r requirements.txt
$ python3 setup.py bdist_wheel

MacOS:

$ brew install autoconf automake libtool pkg-config
$  git clone https://github.com/aouinizied/nfstream.git
$  cd nfstream
$  python3 -m pip install -r requirements.txt
$  python3 setup.py bdist_wheel