netpack-gpack is released

netpack (gpack package) is useful to measure and track traffic through your site, using inherently libpcap and its own wrapped interface named gpcap.

It includes:
* gPackCount - the Packet counter utility
Shows IP packet structure, especially TCP details.

It uses gobj, gobjeio and gpcap libraries, including in this package.


REQUIREMENTS
++++++++++++
* compiles in Linux; requires g++, make, flex and bison
* run gpackcount as root (or otherwise just an interface test mode)


Main site:
http://moreira.dnsalias.org/projects/netpack/

Mirror:
http://millis.org.uk/projects/netpack/

A sample of netpack in Debian sarge

For those who cannot wait for an official package of Debian, here goes a nice how-to suited for guys with some experience on Linux.
Requirement: libpcap0.7; your Kernel support (CONFIG_PACKET, for Kernels >= 2.2, according green report below; or attached an interface in promiscuous mode on Kernel < 2.2).

frog:~# grep CONFIG_PACKET=[my] /boot/config-2.6.8-2-386 ; [ $? = 0 ] && echo "Kernel `uname -a`: Supports pcap"
CONFIG_PACKET=m
Kernel Linux frog 2.6.8-2-386 #1 Thu May 19 17:40:50 JST 2005 i686 GNU/Linux: Supports pcap

Check the installed libpcap0.7:

frog:~# dpkg -l | grep pcap
ii libpcap0.7 0.7.2-7 System interface for user-level packet captu
ii libpcap0.7-dev 0.7.2-7 Development library and header files for lib
ii libpcap0.8 0.9.5-1 System interface for user-level packet captu
frog:~# ldconfig -p | grep pcap
libpcap.so.0.8 (libc6) => /usr/lib/libpcap.so.0.8
libpcap.so.0.7 (libc6) => /usr/lib/libpcap.so.0.7
frog:~# file ~guest/gpackcount_static
/home/guest/gpackcount_static: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.4.1, dynamically linked (uses shared libs), for GNU/Linux 2.4.1, stripped

For compiling netpack package by your own you will need libpcap0.7-dev, but if you download a binary package of netpack you will not need this.
Supose your Debian box is connecting a share in a windowzed host, via Samba-client.

frog:~# mount -t smbfs
//luisa/huge on /net/luisa/huge type smbfs (rw)

This indicates a share called huge is mounted locally at /net/luisa/huge. You can sniff the raw-text transfer protocol (NetBIOS, in particular microsoft-ds) using the following command:

~guest/gpackcount_static packets -g 445,0,0,445 -v

In my case the windowzed PC has a replica of my Freesco-box (moreira.dnsalias.org/linux), and I synch information regularly from the Freesco-box to this PC using a Debian guest (named frog). I use rsync (that is actually also installed at the Freesco-box, though this is only a nice-to-have: it improves even more the transfer speed) -- rsync is the Rolls Royce of a synchronization tool. It also allows compressed file-transfer (using '-z'). In the following example you can see a folder named 'linux' transferred to the PC using the Samba share.

frog:~# grep smbfs /etc/fstab
//luisa/huge /net/luisa/huge smbfs rw,noauto,username=henrique,password=your_windowzed_pass 0 0
frog:~# mount /net/luisa/huge
frog:~# cd /net/luisa/huge/pub/public_html/org/linux
frog:/net/luisa/huge/pub/public_html/org/linux# rsync -v -v -t -r --stats root@maxtor:/www/org/linux . 2> /tmp/rep

The bold line is the customized Samba client I have configured on the Debian host.
Be sure to have the 'smbfs' package installed, otherwise the mount command will fail (apt-get install smbfs).

Note: I am currently a little bit skeptic about the 'lfs' option on the Samba client.

The rsync command is shown above.

The gpackcount will show the file-transfer contents, excluding the mounting.

History about netpack

How netpack started

Since I have used Freesco (and Kernel provided by it, 2.0.x, doesn't support interface statistics per byte), I started using iptotal. It provided a nice output, by using cgi and rrdtool (a database supporting round-robin data). You can see those graphs on the following link (if my own freesco-box is down by some reason)Quickly I started to enjoy the way iptotal was able to get the octets the Kernel processed, even without direct Kernel support. I did learn about libpcap, and suddenly started to enjoy immensely all things we could do by using this magic library. Nothing about magic, really: it was just using the Kernel Netlink capabilities (Netlink socket is a special IPC used for transferring information between kernel and user-space processes; comprehensive details at the Linux Journal; an interesting genric idea is also published at Wikipedia). pcap stands for Packet Capture library, and is has a quite generic implementation of a packet sniffer and filter, the latter also known as net/bpf, or just BPF. I put BPF aside, since I was not interested in filtering packets on the Kernel level. That would be too much for my knowledge and out of scope.
I become really interested in the ways I could snif data through my own network: that way I could control better the trojans and parasites around -- since wasting money to buy an anti-virus was out of the question.
So I have learn the basic hooks necessary to bring pcap alive: I got amazed by the knowledge required to do such a task. Tricks and adjustments, just to capture a bunch of packets from the Kernel!... [sigh] I thought the best would be to wrap this knowledge in a simple and straight forward interface -- this and a new library (called gpcap) was one step.
The library interfaced libpcap in a simple way for any end-programmer: and I chose C++, to provide an adjusted encapsulation. From gpcap to a small program (called gPackCount) to test it was another step. The README contained the following paragraphs:

There is currently limited functionality provided by *gpackcount*
because this was my start application for testing *gpcap*
...
It uses mainly pcap and gpcap libraries. pcap is GPL (not my own
product), and gpcap is a wrapper for easing the implemented pcap
interfaces. pcap is out of scope in this document, but the basic
idea is to capture packets using either the Kernel CONFIG_PACKET
(PF_PACKET Kernel >=2.2), or attach an interface in promiscuous
mode (Kernel <2.2).

Then gPackCount has grown to comprise more and more services, and includes a self-contained rDNS-cache (with a proprietary caching algorithm). Currently (as of version 0.8) gPackCount has 4932 lines of code, plus three basic libraries:

• libgobj (the basic object handling; lists, etc)
  
• libgobjeio (for the extended generic I/O services)
• libgpcap (pcap eased interface)
  

The gpackcount binary can also be statically linked (RedHat 7.3 version has around 430kb).
Around this utility, the 'NetPack' package was formed as a set of useful services, explained shortly below.

NetPack is a package of the following functionality:

* libgpcap (wrapped pcap functionality, using also libgobj)
DIR: ../.../gprog/gsrc/gpcap

* Autonomous System (AS) tables builder: one .asg and many .ast tables
DIR: gAstCompile

* Packet counter utility
Shows IP packet structure, moreover shows TCP details.
This suits also testing of libgpcap
DIR: gPackCount

The TCP format

TCP Segment Format

Field Name	Size (bytes)	Description
Source Port	2	Source Port: The 16-bit port number of the process that originated the TCP segment on the source device.
Destination Port	2	Destination Port: The 16-bit port number of the process that is the ultimate intended recipient of the message on the destination device. This will usually be a well-known / registered port number for a client request, or a temporary port number in the case of server replies.
Sequence Number	4	Sequence Number: For normal transmissions, the sequence number of the first byte of data in this segment. In a connection request ( SYN ) message, this carries the initial sequence number (ISN) of the source TCP. The first byte of data will be given the next sequence number after the contents of this field, as described in the sequence number synchronization.
Acknowledgment Number	4	Acknowledgment Number: When the ACK bit is set, this segment is serving as an acknowledgment (in addition to other possible duties) and this field contains the sequence number the source is next expecting the destination to send.
Data Offset	1/2 (4 bits)	Data Offset: Specifies the number of 32-bit words of data in the TCP header. In other words, this value times four equals the number of bytes in the header, which must always be a multiple of four. It is called a “data offset” since it indicates by how many 32-bit words the start of the data is offset from the beginning of the TCP segment.
Reserved	3/4 (6 bits)	Reserved: 6 bits reserved for future use; sent as zero.
Control Bits	3/4 (6 bits)
Window	2	Window: Indicates the number of octets of data the sender of this segment is willing to accept from the receiver at one time. This normally corresponds to the current size of the buffer allocated to accept data for this connection. This field is, in other words, the current receive window size for the device sending this segment, which is also the send window for the recipient of the segment.
Checksum	2	Checksum: A 16-bit checksum for data integrity protection, computed over the entire TCP datagram plus a special “pseudo header” of fields. It is used to protect the entire TCP segment against not just errors in transmission, but also errors in delivery. Optional alternate checksum methods are also supported.
Urgent Pointer	2	Urgent Pointer: Used in conjunction with the URG control bit for priority data transfer. This field contains the sequence number of the last byte of urgent data.
Options	Variable
Padding	Variable	Padding: If the Options field is not a multiple of 32 bits in length, enough zeroes are added to pad the header so it is a multiple of 32 bits.
Data	Variable	Data: The bytes of data being sent in the segment.

This first article is only about the TCP segment format.
The netpack package is all focus on TCP (not UDP).