h_ertt(4) Enhanced Round Trip Time Khelp module


In netinet/khelp/h_ertt.h


The Khelp module works within the khelp(9) framework to provide TCP with a per-connection, low noise estimate of the instantaneous RTT. The implementation attempts to be robust in the face of delayed acknowledgements, TCP Segmentation Offload (TSO), receivers who manipulate TCP timestamps and lack of the TCP timestamp option altogether.

TCP receivers using delayed acknowledgements either acknowledge every second packet (reflecting the time stamp of the first) or use a timeout to trigger the acknowledgement if no second packet arrives. If the heuristic used by determines that the receiver is using delayed acknowledgements, it measures the RTT using the second packet (the one that triggers the acknowledgement). It does not measure the RTT if the acknowledgement is for the first packet, since it cannot be accurately determined.

When TSO is in use, will momentarily disable TSO whilst marking a packet to use for a new measurement. The process has negligible impact on the connection.

associates the following struct with each connection's TCP control block:

struct ertt {
        TAILQ_HEAD(txseginfo_head, txseginfo) txsegi_q; /* Private. */
        long            bytes_tx_in_rtt;                /* Private. */
        long            bytes_tx_in_marked_rtt;
        unsigned long   marked_snd_cwnd;
        int             rtt;
        int             maxrtt;
        int             minrtt;
        int             dlyack_rx;                      /* Private. */
        int             timestamp_errors;               /* Private. */
        int             markedpkt_rtt;                  /* Private. */
        uint32_t        flags;

The fields marked as private should not be manipulated by any code outside of the implementation. The non-private fields provide the following data:

The number of bytes transmitted in the markedpkt_rtt
The value of cwnd for the marked rtt measurement.
The most recent RTT measurement.
The longest RTT measurement that has been taken.
The shortest RTT measurement that has been taken.
The ERTT_NEW_MEASUREMENT flag will be set by the implementation when a new measurement is available. It is the responsibility of consumers to unset the flag if they wish to use it as a notification method for new measurements.


Development and testing of this software were made possible in part by grants from the FreeBSD Foundation and Cisco University Research Program Fund at Community Foundation Silicon Valley.


The module first appeared in Fx 9.0 .

The module was first released in 2010 by David Hayes whilst working on the NewTCP research project at Swinburne University of Technology's Centre for Advanced Internet Architectures, Melbourne, Australia. More details are available at:



An -nosplit The Khelp module and this manual page were written by An David Hayes Aq [email protected] .


The module maintains enhanced RTT estimates for all new TCP connections created after the time at which the module was loaded. It might be beneficial to see if it is possible to have the module only affect connections which actually care about ERTT estimates.