man lmc (4): device driver for

SYNOPSIS

To wire this driver into your kernel, add the following line to your kernel configuration file:

device lmc

Alternatively, to load this module at boot time, add

if_lmc_load="YES"

to /boot/loader.conf see loader.conf5.

To wire a line protocol into your kernel, add:

options NETGRAPH device sppp

It is not necessary to wire line protocols into your kernel, they can be loaded later with kldload(8). The driver can send and receive raw IP packets even if neither SPPP nor Netgraph are configured into the kernel. Netgraph and SPPP can both be enabled; Netgraph will be used if the rawdata hook is connected.

DESCRIPTION

This is an open-source UNIX device driver for PCI-bus WAN interface cards. It sends and receives packets in HDLC frames over synchronous circuits. A generic PC plus UNIX plus some LMC / SBE cards makes an open router. This driver works with Fx , Nx , Ox , Bs x and Linux OSs. It has been tested on i386 (SMP 32-bit little-endian) and Sparc (64-bit big-endian) architectures.

The driver works with the following cards:

SBE wanADAPT-HSSI (LMC5200)
High Speed Serial Interface, EIA612/613, 50-pin connector, 0 to 52 Mb/s, DTE only.
SBE wanADAPT-T3 (LMC5245)
T3: two 75-ohm BNC connectors, C-Parity or M13 Framing, 44.736 Mb/s, up to 950 ft.
SBE wanADAPT-SSI (LMC1000)
Synchronous Serial Interface, V.35, X.21, EIA449, EIA530(A), EIA232, 0 to 10 Mb/s, DTE or DCE.
SBE wanADAPT-T1E1 (LMC1200)
T1 or E1: RJ45 conn, 100 or 120 ohms, T1-ESF-B8ZS, T1-SF-AMI, E1-(many)-HDB3, 1.544 Mb/s or 2.048 Mb/s, up to 6 Kft.

Cards contain a high-performance PCI interface, an HDLC function and either integrated modems (T1, T3) or modem interfaces (HSSI and SSI).

PCI: The PCI interface is a DEC 21140A "Tulip" Fast Ethernet chip. This chip has an efficient PCI implementation with scatter/gather DMA, and can run at 100 Mb/s full duplex (twice as fast as needed here).
HDLC: The HDLC functions (ISO-3309: flags, bit-stuffing, CRC) are implemented in a Field Programmable Gate Array (FPGA) which talks to the Ethernet chip through a Media Independent Interface (MII). The hardware in the FPGA translates between Ethernet packets and HDLC frames on-the-fly; think it as a WAN PHY chip for Ethernet.
Modem: The modem chips are the main differences between cards. HSSI cards use ECL10K chips to implement the EIA-612/613 interface. T3 cards use a TranSwitch TXC-03401 framer chip. SSI cards use Linear Technology LTC1343 modem interface chips. T1 cards use a BrookTree/Conexant/Mindspeed Bt8370 framer and line interface chip.

Line protocols exist above device drivers and below internet protocols. They typically encapsulate packets in HDLC frames and deal with higher-level issues like protocol multiplexing and security. This driver is compatible with several line protocol packages:

Netgraph: netgraph(4) implements many basic packet-handling functions as kernel loadable modules. They can be interconnected in a graph to implement many protocols. Configuration is done from userland without rebuilding the kernel. Packets are sent and received through this interface if the driver's rawdata hook is connected, otherwise the ifnet interface (SPPP and RawIP) is used. ASCII configuration control messages are not currently supported.
SPPP: sppp(4) implements Synchronous-PPP, Frame-Relay and Cisco-HDLC in the kernel.
RawIP: This null line protocol, built into the driver, sends and receives raw IPv4 and IPv6 packets in HDLC frames (aka IP-in-HDLC) with no extra bytes of overhead and no state at the end points.

EXAMPLES

ifconfig and lmcconfig

The program lmcconfig(8) manipulates interface parameters beyond the scope of ifconfig(8). In normal operation only a few arguments are needed:

-X: selects the external SPPP line protocol package.
-x: selects the built-in RawIP line protocol package.
-Z: selects PPP line protocol.
-z: selects Cisco-HDLC line protocol.
-F: selects Frame-Relay line protocol.

lmcconfig lmc0: displays interface configuration and status.
lmcconfig lmc0 -D: enables debugging output from the device driver only.
ifconfig lmc0 debug: enables debugging output from the device driver and from the line protocol module above it. Debugging messages that appear on the console are also written to file /var/log/messages Caution when things go very wrong, a torrent of debugging messages can swamp the console and bring a machine to its knees.

Operation

Activate a PPP link using SPPP and Netgraph with:

ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2

Activate a PPP link using only SPPP with:

lmcconfig lmc0 -XYZ
ifconfig lmc0 10.0.0.1 10.0.0.2

Activate a Cisco-HDLC link using SPPP and Netgraph with:

ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2 link2

Activate a Cisco-HDLC link using only SPPP with:

lmcconfig lmc0 -XYz
ifconfig lmc0 10.0.0.1 10.0.0.2

Activate a Cisco-HDLC link using only Netgraph with:

ngctl mkpeer lmc0: cisco rawdata downstream
ngctl mkpeer lmc0:rawdata iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2

Activate a Frame-Relay DTE link using SPPP with:

lmcconfig lmc0 -XYF
ifconfig lmc0 10.0.0.1 10.0.0.2

(SPPP implements the ANSI T1.617 annex D LMI.)

Activate a Frame-Relay DTE link using Netgraph with:

ngctl mkpeer  lmc0: frame_relay rawdata downstream
ngctl mkpeer  lmc0:rawdata lmi dlci0 auto0
ngctl connect lmc0:rawdata dlci0 dlci1023 auto1023
ngctl mkpeer  lmc0:rawdata rfc1490 dlci500 downstream
ngctl mkpeer  lmc0:rawdata.dlci500 iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2

This is ONE possible Frame Relay configuration; there are many.

Activate a RAWIP link using only the driver with:

lmcconfig lmc0 -x
ifconfig lmc0 10.0.0.1 10.0.0.2

Activate a RAWIP link using Netgraph with:

ngctl mkpeer lmc0: iface rawdata inet
ifconfig ng0 10.0.0.1 10.0.0.2

If the driver is unloaded and then loaded, reconnect hooks by:

"ngctl connect lmc0: ng0: rawdata inet"

TESTING

Testing with Loopbacks

Testing with loopbacks requires only one card. Packets can be looped back at many points: in the PCI chip, in the modem chips, through a loopback plug, in the local external equipment, or at the far end of a circuit.

Activate the card with ifconfig(8):

"ifconfig lmc0 10.0.0.1 10.0.0.1"

All cards can be looped through the PCI chip. Cards with internal modems can be looped through the modem framer and the modem line interface. Cards for external modems can be looped through the driver/receiver chips. See lmcconfig(8) for details.

Loopback plugs test everything on the card.

HSSI

Loopback plugs can be ordered from SBE (and others). Transmit clock is normally supplied by the external modem. When an HSSI card is operated with a loopback plug, the PCI bus clock must be used as the transmit clock, typically 33 MHz. When testing an HSSI card with a loopback plug, configure it with lmcconfig(8):

"lmcconfig lmc0 -a 2"

``-a 2 '' selects the PCI bus clock as the transmit clock.

T3

Connect the two BNC jacks with a short coax cable.

SSI

Loopback plugs can be ordered from SBE (only). Transmit clock is normally supplied by the external modem. When an SSI card is operated with a loopback plug, the on-board clock synthesizer must be used. When testing an SSI card with a loopback plug, configure it with lmcconfig(8):

"lmcconfig lmc0 -E -f 10000000"

-E puts the card in DCE mode to source a transmit clock. ``-f 10000000 '' sets the internal clock source to 10 Mb/s.

T1/E1

A loopback plug is a modular plug with two wires connecting pin 1 to pin 4 and pin 2 to pin 5.

One can also test by connecting to a local modem (HSSI and SSI) or NI (T1 and T3) configured to loop back. Cards can generate signals to loopback remote equipment so that complete circuits can be tested; see lmcconfig(8) for details.

Testing with a Modem

Testing with a modem requires two cards of different types.

T3/HSSI: If you have a T3 modem with an HSSI interface (made by Digital Link, Larscom, Kentrox etc.) then use an HSSI card in one machine and a T3 card in the other machine. The T3 coax cables must use the null modem configuration (see below).
T1/V.35: If you have a T1 (or E1) modem with a V.35, X.21 or EIA530 interface, then use an SSI card in one machine and a T1 card in the other machine. Use a T1 null modem cable (see below).

Testing with a Null Modem Cable

Testing with a null modem cable requires two cards of the same type.

HSSI

Three-meter HSSI null-modem cables can be ordered from SBE. In a pinch, a 50-pin SCSI-II cable up to a few meters will work as a straight HSSI cable (not a null modem cable). Longer cables should be purpose-built HSSI cables because the cable impedance is different. Transmit clock is normally supplied by the external modem. When an HSSI card is connected by a null modem cable, the PCI bus clock can be used as the transmit clock, typically 33 MHz. When testing an HSSI card with a null modem cable, configure it with lmcconfig(8):

"lmcconfig lmc0 -a 2"

``-a 2 '' selects the PCI bus clock as the transmit clock.

T3

T3 null modem cables are just 75-ohm coax cables with BNC connectors. TX OUT on one card should be connected to RX IN on the other card. In a pinch, 50-ohm thin Ethernet cables usually work up to a few meters, but they will not work for longer runs --- 75-ohm coax is required

SSI

Three-meter SSI null modem cables can be ordered from SBE. An SSI null modem cable reports a cable type of V.36/EIA449. Transmit clock is normally supplied by the external modem. When an SSI card is connected by a null modem cable, an on-board clock synthesizer is used. When testing an SSI card with a null modem cable, configure it with lmcconfig(8):

"lmcconfig lmc0 -E -f 10000000"

-E puts the card in DCE mode to source a transmit clock. ``-f 10000000 '' sets the internal clock source to 10 Mb/s.

T1/E1

A T1 null modem cable has two twisted pairs that connect pins 1 and 2 on one plug to pins 4 and 5 on the other plug. Looking into the cable entry hole of a plug, with the locking tab oriented down, pin 1 is on the left. A twisted pair Ethernet cable makes an excellent straight T1 cable. Alas, Ethernet cross-over cables do not work as T1 null modem cables.

OPERATION NOTES

Packet Lengths

Maximum transmit and receive packet length is unlimited. Minimum transmit and receive packet length is one byte.

Cleaning up after one packet and setting up for the next packet involves making several DMA references. This can take longer than the duration of a short packet, causing the adapter to fall behind. For typical PCI bus traffic levels and memory system latencies, back-to-back packets longer than about 20 bytes will always work (53 byte cells work), but a burst of several hundred back-to-back packets shorter than 20 bytes will cause packets to be dropped. This usually is not a problem since an IPv4 packet header is at least 20 bytes long.

This device driver imposes no constraints on packet size. Most operating systems set the default Maximum Transmission Unit (MTU) to 1500 bytes; the legal range is usually (72..65535). This can be changed with

"ifconfig lmc0 mtu 2000"

SPPP enforces an MTU of (128..far-end-MRU) for PPP and 1500 bytes for Cisco-HDLC. RAWIP sets the default MTU to 4032 bytes, but it can be changed to anything.

BPF - Berkeley Packet Filter

This driver has hooks for bpf(4), the Berkeley Packet Filter. The line protocol header length reported to BPF is four bytes for SPPP and P2P line protocols and zero bytes for RawIP.

To include BPF support into your kernel, add the following line to conf/YOURKERNEL

"device bpf"

To test the BPF kernel interface, bring up a link between two machines, then run ping(8) and tcpdump(1):

"ping 10.0.0.1"

and in a different window:

"tcpdump -i lmc0"

The output from tcpdump(1) should look like this:

03:54:35.979965 10.0.0.2 > 10.0.0.1: icmp: echo request
03:54:35.981423 10.0.0.1 > 10.0.0.2: icmp: echo reply

Line protocol control packets will appear among the ping(8) packets occasionally.

Device Polling

A T3 receiver can generate over 100K interrupts per second, this can cause a system to ``live-lock'' spend all of its time servicing interrupts. Fx has a polling mechanism to prevent live-lock.

Fx Ns 's mechanism permanently disables interrupts from the card and instead the card's interrupt service routine is called each time the kernel is entered (syscall, timer interrupt, etc.) and from the kernel idle loop; this adds some latency. The driver is permitted to process a limited number of packets. The percentage of the CPU that can be consumed this way is settable.

See the polling(4) manpage for details on how to enable the polling mode.

SNMP: Simple Network Management Protocol

This driver is aware of what is required to be a Network Interface Object managed by an Agent of the Simple Network Management Protocol. The driver exports SNMP-formatted configuration and status information sufficient for an SNMP Agent to create MIBs for:

"RFC-2233: Interfaces group" ,
"RFC-2496: DS3 interfaces" ,
"RFC-2495: DS1/E1 interfaces" ,
"RFC-1659: RS232-like interfaces" .

An SNMP Agent is a user program, not a kernel function. Agents can retrieve configuration and status information by using Netgraph control messages or ioctl(2) system calls. User programs should poll sc->cfg.ticks which increments once per second after the SNMP state has been updated.

HSSI and SSI LEDs

The card should be operational if all three green LEDs are on (the upper-left one should be blinking) and the red LED is off. All four LEDs turn on at power-on and module unload.

"RED" Ta upper-right Ta Transmit clock
"GREEN" Ta upper-left Ta Device driver is alive if blinking
"GREEN" Ta lower-right Ta Modem signals are good
"GREEN" Ta lower-left Ta Cable is plugged in (SSI only)

T1E1 and T3 LEDs

The card should be operational if the upper-left green LED is blinking and all other LEDs are off. For the T3 card, if other LEDs are on or blinking, try swapping the coax cables! All four LEDs turn on at power-on and module unload.

"RED" Ta upper-right Ta Received signal is wrong
"GREEN" Ta upper-left Ta Device driver is alive if blinking
"BLUE" Ta lower-right Ta Alarm Information Signal (AIS)
"YELLOW" Ta lower-left Ta Remote Alarm Indication (RAI)

"The green" Ta LED blinks if the device driver is alive.
"The red" Ta LED blinks if an outward loopback is active.
"The blue" Ta LED blinks if sending AIS, on solid if receiving AIS.
"The yellow" Ta LED blinks if sending RAI, on solid if receiving RAI.

E1 Framing

Phone companies usually insist that customers put a Frame Alignment Signal (FAS) in time slot 0. A Cyclic Redundancy Checksum (CRC) can also ride in time slot 0. Channel Associated Signalling (CAS) uses Time Slot 16. In telco-speak signalling is on/off hook, ringing, busy, etc. Signalling is not needed here and consumes 64 Kb/s. Only use E1-CAS formats if the other end insists on it! Use E1-FAS+CRC framing format on a public circuit. Depending on the equipment installed in a private circuit, it may be possible to use all 32 time slots for data (E1-NONE).

T3 Framing

M13 is a technique for multiplexing 28 T1s into a T3. Muxes use the C-bits for speed-matching the tributaries. Muxing is not needed here and usurps the FEBE and FEAC bits. Only use T3-M13 format if the other end insists on it! Use T3-CParity framing format if possible. Loop Timing, Fractional T3, and HDLC packets in the Facility Data Link are not supported.

T1 & T3 Frame Overhead Functions

Performance Report Messages (PRMs) are enabled in T1-ESF.
Bit Oriented Protocol (BOP) messages are enabled in T1-ESF.
In-band loopback control (framed or not) is enabled in T1-SF.
Far End Alarm and Control (FEAC) msgs are enabled in T3-CPar.
Far End Block Error (FEBE) reports are enabled in T3-CPar.
Remote Alarm Indication (RAI) is enabled in T3-Any.
Loopbacks initiated remotely time out after 300 seconds.

T1/E1 'Fractional' 64 kb/s Time Slots

T1 uses time slots 24..1; E1 uses time slots 31..0. E1 uses TS0 for FAS overhead and TS16 for CAS overhead. E1-NONE has no overhead, so all 32 TSs are available for data. Enable/disable time slots by setting 32 1s/0s in a config param. Enabling an E1 overhead time slot, or enabling TS0 or TS25-TS31 for T1, is ignored by the driver, which knows better. The default TS param, 0xFFFFFFFF, enables the maximum number of time slots for whatever frame format is selected. 56 Kb/s time slots are not supported.

T1 Raw Mode

Special gate array microcode exists for the T1/E1 card. Each T1 frame of 24 bytes is treated as a packet. A raw T1 byte stream can be delivered to main memory and transmitted from main memory. The T1 card adds or deletes framing bits but does not touch the data. ATM cells can be transmitted and received this way, with the software doing all the work. But that is not hard; after all it is only 1.5 Mb/s second!

T3 Circuit Emulation Mode

Special gate array microcode exists for the T3 card. Each T3 frame of 595 bytes is treated as a packet. A raw T3 signal can be packetized transported through a packet network (using some protocol) and then reconstituted as a T3 signal at the far end. The output transmitter's bit rate can be controlled from software so that it can be frequency locked to the distant input signal.

HSSI and SSI Transmit Clocks

Synchronous interfaces use two transmit clocks to eliminate skew caused by speed-of-light delays in the modem cable. DCEs (modems) drive ST, Send Timing, the first transmit clock. DTEs (hosts) receive ST and use it to clock transmit data, TD, onto the modem cable. DTEs also drive a copy of ST back towards the DCE and call it TT, Transmit Timing, the second transmit clock. DCEs receive TT and TD and use TT to clock TD into a flip flop. TT experiences the same delay as (and has no skew relative to) TD. Thus, cable length does not affect data/clock timing.

HISTORY

An Ron Crane had the idea to use a Fast Ethernet chip as a PCI interface and add an Ethernet-to-HDLC gate array to make a WAN card. An David Boggs designed the Ethernet-to-HDLC gate array and PC cards. We did this at our company, LAN Media Corporation (LMC) SBE Corp. acquired LMC and continues to make the cards.

Since the cards use Tulip Ethernet chips, we started with An Matt Thomas Ns ' ubiquitous de(4) driver. An Michael Graff stripped out the Ethernet stuff and added HSSI stuff. An Basil Gunn ported it to Solaris (lost) and Rob Braun ported it to Linux An Andrew Stanley-Jones added support for three more cards and wrote the first version of lmcconfig(8). An David Boggs rewrote everything and now feels responsible for it.

AUTHORS

An David Boggs Aq [email protected]