# Linux Installation and Configuration of Intel® Fibre Channel Over Ethernet (FCoE)

SUSE* Linux Enterprise Server 11 FCoE Installation

Red hat* Enterprise Linux 6 FCoE Installation

Linux FCoE Configuration

## SUSE* Linux Enterprise Server 11 (SLES 11) FCoE Installation

### Select FCoE During OS Installation

1. In Installation Settings, select FCoE Package
2. Click Details to enter a new window. Select additional installation package
3. Enter Search tab
4. In Search box, type "fcoe" and click "Search"
5. Select fcoe-utils package and click Accept
6. Click OK for questions to get back to "Installation Settings"

### Using Yast to install after OS Installation

| NOTE: If FCoE is installed during OS installation, then this is not required.

1. Select Software in the left pane
2. Select Software Management in the right pane
3. Type "fcoe" in the search box, then click Search button
4. "open-fcoe" package will show in right pane
5. Select "open-fcoe" package, then click Accept
6. Verify "Automatic Changes" and click OK
7. Click Install to install OS

## Red Hat* Enterprise Linux 6 (RHEL 6) FCoE Installation

### Select FCoE During OS Installation

1. Select your desired installation profile (Basic Server, Database Server, Web Server, etc).
2. Set up your repositories as desired, and select the Customize now button before clicking Next to go to the software package selection screen.
3. Under Base System, select FCoE Storage Client. Make sure that the optional package Intel LLDP Agent is included under Optional packages.
4. Continue with OS installation.

### Using YUM to install after OS Installation

| NOTE: If FCoE is installed during OS installation, then these steps are not required.

1. Either configure the Red Hat Network (RHN) or mount the installation DVD. Please refer to the Red Hat documentation to learn how to register your repositories with YUM.
2. root# yum -y install fcoe*
   1. You will notice several dependency packages being automatically installed.

## Linux FCoE Configuration

| NOTE: The following examples assume that the network interface named "eth3" is the FCoE interface. Your interface name is likely to be different. Please substitute your interface name when "eth3" is used below.

### Verifying the Network Interface

After the FCoE package has been installed activate the interface. Then ensure that link is established and active at 10G link speed to the network switch.

> root# ifconfig eth3 up

> root# ethtool eth3

> Settings for eth3:

> Speed: 10000Mb/s
> Link detected: yes

| NOTE: If you do not see link detected, check cabling and correct before proceeding.

### Configuring Link Layer Discovery Protocal agent daemon (lldpad)

lldpad is a utility used to initiate and configure the Link Layer Discovery Protocol (LLDP). LLDP enables the IEEE Data Center Bridging extensions (DCBx) so that FCoE VLAN discovery, parameter negotiations and session initiation can be accomplished.

1. Start lldpad service and configure to start at boot time.

> root# service lldpad start

> Starting lldpad: [done] [ OK ]

> root# chkconfig lldpad on

> | NOTE: There is no output from this command but it will enable lldpad to automatically start when the system is booted.

2. Adjust DCB settings on every physical interface to be used for FCoE with these commands:

   | NOTE: dcbtool makes change to /var/lib/lldpad/lldpad.conf.

   root# dcbtool sc eth3 dcb on

   > root# dcbtool sc eth3 dcb on
   > Version: 2
   > Command: Set Config
   > Feature: DCB State
   > Port: eth3
   > Status: Successful

   root# dcbtool sc eth3 app:0 e:1

   > root# dcbtool sc eth3 app:0 e:1
   > Version: 2
   > Command: Set Config
   > Feature: Application FCoE
   > Port: eth3
   > Status: Successful

   root# dcbtool sc eth3 pfc e:1 a:1 w:1

   > root# dcbtool sc eth3 pfc e:1 a:1 w:1
   > Version: 2
   > Command: Set Config
   > Feature: Priority Flow Control
   > Port: eth3
   > Status: Successful

   root# dcbtool go eth3 pfc

   > root# dcbtool go eth3 pfc
   > Version: 2
   > Command: Get Oper
   > Feature: Priority Flow Control
   > Port: eth3
   > Status: Successful
   > Oper Version: 0
   > Max Version: 0
   > Errors: 0x00 - none
   > Oper Mode: true
   > Syncd: true
   > pfcup: 0 0 0 1 0 0 0 0

   root# dcbtool go eth3 app:0

   > root# dcbtool go eth3 app:0
   > Version: 2
   > Command: Get Oper
   > Feature: Application FCoE
   > Port: eth3
   > Status: Successful
   > Oper Version: 0
   > Max Version: 0
   > Errors: 0x00 - none
   > Oper Mode: true
   > Syncd: true
   > appcfg: 08

### Configuring FCoE

1. Create a configuration file for the VLAN interface. To copy cfg file, run command:

   | NOTE: The sample file is cfg-ethx. Copy it to a file which reflects your FCoE network interface name (eg. cfg-eth3).

> root# cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth3

2. Start fcoe service and configure to start at boot time.

> root# service fcoe start

> > Starting FCoE initiator service: [ OK ]

> root# service fcoe status

> > Created interfaces: eth3.100-fcoe

> > | NOTE: In the example above, an FCoE interface was created for eth3 using VLAN 100. "No interfaces added" indicates that the FIPVLAN protocol was not able to acquire a vlan and therefore an interface was not created. This must be corrected before continuing. This issue may be because you are not connected to a DCB enabled switch or DCB is disabled on this switch port. Contact your network administrator for further help.

> root# chkconfig fcoe on

> > | NOTE: There is no output from this command but it will enable FCoE to automatically start when the system is booted.

1. Verify that a vlan interface has been created.

> root# ifconfig -a

> > eth3 Link encap:Ethernet HWaddr 00:1B:21:60:62:75
> > ...
> > eth3.100-fcoe Link encap:Ethernet HWaddr 00:1B:21:60:62:75

> > | NOTE: The interface name "eth3.100-fcoe" indicates that the FCoE Initialization Protocol (FIP) negotiated with the switch and established an FCoE session using VLAN 100. This process usually happens automatically for all DCBx enabled switches.

> No FCoE ports configured

> > | NOTE: This message indicates that the FIP VLAN Discovery failed. Please see your network administrator and ensure that this port is configured for DCBx and FCoE.

4. Check that vlan interface can obtain FC-ID. Run command

> > root# fcoeadm -i

> > > Description: 82599EB 10-Gigabit Network Connection
> > > Revision: 01
> > > Manufacturer: Intel Corporation
> > > Serial Number: 001B21606274
> > > Driver: ixgbe 2.0.62-k2
> > > Number of Ports: 2
> > > Symbolic Name: fcoe v0.1 over eth3.100-fcoe
> > > OS Device Name: host3
> > > Node Name: 0x1000001B21606277
> > > Port Name: 0x2000001B21606277
> > > FabricName: 0x2064000DECA33A01
> > > Speed: 10 Gbit
> > > Supported Speed: 10 Gbit
> > > MaxFrameSize: 2112
> > > FC-ID (Port ID): 0x4A0064
> > > State: Online

> > > | NOTE: The important values to note are that the state is "Online", the Port Name and the FC-ID. Your storage administrator may need the Port Name and the FC-ID in order to identify your connection when allocating zones for you.

> If you see the FCoE interface and an FCID from the switch, you are ready to zone and provision your storage. Please see your storage administrator. Once your storage administrator has properly provisioned your storage, you may continue to configure.

5. Verify that you have been assigned on a storage target.

> root# fcoeadm -t
> Interface: eth3.100-fcoe
> ...
> FC-ID (Port ID): 0x4A06EF
> State: Online
> LUN ID Device Name Capacity Block Size Description
> ------ ----------- ---------- ---------- ----------------------------
> 0 /dev/sdb 25.00 GB 512 DGC RAID 10 (rev 0429)

> | NOTE: This example shows that a LUN has been provisioned and assigned to /dev/sdb. This may happen automatically but usually you must request a LUN from your storage administrator.

6. Verify that the file system has identified the SAN storage.

> root# lsscsi
> [0:0:0:0] cd/dvd TSSTcorp CDDVDW SH-S223L SB02 /dev/sr1
> [1:0:0:0] disk ATA ST3160811AS 3.AA /dev/sda
> [2:0:1:0] cd/dvd MATSHITA DVD-ROM SR-8178 PZ16 /dev/sr0
> [3:0:0:0] disk DGC RAID 10 0429 /dev/sdb

### Basic FCoE Commands

> show interface info: fcoeadm -i

> > root# fcoeadm -i
> > Description: 82599EB 10-Gigabit Network Connection
> > Revision: 01
> > Manufacturer: Intel Corporation
> > Serial Number: 001B21606274
> > Driver: ixgbe 2.0.62-k2
> > Number of Ports: 2
> > Symbolic Name: fcoe v0.1 over eth3.100-fcoe
> > OS Device Name: host3
> > Node Name: 0x1000001B21606277
> > Port Name: 0x2000001B21606277
> > FabricName: 0x2064000DECA33A01
> > Speed: 10 Gbit
> > Supported Speed: 10 Gbit
> > MaxFrameSize: 2112
> > FC-ID (Port ID): 0x4A0064
> > State: Online

> show LUNs: fcoeadm -l

> > root# fcoeadm -l
> > Interface: eth3.100-fcoe
> > Roles: FCP Target
> > Node Name: 0x50060160C46029CC
> > Port Name: 0x50060160446029CC
> > Target ID: 0
> > MaxFrameSize: 2048
> > OS Device Name: rport-3:0-1
> > FC-ID (Port ID): 0x4A06EF
> > State: Online
> > LUN #0 Information:
> > OS Device Name: /dev/sdb
> > Description: DGC RAID 10 (rev 0429)
> > Ethernet Port FCID: 0x4A0064
> > Target FCID: 0x4A06EF
> > Target ID: 0
> > LUN ID: 0
> > Capacity: 25.00 GB
> > Capacity in Blocks: 52428799
> > Block Size: 512 bytes
> > Status: Attached

> show targets: fcoeadm -t

> > root# fcoeadm -t
> > Interface: eth3.100-fcoe
> > Roles: FCP Target
> > Node Name: 0x50060160C46029CC
> > Port Name: 0x50060160446029CC
> > Target ID: 0
> > MaxFrameSize: 2048
> > OS Device Name: rport-3:0-1
> > FC-ID (Port ID): 0x4A06EF
> > State: Online
> > LUN ID Device Name Capacity Block Size Description
> > ------ ----------- ---------- ---------- ----------------------------
> > 0 /dev/sdb 25.00 GB 512 DGC RAID 10 (rev 0429)

> show statistics: fcoeadm -s

> > root# fcoeadm -s eth3.100-fcoe 1
> > eth3.100-fcoe interval: 1 Err Inv IvTx Link Cntl Input Input Output Output
> > Seconds TxFrames TxBytes RxFrames RxBytes Frms CRC Byte Fail Reqs Requests MBytes Requests MBytes
> > ------- --------- ------------ --------- ------------ ---- ---- ---- ---- ---- --------- --------- --------- ---------
> > 0 102 5424 103 6444 0 0 0 0 6 79 526 0 0
> > 1 102 5424 103 6444 0 0 0 0 6 79 526 0 0
> > 2 102 5424 103 6444 0 0 0 0 6 79 526 0 0