Dokumentace systému SUSE Linux Enterprise Server › Storage Administration Guide › Fibre Channel Storage over Ethernet Networks: FCoE

Applies to SUSE Linux Enterprise Server 12

16 Fibre Channel Storage over Ethernet Networks: FCoE

16.1 Installing FCoE and the YaST FCoE Client
16.2 Configuring FCoE Interfaces during the Installation
16.3 Managing FCoE Services with YaST
16.4 Configuring FCoE with Commands
16.5 Managing FCoE Instances with the FCoE Administration Tool
16.6 Setting Up Partitions for an FCoE Initiator Disk
16.7 Creating a File System on an FCoE Initiator Disk
16.8 Additional Information

Many enterprise data centers rely on Ethernet for their LAN and data traffic, and on Fibre Channel networks for their storage infrastructure. Open Fibre Channel over Ethernet (FCoE) Initiator software allows servers with Ethernet adapters to connect to a Fibre Channel storage subsystem over an Ethernet network. This connectivity was previously reserved exclusively for systems with Fibre Channel adapters over a Fibre Channel fabric. The FCoE technology reduces complexity in the data center by aiding network convergence. This helps to preserve your existing investments in a Fibre Channel storage infrastructure and to simplify network management.

Figure 16.1: Open Fibre Channel over Ethernet SAN #

Open-FCoE allows you to run the Fibre Channel protocols on the host, instead of on proprietary hardware on the host bus adapter. It is targeted for 10 Gbps (gigabit per second) Ethernet adapters, but can work on any Ethernet adapter that supports pause frames. The initiator software provides a Fibre Channel protocol processing module as well as an Ethernet based transport module. The Open-FCoE module acts as a low-level driver for SCSI. The Open-FCoE transport uses net_device to send and receive packets. Data Center Bridging (DCB) drivers provide the quality of service for FCoE.

FCoE is an encapsulation protocol that moves the Fibre Channel protocol traffic over Ethernet connections without changing the Fibre Channel frame. This allows your network security and traffic management infrastructure to work the same with FCoE as it does with Fibre Channel.

You might choose to deploy Open-FCoE in your enterprise if the following conditions exist:

Your enterprise already has a Fibre Channel storage subsystem and administrators with Fibre Channel skills and knowledge.
You are deploying 10 Gbps Ethernet in the network.

This section describes how to set up FCoE in your network.

16.1 Installing FCoE and the YaST FCoE Client #

You can set up FCoE disks in your storage infrastructure by enabling FCoE at the switch for the connections to a server. If FCoE disks are available when the SUSE Linux Enterprise Server operating system is installed, the FCoE Initiator software is automatically installed at that time.

If the FCoE Initiator software and YaST FCoE Client software are not installed, use the following procedure to manually install them on an existing system:

Log in to the server as the root user.
In YaST, select SoftwareSoftware Management.
Search for and select the following FCoE packages:
- open-fcoe
- yast2-fcoe-client
For example, type fcoe in the Search field, click Search to locate the software packages, then select the check box next to each software package that you want to install.
Click Accept, then click Continue to accept the automatic changes.

16.2 Configuring FCoE Interfaces during the Installation #

The YaST installation for SUSE Linux Enterprise Server allows you to configure FCoE disks during the operating system installation if FCoE is enabled at the switch for the connections between the server and the Fibre Channel storage infrastructure. Some system BIOS types can automatically detect the FCoE disks, and report the disks to the YaST Installation software. However, automatic detection of FCoE disks is not supported by all BIOS types. To enable automatic detection in this case, you can add the withfcoe option to the kernel command line when you begin the installation:

withfcoe=1

When the FCoE disks are detected, the YaST installation offers the option to configure FCoE instances at that time. On the Disk Activation page, select Configure FCoE Interfaces to access the FCoE configuration. For information about configuring the FCoE interfaces, see Section 16.3, “Managing FCoE Services with YaST”.

16.3 Managing FCoE Services with YaST #

You can use the YaST FCoE Client Configuration option to create, configure, and remove FCoE interfaces for the FCoE disks in your Fibre Channel storage infrastructure. To use this option, the FCoE Initiator service (the fcoemon daemon) and the Link Layer Discovery Protocol agent daemon (lldpad) must be installed and running, and the FCoE connections must be enabled at the FCoE-capable switch.

Log in as the root user, then launch YaST.
In YaST, select Network ServicesFCoE Client Configuration.
The Fibre Channel over Ethernet Configuration dialog box provides three tabs:
- Services
- Interfaces
- Configuration
On the Services tab, view or modify the FCoE service and Lldpad (Link Layer Discovery Protocol agent daemon) service start time as necessary.
- FCoE Service Start: Specifies whether to start the Fibre Channel over Ethernet service fcoemon daemon at the server boot time or manually. The daemon controls the FCoE interfaces and establishes a connection with the lldpad daemon. The values are When Booting (default) or Manually.
- Lldpad Service Start: Specifies whether to start the Link Layer Discovery Protocol agent lldpad daemon at the server boot time or manually. The lldpad daemon informs the fcoemon daemon about the Data Center Bridging features and the configuration of the FCoE interfaces. The values are When Booting (default) or Manually.
If you modify a setting, click OK to save and apply the change.
On the Interfaces tab, view information about all of the detected network adapters on the server, including information about VLAN and FCoE configuration. You can also create an FCoE VLAN interface, change settings for an existing FCoE interface, or remove an FCoE interface.
View FCoE Information.
The FCoE Interfaces table displays the following information about each adapter:
- Device Name: Specifies the adapter name such as eth4.
- Model: Specifies the adapter model information.
- FCoE VLAN Interface.
  - Interface Name: If a name is assigned to the interface, such as eth4.200, FCoE is available on the switch, and the FCoE interface is activated for the adapter.
  - Not Configured: If the status is not configured, FCoE is enabled on the switch, but an FCoE interface has not been activated for the adapter. Select the adapter, then click Create FCoE VLAN Interface to activate the interface on the adapter.
  - Not Available: If the status is not available, FCoE is not possible for the adapter because FCoE has not been enabled for that connection on the switch.
- FCoE Enable: Specifies whether FCoE is enabled on the switch for the adapter connection. (yes or no)
- DCB Required: Specifies whether the adapter requires Data Center Bridging. (yes or no)
- Auto VLAN: Specifies whether automatic VLAN configuration is enabled for the adapter. (yes or no)
- DCB Capable: Specifies whether the adapter supports Data Center Bridging. (yes or no)
Change FCoE Settings.
Select an FCoE VLAN interface, then click Change Settings at the bottom of the page to open the Change FCoE Settings dialog box.
- FCoE Enable: Enable or disable the creation of FCoE instances for the adapter. Values are yes or no.
- DCB Required: Specifies whether Data Center Bridging is required for the adapter. Values are yes (default) or no. DCB is usually required.
- Auto VLAN: Specifies whether the fcoemon daemon creates the VLAN interfaces automatically. Values are yes or no.
If you modify a setting, click Next to save and apply the change. The settings are written to the /etc/fcoe/ethX file. The fcoemon daemon reads the configuration files for each FCoE interface when the daemon is initialized. There is a file for every FCoE interface.
Create FCoE VLAN Interfaces.
Select an adapter that has FCoE enabled but is not configured, then click Yes to configure the FCoE interface. The assigned interface name appears in the list, such as eth5.200.
Remove FCoE Interface.
Select the FCoE interface that you want to remove, click Remove Interface at the bottom of the page, then click Continue to confirm. The FCoE Interface value changes to not configured.
On the Configuration tab, view or modify the general settings for the FCoE system service.
- Debug: Enables or disables debugging messages from the FCoE service script and fcoemon daemon. The values are Yes or No (default).
- Use syslog: Specifies whether messages are sent to the system log (/var/log/syslog). The values are Yes (default) or No.
If you modify a setting, click OK to save and apply the change. The settings are written to the /etc/fcoe/config file.
Click OK to save and apply changes.

16.4 Configuring FCoE with Commands #

Log in to the server as the root user, then open a terminal console.
Use YaST to configure the Ethernet network interface card, such as eth2.
Start the Link Layer Discovery Protocol agent daemon (lldpad).
```
rclldpad start
```

Enable Data Center Bridging on your Ethernet adapter.

dcbtool sc eth2 dcb on
  Version:       2
  Command:       Set Config
  Feature:       DCB State
  Port:          eth2
  Status:        Successful

Enable and set the Priority Flow Control (PFC) settings for Data Center Bridging.
```
dcbtool sc eth<x> pfc e:1 a:1 w:1
```
Argument setting values are:
e:<0|1>
Controls feature enable.
a:<0|1>
Controls whether the feature is advertised via Data Center Bridging Exchange protocol to the peer.
w:<0|1>
Controls whether the feature is willing to change its operational configuration based on what is received from the peer.

Enable the Data Center Bridging to accept the switch’s priority setting for FCoE.

dcbtool sc eth2 app:fcoe e:1
  Version:       2
  Command:       Set Config
  Feature:       Application FCoE
  Port:          eth2
  Status:        Successful

Copy the default FCoE configuration file to /etc/fcoe/cfg-eth2.
```
cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth2
```

Start the FCoE Initiator service.

rcfcoe start
  Starting FCoE initiator service

Set up the Link Layer Discovery Protocol agent daemon (lldpad) and the FCoE Initiator service to start when booting.
```
systemctl enable llpad.service fcoe.service
```

16.5 Managing FCoE Instances with the FCoE Administration Tool #

The fcoeadm utility is the Fibre Channel over Ethernet (FCoE) management tool for the Open-FCoE project. It can be used to create, destroy, and reset an FCoE instance on a given network interface. The fcoeadm utility sends commands to a running fcoemon process via a socket interface. For information about fcoemon, see the fcoemon(8) man page.

The fcoeadm utility allows you to query the FCoE instances about the following:

Interfaces
Target LUNs
Port statistics

The fcoeadm utility is part of the fcoe-utils package. It is maintained by the Open-FCoE project (http://open-fcoe.org/).

Syntax

Fiber Channel over Ethernet Administration version 1.0.12.

fcoeadm
  [-c|--create] [<ethX>]
  [-d|--destroy] [<ethX>]
  [-r|--reset] [<ethX>]
  [-S|--Scan] [<ethX>]
  [-i|--interface] [<ethX>]
  [-t|--target] [<ethX>]
  [-l|--lun] [<ethX>]
  [-s|--stats <ethX>] [<interval>]
  [-v|--version]
  [-h|--help]

Options

-c , --create <ethX>

Creates an FCoE instance based on the specified network interface. If an fcoemon configuration file does not exist for the Open-FCoE service daemon interface (/etc/fcoe/cfg-ethx; see fcoemon(8) man page), the created FCoE instance does not require Data Center Bridging.

Example: To create an FCoE instance on eth2.101:

fcoeadm -c eth2.101

-d , --destroy <ethX>

Destroys an FCoE instance on the specified network interface. This does not destroy FCoE instances created by fipvlan.

Example: To destroy an FCoE instance on eth2.101:

fcoeadm -d eth2.101

-h , --help

Displays the usage message of the fcoeadm command.

-i , --interface [<ethX>]

Shows information about the FCoE instance on the specified network interface. If no network interface is specified, it shows information for all FCoE instances.

Examples.

To show information about all of the adapters and their ports that have FCoE instances created:

fcoeadm -i

To show information about all of the FCoE instances on interface eth3:

fcoeadm -i eth3

-l , --lun [<ethX>]

Shows detailed information about the discovered SCSI LUNs associated with the FCoE instance on the specified network interface. If no network interface is specified, it shows information about SCSI LUNs from all FCoE instances.

Examples.

To show detailed information about all of the LUNs discovered on all FCoE connections:

fcoeadm -l

To show detailed information about all of the LUNs discovered on a specific connections, such as eth3.101:

fcoeadm -l eth3.101

-r , --reset <ethX>

Resets the FCoE instance on the specified network interface. This does not reset FCoE instances created by fipvlan.

Example: To reset the FCoE instance on eth2.101:

fcoeadm -r eth2.101

-s , --stats <ethX> [interval]

Shows the statistics (including FC4 statistics) of the FCoE instance on the specified network interface. It displays one line per given time interval. Specify the interval value in whole integers greater than 0. The interval value is the elapsed time of the interval in seconds. If an interval is not specified, the default interval is 1 second.

Examples:

You can show statistics information about a specific eth3 port that has FCoE instances. The statistics are displayed one line per time interval. The default interval of one second is not specified in the command.

fcoeadm -s eth3

To show statistics information about a specific eth3 port that has FCoE instances, at an interval of 3 seconds. The statistics are displayed one line per time interval.

fcoeadm -s eth3 3

-S , --Scan <ethX>

Rescans for new targets and LUN for the specified network interface. This does not rescan any NPIV (N_Port ID Virtualization) instances created on the same port, and does not rescan any FCoE instances created by fipvlan.

-t , --target [<ethX>]

Shows information about the discovered targets associated with the FCoE instance on the specified network interface. If no network interface is specified, it shows information about discovered targets from all FCoE instances.

Examples.

You can show information about all of the discovered targets from all of the ports that have FCoE instances. They might be on different adapter cards. After each discovered target, any associated LUNs are listed.

fcoeadm -t

You can show information about all of the discovered targets from a given eth3 port having FCoE instance. After each discovered target, any associated LUNs are listed.

fcoeadm -t eth3

-v , --version

Displays the version of the fcoeadm command.

FCoE Response Examples

View FCoE Initiator Status for FC-ID Node/Port Number

fcoeadm -i eth0.201
  Description:      82599EB 10-Gigabit SFI/SFP+ Network Connection 
  Revision:         01
  Manufacturer:     Intel Corporation 
  Serial Number:    001B219B258C 
  Driver:           ixgbe 3.3.8-k2 
  Number of Ports:  1      

      Symbolic Name:     fcoe v0.1 over eth0.201     
      OS Device Name:    host8     
      Node Name:         0x1000001B219B258E     
      Port Name:         0x2000001B219B258E     
      FabricName:        0x2001000573D38141     
      Speed:             10 Gbit     
      Supported Speed:   10 Gbit     
      MaxFrameSize:      2112     
      FC-ID (Port ID):   0x790003     
      State:             Online

View FCoE Targets for FC-ID Node/Port Number

fcoeadm -t eth0.201  
  Interface:        eth0.201 
  Roles:            FCP Target 
  Node Name:        0x200000D0231B5C72 
  Port Name:        0x210000D0231B5C72 
  Target ID:        0 
  MaxFrameSize:     2048 
  OS Device Name:   rport-8:0-7 
  FC-ID (Port ID):  0x79000C 
  State:            Online  

LUN ID  Device Name   Capacity   Block Size  Description 
------  -----------  ----------  ----------  ----------------------------     
    40  /dev/sdqi     792.84 GB      512     IFT DS S24F-R2840-4 (rev 386C)
    72  /dev/sdpk     650.00 GB      512     IFT DS S24F-R2840-4 (rev 386C)
   168  /dev/sdgy       1.30 TB      512     IFT DS S24F-R2840-4 (rev 386C)

16.6 Setting Up Partitions for an FCoE Initiator Disk #

You can use the fdisk(8) command to set up partitions for an FCoE initiator disk.

fdisk /dev/sdc
  Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel.
  Building a new DOS disklabel with disk identifier 0xfc691889.
  Changes will remain in memory only, until you decide to write them.
  After that, of course, the previous content won’t be recoverable.

  Warning: Invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

  Command (n for help): n
  Command action
     e    extended
     p    primary partition (1-4)

  p
     Partition number (1-4): 4
     First cylinder (1-1017, default 1): 
       Using default value 1
     Last cylinder, *cylinders or *size(K,M,G) (1-1017, default 1017):
       Using default value 1017

  Command (n for help): w
     The partition table has been altered!

     Calling loctl() to re-read partition table.
     Syncing disks.

16.7 Creating a File System on an FCoE Initiator Disk #

You can use the mkfs(8) command to create a file system on an FCoE initiator disk.

mkfs /dev/sdc
  mke2fs 1.41.9 (22-Aug-2011)
  /dev/sdc is entire device, not just one partition!
  Proceed anyway? (y, n) y
  Filesystem label=
  OS type: Linux
  Block size=4096 (log-2)
  262144 inodes, 1048576 blocks
  52428 blocks (5.00%) reserved for the super user
  First data block=0
  Maximum filesystem blocks=1073741824
  32 block groups
  32768 blocks per group, 32768 fragments per group
  8192 inodes per group
  Superblock backups stored on blocks:
          32768, 98304, 163840, 229376, 294912, 819200, 804736

  Writing inode tables: done
  Writing superblocks and filesystem accounting information: done

  This filesystem will be automatically checked every 27 mounts or
  180 days, whichever comes first. Use tune2fs -c or -i to override.

16.8 Additional Information #

For information, see the follow documentation:

For information about the Open-FCoE service daemon, see the fcoemon(8)man page.
For information about the Open-FCoE Administration tool, see the fcoeadm(8) man page.
For information about the Data Center Bridging Configuration tool, see the dcbtool(8) man page.
For information about the Link Layer Discovery Protocol agent daemon, see the lldpad(8) man page.
Open Fibre Channel over Ethernet Quick Start (http://www.open-fcoe.org/open-fcoe/wiki/quickstart).

Print this page