14 Mass Storage over IP Networks: iSCSI

Install the iSCSI target software on the server where you want to create iSCSI target devices.

Install the iSCSI initiator software on each server where you want to access the target devices that you set up on the iSCSI target server.

The iSCSI target configuration exports existing block devices to iSCSI initiators. You must prepare the storage space you want to use in the target devices by setting up unformatted partitions or devices by using the Partitioner in YaST, or by partitioning the devices from the command line. iSCSI LIO targets can use unformatted partitions with Linux, Linux LVM, or Linux RAID file system IDs.

Important

After you set up a device or partition for use as an iSCSI target, you never access it directly via its local path. Do not specify a mount point for it when you create it.

To configure the iSCSI target, run the iSCSI Target module in YaST. The configuration is split into three tabs. In the Service tab, select the start mode and the firewall settings. If you want to access the iSCSI target from a remote machine, select Open Port in Firewall. If an iSNS server should manage the discovery and access control, activate iSNS Access Control and enter the IP address of your iSNS server. You cannot use hostnames or DNS names; you must use the IP address. For more about iSNS, read Chapter 13, iSNS for Linux.

The Global tab provides settings for the iSCSI server. The authentication set here is used for the discovery of services, not for accessing the targets. If you do not want to restrict the access to the discovery, use No Authentication.

If authentication is needed, there are two possibilities to consider. One is that an initiator must prove that it has the permissions to run a discovery on the iSCSI target. This is done with Incoming Authentication. The other is that the iSCSI target must prove to the initiator that it is the expected target. Therefore, the iSCSI target can also provide a user name and password. This is done with Outgoing Authentication. Find more information about authentication in RFC 3720 (http://www.ietf.org/rfc/rfc3720.txt).

The targets are defined in the Targets tab. Use Add to create a new iSCSI target. The first dialog box asks for information about the device to export.

The next menu configures the access restrictions of the target. The configuration is very similar to the configuration of the discovery authentication. In this case, at least an incoming authentication should be setup.

Next finishes the configuration of the new target, and brings you back to the overview page of the Target tab. Activate your changes by clicking Finish.

To create a target device:

Configure an iSCSI target in /etc/ietd.conf. All parameters in this file before the first Target declaration are global for the file. Authentication information in this portion has a special meaning—it is not global, but is used for the discovery of the iSCSI target.

If you have access to an iSNS server, you should first configure the file to tell the target about this server. The address of the iSNS server must always be given as an IP address. You cannot specify the DNS name for the server. The configuration for this functionality looks like the following:

iSNSServer 192.168.1.111
iSNSAccessControl no

This configuration makes the iSCSI target register itself with the iSNS server, which in turn provides the discovery for initiators. For more about iSNS, see Chapter 13, iSNS for Linux. The access control for the iSNS discovery is not supported. Keep iSNSAccessControl no.

All direct iSCSI authentication can be done in two directions. The iSCSI target can require the iSCSI initiator to authenticate with the IncomingUser, which can be added multiple times. The iSCSI initiator can also require the iSCSI target to authenticate. Use OutgoingUser for this. Both have the same syntax:

IncomingUser <username> <password>
OutgoingUser <username> <password>

The authentication is followed by one or more target definitions. For each target, add a Target section. This section always starts with a Target identifier followed, by definitions of logical unit numbers:

Target iqn.yyyy-mm.<reversed domain name>[:identifier]
          Lun 0 Path=/dev/mapper/system-v3
          Lun 1 Path=/dev/hda4
          Lun 2 Path=/var/lib/xen/images/xen-1,Type=fileio

In the Target line, yyyy-mm is the date when this target is activated, and identifier is freely selectable. Find more about naming conventions in RFC 3722 (http://www.ietf.org/rfc/rfc3722.txt). Three different block devices are exported in this example. The first block device is a logical volume (see also Chapter 4, LVM Configuration), the second is an IDE partition, and the third is an image available in the local file system. All these look like block devices to an iSCSI initiator.

Before activating the iSCSI target, add at least one IncomingUser after the Lun definitions. It does the authentication for the use of this target.

To activate all your changes, restart the iscsitarget daemon with sudo systemctl restart open-iscsi.service. Check your configuration in the /proc file system:

cat /proc/net/iet/volume
tid:1 name:iqn.2006-02.com.example.iserv:systems
        lun:0 state:0 iotype:fileio path:/dev/mapper/system-v3
        lun:1 state:0 iotype:fileio path:/dev/hda4
        lun:2 state:0 iotype:fileio path:/var/lib/xen/images/xen-1

There are many more options that control the behavior of the iSCSI target. For more information, see the man page of ietd.conf.

Active sessions are also displayed in the /proc file system. For each connected initiator, an extra entry is added to /proc/net/iet/session:

cat /proc/net/iet/session
tid:1 name:iqn.2006-02.com.example.iserv:system-v3
   sid:562949957419520 initiator:iqn.2005-11.de.suse:cn=rome.example.com,01.9ff842f5645
      cid:0 ip:192.168.178.42 state:active hd:none dd:none
   sid:281474980708864 initiator:iqn.2006-02.de.suse:01.6f7259c88b70
      cid:0 ip:192.168.178.72 state:active hd:none dd:none

When changes to the iSCSI target configuration are necessary, you must always restart the target to activate changes that are done in the configuration file. Unfortunately, all active sessions are interrupted in this process. To maintain an undisturbed operation, the changes should be done in the main configuration file /etc/ietd.conf, but also made manually to the current configuration with the administration utility ietadm.

To create a new iSCSI target with a LUN, first update your configuration file. The additional entry could be:

Target iqn.2006-02.com.example.iserv:system2
          Lun 0 Path=/dev/mapper/system-swap2
          IncomingUser joe secret

To set up this configuration manually, proceed as follows:

It is also possible to delete active connections. First, check all active connections with the command cat /proc/net/iet/session. This might look like:

cat /proc/net/iet/session
tid:1 name:iqn.2006-03.com.example.iserv:system
        sid:281474980708864 initiator:iqn.1996-04.com.example:01.82725735af5
                cid:0 ip:192.168.178.72 state:active hd:none dd:none

To delete the session with the session ID 281474980708864, use the command ietadm --op delete --tid=1 --sid=281474980708864 --cid=0. Be aware that this makes the device inaccessible on the client system and processes accessing this device are likely to hang.

ietadm can also be used to change various configuration parameters. Obtain a list of the global variables with ietadm --op show --tid=1 --sid=0. The output looks like:

InitialR2T=Yes
ImmediateData=Yes
MaxConnections=1
MaxRecvDataSegmentLength=8192
MaxXmitDataSegmentLength=8192
MaxBurstLength=262144
FirstBurstLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=20
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
ErrorRecoveryLevel=0
HeaderDigest=None
DataDigest=None
OFMarker=No
IFMarker=No
OFMarkInt=Reject
IFMarkInt=Reject

All of these parameters can be easily changed. For example, if you want to change the maximum number of connections to two, use

ietadm --op update --tid=1 --params=MaxConnections=2. 

In the file /etc/ietd.conf, the associated line should look like MaxConnections 2.

Warning

The changes that you make with the ietadm utility are not permanent for the system. These changes are lost at the next reboot if they are not added to the /etc/ietd.conf configuration file. Depending on the usage of iSCSI in your network, this might lead to severe problems.

There are several more options available for the ietadm utility. Use ietadm -h to find an overview. The abbreviations there are target ID (tid), session ID (sid), and connection ID (cid). They can also be found in /proc/net/iet/session.

The iSCSI Initiator Overview in YaST is divided into three tabs:

Both the discovery and the configuration of iSCSI connections require a running iscsid. When running the discovery the first time, the internal database of the iSCSI initiator is created in the directory /var/lib/open-iscsi.

If your discovery is password protected, provide the authentication information to iscsid. Because the internal database does not exist when doing the first discovery, it cannot be used at this time. Instead, the configuration file /etc/iscsid.conf must be edited to provide the information. To add your password information for the discovery, add the following lines to the end of /etc/iscsid.conf:

discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = <username>
discovery.sendtargets.auth.password = <password>

The discovery stores all received values in an internal persistent database. In addition, it displays all detected targets. Run this discovery with the following command:

iscsiadm -m discovery --type=st --portal=<targetip>

The output should look like the following:

10.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems

To discover the available targets on a iSNS server, use the following command:

iscsiadm --mode discovery --type isns --portal <targetip> 

For each target defined on the iSCSI target, one line appears. For more information about the stored data, see Section 14.3.3, “The iSCSI Client Databases”.

The special --login option of iscsiadm creates all needed devices:

iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --login

The newly generated devices show up in the output of lsscsi and can now be accessed by mount.

All information that was discovered by the iSCSI initiator is stored in two database files that reside in /var/lib/open-iscsi. There is one database for the discovery of targets and one for the discovered nodes. When accessing a database, you first must select if you want to get your data from the discovery or from the node database. Do this with the -m discovery and -m node parameters of iscsiadm. Using iscsiadm with one of these parameters gives an overview of the stored records:

iscsiadm -m discovery
10.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems

The target name in this example is iqn.2006-02.com.example.iserv:systems. This name is needed for all actions that relate to this special data set. To examine the content of the data record with the ID iqn.2006-02.com.example.iserv:systems, use the following command:

iscsiadm -m node --targetname iqn.2006-02.com.example.iserv:systems
node.name = iqn.2006-02.com.example.iserv:systems
node.transport_name = tcp
node.tpgt = 1
node.active_conn = 1
node.startup = manual
node.session.initial_cmdsn = 0
node.session.reopen_max = 32
node.session.auth.authmethod = CHAP
node.session.auth.username = joe
node.session.auth.password = ********
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 0
node.session.err_timeo.abort_timeout = 10
node.session.err_timeo.reset_timeout = 30
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
....

To edit the value of one of these variables, use the command iscsiadm with the update operation. For example, if you want iscsid to log in to the iSCSI target when it initializes, set the variable node.startup to the value automatic:

iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems -p ip:port --op=update --name=node.startup --value=automatic

Remove obsolete data sets with the delete operation If the target iqn.2006-02.com.example.iserv:systems is no longer a valid record, delete this record with the following command:

iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems -p ip:port --op=delete

Important

Use this option with caution because it deletes the record without any additional confirmation prompt.

To get a list of all discovered targets, run the iscsiadm -m node command.

In SLES 10, you could add the hotplug option to your device in the /etc/fstab file to mount iSCSI targets. For example:

/dev/disk/by-uuid-blah /oracle/db   ext3       hotplug,rw            0 2

For SLES 11, the hotplug option no longer works. Use the nofail option instead. For example:

/dev/sdb1   /mnt/mountpoint   ext3   acl,user,nofail   0 0

For information, see TID 7004427: /etc/fstab entry does not mount iSCSI device on boot up (http://www.suse.com/support/php/search.do?cmd=displayKC&docType=kc&externalId=7004427).

A firewall might drop packets if it gets to busy. The default for the SUSE Firewall is to drop packets after three minutes. If you find that iSCSI traffic packets are being dropped, you can consider configuring the SUSE Firewall to queue packets instead of dropping them when it gets too busy.

Use the troubleshooting tips in this section when using LVM on iSCSI targets.

When Open-iSCSI starts, it can mount the targets even if the option node.startup option is set to manual in the /etc/iscsi/iscsid.conf file if you manually modified the configuration file.

Check the /etc/iscsi/nodes/<target_name>/<ip_address,port>/default file. It contains a node.startup setting that overrides the /etc/iscsi/iscsid.conf file. Setting the mount option to manual by using the YaST interface also sets the node.startup = manual in the /etc/iscsi/nodes/<target_name>/<ip_address,port>/default files.