Issues using mhvtl with freebsd and iscsi

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Issues using mhvtl with freebsd and iscsi

obdevel
Hi

Apologies for spamming the list with such a long first post, as I can't definitively point the finger at a mhvtl problem, but I'd appreciate some help.

I have mhvtl running on Centos 7 under VMWare, exported via tgt iscsi to a number of other platforms. Standard config created using mhvtlgui. Tested on several other linuxes (32 & 64 bit) and Win7/2012R2 which all work absolutely fine, as you'd expect. Tape drive access works fine on FreeBSD 10.1 but I get strangeness operating the tape library.

Clearly, the problem may well be in tgt or the FreeBSD iscsi initiator, but any pointers for debugging would be welcomed.

Target environment:
[root@centos tmp]# uname -a
Linux centos 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@centos tmp]# vtlcmd -v
Version: 1.5.3-git-f993138
[root@centos tmp]# lsscsi -g
[0:0:0:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sda   /dev/sg0 
[0:0:1:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sdb   /dev/sg1 
[2:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0   /dev/sg2 
[3:0:0:0]    mediumx STK      L700             0105  /dev/sch0  /dev/sg12
[3:0:1:0]    tape    IBM      ULT3580-TD5      0105  /dev/st0   /dev/sg3 
[3:0:2:0]    tape    IBM      ULT3580-TD5      0105  /dev/st1   /dev/sg4 
[3:0:3:0]    tape    IBM      ULT3580-TD4      0105  /dev/st2   /dev/sg5 
[3:0:4:0]    tape    IBM      ULT3580-TD4      0105  /dev/st3   /dev/sg6 
[3:1:0:0]    mediumx IBM      03584L22         4.02  /dev/sch1  /dev/sg13
[3:1:0:1]    tape    IBM      ULT3580-TD4      252D  /dev/st4   /dev/sg7 
[3:1:0:2]    tape    IBM      ULT3580-TD4      252D  /dev/st5   /dev/sg8 
[3:1:0:3]    tape    IBM      ULT3580-TD4      252D  /dev/st6   /dev/sg9 
[3:1:0:4]    tape    IBM      ULT3580-TD4      252D  /dev/st7   /dev/sg10
[3:1:0:5]    tape    IBM      ULT3580-TD4      252D  /dev/st8   /dev/sg11
[root@centos tmp]# tgtadm --version
1.0.46

Initiator environment:
root@freebsd:/tmp # uname -a
FreeBSD freebsd 10.1-RELEASE-p16 FreeBSD 10.1-RELEASE-p16 #0: Tue Jul 28 12:04:19 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
root@freebsd:/tmp # iscsictl -L
Target name                          Target portal    State
iqn.1994-05.com.redhat:dfa419db7c60:mhvtl:stgt:1 192.168.97.148   Connected: ch0 sa0 sa2 sa3 sa5 
iqn.1994-05.com.redhat:dfa419db7c60:mhvtl:stgt:2 192.168.97.148   Connected: ch1 sa1 sa4 sa6 sa7 sa8 
root@freebsd:/tmp # camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00>   at scbus1 target 0 lun 0 (pass0,cd0)
<VMware, VMware Virtual S 1.0>     at scbus2 target 0 lun 0 (pass1,da0)
<IET Controller 0001>              at scbus3 target 0 lun 0 (pass2)
<STK L700 0105>                    at scbus3 target 0 lun 1 (ch0,pass4)
<IBM ULT3580-TD5 0105>             at scbus3 target 0 lun 2 (sa0,pass6)
<IBM ULT3580-TD5 0105>             at scbus3 target 0 lun 3 (sa2,pass8)
<IBM ULT3580-TD4 0105>             at scbus3 target 0 lun 4 (sa3,pass9)
<IBM ULT3580-TD4 0105>             at scbus3 target 0 lun 5 (sa5,pass11)
<IET Controller 0001>              at scbus4 target 0 lun 0 (pass3)
<IBM 03584L22 4.02>                at scbus4 target 0 lun 1 (ch1,pass5)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 2 (sa1,pass7)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 3 (sa4,pass10)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 4 (sa6,pass12)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 5 (sa7,pass13)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 6 (sa8,pass14)

Comparing inventory output on the two platforms:

[root@centos tmp]# mtx -f /dev/sg12 status
  Storage Changer /dev/sg12:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Full (Storage Element 1 Loaded):VolumeTag = E01001L4                            
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
      Storage Element 1:Empty
      Storage Element 2:Full :VolumeTag=E01002L4                            
      Storage Element 3:Full :VolumeTag=E01003L4                            
      Storage Element 4:Full :VolumeTag=E01004L4                            
      Storage Element 5:Full :VolumeTag=E01005L4                            
      Storage Element 6:Full :VolumeTag=E01006L4                            
      Storage Element 7:Full :VolumeTag=E01007L4                            
      Storage Element 8:Full :VolumeTag=E01008L4                            
      Storage Element 9:Full :VolumeTag=E01009L4                            
      Storage Element 10:Full :VolumeTag=E01010L4                            
      Storage Element 11:Full :VolumeTag=E01011L4                            
      Storage Element 12:Full :VolumeTag=E01012L4                            
      Storage Element 13:Full :VolumeTag=E01013L4                            
      Storage Element 14:Full :VolumeTag=E01014L4                            
      Storage Element 15:Full :VolumeTag=E01015L4                            
      Storage Element 16:Full :VolumeTag=E01016L4                            
      Storage Element 17:Full :VolumeTag=E01017L4                            
      Storage Element 18:Full :VolumeTag=E01018L4                            
      Storage Element 19:Full :VolumeTag=E01019L4                            
      Storage Element 20:Full :VolumeTag=E01020L4                            
      Storage Element 21:Empty
      Storage Element 22:Full :VolumeTag=CLN101L4                            
      Storage Element 23:Full :VolumeTag=CLN102L5                            
      Storage Element 24:Empty
      Storage Element 25:Empty
      Storage Element 26:Empty
      Storage Element 27:Empty
      Storage Element 28:Empty
      Storage Element 29:Empty
      Storage Element 30:Full :VolumeTag=F01030L5                            
      Storage Element 31:Full :VolumeTag=F01031L5                            
      Storage Element 32:Full :VolumeTag=F01032L5                            
      Storage Element 33:Full :VolumeTag=F01033L5                            
      Storage Element 34:Full :VolumeTag=F01034L5                            
      Storage Element 35:Full :VolumeTag=F01035L5                            
      Storage Element 36:Full :VolumeTag=F01036L5                            
      Storage Element 37:Full :VolumeTag=F01037L5                            
      Storage Element 38:Full :VolumeTag=F01038L5                            
      Storage Element 39:Full :VolumeTag=F01039L5                            
      Storage Element 40 IMPORT/EXPORT:Empty
      Storage Element 41 IMPORT/EXPORT:Empty
      Storage Element 42 IMPORT/EXPORT:Empty
      Storage Element 43 IMPORT/EXPORT:Empty

root@freebsd:/tmp # chio -f /dev/ch0 status
picker 0: <INEAB,EXENAB,ACCESS,IMPEXP>
slot 0: <INEAB,EXENAB,ACCESS,IMPEXP>
slot 0:  serial number: <>
slot 0: 
slot 0: 
slot 0: 
slot 0: <ACCESS,EXCEPT,IMPEXP> serial number: <>
slot 0:  serial number: <>
slot 0: 
slot 0:  serial number: <>
slot 0: 
slot 0: <EXENAB,ACCESS> serial number: <>
slot 0: 
slot 0: 
slot 0: 
slot 0: <FULL>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,EXENAB,ACCESS,FULL>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,ACCESS,EXCEPT>
slot 0: <INEAB,EXENAB,EXCEPT,FULL>
slot 24: 
slot 0:  serial number: <>
slot 0:  serial number: <>
slot 0: <EXENAB,ACCESS,EXCEPT> serial number: <>
slot 0: <INEAB,EXCEPT>
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
portal 0: <ACCESS,FULL> serial number: <IBM     ULT3580-TD5     XYZZY_A1  >
portal 1: <ACCESS> serial number: <IBM     ULT3580-TD5     XYZZY_A2  >
portal 2: <ACCESS> serial number: <IBM     ULT3580-TD4     XYZZY_A3  >
portal 3: <ACCESS> serial number: <IBM     ULT3580-TD4     XYZZY_A4  >
drive 0: <ACCESS>
drive 1: <ACCESS,FULL>
drive 2: <ACCESS,FULL>
drive 3: <ACCESS,FULL>

As you can see, the FreeBSD output is incorrect/garbled/whatever.

The only debug I have so far is a lot of mhvtl CDB output in dmesg and the odd console syslog message from tgtd:
"Message from syslogd@centos at Sep  5 19:56:22 ...
 tgtd:

Any thoughts ?

Thank you.

Reply | Threaded
Open this post in threaded view
|

Re: Issues using mhvtl with freebsd and iscsi

Mark Harvey
Administrator
One thing you could try is configure iSCSI initiator from centOS to itself and try mtx to the iSCSI devices. 

Can you post the tgtd setup please. 

Sent from my autocorrecting iPhone

On 6 Sep 2015, at 11:14, obdevel [via mhVTL - A Linux Virtual Tape Library] <[hidden email]> wrote:

Hi

Apologies for spamming the list with such a long first post, as I can't definitively point the finger at a mhvtl problem, but I'd appreciate some help.

I have mhvtl running on Centos 7 under VMWare, exported via tgt iscsi to a number of other platforms. Standard config created using mhvtlgui. Tested on several other linuxes (32 & 64 bit) and Win7/2012R2 which all work absolutely fine, as you'd expect. Tape drive access works fine on FreeBSD 10.1 but I get strangeness operating the tape library.

Clearly, the problem may well be in tgt or the FreeBSD iscsi initiator, but any pointers for debugging would be welcomed.

Target environment:
[root@centos tmp]# uname -a
Linux centos 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@centos tmp]# vtlcmd -v
Version: 1.5.3-git-f993138
[root@centos tmp]# lsscsi -g
[0:0:0:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sda   /dev/sg0 
[0:0:1:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sdb   /dev/sg1 
[2:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0   /dev/sg2 
[3:0:0:0]    mediumx STK      L700             0105  /dev/sch0  /dev/sg12
[3:0:1:0]    tape    IBM      ULT3580-TD5      0105  /dev/st0   /dev/sg3 
[3:0:2:0]    tape    IBM      ULT3580-TD5      0105  /dev/st1   /dev/sg4 
[3:0:3:0]    tape    IBM      ULT3580-TD4      0105  /dev/st2   /dev/sg5 
[3:0:4:0]    tape    IBM      ULT3580-TD4      0105  /dev/st3   /dev/sg6 
[3:1:0:0]    mediumx IBM      03584L22         4.02  /dev/sch1  /dev/sg13
[3:1:0:1]    tape    IBM      ULT3580-TD4      252D  /dev/st4   /dev/sg7 
[3:1:0:2]    tape    IBM      ULT3580-TD4      252D  /dev/st5   /dev/sg8 
[3:1:0:3]    tape    IBM      ULT3580-TD4      252D  /dev/st6   /dev/sg9 
[3:1:0:4]    tape    IBM      ULT3580-TD4      252D  /dev/st7   /dev/sg10
[3:1:0:5]    tape    IBM      ULT3580-TD4      252D  /dev/st8   /dev/sg11
[root@centos tmp]# tgtadm --version
1.0.46

Initiator environment:
root@freebsd:/tmp # uname -a
FreeBSD freebsd 10.1-RELEASE-p16 FreeBSD 10.1-RELEASE-p16 #0: Tue Jul 28 12:04:19 UTC 2015     [hidden email]:/usr/obj/usr/src/sys/GENERIC  amd64
root@freebsd:/tmp # iscsictl -L
Target name                          Target portal    State
iqn.1994-05.com.redhat:dfa419db7c60:mhvtl:stgt:1 192.168.97.148   Connected: ch0 sa0 sa2 sa3 sa5 
iqn.1994-05.com.redhat:dfa419db7c60:mhvtl:stgt:2 192.168.97.148   Connected: ch1 sa1 sa4 sa6 sa7 sa8 
root@freebsd:/tmp # camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00>   at scbus1 target 0 lun 0 (pass0,cd0)
<VMware, VMware Virtual S 1.0>     at scbus2 target 0 lun 0 (pass1,da0)
<IET Controller 0001>              at scbus3 target 0 lun 0 (pass2)
<STK L700 0105>                    at scbus3 target 0 lun 1 (ch0,pass4)
<IBM ULT3580-TD5 0105>             at scbus3 target 0 lun 2 (sa0,pass6)
<IBM ULT3580-TD5 0105>             at scbus3 target 0 lun 3 (sa2,pass8)
<IBM ULT3580-TD4 0105>             at scbus3 target 0 lun 4 (sa3,pass9)
<IBM ULT3580-TD4 0105>             at scbus3 target 0 lun 5 (sa5,pass11)
<IET Controller 0001>              at scbus4 target 0 lun 0 (pass3)
<IBM 03584L22 4.02>                at scbus4 target 0 lun 1 (ch1,pass5)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 2 (sa1,pass7)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 3 (sa4,pass10)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 4 (sa6,pass12)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 5 (sa7,pass13)
<IBM ULT3580-TD4 252D>             at scbus4 target 0 lun 6 (sa8,pass14)

Comparing inventory output on the two platforms:

[root@centos tmp]# mtx -f /dev/sg12 status
  Storage Changer /dev/sg12:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Full (Storage Element 1 Loaded):VolumeTag = E01001L4                            
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
      Storage Element 1:Empty
      Storage Element 2:Full :VolumeTag=E01002L4                            
      Storage Element 3:Full :VolumeTag=E01003L4                            
      Storage Element 4:Full :VolumeTag=E01004L4                            
      Storage Element 5:Full :VolumeTag=E01005L4                            
      Storage Element 6:Full :VolumeTag=E01006L4                            
      Storage Element 7:Full :VolumeTag=E01007L4                            
      Storage Element 8:Full :VolumeTag=E01008L4                            
      Storage Element 9:Full :VolumeTag=E01009L4                            
      Storage Element 10:Full :VolumeTag=E01010L4                            
      Storage Element 11:Full :VolumeTag=E01011L4                            
      Storage Element 12:Full :VolumeTag=E01012L4                            
      Storage Element 13:Full :VolumeTag=E01013L4                            
      Storage Element 14:Full :VolumeTag=E01014L4                            
      Storage Element 15:Full :VolumeTag=E01015L4                            
      Storage Element 16:Full :VolumeTag=E01016L4                            
      Storage Element 17:Full :VolumeTag=E01017L4                            
      Storage Element 18:Full :VolumeTag=E01018L4                            
      Storage Element 19:Full :VolumeTag=E01019L4                            
      Storage Element 20:Full :VolumeTag=E01020L4                            
      Storage Element 21:Empty
      Storage Element 22:Full :VolumeTag=CLN101L4                            
      Storage Element 23:Full :VolumeTag=CLN102L5                            
      Storage Element 24:Empty
      Storage Element 25:Empty
      Storage Element 26:Empty
      Storage Element 27:Empty
      Storage Element 28:Empty
      Storage Element 29:Empty
      Storage Element 30:Full :VolumeTag=F01030L5                            
      Storage Element 31:Full :VolumeTag=F01031L5                            
      Storage Element 32:Full :VolumeTag=F01032L5                            
      Storage Element 33:Full :VolumeTag=F01033L5                            
      Storage Element 34:Full :VolumeTag=F01034L5                            
      Storage Element 35:Full :VolumeTag=F01035L5                            
      Storage Element 36:Full :VolumeTag=F01036L5                            
      Storage Element 37:Full :VolumeTag=F01037L5                            
      Storage Element 38:Full :VolumeTag=F01038L5                            
      Storage Element 39:Full :VolumeTag=F01039L5                            
      Storage Element 40 IMPORT/EXPORT:Empty
      Storage Element 41 IMPORT/EXPORT:Empty
      Storage Element 42 IMPORT/EXPORT:Empty
      Storage Element 43 IMPORT/EXPORT:Empty

root@freebsd:/tmp # chio -f /dev/ch0 status
picker 0: <INEAB,EXENAB,ACCESS,IMPEXP>
slot 0: <INEAB,EXENAB,ACCESS,IMPEXP>
slot 0:  serial number: <>
slot 0: 
slot 0: 
slot 0: 
slot 0: <ACCESS,EXCEPT,IMPEXP> serial number: <>
slot 0:  serial number: <>
slot 0: 
slot 0:  serial number: <>
slot 0: 
slot 0: <EXENAB,ACCESS> serial number: <>
slot 0: 
slot 0: 
slot 0: 
slot 0: <FULL>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,EXENAB,ACCESS,FULL>
slot 0: <INEAB,IMPEXP>
slot 0: <INEAB,ACCESS,EXCEPT>
slot 0: <INEAB,EXENAB,EXCEPT,FULL>
slot 24: 
slot 0:  serial number: <>
slot 0:  serial number: <>
slot 0: <EXENAB,ACCESS,EXCEPT> serial number: <>
slot 0: <INEAB,EXCEPT>
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
slot 0: 
portal 0: <ACCESS,FULL> serial number: <IBM     ULT3580-TD5     XYZZY_A1  >
portal 1: <ACCESS> serial number: <IBM     ULT3580-TD5     XYZZY_A2  >
portal 2: <ACCESS> serial number: <IBM     ULT3580-TD4     XYZZY_A3  >
portal 3: <ACCESS> serial number: <IBM     ULT3580-TD4     XYZZY_A4  >
drive 0: <ACCESS>
drive 1: <ACCESS,FULL>
drive 2: <ACCESS,FULL>
drive 3: <ACCESS,FULL>

As you can see, the FreeBSD output is incorrect/garbled/whatever.

The only debug I have so far is a lot of mhvtl CDB output in dmesg and the odd console syslog message from tgtd:
"Message from syslogd@centos at Sep  5 19:56:22 ...
 tgtd:

Any thoughts ?

Thank you.




If you reply to this email, your message will be added to the discussion below:
http://mhvtl-a-linux-virtual-tape-library.966029.n3.nabble.com/Issues-using-mhvtl-with-freebsd-and-iscsi-tp4025932.html
To start a new topic under mhVTL - A Linux Virtual Tape Library, email [hidden email]
To unsubscribe from mhVTL - A Linux Virtual Tape Library, click here.
NAML
Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: Issues using mhvtl with freebsd and iscsi

obdevel
Thanks for the suggestions.

At this time I don't know whether it's the freebsd changer program (chio) sending strange CDBs or tgt messing with them.

>> One thing you could try is configure iSCSI initiator from centOS to itself and try mtx to the iSCSI devices.
Works perfectly

>> Can you post the tgtd setup please.
Default configuration using the mhvtl-gui. I haven't changed this, nor do I really know how to. Time to learn more I guess :)
default-driver iscsi
	backing-store /dev/sg12
	backing-store /dev/sg3
	backing-store /dev/sg4
	backing-store /dev/sg5
	backing-store /dev/sg6
        bs-type sg
        device-type pt

	backing-store /dev/sg13
	backing-store /dev/sg7
	backing-store /dev/sg8
	backing-store /dev/sg9
	backing-store /dev/sg10
	backing-store /dev/sg11
        bs-type sg
        device-type pt

Here are the CDB entries from /var/log/messages when performing an inventory on the first library in my config:

1. using mtx(1) on the linux host direct to the vtl: (mtx -f /dev/sg12 status)
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731334) (delay 1000005): 12 00 00 00 38 00
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731335) (delay 405): 1a 08 1d 00 88 00
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731336) (delay 405): b8 12 03 e8 00 27 00 00 0f e8 00 00
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731337) (delay 405): b8 13 00 0a 00 04 00 00 0f e8 00 00
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731338) (delay 405): b8 14 01 f4 00 04 00 00 0f e8 00 00
Sep  7 02:18:31 centos vtllibrary[8856]: CDB (1731339) (delay 405): b8 11 00 01 00 01 00 00 0f e8 00 00

2. using chio(1) on the freebsd host via tgt/iscsi: (chio -f /dev/ch0 status)
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731407) (delay 443605): 1a 08 1d 00 20 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731408) (delay 405): 1a 08 1f 00 20 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731409) (delay 405): b8 00 00 01 00 01 03 00 04 00 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731410) (delay 805): b8 00 00 01 00 01 03 00 00 20 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731411) (delay 805): b8 00 03 e8 00 01 03 00 04 00 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731412) (delay 1205): b8 00 03 e8 00 27 03 00 02 80 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731413) (delay 1605): b8 00 00 0a 00 01 03 00 04 00 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731414) (delay 1205): b8 00 00 0a 00 04 03 00 00 d8 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731415) (delay 1205): b8 00 01 f4 00 01 03 00 04 00 00 00
Sep  7 02:31:49 centos vtllibrary[8856]: CDB (1731416) (delay 1205): b8 00 01 f4 00 04 03 00 00 50 00 00

Why should they be so different for essentially the same operation ?

Lastly, the CDBs generated as the freebsd kernel discovers the devices and creates its local special files (sequentially, between the previous two operations, using #iscsictl -A -d 192.168.97.148):
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731340) (delay 662805): 12 00 00 00 24 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731341) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731342) (delay 805): 12 00 00 00 40 00
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731344) (delay 1205): 12 01 00 00 ff 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731343) (delay 5): 12 00 00 00 3a 00
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731345) (delay 805): 12 01 80 00 ff 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731346) (delay 405): 12 01 00 00 ff 00
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731347) (delay 1205): 00 00 00 00 00 00
Sep  7 02:27:42 centos vtllibrary[8856]: CDB (1731349) (delay 1205): 1a 08 1d 00 20 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731348) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731351) (delay 1605): 12 01 80 00 ff 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731352) (delay 1205): 00 00 00 00 00 00
Sep  7 02:27:42 centos vtllibrary[8861]: CDB (1731353) (delay 1205): 1a 08 1d 00 20 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731354) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731350) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731355) (delay 5): 12 00 00 00 40 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731356) (delay 5): 12 00 00 00 40 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731358) (delay 1205): 12 01 00 00 ff 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731357) (delay 1205): 12 01 00 00 ff 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731360) (delay 805): 12 01 83 00 fc 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731359) (delay 805): 12 01 83 00 fc 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731362) (delay 805): 12 01 80 00 ff 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731361) (delay 805): 12 01 80 00 ff 00
Sep  7 02:27:43 centos vtltape[8828]: CDB (1731363) (delay 1205): 00 00 00 00 00 00
Sep  7 02:27:43 centos vtltape[8797]: CDB (1731364) (delay 1205): 00 00 00 00 00 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731365) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731367) (delay 5): 12 00 00 00 40 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731368) (delay 405): 12 01 00 00 ff 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731369) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731370) (delay 405): 12 01 80 00 ff 00
Sep  7 02:27:44 centos vtltape[8801]: CDB (1731371) (delay 405): 00 00 00 00 00 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731366) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731373) (delay 1205): 12 00 00 00 40 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731374) (delay 1205): 12 01 00 00 ff 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731375) (delay 805): 12 01 83 00 fc 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731376) (delay 1205): 12 01 80 00 ff 00
Sep  7 02:27:44 centos vtltape[8831]: CDB (1731377) (delay 805): 00 00 00 00 00 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731372) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731379) (delay 805): 12 00 00 00 40 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731380) (delay 805): 12 01 00 00 ff 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731381) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731382) (delay 405): 12 01 80 00 ff 00
Sep  7 02:27:45 centos vtltape[8805]: CDB (1731383) (delay 405): 00 00 00 00 00 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731378) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731385) (delay 1205): 12 00 00 00 40 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731386) (delay 1205): 12 01 00 00 ff 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731387) (delay 1205): 12 01 83 00 fc 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731388) (delay 1205): 12 01 80 00 ff 00
Sep  7 02:27:45 centos vtltape[8837]: CDB (1731389) (delay 1205): 00 00 00 00 00 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731390) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731391) (delay 5): 12 00 00 00 40 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731392) (delay 405): 12 01 00 00 ff 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731393) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731394) (delay 405): 12 01 80 00 ff 00
Sep  7 02:27:46 centos vtltape[8841]: CDB (1731395) (delay 405): 00 00 00 00 00 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731384) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731397) (delay 805): 12 00 00 00 40 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731398) (delay 805): 12 01 00 00 ff 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731399) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731400) (delay 405): 12 01 80 00 ff 00
Sep  7 02:27:46 centos vtltape[8820]: CDB (1731401) (delay 405): 00 00 00 00 00 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731396) (delay 1000005): 12 00 00 00 24 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731402) (delay 805): 12 00 00 00 40 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731403) (delay 805): 12 01 00 00 ff 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731404) (delay 405): 12 01 83 00 fc 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731405) (delay 405): 12 01 80 00 ff 00
Sep  7 02:27:47 centos vtltape[8846]: CDB (1731406) (delay 405): 00 00 00 00 00 00

Thanks again Mark.
Reply | Threaded
Open this post in threaded view
|

Re: Issues using mhvtl with freebsd and iscsi

obdevel
Ok, so I'd forgotten that mtx is available for freebsd. The binary distribution doesn't work (assume 32 bit) so I compiled it up and it works perfectly :)

So, unless you're interested in pursuing this as a purely intellectual exercise, I think we can put this issue to bed and leave the solution/workaround for posterity. Happy to assist if you want more info.

One more nugget for anyone wanting to talk to tape devices on freebsd, buffers passed to the read() and write() system calls need to be aligned on page boundaries. You can't just allocate on the stack, e.g.

        char *buff, *label
        // allocate memory on page boundary
        pagesize = sysconf(_SC_PAGESIZE);
        posix_memalign((void **)&buff, pagesize, BLOCKSIZE);
        posix_memalign((void **)&label, pagesize, LABELSIZE);

And lastly, tgt performance is not great. I can easily get 200MB/s throughput to a local virtual tape device with v low cpu usage. From other hosts using tgt/iscsi (over the vmware internal network) I'm lucky to see 40MB/s and tgtd uses 50% cpu time.

Thanks again.