ANNOUNCE: 0.19-0Beta release.

classic Classic list List threaded Threaded
Locked 24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
This post was updated on .
In time for Remembrance Day.

This is a pre-build/release of code with the new 'personality module' framework.

https://sites.google.com/site/linuxvtl2/home#vtl-download-devel

- Based on fixes which will be included in 0.18.12 (when released)
- The 0.19 branch will be to develop the idea of a 'Personality Modules' which will be a 'plugin' style module per device type.
  Currently, IBM LTO4, LTO5, IBM 03592, STK T10K and AIT are partially implemented.

No change to the kernel module is required if you already have the 0.18-11 release installed.

Lightly tested with NetBackup 6.5.6 and NetBackup 7.0.

I've not tested the Encryption changes.

Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

[nia]
Administrator
Good news Mark ! ..

Thanks
n.i.a
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

perisea
In reply to this post by Mark Harvey
That's great Mark.

But i can't download the package from https://sites.google.com/site/linuxvtl2/, could you send me a copy please? My email is wangye@huaweisymantec.com.

Thanks.
nia
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

nia
Administrator
I have attached the file  .. You can just download here ..

mhvtl-2010-11-11.tgz
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
For ease of install, I've built RPMS for this first beta.

mhvtl-0.19-0Beta.i586.rpm
mhvtl-0.19-0Beta.src.rpm
mhvtl-0.19-0Beta.x86_64.rpm

Please note: I am after any feedback on change of behaviour.

I have only implemented 'personality modules' for the following drive types:
LTO4
LTO5
AIT4
STK T10000A and T10000B
IBM 03592 J1A, E05 and E06

The personality modules are incomplete. i.e. The LTO5 does not support all LTO5 functionality..

So any testing of these drive types and reports of what is working / what is not working will be appreciated.

Cheers
Mark
Regards from Australia
Mark Harvey
nia
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

nia
Administrator
In reply to this post by Mark Harvey
Mark Harvey wrote
- The 0.19 branch will be to develop the idea of a 'Personality Modules' which will be a 'plugin' style module per device type.

Mark, will Library robot devices ever be implemented with same 'Personality Modules' as well ?

I suspect some library robots may behave different than others .. Not sure if true ...

Thanks
n.i.a
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
Yes, the library module will also contain a personality module..

Most people are more concerned with drive type emulation rather than the library (once they find a library type that works for them/their backup application). So the tape drives are first off the list.

I'm planning on the whole '0.19' branch as being a stepping stone and incremental steps for the introduction & implementation of the personality module idea.

I'll attempt to keep each '0.19-XX' release functional..

Cheers
Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

perisea
In reply to this post by nia
n.i.a wrote
I have attached the file  .. You can just download here ..

mhvtl-2010-11-11.tgz
I've got the source project from Mark.
You are very enthusiastic nia, thank you all the same.
nia
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

nia
Administrator
In reply to this post by Mark Harvey
Mark Harvey wrote
Please note: I am after any feedback on change of behaviour.
I have only implemented 'personality modules' for the following drive types:
LTO4
LTO5
AIT4
STK T10000A and T10000B
IBM 03592 J1A, E05 and E06
Mark, any chance for a quick patch to include LTO-1 in new personality modules so I can test with TSM.  As you are maybe aware, TSM does not like anything other than LTO-1 if using IBM Library and and T10000A, T10000B if using STK .. Issue is still pending See Here.. I have already have a setup running using LTO-1 and wanted to upgrade to  0.19-0Beta release and still keep LTO-1 for now.
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
Yep, I'll post a 2nd 'beta' shortly for testing.
Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
OK, Beta2..

It contains personality module for IBM LTO 1/2/3/4 & 5 media.

mhvtl-2010-11-16.tgz

Ongoing conversion across to new format.

Please advise if it works or if you experience any problems with this.

Many thanks
Mark
Regards from Australia
Mark Harvey
nia
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

nia
Administrator
Thanks Mark ..  
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

ap2010
In reply to this post by Mark Harvey
Just tried the second beta out. I am using an LTO1 drive. However, when I move a tape into the drive with mtx it hangs.

This is the debug output in the messages log:

Nov 16 09:36:46 orange vtllibrary[4638]: smc_move_medium: MOVE MEDIUM (743) **
Nov 16 09:36:46 orange vtllibrary[4638]: Moving from slot 1031 to Slot 1 using transport 256, Invert media: no
Nov 16 09:36:46 orange vtllibrary[4638]: move_slot2drive: About to send cmd: 'lload UD1010L1' to drive 11
Nov 16 09:36:46 orange vtltape[4606]: processMessageQ: Q snd_id 10 msg : lload UD1010L1
Nov 16 09:36:46 orange vtltape[4606]: load_tape: Opening file/media /opt/mhvtl/UD1010L1
Nov 16 09:36:46 orange vtltape[4606]: read_header: Reading header 3956 at offset 93453245, type: FILEMARK, size: 0
Nov 16 09:36:46 orange kernel: vtltape[4606]: segfault at 0 ip (null) sp bfad5b7c error 14 in libc-2.11.2.so[110000+172000]
Nov 16 09:36:46 orange vtltape[4606]: read_header: Reading header 0 at offset 0, type: DATA, size: 64512
Nov 16 09:36:46 orange vtltape[4606]: loadTape: Media 'LTO1 Data' loaded with S/No. : UD1010L1_1267815081

What I find a bit confusing is that the library itself has device nr 11, the actual tape drive is 12. The third line wants to insert a tape in drive 11. I don't know if this is of any use.

Albert
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

ap2010
In reply to this post by Mark Harvey
Just contacted Mark,

as I he didn't had access to this forum at the moment he asked me people NOT to use beta2 as it segfaults.

In particular it segfaults with the LTO1 personality in vtltape.

To be continued ...


Albert
nia
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

nia
Administrator
Thanks Albert .. I did the same and had unexpected results as well .. Did not know what was going on at first but now that I see your confirmation, I went back to 0.18-11.

Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
Identified root cause of segfault in IBM LTO1/2/3 personality module.

Attached is 'Beta3' with this fixed.

Testing for those who like the bleeding edge :)

mhvtl-2010-11-17-0.19.0Beta3-git-0.19-beta3.tgz

Reports of the success/failure of any testing most welcome.

Cheers
Mark
Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

ap2010
No good news.

loading a tape works. Then I wrote data to an empty tape, looks to be working. Rewind tape and tried to read it:

[root@orange ~]# mt -f /dev/nst0 rewind
[root@orange ~]# tar tvzf /dev/nst0
tar: /dev/nst0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

gzip: stdin: unexpected end of file
tar: Child returned status 2
tar: Exiting with failure status due to previous errors

I am a bit worried about the "Cannot allocate memory" error.

Then

mtx -f /dev/changer unload

command gives:

vtltape: processMessageQ: Q snd_id 10 msg : unload
vtltape: updateMAM: updateMAM(0)
Segmentation fault
You have new mail in /var/spool/mail/root

Albert

All this done with:

loading mhvtl kernel module and starting all executables by hand:

modprobe mhvtl
vtltape -q 11 -d -v -v -v
vtllibrary -q 10 -d -v -v -v

Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

Mark Harvey
Administrator
Oh dear.. Brown paper bag time...

I fixed the test for 'clear_WORM()' in the beta3 release and forgot about the 'set_WORM()' :(

Time to bring out beta4..

Thanks for your testing & bug reports.
Regards from Australia
Mark Harvey
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

ap2010
Hi Mark,

following your remark I patched vtltape accordingly:

--- vtltape.c.org 2010-11-18 08:48:19.131155228 +0100
+++ vtltape.c 2010-11-18 08:44:56.991155481 +0100
@@ -1839,7 +1839,8 @@
  }
  rewind_tape(sam_stat);
  }
- lu_ssc.pm->set_WORM();
+ if (lu_ssc.pm->set_WORM)
+ lu_ssc.pm->set_WORM();
 
  MHVTL_DBG(1, "Write Once Read Many (WORM) media loaded");
 // set_worm_mode_pg();


Indeed, this seems to work much better now, at least I could a a manual backup/restore and bacula was able to use it properly.

By the way, if you startup vtltape or vtllibrary by hand you'll see the following error:

oom_adjust: Can't adjust oom-killer's pardon /proc/3284/oom_adj, Permission denied

The code  is actually found in vtllib.c and it stems from the fact that vtltape and vtllibrary run as user vtl, while this parameter is only writeable as user root:

-rw-r--r-- 1 root root 0 2010-11-18 09:17 oom_adj

mvhtl works fine, but I think this should be fixed.

Albert


Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: 0.19-0Beta release.

ap2010
Seems there is more to fix than the little patch.

If I use mhvtl manual (loading a tape, writing to it etc) it seems to work.

bacula has more difficulties:

Nov 18 15:34:25 orange vtltape[1476]: processMessageQ: Q snd_id 10 msg : unload
Nov 18 15:34:25 orange kernel: vtltape[1476]: segfault at 0 ip (null) sp bfd594cc error 14 in libvtlscsi.so[101000+f000]
12