Re: sa(4) driver changes available for test

From: Dan Langille <dan_at_langille.org>
Date: Sun, 1 Mar 2015 19:41:07 -0500
> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry <ken_at_FreeBSD.ORG> wrote:
> 
> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
>> 
>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry <ken_at_freebsd.org> wrote:
>>> 
>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>>>> 
>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry <ken_at_freebsd.org> wrote:
>>>>> 
>>>>> 
>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver
>>>>> that I'm planning to commit in the near future.
>>>>> 
>>>>> A description of the changes is here and below in this message.
>>>>> 
>>>>> If you have tape hardware and the inclination, I'd appreciate testing and
>>>>> feedback.
>>>> 
>>>> I have a DLT 8000 and an SDLT 220.
>>>> 
>>>> I don't have anything running current, but I have a spare machine which I could use for testing.
>>>> 
>>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula.
>>>> 
>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
>>>> 
>>> 
>>> Actually, yes.  Bacula is a bit tricky to configure, so your trying it out
>>> would be helpful if you have the time.
>>> 
>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
>>> claim to support long position information for the SCSI READ POSITION
>>> command.
>>> 
>>> You can see what I'm talking about by doing:
>>> 
>>> mt eod
>>> mt status
>>> 
>>> On my DDS-4 tape drive, this shows:
>>> 
>>> # mt -f /dev/nsa3 status
>>> Drive: sa3: <SEAGATE DAT    06240-XXX 8071> Serial Number: HJ00YWY
>>> ---------------------------------
>>> Mode      Density              Blocksize      bpi      Compression
>>> Current:  0x26:DDS-4           1024 bytes     97000    enabled (DCLZ)
>>> ---------------------------------
>>> Current Driver State: at rest.
>>> ---------------------------------
>>> Partition:   0      Calc File Number:  -1     Calc Record Number: -1
>>> Residual:    0  Reported File Number:  -1 Reported Record Number: -1
>>> Flags: None
>>> 
>>> But on an LTO-5, which will give long position information, I get:
>>> 
>>> [root_at_doc ~]# mt status
>>> Drive: sa0: <IBM ULTRIUM-HH5 E4J1>
>>> ---------------------------------
>>> Mode      Density              Blocksize      bpi      Compression
>>> Current:  0x58:LTO-5           variable       384607   enabled (0x1)
>>> ---------------------------------
>>> Current Driver State: at rest.
>>> ---------------------------------
>>> Partition:   0      Calc File Number:   2     Calc Record Number: -1
>>> Residual:    0  Reported File Number:   2 Reported Record Number: 32373
>>> Flags: None
>>> 
>>> That, in combination with the changes I made to the position information
>>> code in the driver, mean that even the old MTIOCGET ioctl should return an
>>> accurate file number at end of data.  e.g., on the LTO-5:
>>> 
>>> [root_at_doc ~]# mt ostatus
>>> Mode      Density              Blocksize      bpi      Compression
>>> Current:  0x58:LTO-5           variable       384607   0x1
>>> ---------available modes---------
>>> 0:        0x58:LTO-5           variable       384607   0x1
>>> 1:        0x58:LTO-5           variable       384607   0x1
>>> 2:        0x58:LTO-5           variable       384607   0x1
>>> 3:        0x58:LTO-5           variable       384607   0x1
>>> ---------------------------------
>>> Current Driver State: at rest.
>>> ---------------------------------
>>> File Number: 2  Record Number: -1       Residual Count -1
>>> 
>>> So the thing to try, in addition to just making sure that Bacula continues
>>> to work properly, is to try setting this for the tape drive in
>>> bacula-sd.conf:
>>> 
>>> Hardware End of Medium = yes
>>> 
>>> It looks like the Bacula tape program (btape) has a test mode, and it would
>>> be good to run through the tests on one of the tape drives and see whether
>>> they work, and whether the results are different before and after the
>>> changes.  I'm not sure how to enable the test mode.
>> 
>> I have this in /usr/local/etc/bacula/bacula-sd.conf
>> 
>> Device {
>>  Name                    = DLT
>>  Description             = "QUANTUM DLT7000 1624"
>>  Media Type              = DLT
>>  Archive Device          = /dev/nsa1
>> 
>>  Autochanger             = YES
>>  Drive Index             = 0
>> 
>>  Offline On Unmount      = no
>>  Hardware End of Medium  = yes
>>  BSF at EOM              = yes
>>  Backward Space Record   = no
>>  Fast Forward Space File = no
>>  TWO EOF                 = yes
>> }
>> 
>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model.
>> 
>> Here's the test I ran tonight:
>> 
>> [root_at_cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1                                                                                                
>> Tape block granularity is 1024 bytes.
>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing.
>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
>> *test
>> 
>> === Write, rewind, and re-read test ===
>> 
>> I'm going to write 10000 records and an EOF
>> then write 10000 records and an EOF, then rewind,
>> and re-read the data to verify that it is correct.
>> 
>> This is an *essential* feature ...
>> 
>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1210-0 Rewind OK.
>> 10000 blocks re-read correctly.
>> Got EOF on tape.
>> 10000 blocks re-read correctly.
>> === Test Succeeded. End Write, rewind, and re-read test ===
>> 
>> btape: btape.c:1277-0 Block position test
>> btape: btape.c:1289-0 Rewind OK.
>> Reposition to file:block 0:4
>> Block 5 re-read correctly.
>> Reposition to file:block 0:200
>> Block 201 re-read correctly.
>> Reposition to file:block 0:9999
>> Block 10000 re-read correctly.
>> Reposition to file:block 1:0
>> Block 10001 re-read correctly.
>> Reposition to file:block 1:600
>> Block 10601 re-read correctly.
>> Reposition to file:block 1:9999
>> Block 20000 re-read correctly.
>> === Test Succeeded. End Write, rewind, and re-read test ===
>> 
>> 
>> 
>> === Append files test ===
>> 
>> This test is essential to Bacula.
>> 
>> I'm going to write one record  in file 0,
>>                   two records in file 1,
>>             and three records in file 2
>> 
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1420-0 Now moving to end of medium.
> 
> This is the critical piece.  The test moves the tape to the end of the
> medium.  With hardware position information, you can tell what filemark
> you're on.  Without it, you can't.
> 
>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0.
>> We should be in file 3. I am at file 0. This is NOT correct!!!!
>> 
>> Append test failed. Attempting again.
>> Setting "Hardware End of Medium = no
>>    and "Fast Forward Space File = no
>> and retrying append test.
> 
> This is not surprsing, given that the drive doesn't support long read
> position data.  (It's a SCSI-2 device.)  So that means that Bacula will
> need to do it manually.

Yes, I have nothing newer than SCSI-2.  Even my SDLT is SCSI-2 but that
tape library is hooked up to a different computer and was doing backups today.

> === Append files test ===
>> 
>> This test is essential to Bacula.
>> 
>> I'm going to write one record  in file 0,
>>                   two records in file 1,
>>             and three records in file 2
>> 
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1420-0 Now moving to end of medium.
>> btape: btape.c:625-0 Moved to end of medium.
>> We should be in file 3. I am at file 3. This is correct!
>> 
>> Now the important part, I am going to attempt to append to the tape.
>> 
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> Done appending, there should be no I/O errors
>> 
>> Doing Bacula scan of blocks:
>> 1 block of 64448 bytes in file 1
>> End of File mark.
>> 2 blocks of 64448 bytes in file 2
>> End of File mark.
>> 3 blocks of 64448 bytes in file 3
>> End of File mark.
>> 1 block of 64448 bytes in file 4
>> End of File mark.
>> Total files=4, blocks=7, bytes = 451,136
>> End scanning the tape.
>> We should be in file 4. I am at file 4. This is correct!
>> 
>> 
>> It looks like the test worked this time, please add:
>> 
>>    Hardware End of Medium = No
>> 
>>    Fast Forward Space File = No
>> to your Device resource in the Storage conf file.
>> 
>> The above Bacula scan should have output identical to what follows.
>> Please double check it ...
>> === Sample correct output ===
>> 1 block of 64448 bytes in file 1
>> End of File mark.
>> 2 blocks of 64448 bytes in file 2
>> End of File mark.
>> 3 blocks of 64448 bytes in file 3
>> End of File mark.
>> 1 block of 64448 bytes in file 4
>> End of File mark.
>> Total files=4, blocks=7, bytes = 451,136
>> === End sample correct output ===
>> 
>> If the above scan output is not identical to the
>> sample output, you MUST correct the problem
>> or Bacula will not be able to write multiple Jobs to 
>> the tape.
>> 
>> Skipping read backwards test because BSR turned off.
>> 
>> 
>> === Forward space files test ===
>> 
>> This test is essential to Bacula.
>> 
>> I'm going to write five files then test forward spacing
>> 
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
>> btape: btape.c:1909-0 Wrote block to device.
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1634-0 Now forward spacing 1 file.
>> We should be in file 1. I am at file 1. This is correct!
>> btape: btape.c:1646-0 Now forward spacing 2 files.
>> We should be in file 3. I am at file 3. This is correct!
>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1659-0 Now forward spacing 4 files.
>> We should be in file 4. I am at file 4. This is correct!
>> 
>> btape: btape.c:1677-0 Now forward spacing 1 more file.
>> We should be in file 5. I am at file 5. This is correct!
>> 
>> === End Forward space files test ===
>> 
>> 
>> Ah, I see you have an autochanger configured.
>> To test the autochanger you must have a blank tape
>> that I can write on in Slot 1.
>> 
>> Do you wish to continue with the Autochanger test? (y/n): y
>> 
>> 
>> === Autochanger test ===
>> 
>> 3301 Issuing autochanger "loaded" command.
>> Nothing loaded in the drive. OK.
>> 3303 Issuing autochanger "load 1 0" command.
>> 3303 Autochanger "load 1 0" status is OK.
>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
>> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1)
>> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1)
>> 
>> The test autochanger worked!!
> 
> Great, thanks for running the test!  Looks like things are working as well
> as they were before.
> 
>>> 
>>>> I'll let the other Bacula devs know about this.  They deal with the hardware.  I work on PostgreSQL.
>>>> 
>>> 
>>> Thanks!  If there are additional features they would like out of the tape
>>> driver, I'm happy to talk about it.  (Or help if they'd like to use the new
>>> status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.)
>> 
>> Errors are interesting to me.  Especially corrected errors. They are a good indicator of tape quality.
>> 
> 
> Yes.  At least on modern drives, there is a good bit available in the log
> pages.  You can try seeing what logs your drive supports by installing the
> sg3_utils package and running 'sg_logs sa1 --all'.
> 
> Given the large amount of data available on some drives, and the difficulty
> distilling it down to a clear good/bad, I probably won't stick that in the
> 'mt status' output.
> 
> But you can certainly take a look at it and have an idea of whether your
> particular tape/drive are experiencing issues.

That's a lot of output: https://gist.github.com/dlangille/0e15a7fbf7acab56fd32

I will run some Bacula jobs soon.  I'm still setting up config files.

— 
Dan Langille
http://langille.org/
Received on Sun Mar 01 2015 - 23:41:09 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:56 UTC