Re: sa(4) driver changes available for test

> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry <ken_at_freebsd.org> wrote:
> 
> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>> 
>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry <ken_at_freebsd.org> wrote:
>>> 
>>> 
>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver
>>> that I'm planning to commit in the near future.
>>> 
>>> A description of the changes is here and below in this message.
>>> 
>>> If you have tape hardware and the inclination, I'd appreciate testing and
>>> feedback.
>> 
>> I have a DLT 8000 and an SDLT 220.
>> 
>> I don't have anything running current, but I have a spare machine which I could use for testing.
>> 
>> Do you see any value is tests with that hardware? I'd be testing it via Bacula.
>> 
>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
>> 
> 
> Actually, yes.  Bacula is a bit tricky to configure, so your trying it out
> would be helpful if you have the time.
> 
> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
> claim to support long position information for the SCSI READ POSITION
> command.
> 
> You can see what I'm talking about by doing:
> 
> mt eod
> mt status
> 
> On my DDS-4 tape drive, this shows:
> 
> # mt -f /dev/nsa3 status
> Drive: sa3: <SEAGATE DAT    06240-XXX 8071> Serial Number: HJ00YWY
> ---------------------------------
> Mode      Density              Blocksize      bpi      Compression
> Current:  0x26:DDS-4           1024 bytes     97000    enabled (DCLZ)
> ---------------------------------
> Current Driver State: at rest.
> ---------------------------------
> Partition:   0      Calc File Number:  -1     Calc Record Number: -1
> Residual:    0  Reported File Number:  -1 Reported Record Number: -1
> Flags: None
> 
> But on an LTO-5, which will give long position information, I get:
> 
> [root_at_doc ~]# mt status
> Drive: sa0: <IBM ULTRIUM-HH5 E4J1>
> ---------------------------------
> Mode      Density              Blocksize      bpi      Compression
> Current:  0x58:LTO-5           variable       384607   enabled (0x1)
> ---------------------------------
> Current Driver State: at rest.
> ---------------------------------
> Partition:   0      Calc File Number:   2     Calc Record Number: -1
> Residual:    0  Reported File Number:   2 Reported Record Number: 32373
> Flags: None
> 
> That, in combination with the changes I made to the position information
> code in the driver, mean that even the old MTIOCGET ioctl should return an
> accurate file number at end of data.  e.g., on the LTO-5:
> 
> [root_at_doc ~]# mt ostatus
> Mode      Density              Blocksize      bpi      Compression
> Current:  0x58:LTO-5           variable       384607   0x1
> ---------available modes---------
> 0:        0x58:LTO-5           variable       384607   0x1
> 1:        0x58:LTO-5           variable       384607   0x1
> 2:        0x58:LTO-5           variable       384607   0x1
> 3:        0x58:LTO-5           variable       384607   0x1
> ---------------------------------
> Current Driver State: at rest.
> ---------------------------------
> File Number: 2  Record Number: -1       Residual Count -1
> 
> So the thing to try, in addition to just making sure that Bacula continues
> to work properly, is to try setting this for the tape drive in
> bacula-sd.conf:
> 
>  Hardware End of Medium = yes
> 
> It looks like the Bacula tape program (btape) has a test mode, and it would
> be good to run through the tests on one of the tape drives and see whether
> they work, and whether the results are different before and after the
> changes.  I'm not sure how to enable the test mode.

I have this in /usr/local/etc/bacula/bacula-sd.conf

Device {
  Name                    = DLT
  Description             = "QUANTUM DLT7000 1624"
  Media Type              = DLT
  Archive Device          = /dev/nsa1

  Autochanger             = YES
  Drive Index             = 0

  Offline On Unmount      = no
  Hardware End of Medium  = yes
  BSF at EOM              = yes
  Backward Space Record   = no
  Fast Forward Space File = no
  TWO EOF                 = yes
}

FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model.

Here's the test I ran tonight:

[root_at_cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1                                                                                                
Tape block granularity is 1024 bytes.
btape: butil.c:287-0 Using device: "/dev/nsa1" for writing.
btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 10000 records and an EOF
then write 10000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1210-0 Rewind OK.
10000 blocks re-read correctly.
Got EOF on tape.
10000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===

btape: btape.c:1277-0 Block position test
btape: btape.c:1289-0 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:9999
Block 10000 re-read correctly.
Reposition to file:block 1:0
Block 10001 re-read correctly.
Reposition to file:block 1:600
Block 10601 re-read correctly.
Reposition to file:block 1:9999
Block 20000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1420-0 Now moving to end of medium.
btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0.
We should be in file 3. I am at file 0. This is NOT correct!!!!

Append test failed. Attempting again.
Setting "Hardware End of Medium = no
    and "Fast Forward Space File = no
and retrying append test.



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1420-0 Now moving to end of medium.
btape: btape.c:625-0 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 4. This is correct!


It looks like the test worked this time, please add:

    Hardware End of Medium = No

    Fast Forward Space File = No
to your Device resource in the Storage conf file.

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to 
the tape.

Skipping read backwards test because BSR turned off.


=== Forward space files test ===

This test is essential to Bacula.

I'm going to write five files then test forward spacing

btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:1907-0 Wrote one record of 64412 bytes.
btape: btape.c:1909-0 Wrote block to device.
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1634-0 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1646-0 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1659-0 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!

btape: btape.c:1677-0 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!

=== End Forward space files test ===


Ah, I see you have an autochanger configured.
To test the autochanger you must have a blank tape
 that I can write on in Slot 1.

Do you wish to continue with the Autochanger test? (y/n): y


=== Autochanger test ===

3301 Issuing autochanger "loaded" command.
Nothing loaded in the drive. OK.
3303 Issuing autochanger "load 1 0" command.
3303 Autochanger "load 1 0" status is OK.
btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1)
btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1)

The test autochanger worked!!

*

> 
>> I'll let the other Bacula devs know about this.  They deal with the hardware.  I work on PostgreSQL.
>> 
> 
> Thanks!  If there are additional features they would like out of the tape
> driver, I'm happy to talk about it.  (Or help if they'd like to use the new
> status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.)

Errors are interesting to me.  Especially corrected errors. They are a good indicator of tape quality.

— 
Dan Langille
http://langille.org/