NetWorker: mminfo shows savesets that span multiple media volumes as valid when there are missing sections


   Article Number:     540263                                   Article Version: 2     Article Type:    Break Fix 




NetWorker Series,NetWorker,NetWorker Module





This issue affects savesets where the saveset spans multiple media volumes (tape volumes or file type device volumes) and scanner has been used to recreate the media database entries for the saveset.   
    Attempts to clone affected saveset(s) loop with the following message:   
    nsrrecopy: NSR server {hostname}: Cannot find the volume needed     
      nsrrecopy: waiting 30 seconds then retrying...
    The mminfo command for the scanned saveset shows the saveset as valid and complete even if sections of the saveset are missing or have not yet been scanned:   
    mminfo -avot -q ssid={ssid}     
      volume type client date time size ssid fl lvl name     
      000002L5 LTO Ultrium-5 {client_name} 25/06/2019 20:00:42 1806 GB {ssid} mb full {saveset_name}     
      000003L5 LTO Ultrium-5 {client_name} 25/06/2019 20:00:42 1640 GB {ssid} hb full {saveset_name}     
      000005L5 LTO Ultrium-5 {client_name} 25/06/2019 20:00:42 2119 GB {ssid} mb full {saveset_name}     
      000008L5 LTO Ultrium-5 {client_name} 25/06/2019 20:00:42 2341 GB {ssid} tb full {saveset_name}     
      000012L5 LTO Ultrium-5 {client_name} 25/06/2019 20:00:42 1800 GB {ssid} mb full {saveset_name}
    Note: The fl (ssflags) column includes "i" in the flags if a saveset is marked as incomplete.   
    To confirm if there is an issue, check the scanner outputs when scanning the tapes using scanner -m {device_name}, check the output that corresponds to each of the tapes.     
      If the tape containing the header section of the backup (denoted by "h" in the fl (ssflags) column in the
mminfo -avot -q ssid={ssid} output) includes messages simiilar to the following:   
    8851:scanner: ssid {ssid}: continued from `000012L5', scan for synchronization     
      29486:scanner: ssid {ssid}: NOT complete     
      8786:scanner: ssid {ssid}: 3988 GB, 0 file(s)     
      8921:scanner: the following save sets continue on 000005L5:     
      client name  save set             save time     level   size  files   ssid    S     
      {client_name} {saveset_name}      6/25/19 20:00  f  4084464958584      0 {ssid} C
    Then the "real" header section of the saveset is missing and needs to be scanned in.     
      For tapes containing the middle sections of the saveset (denoted by "m" in the fl (ssflags) column in the
mminfo -avot -q ssid={ssid} output), if there is a gap in the complete sequence of tapes (including the header and trailer sections), then there is a missing section of the saveset that needs to be scanned.     
      This issue only occurs after the trailer or tail section of the saveset (denoted by "t" in the fl (ssflags) column in the
mminfo -avot -q ssid={ssid} output) has been been scanned successfully.     
      The scanner output for a tape containing a trailer section will include messages similar to the following:
    8851:scanner: ssid {ssid}: continued from `000005L5', scan for synchronization     
      29485:scanner: ssid 3524423511: scan complete     
      8788:scanner: ssid 3524423511: 0 KB (of 2341 GB, 1 file(s)) read     
      8761:scanner: done with LTO Ultrium-5 tape 000008L5
    To re-confirm that the saveset is incomplete collect the check the mminfo -avVS -q ssid={ssid} output:     
      1) If the total amount of data of the scanned sections does not match the overall saveset size:     
    ssid={ssid} savetime=25/06/2019 20:00:42 (1561489242) {client_name}:{saveset_name}     
        level=full   sflags=vF       size=10502852543348 files=1          insert=19/11/2019
    1806 Gb + 1640 Gb + 2119 Gb + 2341 Gb + 1800 Gb != 10502852543348 bytes   
    then there are one or more missing sections of the saveset on other volumes and the data is not recoverable unless the other sections can be located and scanned to complete the media database information for the saveset.     
      2) If the Clone section of the output does not show the first fragment starting at frag@0:     
Clone #1: cloneid={cloneid}  time=26/06/2019 10:54:10    retent=16/12/2020  flags=     
          frag@0                       volid={volid} file/rec=       {x}/{y}     rn=0 last=14/12/2019
    then the "real" saveset header section is missing      
      If the missing section(s) of the saveset cannot be located on another volume then the only possibility for recovering the data from the available sections of the saveset are through a Data Recovery Bureau.






If a media database is lost due to a disaster and it is not possible to recover the media database from a bootstrap backup, then all media volumes containing backups need to be scanned to rebuild the media database contents.   
    When scanning in savesets that span multiple volumes, scanning in the trailer section of a saveset causes NetWorker to mark the saveset as complete and mark each of the previous sections as a header and one or more middle sections based on the sequence of sections present in the media database at that time.






This is a known issue that will be addressed in a future release.