Here comes second part of my post.  Before I continue, few notes in respect to anonymous questions I got:

  • Yes, DA on feedbackcentral is closed now as target audience has been reached.  No, I won't send you any binaries.
  • NetApp C-mode support in NW9 (replication) is (or was) targeted for GA release.  Same goes for RP and XIO integration.

 

And one part where I have to semi-correct myself - NVE.  NVE has been released to support SDDC buzzword (Software Defined Data Center).  Well, I can understand that for DD, but still not for NW since NW was software anyway.  I still didn't see what difference may be between NVE and NW when I (or you) install it, but it might be some kind of licensing and surely OS settings which are tuned and tweaked (something many customers like to ignore, but they won't miss the opportunity to criticise performance later on). It is still the part I must grasp. I also found that Snapshot on NAS enhancement now has been extended to Business Data Lake apps (see EMC Federation – Business Data Lakes), but I don't use it so I can't comment much on it as I will focus on basic things here.

 

In first part - Part I: What is NetWorker 9? - I gave partial and theoretical overview of what is new.  I say partial as I didn't cover certain things I didn't use and I surely missed some.  Theoretical as I didn't show and how-to except couple of screenshots while I was doing Windows installation (those don't count much).  In last part of Part I: What is NetWorker 9? you also saw some small test env I quickly made for what would be attempt to update NW for 8.1.3.5 to 9.0.0.0.

 

Before upgrade, let's focus on parts we should do the first.  It may sound silly, but one thing is where do you put installation files. These have increased in size big time.  For example, this is Linux ones I will use:

# ll 9.0.0b365/

total 890476

-rw-r----- 1 root root   1730919 Oct 16 16:55 emclicenseserver-3.4.1-2.x86_64_lsb.rpm

-rw-r----- 1 root root 906533215 Oct 16 18:16 nw90_linux_x86_64.tar.gz

The license server is small, but that linux package is huge - that's double than it used to be before.  I'm not even sure why they went through trouble of (g)zipping it since unziped the size is:

-rw-r----- 1 root root 909588480 Oct 16 18:16 nw90_linux_x86_64.tar

... and inside we find:

# tar tvf nw90_linux_x86_64.tar

drwxr-xr-x 0/20              0 2015-09-11 08:45 linux_x86_64/

-rw-r--r-- 0/20        9387587 2015-09-11 08:34 linux_x86_64/lgtoadpt-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       25081042 2015-09-11 08:35 linux_x86_64/lgtobbb-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       57315243 2015-09-11 08:35 linux_x86_64/lgtoclnt-9.0-1.x86_64.rpm

-rw-r--r-- 0/20        1754555 2015-09-11 08:35 linux_x86_64/lgtofr-9.0-1.x86_64.rpm

-rw-r--r-- 0/20        1520861 2015-09-11 08:35 linux_x86_64/lgtoja-9.0-1.x86_64.rpm

-rw-r--r-- 0/20        1464722 2015-09-11 08:35 linux_x86_64/lgtoko-9.0-1.x86_64.rpm

-rw-r--r-- 0/20        2401595 2015-09-11 08:35 linux_x86_64/lgtolicm-9.0-1.x86_64.rpm

-rw-r--r-- 0/20         328750 2015-09-11 08:35 linux_x86_64/lgtoman-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       46192796 2015-09-11 08:36 linux_x86_64/lgtonode-9.0-1.x86_64.rpm

-rw-r--r-- 0/20      267246664 2015-09-11 08:37 linux_x86_64/lgtoserv-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       67930218 2015-09-11 08:37 linux_x86_64/lgtoxtdclnt-9.0-1.x86_64.rpm

-rw-r--r-- 0/20        1481061 2015-09-11 08:37 linux_x86_64/lgtozh-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       60881102 2015-09-11 07:43 linux_x86_64/lgtoclnt_9.0_amd64.deb

-rw-r--r-- 0/20       72485346 2015-09-11 07:44 linux_x86_64/lgtoxtdclnt_9.0_amd64.deb

-rw-r--r-- 0/20       73148329 2015-09-11 08:34 linux_x86_64/lgtonmc-9.0-1.x86_64.rpm

-rw-r--r-- 0/20       87058606 2015-09-11 07:44 linux_x86_64/AvamarClient-linux-rhel4-x86_64-7.2.100-401.rpm

-rw-r--r-- 0/20       52253858 2015-09-11 07:44 linux_x86_64/AvamarClient-linux-sles11-x86_64-7.2.100-401.rpm

-rw-r--r-- 0/20       13991214 2015-09-11 07:44 linux_x86_64/AvamarClient-debian4.0-x86_64-7.2.100-401.deb

-rw-r--r-- 0/20       66226762 2015-09-11 08:33 linux_x86_64/lgtoauthc-9.0-1.x86_64.rpm

-rwxr-xr-x 0/20        1330804 2015-09-11 05:20 linux_x86_64/gstdbunload

-rwxr-xr-x 0/20           8058 2015-09-11 05:20 linux_x86_64/LGTO_METAFILE.linuxx86_64

-rwxr-xr-x 0/20            866 2015-09-11 05:20 linux_x86_64/LGTO_METAFILE_linux86w_sles11_avamarclient.mf

-rwxr-xr-x 0/20            863 2015-09-11 05:20 linux_x86_64/LGTO_METAFILE_linux86w_rhel4_avamarclient.mf

-rwxr-xr-x 0/20          70618 2015-09-11 05:20 linux_x86_64/sd_products.res

The names itself are pretty much clear to decipher what is what. While EMC doesn't recommend to use license manager any longer, it is still part of the package.  Here is rule of thumb guide as per official docs:

Capture28.JPG.jpg

 

Anyway, back to size.  Obviously with client and extended client, total package size has grown.  Then we have Avamar, NMC and authentication service - this takes space.  In the past I used to keep all these on server for code trees in use. Recently, with 8.2.x, I started to use nsrpush - but bad thing there is that nsrpush is is per datazone and I can't share repository between different servers (datazones).  So, in essence, I'm better off if I place this into yum server and script update (or just use NFS share as source as yum server I would just use with Linux).  For the sake of the upgrade, I won't explore possibilities here on what changed with nsrpush, but I will rather use NFS mount which I can mount on both server and storage node to install new code. I personally would like to separate inventory and repository part NW to make possible repository one to be shared between datazones.  Perhaps that is next step.

 

As my server runs on Linux, steps I should follow in theory are:

  • install java on backup server; needed by both server and authentication server.  Required minimum version is 7u60.  You may wish to go for 8 right away (check Java SE Runtime Environment 8 - Downloads).
  • install authentication server (you can install it at the same time as backup server, but you must configure it before your run server, for example /opt/nsr/authc-server/scripts/authc_configure.sh)
  • remove old server code and install new server code (or simply update, for example: rpm -Uvh lgtoclnt*.rpm lgtoxtdclnt*.rpm lgtonode*.rpm lgtoserv*.rpm lgtoauthc*.rpm lgtoadpt*.rpm)
  • install new licensing server
  • remove old storage node and NMC code
  • install new storage node and NMC (this includes conversion via gstdbunload)
  • run some verification tests

 

In my case, I do not have java installed as seen from following check:

# rpm -q --list jre | grep "bin/java"

#

# rpm -qa | grep jre

#

So, I downloaded the jre and installed it:

# rpm -ivh ./jre-8u60-linux-x64.rpm

Preparing...                ########################################### [100%]

   1:jre1.8.0_60            ########################################### [100%]

Unpacking JAR files...

  plugin.jar...

  javaws.jar...

  deploy.jar...

  rt.jar...

  jsse.jar...

  charsets.jar...

  localedata.jar...

  jfxrt.jar...

 

# rpm -qa | grep jre

jre1.8.0_60-1.8.0_60-fcs.x86_64

 

# rpm -q --list jre1.8.0_60-1.8.0_60-fcs.x86_64 | grep "bin/java"

/usr/java/jre1.8.0_60/bin/java

/usr/java/jre1.8.0_60/bin/javaws

 

# java -version

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

 

OK, java is in place.  Now, before I stop old server, it is recommended to run some checks in respect to existing DBs.  For example, I know it has been said browse and retention management has changed.  So, let's see how this get's handled during upgrade.  I never use setup where I have different browse and retention, but some people do.  To see what those folks are about to face, I will select two ssids and change them in following way:

  • first ssid I will extend retention so that browse<retention for both original and cloned ssid
  • second ssid I will extended retention so that browse<retention only for clone ssid

 

So, let's pick one client.  Here I shows you ssids sorted by date and cut off by head (but enough to show you expiration of whole cycle):

Capture29.JPG.jpg

 

What is not visible on this screenshot is that I run this on 18th around 11:20, but that doesn't matter much right now.  So, I modified two ssids as I said I would:

Capture30.JPG.jpg

 

I was under impression that different retentions could be made between cloned ssids, but perhaps I was wrong (or perhaps that applies to tapes or anything else except DD Boost device based ssids).  I really don't know since I don't use such setup.  What I see is that action on two ssids ended up with same outcome even in one case this action was limited.  Second thing I see is that when using manual time policy manipulation with "+ amount of time" approach (which + amount since current time of action), retention field is not rounded to that date to 23:59:59.

 

EDIT: Not using it means my brain needs more time to reflect the things... the reason why I didn't see it was that I should check clretent field instead.  When checking clretent field, we can se both actions executed as they should be:

Capture31.JPG.jpg

I'm also going to capture status of mdb before with mminfo and compare it with later status after upgrade:

Capture32.JPG.jpg

With this in place, let's rock.  Update guide says you should make sure DB is in order.  With my test environment I know it is as it is fresh and new, almost a baby, but in any real situation you should make sure this really is in order so run following:

  • nsrim -X (this will do cross-check of index and mdb)
  • nsrck -m (this will do mdb check - it will never report anything on STDIO - check daemon.raw for any messages)
  • docs suggest to run nsrls -m and nsrports as well to record what you had before - I normally never do that, but ok

# nsrls -m

 

 

Database id 0: /nsr/mm/mmvolume6

Fid |      Size |     Count | Name

------------------------------------------

   0 |     32 KB |        17 | vol

   1 |    424 KB |       472 | ss

   2 |      8 KB |         6 | clients

# nsrports

Service ports: 7937-9936

Connection ports: 0-0

  • capture mminfo -B output - in case that you have to do DR.  I would suggest to stop ALL NW backups, run index group (or any group that will give you bootstrap backup) and then capture it.

 

So, here are the steps as I did them (assuming you disabled backups previously):

  • (those extra paranoid may wish to reboot their infra and double-checks logs before proceeding any further)
  • stop NetWorker server by running nsr_shutdown
  • stop NMC by running /etc/init.d/gst stop
  • stop storage node by running nsr_shutdown
  • prepare packages you need and check what you have on server

Capture33.JPG.jpg

  • I will use rpm -Uvh in this case to update/install packages I need.

Capture34.JPG.jpg

  • At this point, EMC (me too) recommended to delete content of /nsr/tmp

Capture35.JPG.jpg

  • To configure authentication, as seen above, we must run /opt/nsr/authc-server/scripts/authc_configure.sh. So, let's do it.

Capture36.JPG.jpg

As seen above, first we specify where java is.  By default, there will be symlink in place of /usr/java/latest. Then we configure Apache Tomcat configuration and finally admin password to enter NMC.

  • At this point I could run NetWorker server and install new licensing package, but I won't.  Instead I will update storage and NMC first. One thing to check is gstdbunload presence.  This binary should be part of 8.1.3.5 and it is - check for existence of /opt/lgtonmc/bin/gstdbunload.  Now, official docs do not know which version you have so they tell you to copy this binary from NW9 tarball - these two do not appear to be the same:

[root@censorship linux_x86_64]# ll /opt/lgtonmc/bin/gstdbunload /nwinsta/9.0.0b365/linux_x86_64/gstdbunload

-rwxr-xr-x 1 root games 1330804 Sep 11 05:20 /nwinsta/9.0.0b365/linux_x86_64/gstdbunload

-r-xr-xr-x 1 root root   345864 Sep 30 03:21 /opt/lgtonmc/bin/gstdbunload

The difference in size comes from NW9 being not stripped, but you also see that that these were linked using different kernels so if you strip the binary some small difference still remains:

Capture37.JPG.jpg

So, which one to use?  I think both are ok and you can use the one you already have and if you don't have, just copy it to /opt/lgtonmc/bin and you will be fine.

 

Theory says I should create dump folder where converted DB will be.  My DB was almost empty, I didn't even configure monitoring not LDAP auth so I wonder if there is anything to dump, but I will use /tmp/H as dump folder.  Here is the output when using native gstdbunload (the one coming with 8.1.3.5):

[root@censhorship linux_x86_64]# rm -rf /tmp/H/*

[root@censorship linux_x86_64]# /opt/lgtonmc/bin/gstdbunload /tmp/H

SQL Anywhere Unload Utility Version 12.0.1.3798

Connecting and initializing

Unloading "lgto"."bootstrap_config" into ./unload/720.dat

509 rows written

Unloading "lgto"."gst_object_properties" into ./unload/721.dat

161 rows written

Unloading "lgto"."gst_internal_max_key" into ./unload/723.dat

1 rows written

Unloading "lgto"."gst_group" into ./unload/726.dat

4 rows written

Unloading "lgto"."gst_gwtmeta_malleable" into ./unload/727.dat

1 rows written

Unloading "lgto"."gst_host" into ./unload/728.dat

5 rows written

Unloading "lgto"."gst_host_aliases" into ./unload/729.dat

0 rows written

Unloading "lgto"."gst_hierarchy_host" into ./unload/730.dat

5 rows written

Unloading "lgto"."gst_lic_sig" into ./unload/731.dat

0 rows written

Unloading "lgto"."gst_lic_sig_intercepts" into ./unload/732.dat

0 rows written

Unloading "lgto"."gst_managed_app" into ./unload/733.dat

8 rows written

Unloading "lgto"."gst_app_feature" into ./unload/734.dat

4 rows written

Unloading "lgto"."gst_managed_event" into ./unload/735.dat

0 rows written

Unloading "lgto"."gst_sub_annotation" into ./unload/736.dat

0 rows written

Unloading "lgto"."gst_poll_private" into ./unload/737.dat

0 rows written

Unloading "lgto"."gst_preference_definition" into ./unload/738.dat

0 rows written

Unloading "lgto"."gst_user" into ./unload/743.dat

4 rows written

Unloading "lgto"."gst_user_login" into ./unload/744.dat

4 rows written

Unloading "lgto"."gst_user_malleable" into ./unload/745.dat

4 rows written

Unloading "lgto"."gst_group_user_members" into ./unload/746.dat

5 rows written

Unloading "lgto"."gst_license_updates" into ./unload/747.dat

1 rows written

Unloading "lgto"."gst_retention_policy" into ./unload/748.dat

5 rows written

Unloading "lgto"."media_save_set" into ./unload/756.dat

0 rows written

Unloading "lgto"."save_set_annotations" into ./unload/757.dat

0 rows written

Unloading "lgto"."volume" into ./unload/759.dat

0 rows written

Unloading "lgto"."completion_save_group" into ./unload/761.dat

0 rows written

Unloading "lgto"."completion_save_set" into ./unload/762.dat

0 rows written

Unloading "lgto"."completion_save_set_output" into ./unload/763.dat

0 rows written

Unloading "lgto"."completion_media_save_set_link" into ./unload/764.dat

0 rows written

Unloading "lgto"."server_timesheets" into ./unload/765.dat

0 rows written

Unloading "lgto"."gst_event_priorities" into ./unload/767.dat

35 rows written

Unloading "lgto"."gst_audit" into ./unload/770.dat

233 rows written

Unloading "lgto"."gst_ownership" into ./unload/771.dat

150 rows written

Unloading "lgto"."gst_group_group_members" into ./unload/772.dat

0 rows written

Unloading "lgto"."gst_reports" into ./unload/773.dat

101 rows written

Unloading "lgto"."gst_report_hierarchy" into ./unload/774.dat

98 rows written

Unloading "lgto"."gst_report_definition" into ./unload/775.dat

63 rows written

Unloading "lgto"."gst_report_drilldown_definition" into ./unload/776.dat

81 rows written

Unloading "lgto"."gst_system_options" into ./unload/781.dat

9 rows written

Unloading "lgto"."gst_server_configuration" into ./unload/782.dat

4 rows written

Unloading "lgto"."gst_sub_poll_private" into ./unload/783.dat

0 rows written

Unloading "lgto"."clone" into ./unload/784.dat

0 rows written

Unloading "lgto"."clone_volume_link" into ./unload/785.dat

0 rows written

Unloading "lgto"."gst_device_activity" into ./unload/787.dat

0 rows written

Unloading "lgto"."gst_inactive_files" into ./unload/789.dat

0 rows written

Unloading "lgto"."dedupe_save_set" into ./unload/791.dat

0 rows written

Unloading "lgto"."gst_group_ext_role_members" into ./unload/793.dat

0 rows written

Unloading "lgto"."gst_recover_job" into ./unload/795.dat

0 rows written

Unloading "lgto"."gst_cloud_save_session" into ./unload/797.dat

0 rows written

Unloading "lgto"."gst_cloud_recover_session" into ./unload/798.dat

0 rows written

Unloaded 50 tables of 56

Unloading "lgto"."data_domain_save_set" into ./unload/799.dat

0 rows written

Unloading "lgto"."gst_clone_job" into ./unload/801.dat

0 rows written

Unloading "lgto"."gst_clone_save_set" into ./unload/802.dat

0 rows written

Unloading "lgto"."gst_certificates_PSO" into ./unload/804.dat

0 rows written

Unloading "lgto"."gst_data_protection_policy" into ./unload/805.dat

0 rows written

Unloading "lgto"."gst_data_protection_saveset" into ./unload/806.dat

0 rows written

136431:gstdbunload: Database unload completed successfully.

 

[root@censorshup linux_x86_64]# ll /tmp/H/

total 40

-rw-r----- 1 root root  4325 Oct 18 18:28 reload.log

-rw------- 1 root root 28018 Oct 18 18:28 reload.sql

drwxr-x--- 2 root root  4096 Oct 18 18:28 unload

 

[root@censorshup linux_x86_64]# ll -R /tmp/H/

/tmp/H/:

total 40

-rw-r----- 1 root root  4325 Oct 18 18:28 reload.log

-rw------- 1 root root 28018 Oct 18 18:28 reload.sql

drwxr-x--- 2 root root  4096 Oct 18 18:28 unload

 

 

/tmp/H/unload:

total 804

-rwxr-x--- 1 root root  40670 Oct 18 18:28 720.dat

-rwxr-x--- 1 root root  11667 Oct 18 18:28 721.dat

-rwxr-x--- 1 root root     10 Oct 18 18:28 723.dat

-rwxr-x--- 1 root root    390 Oct 18 18:28 726.dat

-rwxr-x--- 1 root root   1802 Oct 18 18:28 727.dat

-rwxr-x--- 1 root root    215 Oct 18 18:28 728.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 729.dat

-rwxr-x--- 1 root root     85 Oct 18 18:28 730.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 731.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 732.dat

-rwxr-x--- 1 root root   1112 Oct 18 18:28 733.dat

-rwxr-x--- 1 root root    223 Oct 18 18:28 734.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 735.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 736.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 737.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 738.dat

-rwxr-x--- 1 root root    438 Oct 18 18:28 743.dat

-rwxr-x--- 1 root root    287 Oct 18 18:28 744.dat

-rwxr-x--- 1 root root  10914 Oct 18 18:28 745.dat

-rwxr-x--- 1 root root     48 Oct 18 18:28 746.dat

-rwxr-x--- 1 root root     33 Oct 18 18:28 747.dat

-rwxr-x--- 1 root root     82 Oct 18 18:28 748.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 756.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 757.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 759.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 761.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 762.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 763.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 764.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 765.dat

-rwxr-x--- 1 root root    829 Oct 18 18:28 767.dat

-rwxr-x--- 1 root root  77816 Oct 18 18:28 770.dat

-rwxr-x--- 1 root root   1417 Oct 18 18:28 771.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 772.dat

-rwxr-x--- 1 root root   5343 Oct 18 18:28 773.dat

-rwxr-x--- 1 root root   1058 Oct 18 18:28 774.dat

-rwxr-x--- 1 root root 589096 Oct 18 18:28 775.dat

-rwxr-x--- 1 root root  10196 Oct 18 18:28 776.dat

-rwxr-x--- 1 root root    837 Oct 18 18:28 781.dat

-rwxr-x--- 1 root root    140 Oct 18 18:28 782.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 783.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 784.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 785.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 787.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 789.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 791.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 793.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 795.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 797.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 798.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 799.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 801.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 802.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 804.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 805.dat

-rwxr-x--- 1 root root      0 Oct 18 18:28 806.dat

[root@censorship linux_x86_64]#

Note that this runs only when NMC daemons are down so don't think for a moment I started daemons and didn't say it   Now, let's update daemons.

Capture38.JPG.jpg

You will noticed that I forgot to update manuals so I did it at next step - it happens (actually I did the same thing on server as well)

  • At this point, I can run NSR daemon on storage node /NMC host.  Actually, I have to  because running  nmc_config requires that. The problem is that nmc_config also requires daemons on server to be started we will first runs NSR daemons on storage node, then NSR daemons on server and finally run this script on storage node (which happens to host NMC - otherwise bad idea).
  • After running /etc/init.d/networker start you won't see much - it will just run.  Some details you can see from daemon.raw, but nothing special.

[root@<NSR_SN_NODE> linux_x86_64]# /etc/init.d/networker start

[root@<NSR_SN_NODE> linux_x86_64]# /etc/init.d/networker status

+--o nsrexecd (50181)

[root@<NSR_SN_NODE> linux_x86_64]# ps -ef | grep nsr

root     50181     1  0 18:59 ?        00:00:00 /usr/sbin/nsrexecd

root     52160 37434  0 19:00 pts/0    00:00:00 grep nsr

[root@<NSR_SN_NODE> linux_x86_64]# nsrrpcinfo -p localhost

   program vers proto   port

    100000    2   tcp   7938  nsrportmapper

    100000    2   udp   7938  nsrportmapper

    390436    1   tcp   9437  nsrexecd

    390435    1   tcp   8955  nsrexecd

    390113    1   tcp   7937  nsrexecd

    390440    1   tcp   8885  nsrexecd_nsrfa

[root@<NSR_SN_NODE> linux_x86_64]# ls -ltra /nsr

total 56

drwx------  2 root root 16384 Dec 23  2013 lost+found

-rw-r-----  1 root root    50 Apr  7  2014 nsrrc

drwxrwxrwt  4 root root  4096 Oct 14 18:57 dedup

drwxr-xr-x  3 root root  4096 Oct 14 18:57 res

drwxr-xr-x 10 root root  4096 Oct 14 18:57 .

drwx------  5 root root  4096 Oct 14 23:28 cores

drwxrwxrwt  2 root root  4096 Oct 15 00:00 applogs

dr-xr-xr-x 25 root root  4096 Oct 18 17:57 ..

drwxr-xr-x  5 root root  4096 Oct 18 18:59 tmp

drwxr-x---  2 root root  4096 Oct 18 18:59 run

drwxr-x---  3 root root  4096 Oct 18 18:59 logs

[root@<NSR_SN_NODE> linux_x86_64]# ls -ltra /nsr/logs/

total 836

drwxr-xr-x 10 root root   4096 Oct 14 18:57 ..

drwxrwxrwt  2 root root   4096 Oct 18 18:59 adhoc

drwxr-x---  3 root root   4096 Oct 18 18:59 .

-rw-r-----  1 root root 837441 Oct 18 19:01 daemon.raw

[root@<NSR_SN_NODE> linux_x86_64]# nsr_render_log /nsr/logs/daemon.raw

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Product:      NetWorker

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Build number: 365

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 06:59:49 PM  1 5 0 3520366368 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

90524 10/18/2015 06:59:50 PM  2 1 34 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapinst' because "Numerical result out of range".

90525 10/18/2015 06:59:50 PM  2 5 0 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR warning Authentication Warning: Removing group 'sapinst' from the credential for user 'root'.

90524 10/18/2015 06:59:50 PM  2 1 34 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapinst' because "Numerical result out of range".

90525 10/18/2015 06:59:50 PM  2 5 0 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR warning Authentication Warning: Removing group 'sapinst' from the credential for user 'root'.

90524 10/18/2015 06:59:50 PM  2 1 34 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapinst' because "Numerical result out of range".

90525 10/18/2015 06:59:50 PM  2 5 0 3246372608 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR warning Authentication Warning: Removing group 'sapinst' from the credential for user 'root'.

101330 10/18/2015 07:01:15 PM  0 0 0 3267352320 50181 0 <NSR_SN_NODE>.corp.vattenfall.com nsrexecd NSR info The security audit log is using a cached copy of the resource found on the host '<NSR_BCK_NODE>.corp.vattenfall.com' for initialization.

[root@<NSR_SN_NODE> linux_x86_64]#

When I look above, I see that out of the blue there is nsrexecd_nsrfa in rpcinfo output. If I remember correctly, nsrfa RPCs are related to jobs so it comes as surprise to see it here in what right now is nothing more than client running.  Another thing is /nsr/logs/adhoc folder.  Not sure if this is separate log area for ad hoc jobs or some debug remains.

  • starting server is where thrill is - so let's give it a try as well.  You will notice some new kids on the block:

Capture39.JPG.jpg

And after some time after nsrmmds started this changed to:

Capture40.JPG.jpg

Here is what I see on RPC front on backup server:

Capture41.JPG.jpg

My nsr folder does look a bit different and there some new folders there:

Capture42.JPG.jpg

authc is easy to decipher due to nsrtomcat ownership (Apache Tomcat and authentication service).What about that rabbitmq?  Who's the rabbit?  I suspect this is message queue (mq) which is also something new in NW9.   As per docs:

The message queue adapter is a component on the NetWorker server that provides programs access to a NetWorker server over a message bus. The message queue adapter daemon (nsrmqd) translates the communications that occur between a message protocol and NetWorker RPC functions. Backup and Recovery Manager and Hyper-V FLR in NetWorker Module for Microsoft (NMM) use the message queue adapter to communicate with the NetWorker server and NetWorker server components.

Fact that installation will add nsrtomcat into passwd group file on Linux should keep on alert all those using cfengine to keep an eye on these files from any undesigned alterations.

  • we also said previously that media database format changed. For example, I can see this now:

Capture43.JPG.jpg

mmvolume6.Oct_18_15 is mdb copy (in case something goes wrong) and next you see (inside mmvolrel) content of media database as it is now (and nsrls -m won't work any longer so why EMC recommended to capture that output before is beyond my understanding, but most likely data may help them during conversion troubleshooting).

  • logs folder changed a bit as well.. for example I have following:

Capture44.JPG.jpg

One things which surprised me was size of daemon.raw as it seems to be populated with something all the time (as no backup run since I started daemons). It turns out my log was flooded with messages about sapinst group for some reason... logs like this:

90524 10/18/2015 07:25:07 PM  2 1 34 3210733312 23821 0 <NSR_BCK_NODE> nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapins

t' because "Numerical result out of range".

90525 10/18/2015 07:25:07 PM  2 5 0 3210733312 23821 0 <NSR_BCK_NODE> nsrexecd NSR warning Authentication Warning: Removing group 'sapinst' from the credential for user 'root'.

90524 10/18/2015 07:25:07 PM  2 1 34 3221223168 23821 0 <NSR_BCK_NODE> nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapins

t' because "Numerical result out of range".

90525 10/18/2015 07:25:07 PM  2 5 0 3221223168 23821 0 <NSR_BCK_NODE> nsrexecd NSR warning Authentication Warning: Removing group 'sapinst' from the credential for user 'root'.

90524 10/18/2015 07:25:08 PM  2 1 34 3221223168 23821 0 <NSR_BCK_NODE> nsrexecd SYSTEM warning Authentication Warning: Group verification for user 'root' failed. Cannot get group information for group 'sapins

t' because "Numerical result out of range".

I didn't check why is it so nor did I dig into this any further, but I did quick check and one thing I see is that root user reports sapinst to contain this user, but this is not listed in /etc/group so....:

Capture45.JPG.jpg

I also had bunch of "/bin/sh: /bin/logger: No such file or directory".  Indeed, it is is in /usr/bin so nothing that symlink can't fix. Anyway, if I exclude sapinst from logs, I will get following:

[root@<NSR_BCK_NODE> logs]# nsr_render_log /nsr/logs/daemon.raw | grep -v sapinst

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Build number: 365

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:24:27 PM  1 5 0 3765520160 23821 0 <NSR_BCK_NODE> nsrexecd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Build number: 365

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

127106 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Startup in progress; starting all daemons...

144355 10/18/2015 07:24:28 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon rmq-epmd is running.

144355 10/18/2015 07:24:30 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon rmq is running.

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq: 

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:               RabbitMQ 3.2.4. Copyright (C) 2007-2013 GoPivotal, Inc.

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:   ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:   ##  ##

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:   ##########  Logs: /opt/nsr/rabbitmq-server-3.2.4/sbin/../var/log/rabbitmq/rabbit@<NSR_BCK_NODE>.log

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:   ######  ##        /opt/nsr/rabbitmq-server-3.2.4/sbin/../var/log/rabbitmq/rabbit@<NSR_BCK_NODE>-sasl.log

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:   ##########

0 10/18/2015 07:24:31 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:               Starting broker...

0 10/18/2015 07:24:32 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice rmq:  completed with 0 plugins.

144355 10/18/2015 07:24:32 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon authservice is running.

0 10/18/2015 07:24:32 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice authservice: Tomcat access log purging task is started.

0 10/18/2015 07:24:32 PM  1 5 0 3448100608 23938 0 <NSR_BCK_NODE> nsrctld NSR notice authservice: Tomcat access log purging task is done.

144355 10/18/2015 07:24:34 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon nsrd is running.

83271 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup in process (step 1 of 5); opening NetWorker server RAP database...

83272 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup in process (step 2 of 5); initializing RAP handlers...

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Server Notice: started

134164 10/18/2015 07:24:34 PM  1 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Query of nsrd resource 'NSR' for attribute '<null>' discovered attribute missing. Defaulting internally cached value to 1.

78366 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Beginning check of client attributes

78367 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Finished check of client attributes

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Platinum' has been created

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Gold' has been created

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Silver' has been created

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Bronze' has been created

71193 10/18/2015 07:24:34 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Server Protection' has been created

83273 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup in process (step 3 of 5); checking resource types in the RAP database...

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.804826 preserve_group_membership(): client '<NSR_BCK_NODE>' updating pre policy group info with index_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.807475 preserve_group_membership(): client '<NSR_SN_NODE>' updating pre policy group info with index_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.810779 preserve_group_membership(): client '<NSR_CLIENT1>' updating pre policy group info with FS_PRD_SiteA_FE_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.812801 preserve_group_membership(): client '<NSR_CLIENT2>' updating pre policy group info with FS_PRD_SiteA_FE_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.820258 preserve_group_membership(): client '<NSR_SN_NODE>' updating pre policy group info with NMC_SYB_PRD_SiteA_BE_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.821017 preserve_group_membership(): client '<NSR_CLIENT3>' updating pre policy group info with FS_PRD_SiteB_FE_1

0 10/18/2015 07:24:34 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice 10/18/15 19:24:34.825663 preserve_group_membership(): client '<NSR_CLIENT4>' updating pre policy group info with FS_PRD_SiteA_FE_1

88795 10/18/2015 07:24:34 PM  1 3 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd RAP notice Deleting old NSR directive resource 'NetWare standard directives'.

88795 10/18/2015 07:24:34 PM  1 3 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd RAP notice Deleting old NSR directive resource 'NetWare with compression directives'.

83274 10/18/2015 07:24:35 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup in process (step 4 of 5); registering RPC services...

91567 10/18/2015 07:24:35 PM  1 5 0 840943360 24108 0 <NSR_BCK_NODE> nsrd NSR notice Configuring the nsrlogd process on host '<NSR_BCK_NODE>'.

140522 10/18/2015 07:24:35 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Loading authentication configuration file

143380 10/18/2015 07:24:35 PM  1 5 0 830453504 24108 0 <NSR_BCK_NODE> nsrd NSR notice Connected to the message bus.

144355 10/18/2015 07:24:36 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon connectemc is running.

144355 10/18/2015 07:24:38 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon eventservice is running.

127107 10/18/2015 07:24:48 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Startup complete

144358 10/18/2015 07:24:51 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Starting daemon hyperv-flr-ui because it has been enabled.

144355 10/18/2015 07:24:51 PM  1 5 0 3604363040 23938 0 <NSR_BCK_NODE> nsrctld NSR notice Daemon hyperv-flr-ui is running.

83275 10/18/2015 07:25:01 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup in process (step 5 of 5); starting all daemons...

136473 10/18/2015 07:25:02 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Media database will be migrated from legacy to relational storage.

112189 10/18/2015 07:25:02 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice media db is checking and rebuilding btrees

9870 10/18/2015 07:25:02 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice media db is consistency checking the database

136482 10/18/2015 07:25:02 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Migrating client IDs to new database

136483 10/18/2015 07:25:03 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Migrating volumes to new database

136484 10/18/2015 07:25:04 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Migrating save sets to new database

98645 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice NetWorker dispatcher is active.

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Build number: 365

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:25:06 PM  1 5 0 4145030944 25190 0 <NSR_BCK_NODE> nsrdispd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Build number: 365

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:25:08 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Build number: 365

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:25:09 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

134863 10/18/2015 07:25:09 PM  1 0 0 1149306656 24114 0 <NSR_BCK_NODE> nsrlmc NSR notice CLP/ELMS server for basic license information found the following: 'Can't find license file.'.

134863 10/18/2015 07:25:09 PM  1 0 0 3524245280 24115 0 <NSR_BCK_NODE> nsrlmc NSR notice CLP/ELMS server for basic license information found the following: 'Can't find license file.'.

83691 10/18/2015 07:25:09 PM  1 9 0 2115737344 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Opening Jobs database

91976 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice Running nsrck to check the client file indexes.

71191 10/18/2015 07:25:09 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Warning event: CLP/ELMS server for basic license information found the following: 'Can't find license file.'. Please obtain the license file.  

71191 10/18/2015 07:25:09 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Warning event: CLP/ELMS server for basic license information found the following: 'Can't find license file.'. Please obtain the license file.  

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Build number: 365

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:25:09 PM  1 5 0 3134723872 25183 0 <NSR_BCK_NODE> nsrindexd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

136471 10/18/2015 07:25:10 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Media database is creating database indexes.

116722 10/18/2015 07:25:10 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice Media database is performing extended consistency checks.

116723 10/18/2015 07:25:10 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice Media database is checking volumes.

71193 10/18/2015 07:25:10 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Database Notice: Media database is performing extended consistency checks.

71193 10/18/2015 07:25:10 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_BCK_NODE>' 

98282 10/18/2015 07:25:10 PM  1 2 0 3580196608 23821 0 <NSR_BCK_NODE> nsrexecd RPC notice Process nsrlogd successfully configured on host '<NSR_BCK_NODE>' for server '<NSR_BCK_NODE>'.

71193 10/18/2015 07:25:10 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info client Info: Process nsrlogd successfully configured on host '<NSR_BCK_NODE>' for server '<NSR_BCK_NODE>'.

98282 10/18/2015 07:25:10 PM  1 2 0 3580196608 23821 0 <NSR_BCK_NODE> nsrexecd RPC notice Process nsrlogd successfully configured on host '<NSR_BCK_NODE>' for server '<NSR_BCK_NODE>'.

114595 10/18/2015 07:25:11 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice Media database is cross-checking save sets.

115712 10/18/2015 07:25:11 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice Starting vacuum of jobs database on startup.

134217 10/18/2015 07:25:11 PM  1 5 0 1971304192 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice Connected to message bus (tries=1).

115713 10/18/2015 07:25:12 PM  1 5 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd NSR notice Finished vacuum of jobs database on startup in 0 min 1 sec.

69834 10/18/2015 07:25:12 PM  1 9 0 2271999776 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Upgrading jobs database from version 8.1.3 to 9.0.0

114596 10/18/2015 07:25:12 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice Media database is cross-checking volumes.

116725 10/18/2015 07:25:13 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice Media database completed extended consistency checks.

71193 10/18/2015 07:25:13 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Database Notice: Media database completed extended consistency checks.

115953 10/18/2015 07:25:13 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Original media database has been renamed to '/nsr/mm/mmvolume6.Oct_18_15'.

115951 10/18/2015 07:25:13 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Media database successfully migrated 6 clients, 17 volumes, and 472 save sets.

107694 10/18/2015 07:25:13 PM  0 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR info Media database cache is enabled.

10106 10/18/2015 07:25:13 PM  1 8 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice media db is open for business

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Build number: 365

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:25:13 PM  1 5 0 4009875232 25178 0 <NSR_BCK_NODE> nsrmmdbd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_BCK_NODE> contains 113746 records occupying 17 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_CLIENT1>' 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_CLIENT1>.eur contains 481628 records occupying 106 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_CLIENT4>' 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_CLIENT4> contains 221387 records occupying 34 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_CLIENT2>' 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_CLIENT2> contains 112270 records occupying 16 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_CLIENT3>' 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_CLIENT3> contains 112961 records occupying 16 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: checking index for client '<NSR_SN_NODE>' 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: /nsr/index/<NSR_SN_NODE> contains 112226 records occupying 17 MB 

71193 10/18/2015 07:25:20 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Index Notice: Completed checking 6 client(s) 

83276 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Startup process completed; server is now fully available

71191 10/18/2015 07:25:38 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Critical event: NetWorker evaluation mode will expire in 27 days and the server will be disabled after that time. 

12107 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Evaluation Edition

12116 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice License Summary:

66441 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice  Available: sv=32767, virt=32767, ndmp=32767

64047 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice  Borrowed:  sv_borrowed=0

66442 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice  Remaining: sv=32761, virt=32767, ndmp=32767

69792 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Connected Clients:(6)

0 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice  <NSR_SN_NODE> <NSR_BCK_NODE> <NSR_CLIENT4> <NSR_CLIENT3> <NSR_CLIENT2>

0 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice  <NSR_CLIENT1>

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice OFC: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle 7, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle 7, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Informix, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Informix, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Microsoft SQL Server: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Sybase, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Sybase, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Microsoft Exchange Server: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for MEDITECH: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Lotus, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Lotus, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Lotus cc:Mail: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for SAP, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for SAP, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for SAP, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for SAP R/3 Microsoft SQL Server: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Microsoft Mail: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Database Toolkit for Informix: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for DB2, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for DB2, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker for EMC Symmetrix for Oracle: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for EMC Symmetrix for SAP/R3 on Oracle: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle, Windows Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Oracle on Linux: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice SharePoint Portal Server for Windows: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Microsoft Applications: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Snapshot Management for Shadow Copy: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Databases and Applications, Windows/Linux Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for Databases and Applications, Unix Client/1: Available=32767, Remaining=32767, Used=0

12128 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NetWorker Module for SnapImage: Available=32767, Remaining=32767, Used=0

91602 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice NSR server host '<NSR_BCK_NODE>' using client ID '98be7053-00000004-561e89ca-561e89c9-00011700-66c877a4' 

38758 10/18/2015 07:25:38 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info server notice: Performing policy migration.

71191 10/18/2015 07:25:38 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Server Info event: NetWorker resource migration to policy is in progress.

88800 10/18/2015 07:25:38 PM  2 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR warning Notification process '/bin/logger -p daemon.notice' failed with status 127.

95642 10/18/2015 07:25:38 PM  1 5 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR notice Request to start 'nsrvmwsd'.

95668 10/18/2015 07:25:40 PM  1 5 0 1857513248 26281 0 <NSR_BCK_NODE> nsrvmwsd NSR notice Started SOAP service on TCP port 8080.

71193 10/18/2015 07:25:41 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: 'Protection Policy' 'Backup' has been created

71192 10/18/2015 07:25:41 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Server event cleared: 

38758 10/18/2015 07:25:41 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info server notice: Policy migration successfully completed. Refer to migration summary logs at '/nsr/logs/migration/migration_summary_Sun-Oct-18-2015-19-25-40.log'.

82327 10/18/2015 07:26:16 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Starting full purge of jobs database

93514 10/18/2015 07:26:16 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Completed full database purge  in 0 min 1 sec. Records purged: 416

71193 10/18/2015 07:26:32 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Info: The nsrsnmd process is started on host '<NSR_BCK_NODE>' as requested.

71193 10/18/2015 07:26:32 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Info: The nsrsnmd process is started on host '<NSR_SN_NODE>' as requested.

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Product:      NetWorker

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Release:      9.0.0.0.Build.365

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Build number: 365

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Build date:   Thu Sep 10 19:59:49 PDT 2015

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Build arch.:  linux86w

0 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice @(#) Build info:   DBG=0,OPT=-O2 -fno-strict-aliasing

33486 10/18/2015 07:26:32 PM  1 5 0 2031679264 29078 0 <NSR_BCK_NODE> nsrsnmd NSR notice Starting nsrsnmd 412049409 with pid 29078.

51624 10/18/2015 07:27:49 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info The nsrsnmd process at storage node <NSR_BCK_NODE> has changed its state from SNMD_STARTED to SNMD_READY

51624 10/18/2015 07:27:49 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info The nsrsnmd process at storage node <NSR_SN_NODE> has changed its state from SNMD_STARTED to SNMD_READY

33526 10/18/2015 07:27:50 PM  1 5 0 2265339680 31012 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #4, with PID 31012, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 2036279072 31017 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #9, with PID 31017, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 3966351136 31014 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #6, with PID 31014, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 3498157856 31013 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #5, with PID 31013, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 3463477024 31010 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #2, with PID 31010, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 596539168 31011 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #3, with PID 31011, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 4263728928 31015 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #7, with PID 31015, at HOST <NSR_BCK_NODE>

33526 10/18/2015 07:27:50 PM  1 5 0 3155924768 31016 0 <NSR_BCK_NODE> nsrmmd NSR notice Start nsrmmd #8, with PID 31016, at HOST <NSR_BCK_NODE>

71193 10/18/2015 07:28:05 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Info: The storage node <NSR_SN_NODE> is ready for use. 

71193 10/18/2015 07:28:05 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Media Info: The storage node <NSR_BCK_NODE> is ready for use. 

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting workflow 'Backup/FS_PRD_SiteB_FE_1'

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'backup' for workflow 'Backup/FS_PRD_SiteB_FE_1'

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'clone' for workflow 'Backup/FS_PRD_SiteB_FE_1'

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting workflow 'Backup/FS_PRD_SiteA_BE_1'

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'backup' for workflow 'Backup/FS_PRD_SiteA_BE_1'

71193 10/18/2015 07:30:51 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'clone' for workflow 'Backup/FS_PRD_SiteA_BE_1'

38758 10/18/2015 07:30:57 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action backup succeeded

38758 10/18/2015 07:30:57 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action backup succeeded

38758 10/18/2015 07:31:01 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action clone succeeded

38758 10/18/2015 07:31:01 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: workflow FS_PRD_SiteA_BE_1 succeeded

38758 10/18/2015 07:31:02 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action clone succeeded

38758 10/18/2015 07:31:02 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: workflow FS_PRD_SiteB_FE_1 succeeded

82327 10/18/2015 08:26:21 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Starting full purge of jobs database

93514 10/18/2015 08:26:21 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Completed full database purge  in 0 min 1 sec. Records purged: 97

134863 10/18/2015 08:26:37 PM  1 0 0 3269433120 10013 0 <NSR_BCK_NODE> nsrlmc NSR notice CLP/ELMS server for basic license information found the following: 'Can't find license file.'.

71191 10/18/2015 08:26:37 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Warning event: CLP/ELMS server for basic license information found the following: 'Can't find license file.'. Please obtain the license file.  

71193 10/18/2015 09:00:04 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting workflow 'Backup/NMC_SYB_PRD_SiteA_BE_1'

71193 10/18/2015 09:00:04 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'backup' for workflow 'Backup/NMC_SYB_PRD_SiteA_BE_1'

71193 10/18/2015 09:00:04 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Protection Policy Notice: Starting action 'clone' for workflow 'Backup/NMC_SYB_PRD_SiteA_BE_1'

38758 10/18/2015 09:00:09 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action backup succeeded

38758 10/18/2015 09:00:14 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: action clone succeeded

38758 10/18/2015 09:00:14 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info policy notice: workflow NMC_SYB_PRD_SiteA_BE_1 succeeded

82327 10/18/2015 09:26:24 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Starting full purge of jobs database

93514 10/18/2015 09:26:24 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Completed full database purge  in 0 min 1 sec. Records purged: 12

134863 10/18/2015 09:27:24 PM  1 0 0 2510513952 56054 0 <NSR_BCK_NODE> nsrlmc NSR notice CLP/ELMS server for basic license information found the following: 'Can't find license file.'.

71191 10/18/2015 09:27:24 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Warning event: CLP/ELMS server for basic license information found the following: 'Can't find license file.'. Please obtain the license file.  

82327 10/18/2015 10:26:29 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Starting full purge of jobs database

93514 10/18/2015 10:26:29 PM  1 9 0 1960814336 25195 0 <NSR_BCK_NODE> nsrjobd JOBS notice Completed full database purge  in 0 min 1 sec. Records purged: 0

134863 10/18/2015 10:28:15 PM  1 0 0 1741334304 36916 0 <NSR_BCK_NODE> nsrlmc NSR notice CLP/ELMS server for basic license information found the following: 'Can't find license file.'.

71191 10/18/2015 10:28:15 PM  0 0 0 1122965280 24108 0 <NSR_BCK_NODE> nsrd NSR info Registration Warning event: CLP/ELMS server for basic license information found the following: 'Can't find license file.'. Please obtain the license file

  • I guess, this is good moment to install that licensing solution as well.  Note that normally (and this is recommended) you would have dedicated machine for license manager (makes even more sense if you have more datazones).

Capture46.JPG.jpg

Well, that's where my experience with license manager stops; I'm testing this with eval server so no license files here.

  • Now we can go back to NMC. Here is the part I did there.  Before you start, it is good to know that this installation will create postgres user and that user must have access to gst dump you created previously with gstdbunload command.

[root@censorship logs]# /opt/lgtonmc/bin/nmc_config

 

 

NOTE

====

Install has detected the configuration file of a previous lgtonmc

package. Install will attempt to read the configuration parameters

in this file and present them as default values where appropriate.

Please modify any value that is incorrect or needs to be changed.

 

 

 

 

 

For optimum security, the NMC server must run the embedded PostgreSQL database server

as a non root user. Specify a local user name that will start the database server.

If you specify a user that does not exist, the configuration process will provide you with

the option to create the user account.

 

 

 

Specify the user for the database server [postgres]: <ENTER>

 

 

 

User postgres is not a local user. Do you want to create this user [n]? y

 

 

 

Specify the directory to use for the LGTOnmc database [/nsr/nmc/nmcdb]: <ENTER>

 

 

 

Do you want to migrate the NMC server data from a previous LGTOnmc 8.x.x release [y]? <ENTER>

 

 

 

Specify the directory that contains the unloaded data [/tmp/H]: <ENTER>

 

 

 

Specify the host name of the NetWorker Authentication Service host: <NSR_BCK_SERVER>

 

 

 

Specify the port number that the NetWorker Authentication Service uses for communication [9090]: <ENTER>

 

 

 

Start the NMC server daemons at end of the configuration [y]? <ENTER>

 

 

 

Creating the installation log in /opt/lgtonmc/logs/install.log.

 

 

 

Performing initialization. Please wait...

 

 

 

ERROR: Database user "postgres" does not have permission to access /tmp/H.

Verify that all directory components have execute permission for this user.

 

 

 

 

Fix the permissions of the directory components in the path and press enter.

 

 

 

 

<AT THIS POINT IN SEPARATE WINDOW I RUN CHMOD -R 777 /tmp/H>

 

Starting migration of data from /tmp/H to /nsr/nmc/nmcdb

and logging output to /opt/lgtonmc/logs/install.log. Please wait...

Starting GST:

GST Services, Version 9.0.Build.365

 

 

 

done.

The installation completed successfully.

[root@censorship logs]#

 

So, it this is?  Before I go to NMC, let's do few more checks.  First, mminfo post check.  It turns out, some things have changed - check this out (I only remove client names):

Capture47.JPG.jpg

 

If you check this carefully you will see that:

  • First 4 entries refer to 2 ssids (and their clones) where I changed retention.  We see now that ssbrowse() and ssretent() are the same and ssbrowse() takes same value as ssretent().
  • I'm not sure about other values.  While they are ok as far as actions goes [where ssbrowse() gets value of ssretent()], I have no clue why there was difference in value in the first place as I didn't do that.  Unless this value for retention changes with every subsequent incremental backup (I didn't test that and as far as I remember, that was not the case in the past, but perhaps recent code changed that or I simply lost the touch - somewhat I would expect more clients and ssids affected in case that level theory holds the water).

 

Let's check NMC. This is my splash screen:

Capture48.JPG.jpg

 

I had similar screen with 8.2.1.4 and it went away with 8.2.1.8.  I assume that GA version won't have JRE detection issues.  Meanwhile, click on "here" link and you will be just fine.  Once I tried to log, my administrator password used was one I set during authentication service on server.  Bare in mind that previously I had different one in old NMC so that part is not the same - just to be clear on that. Once you log, you are greeted with:

Capture49.JPG.jpg

 

 

There are some new cool icons for all you fashion gurus out there.  I personally like it.  Those DD are for Data Domains of course while that shield with N inside stands for NetWorker server.

 

Before you connect to real part of the server configuration, you get new layout in connection window.  This one always felt a bit empty, but now the filled it with some more data. These data refers to various NetWorker resources including ECN area, support resources and similar.  Hardly unknown thing, but there are always new users coming in and having things for them easy is always good thing.

Capture50.JPG.jpg

But real thing starts once you connect to NetWorker administration.  First time you do it, you are asked if you wish to check migration logs; it is up to you if you wish to do it or not (you see the same log being referenced in daemon.raw so most likely you already read the log).

Capture52.JPG.jpg

 

Here is example of my log:

Capture53.JPG.jpg

 

As noted in Part I: What is NetWorker 9?, all your stuff is now part of backup policy and you have predefined ones as well.  At this point, I decided to move AIX->Linux to Part IV of this jumbo article and in Part III I plan to stick a bit more with what we have now and how it works.  For now, I only give you main screen with monitoring tab displayed.

Capture54.JPG.jpg

 

Stay tuned.