1 2 Previous Next 16 Replies Latest reply: Apr 3, 2012 10:46 PM by Carlos Rojas RSS

RPC Errors

twilliamson

We are getting RPC errors:  39078:save: RPC error: RPC send operation failed; errno = An existing connection was forcibly closed by the remote host.   74209:save: Quit signal received.

<ERROR>: Failed with error(s)

 

I follwed the instructions on the TID and I added the registry entry for KeepAliveTime (3360000) and the Environment variable to add NSR_KEEPALIVE_WAIT (10)

 

But we are still seeing RPC errors.

 

There is no firewall enabled on the backup server.

Our network guru says all is well with the network.

 

Any ideas?

Thanks, Tad Williamson

Network Administrator

Oconee Regional Medical Center

  • 1. Re: RPC Errors
    Ahmed Bahaa

    Hi twilliamson,

     

    It seems to me a TCP timeout issue, but firstly are you able to telnet your client using 7939 and then 7938 ports? Just to make sure it is not related to firewalls between server and client.

     

    To avoid any TCP timeout issue, I would suggest the following:

     

    1- Change the group properties, and set "Client Inactivity Timeout" to "0"

    2- Setting Keepalives to Prevent Timeouts

    https://solutions.emc.com/emcsolutionview.asp?id=esg91994

     

    If you are using Windows operating systems for the Backup server and client, use the following recommendations as well:

     

        1- Disable TCP Chimney on Server and Client

                   https://solutions.emc.com/emcsolutionview.asp?id=esg118437

     

        2- Set TCP/IP keep alive settings at OS level.

    https://solutions.emc.com/emcsolutionview.asp?id=esg60759

     

    Waiting your updates.

     

    Thanks,

    Ahmed Bahaa

  • 2. Re: RPC Errors
    Ahmed Bahaa

    Also, If you are using Windows operating systems, There is some recommendations by Microsoft  TCP/IP tuning to avoid any TCP timeout issue. You need to apply those recommendations and changes on the registry of the backup server and the client Windows boxes:

     

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpWindowSize=256000

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\GlobalMaxTcpWindowSize=16777216

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveInterval=1000

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveTime=600000

     

    Thanks,

    Ahmed Bahaa

  • 3. Re: RPC Errors
    Lucky85

    Try to use telnet to connect to port 7937 and check if way is opened…

     

    Regards

    Lukas

  • 4. Re: RPC Errors
    twilliamson

    When I telnet to that port, should I get a reply/info?  I just get a dash with no errors

  • 5. Re: RPC Errors
    twilliamson

    When I telnet to those ports, should I get a response/info?  I just get a dash with no errors.

     

    Thanks, Tad

  • 6. Re: RPC Errors
    Lucky85

    Dash is ok, and mean you are able to connect to port…

  • 7. Re: RPC Errors
    twilliamson

    I can connect to ports 7937 and 7938.  We are trying some of the other suggestions now.

     

    Thanks

  • 8. Re: RPC Errors
    twilliamson

    Thanks, we can connect ot 7937 and 7938.  Trying some other suggestions now.

     

    Thanks, Tad

  • 9. Re: RPC Errors
    Ahmed Bahaa

    Hi twilliamson,

     

    As i thought it is not firewall issue, seems to be TCP time out issue. How is it going now , did you try those recommendations ? if you are using operating system other than Windows , please update me with it and its version.

     

    Thanks,

    Ahmed Bahaa

  • 10. Re: RPC Errors
    Ahmed Bahaa

    Hi twilliamson,

     

    How is this issue going ? Is there any updates ?

     

    Thanks,

    Ahmed Bahaa

  • 11. Re: RPC Errors
    twilliamson

    Thanks for checking.  We are have other issues with the Backups (old Hardware) but we are adding the registry entry and system variable to systems and testing.

     

    We are also having a problem with VSS.  We are seeing errors like:

     

    • relay:VSS SYSTEM SERVICES:\ 5388:save: Failure status of writer MSDEWriter - VSS_WS_FAILED_AT_PREPARE_SNAPSHOT

     

    • relay:VSS SYSTEM SERVICES:\ VSS SYSTEM SERVICES: ERROR: VSS failed to process snapshot, error=0x800423f4: The VSS writer operation failed because of an error that might recur if another shadow copy is created. ** Says to look on Event and Error help link.

     

    • relay:VSS SYSTEM SERVICES:\ 7162:save: save of VSS SYSTEM SERVICES:\ to ormcbackup.ormcinc.org failed

     

          

    • <ERROR> :  Failed with error(s)

     

     

     

    I found a Microsoft KB about a patch and may try this unless you have any other suggestions.

     

    Thanks,

     

     

     

    Tad Williamson

     

    Network Administrator

     

    Oconee Regional Medical Center

     

    twilliamson@ormcinc.org <mailto:twilliamson@ormcinc.org>

     

    478-457-2150

  • 12. Re: RPC Errors
    Ahmed Bahaa

    Hi twilliamson,

     

    As there is Microsoft patch to solve this VSS issues, try it firstly , then if the patch did not solve the problem , we will investigate what is the issue and try to solve. But for the time being update your OS to the latest updates and add the required patches.

     

    Waiting your updates regarding the RPC communication as well.

     

    Thanks,

    Ahmed Bahaa

  • 13. Re: RPC Errors
    Carlos Rojas

    Hi Tad,

     

    What is the OS of that client failing for MSDE writer? I guess it's Windows 2003, right?

     

    There are few known issues for that particular writer. I would suggest you to take a look at the following KB's:

     

    https://solutions.emc.com/emcsolutionview.asp?id=esg111772

     

    You can also try to disable MSDE writer:

     

    https://solutions.emc.com/emcsolutionview.asp?id=esg104603

     

    https://solutions.emc.com/emcsolutionview.asp?id=esg76468

     

    However you should look into Application and System event logs. If you find any event 25 for volsnap then take a look at the following MS KB:

     

    http://support.microsoft.com/kb/826936eps:

     

    32Bit Windows Server 2003 VSS Reset:

    net stop vss

    net stop swprv

    cd %SystemRoot%\system32

    regsvr32 ole32.dll

    regsvr32 vss_ps.dll

    Vssvc /Register

    regsvr32 /i swprv.dll

    regsvr32 /i eventcls.dll

    regsvr32 es.dll

    regsvr32 stdprov.dll

    regsvr32 msxml.dll

    regsvr32 msxml2.dll

    regsvr32 msxml3.dll

     

    64BIT adjustment:

     

    net stop vss

    net stop swprv

    cd %SystemRoot%\system32

    regsvr32 ole32.dll

    regsvr32 vss_ps.dll

    Vssvc /Register

    regsvr32 /i swprv.dll

    regsvr32 /i eventcls.dll

    regsvr32 es.dll

    regsvr32 stdprov.dll

    cd %SystemRoot%\SysWOW64

    regsvr32 msxml.dll

    regsvr32 msxml2.dll

    cd %SystemRoot%\system32

    regsvr32 msxml3.dll

    regsvr32 vssui.dll

     

    And if this doesn't help, I would probably engage Microsoft, however there is a last possible workaround:

     

    http://support.microsoft.com/kb/912414

     

    Thank you.

    Carlos.

     

    Anyway you should also try to re-register VSS dll's. See below the st

  • 14. Re: RPC Errors
    twilliamson

    My apologies for not getting back with you sooner.  We re-aimed the VSS files to a drive with more space on the target/host, but it is still giving us errors on the backups:

     

    VSS SYSTEM SERVICES: ERROR: VSS failed to process snapshot, error=0x800706be: The remote procedure call failed.

    7162:save: save of VSS SYSTEM SERVICES:\ to ormcbackup.ormcinc.org failed
    <ERROR> :  Failed with error(s)

     

    System Writer - ERROR: Failed to save FileGroup files, writer = System Writer
    System Writer - Error saving writer System Writer
    System Writer - ERROR: Aborting backup of saveset VSS SYSTEM FILESET: because of the error with writer System Writer.
    System Writer - Error saving
    7162:save: save of VSS SYSTEM FILESET:\ to ormcbackup.ormcinc.org failed
    <ERROR> :  Failed with error(s)

     

    This only seems to be happening on one of the hosts we are backing up.  I have made sure the BITS, Com+ EventSystem and VSS services are running and it seems the VSS service stops after a while in the backup.

     

    Any Ideas?

    Thanks, Tad

1 2 Previous Next