XenApp 7.5 “Fail worker callback using SPN HOST/ and IP address”

Following the previous post regards XenApp 7.5 Application Delivery Controller (ADC), it also seems that XenApp 7.5 Virtual Deliver Agent (VDA) fails to connect to ADC when hosted on Windows Server 2008 R2 x64 Datacenter. 

Our Setup

Below is the test environment we’re using.

  1. XenApp 7.5 ADC running on Windows Server 2012 x64 Standard (it failed to run on Windows Server 2008 R2 SP1 x64 Datacenter)
    1. 10.20.0.75, cont2012.adtest.local
  2. XenApp 7.5 VDA running on Windows Server 2008 R2 SP1 x64 Datacenter
    1. 10.20.0.72, vda2008.adtest.local
  3. Active Directory Domain Controller (Windows Server 2008 R2 SP1 x64 Datacenter)
    1. 10.20.0.10 (domain: adtest.local)
  4. Both ADC and VDA are connected to AD DC
  5. Windows firewall is OFF on AD DC, ADC and VDA, no other firewall is set up
  6. No antivirus is installed therefore no built-in firewall

Connectivity and security.

  1. Ping works both ways for IPs and FQDNs, DNS resolution has no issues
  2. Kerberos Key Distribution service is enabled and running on AD DC
  3. Registry value on VDA for “ListOfDDCs” is set to cont2012.adtest.local
    1. HKEY_LOCAL_MACHINE\Software\Citrix\VirtualDesktopAgent\ListOfDDCs (REG_SZ)
  4. ADC computer is added to VDA local Administrators group
  5. VDA computer is added to ADC local Administrators group
  6. ADC computer is added to VDA security policy for “Access this computer from the network”
  7. VDA computer is added to ADC security policy for “Access this computer from the network”

Webserver is up and running on ADC on port 80, and is accessible from VDA and AD DC via telnet and web browser.

Troubleshooting

VDA Logs

VDA logging was up as per these Citrix instructions.

BrokerAgent:ConstructAndResolveRegistrarNames: Using IP Addresses; IP 10.20.0.75, Hostname cont2012.adtest.local, m_UseIpv6Registration = False

BrokerAgent:=========>>>>> Attempting registration with following controller(s): cont2012.adtest.local (10.20.0.75)

BrokerAgent:AttemptRegistrationWithSingleDdc: Attempting to talk to controller...
BrokerAgent:AgentHeartBeat m_connectionId = S-1-5-21-3517788518-937966496-1463735470-1123:D3C3710AC76B5DFA810F54CB97E93141:635322141639732680

BrokerAgent:CurrentSettingsVersion is 0; 
BrokerAgent:We are attempting to register with DDC 'cont2012.adtest.local'; Previous successful registration was with DDC ''

BrokerAgent:Sending CurrentSettingsVersion = 0 to DDC to force policy delivery
BrokerAgent:Registration request 7.5.0.4523 Windows 2008 R2 Service Pack 1 Microsoft Windows NT 6.1.7601 Service Pack 1S-1-5-21-3517788518-937966496-1463735470-1123NULL0.

BrokerAgent:request.WorkerCapabilities CBP1_5
BrokerAgent:request.WorkerCapabilities MultiSession
BrokerAgent:Registration multi-session Type MultiSession.
BrokerAgent:AttemptRegistrationWithSingleDdc: Failed to register with http://CONT2012.adtest.local:80/Citrix/CdsController/IRegistrar. WCF Fault with detail CallbackCommunicationError, message 'Fail worker callback using SPN HOST/VDA2008.adtest.local and IP address 10.20.0.72'

BrokerAgent:TimedEventLogWorkItemManager::ProcessWorkItemThreadBody - Processing
BrokerAgent:TimedEventLogWorkItemManager::ProcessWorkItemThreadBody - Sleeping 599999ms
BrokerAgent:AttemptRegistration: Could not register with any controllers. Waiting to try again in 120000 ms. Multi-forest - False

Windows Event Logs on ADC

Citrix Broker Service:

The Citrix Broker Service failed to contact virtual machine 'VDA2008.adtest.local' (IP address ). 

Check that the virtual machine can be contacted from the controller and that any firewall on the virtual machine allows connections from the controller. See Citrix Knowledge Base article CTX126992. 

Error details: 
Exception 'The request channel timed out while waiting for a reply after 00:00:05. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout.' of type 'System.TimeoutException'.

Citrix Desktop Service:

The Citrix Desktop Service cannot connect to the delivery controller 'http://CONT2012.adtest.local:80/Citrix/CdsController/IRegistrar' (IP Address '10.20.0.75')

Check that the system clock is in sync between this machine and the delivery controller. If this does not resolve the problem, please refer to Citrix Knowledge Base article CTX117248 for further information.

Error Details: 
Exception 'Fail worker callback using SPN HOST/VDA2008.adtest.local and IP address 10.20.0.72' of type 'System.ServiceModel.FaultException`1[Citrix.Cds.Protocol.Controller.Fault]'..

Windows Event Logs on VDA

The Citrix Desktop Service has detected that the delivery controller cont2012.adtest.local (IP Address 10.20.0.75) cannot connect to the Service. One possible reason for this is that the 'Access this computer from the network' security policy does not allow the delivery controller server identity to access this machine. 

Please check that a local or group policy is not set incorrectly to disallow access from the delivery controller servers.

XDPing Tool

Citrix XDPing tool was set up to help troubleshoot issues.

Output for ADC below.

XDPing 2.2.0.0
Created by Citrix Systems Engineering and Escalation teams.

Checking version : You are using the latest version.

--------------------------------------------------------------------
Local Machine::

  NetBIOS Name = CONT2012
  OS Version   = Microsoft Windows NT 6.2.9200.0
  Platform     = X64 Platform

  Computer Domain: adtest.local
    Role       = Member Server
    Membership = Verified, SID:S-1-5-21-3517788518-937966496-1463735470-1128 [OK]

--------------------------------------------------------------------
User::

  User Name      = administrator
  User Domain    = ADTEST
  Authentication = Kerberos [OK]
  Groups:
     ADTEST\Domain Users
     Everyone
     BUILTIN\Users
     BUILTIN\Administrators
     NT AUTHORITY\REMOTE INTERACTIVE LOGON
     NT AUTHORITY\INTERACTIVE
     NT AUTHORITY\Authenticated Users
     NT AUTHORITY\This Organization
     LOCAL
     ADTEST\Group Policy Creator Owners
     ADTEST\Domain Computers
     ADTEST\Domain Admins
     ADTEST\Enterprise Admins
     ADTEST\Schema Admins
     ADTEST\Denied RODC Password Replication Group

--------------------------------------------------------------------
Local Machine Time::

  UTC   = 4/4/2014 2:43:33 PM
  Local = 4/4/2014 3:43:33 PM (GMT Daylight Time)
  DST   = Yes
  NtpServer = time.windows.com,0x9

--------------------------------------------------------------------
Domain Controller(s) Time::

Date/Time from adtest.local : 4/4/2014 3:43:33 PM : Time difference (mins): 0 [OK]

--------------------------------------------------------------------
Network Interfaces::

  NIC #0 "Ethernet":
    Network      = Ethernet, 1Gb/s, Up
    MAC          = 00:11:22:D4:89:00
    DNS servers  = 10.20.0.10
    Gateways     = 10.20.0.1
    DHCP server  = 10.20.0.1
    Address #0   = 10.20.0.75/255.255.255.0, Preferred, Origin=Dhcp/OriginDhcp
           Lease = 5400/3063/3063

  NIC #1 "Loopback Pseudo-Interface 1", Loopback:
    Network      = Loopback, 1073Mb/s, Up
    DNS servers  = fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1
    Address #0   = ::1/0.0.0.0, Preferred, Origin=WellKnown/WellKnown
           Lease = 2359/4294967295/4294967295
    Address #1   = 127.0.0.1/, Preferred, Origin=WellKnown/WellKnown
           Lease = 2359/4294967295/4294967295

  NIC #2 "isatap.{5DF39DBE-C24F-4D98-80CE-E324E17C10FB}":
    Network      = Tunnel, 0Gb/s, Down
    MAC          = 00:00:00:00:00:00:00:E0
    DNS servers  = 10.20.0.10
    Address #0   = fe80::5efe:10.20.0.75%14/0.0.0.0, Deprecated, Origin=WellKnow
n/LinkLayerAddress
           Lease = 2299/4294967295/4294967295

  NIC #3 "Local Area Connection* 11":
    Network      = Tunnel, 0Gb/s, Down
    MAC          = 00:00:00:00:00:00:00:E0
    Address #0   = fe80::100:7f:fffe%13/0.0.0.0, Deprecated, Origin=WellKnown/Li
nkLayerAddress
           Lease = 2348/4294967295/4294967295

--------------------------------------------------------------------
WCF Endpoints: CitrixBrokerService::
C:\Program Files\Citrix\Broker\Service\BrokerService.exe
Version Number :7.5.0.4526

XenDesktop version 7.5.0.4526
 wsHttpBinding:
 Citrix.Broker.Admin.SDK.IBrokerAdminService:
 http://localhost/Citrix/BrokerAdminService/v2:
    Ping Service: /Citrix/BrokerAdminService/v2
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Broker.Admin.IBrokerAdminQuery:
 http://localhost/Citrix/BrokerAdminQuery/v1:
    Ping Service: /Citrix/BrokerAdminQuery/v1
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.EnvTest.Interfaces.IEnvTestApi:
 http://localhost/Citrix/BrokerEnvTests/v1:
    Ping Service: /Citrix/BrokerEnvTests/v1
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Controller.IRegistrar:
 http://localhost/Citrix/CdsController/IRegistrar:
    Ping Service: /Citrix/CdsController/IRegistrar
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Controller.ITicketing:
 http://localhost/Citrix/CdsController/ITicketing:
    Ping Service: /Citrix/CdsController/ITicketing
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Controller.IDynamicDataSink:
 http://localhost/Citrix/CdsController/IDynamicDataSink:
    Ping Service: /Citrix/CdsController/IDynamicDataSink
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Controller.INotifyBroker:
 http://localhost/Citrix/CdsController/INotifyBroker:
    Ping Service: /Citrix/CdsController/INotifyBroker
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]

--------------------------------------------------------------------
Controller Services::

  Service  : Licensing services not present [OK]

--------------------------------------------------------------------
DNS Lookups for Local Machine::

  Host Name  : CONT2012.adtest.local
  Address #0 = ::1 (rDNS: CONT2012.adtest.local) [OK]
  Address #1 = 10.20.0.75 (rDNS: CONT2012.adtest.local) [OK]

--------------------------------------------------------------------
Event Log Check::
Invalid query

--------------------------------------------------------------------
Windows Firewall Settings::
  XDPing has detected that the Windows Firewall service is not runnning. Skippin
g firewall check.

--------------------------------------------------------------------
Summary::

    Checking version : You are using the latest version. [OK]

Number of messages reported = 1

Output for VDA below.

XDPing 2.2.0.0
Created by Citrix Systems Engineering and Escalation teams.

Checking version : You are using the latest version.

--------------------------------------------------------------------
Local Machine::

  NetBIOS Name = VDA2008
  OS Version   = Microsoft Windows NT 6.1.7601 Service Pack 1
  Platform     = X64 Platform

  Computer Domain: adtest.local
    Role       = Member Server
    Membership = Verified, SID:S-1-5-21-3517788518-937966496-1463735470-1123 [OK]

--------------------------------------------------------------------
User::

  User Name      = administrator
  User Domain    = ADTEST
  Authentication = Kerberos [OK]
  Groups:
     VDA2008\None
     Everyone
     BUILTIN\Administrators
     BUILTIN\Remote Desktop Users
     BUILTIN\Users
     NT AUTHORITY\REMOTE INTERACTIVE LOGON
     NT AUTHORITY\INTERACTIVE
     NT AUTHORITY\Authenticated Users
     NT AUTHORITY\This Organization
     LOCAL
     ADTEST\Group Policy Creator Owners
     ADTEST\Domain Computers
     ADTEST\Domain Admins
     ADTEST\Enterprise Admins
     ADTEST\Schema Admins
     ADTEST\Denied RODC Password Replication Group

--------------------------------------------------------------------
Local Machine Time::

  UTC   = 4/4/2014 4:17:03 PM
  Local = 4/4/2014 5:17:03 PM (GMT Daylight Time)
  DST   = Yes
  NtpServer = time.windows.com,0x9

--------------------------------------------------------------------
Domain Controller(s) Time::

Date/Time from adtest.local : 4/4/2014 5:17:03 PM : Time difference (mins): 0 [OK]

--------------------------------------------------------------------
Network Interfaces::

  NIC #0 "Local Area Connection":
    Network      = Ethernet, 1Gb/s, Up
    MAC          = 00:11:22:84:5C:D9
    DNS servers  = 10.20.0.10
    Gateways     = 10.20.0.1
    DHCP server  = 10.20.0.1
    Address #0   = 10.20.0.72/255.255.255.0, Preferred, Origin=Dhcp/OriginDhcp
           Lease = 3600/3410/3410

  NIC #1 "Loopback Pseudo-Interface 1", Loopback:
    Network      = Loopback, 1073Mb/s, Up
    DNS servers  = fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1
    Address #0   = ::1/0.0.0.0, Preferred, Origin=WellKnown/LinkLayerAddress
           Lease = 216/4294967295/4294967295
    Address #1   = 127.0.0.1/, Preferred, Origin=WellKnown/WellKnown
           Lease = 216/4294967295/4294967295

  NIC #2 "isatap.{F1C84D44-AE9D-4F04-8853-EFCA3BB4C4E2}":
    Network      = Tunnel, 0Gb/s, Down
    MAC          = 00:00:00:00:00:00:00:E0
    DNS servers  = 10.20.0.10
    Address #0   = fe80::5efe:10.20.0.72%13/0.0.0.0, Deprecated, Origin=WellKnow
n/LinkLayerAddress
           Lease = 170/4294967295/4294967295

  NIC #3 "Local Area Connection* 9":
    Network      = Tunnel, 0Gb/s, Down
    MAC          = 00:00:00:00:00:00:00:E0
    Address #0   = fe80::100:7f:fffe%11/0.0.0.0, Deprecated, Origin=WellKnown/Li
nkLayerAddress
           Lease = 212/4294967295/4294967295

--------------------------------------------------------------------
WCF Endpoints: BrokerAgent::
C:\Program Files\Citrix\Virtual Desktop Agent\BrokerAgent.exe
Version Number :7.1.0.4019

XenDesktop version 7.1.0.4019
 wsHttpBinding:
 Citrix.Cds.Protocol.Worker.ILaunch:
 http://localhost/Citrix/VirtualDesktopAgent/ILaunch:
    Ping Service: /Citrix/VirtualDesktopAgent/ILaunch
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Worker.IDynamicDataQuery:
 http://localhost/Citrix/VirtualDesktopAgent/IDynamicDataQuery:
    Ping Service: /Citrix/VirtualDesktopAgent/IDynamicDataQuery
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Worker.IQueryAgent:
 http://localhost/Citrix/VirtualDesktopAgent/IQueryAgent:
    Ping Service: /Citrix/VirtualDesktopAgent/IQueryAgent
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Worker.IConfiguration:
 http://localhost/Citrix/VirtualDesktopAgent/IConfiguration:
    Ping Service: /Citrix/VirtualDesktopAgent/IConfiguration
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]
 wsHttpBinding:
 Citrix.Cds.Protocol.Worker.ISessionManager:
 http://localhost/Citrix/VirtualDesktopAgent/ISessionManager:
    Ping Service: /Citrix/VirtualDesktopAgent/ISessionManager
      Connect = Tcp to ::1:80 via ::1 ("Loopback Pseudo-Interface 1") [OK]
      Service = Listening [OK]

--------------------------------------------------------------------
Workstation Services::

  Service  : BrokerAgent ("Citrix Desktop Service")
    Status = Win32OwnProcess, Running [OK]
    Prereq =
      LanmanWorkstation (Win32ShareProcess), Running

  Service  : Citrix Encryption Service ("Citrix Encryption Service")
    Status = Win32OwnProcess, Running [OK]

  Service  : cpsvc ("Citrix Print Manager Service")
    Status = Win32OwnProcess, Running [OK]
    Prereq =
      Spooler (Win32OwnProcess, InteractiveProcess), Running
      RpcSs (Win32ShareProcess), Running

--------------------------------------------------------------------
DNS Lookups for Local Machine::

  Host Name  : VDA2008.adtest.local
  Address #0 = ::1 (rDNS: VDA2008.adtest.local) [OK]
  Address #1 = 10.20.0.72 (rDNS: VDA2008.adtest.local) [OK]

--------------------------------------------------------------------
Client Details::
   (Session ID) (Status)    (Name)   (Client IP Address):
       0        WFDisconnected   Services   0.0.0.0
       1        WFConnected   Console   149.112.255.255
       2        WFActive   RDP-Tcp#0   10.96.13.81
       65536        WFListen   ICA-CGP   54.0.1.0
       65537        WFListen   ICA-CGP-1   54.0.1.0
       65538        WFListen   ICA-CGP-2   54.0.1.0
       65539        WFListen   ICA-CGP-3   54.0.1.0
       65540        WFListen   ICA-HTML5   54.0.1.0
       65541        WFListen   ICA-TCP   54.0.1.0
       65542        WFListen   RDP-Tcp   54.0.1.0

   Estimated Latency:           -1
   Estimated Bandwidth:         ???
   Estimated Network Condition: DIALUP_CONDITIONS
   Session Reliability:         False

--------------------------------------------------------------------
Event Log Check::
  No importent XenDesktop events detected in the last hour.

--------------------------------------------------------------------
Windows Firewall Settings::
  XDPing has detected that the Windows Firewall service is not runnning. Skipping firewall check.
--------------------------------------------------------------------
XenDesktop Farm::

  Farm GUID (GPO)   : Not Set
  Farm GUID (local) : NOT SET
  Farm GUID In Use  : NOT SET
--------------------------------------------------------------------
Registry Based Configurations::

Registry based Controller list (ListOfDDCs) : [Not Conigured]
 [Not Conigured]
  It is not possible to enurmerate DDC list from VDA [ERROR]

--------------------------------------------------------------------
Summary::

    Checking version : You are using the latest version. [OK]
    It is not possible to enurmerate DDC list from VDA [ERROR]

Number of messages reported = 2

Workaround

XenApp 7.5 VDA works out of the box on Windows Server 2012 x64.

2 thoughts on “XenApp 7.5 “Fail worker callback using SPN HOST/ and IP address”

  1. The callback communication is one small part of the error, you need to look deeper in the brokerservice.exe logfile
    Windows 2008 R2 Sp1 Datacenter has been tested and works fine. You sure you didn’t accidentally have a policy on there that restricted network access Like = “Access To Computer From The Network” policy?

  2. Thanks for your input Carlos, much appreciated. Yes, I’m pretty confident this wasn’t the issue as computers were added to security policy for “Access this computer from the network”, check the blog post please.

    We followed online installation instructions using Windows Server 2008 R2 SP1 x64 Datacenter, but never got VDA connected to ADC. We followed the same online installation instructions using Windows Server 2012 x64 and had no issues at all.

Leave a Reply

Your email address will not be published. Required fields are marked *