TMG is taking long time to start (over 16 minutes) on Windows 2008 R2

- The following service is taking more than 16 minutes to start and may have stopped responding: Microsoft Forefront TMG Control
– The following service is taking more than 16 minutes to start and may have stopped responding: SQL Server Reporting Services (ISARS)
– The Microsoft Forefront TMG Control service terminated with service-specific error The wait operation timed out..
– The SQL Server Reporting Services (ISARS) service hung on starting.
– The Microsoft Forefront TMG Firewall service depends on the Microsoft Forefront TMG Control service which failed to start
– The Microsoft Forefront TMG Managed Control service depends on the Microsoft Forefront TMG Control service which failed to start
– The Microsoft Forefront TMG Job Scheduler service depends on the Microsoft Forefront TMG Control service which failed to start

Server dependencies
Then I checked the service startup dependencies comparing the working servers against the nonworking servers; I found out that the servers where the 16 minutes delay happens the dependencies where not setup properly.
Changed the startup dependencies according to the succesfull servers (default+http+keyiso) with the following command:

sc config isactrl depend= RasMan/SSTPSVC/FwEng/ISASTG/bfe/mpssvc/HTTP/KeyIso

Rebooted the server and Problem solved!

Cause:
The issue is that while the Control service is starting (has not reported the started state back to SCM), its actions might require to start additional service(s).
This causes a Service Control Manager deadlock due to which services cannot start right on time. This causes the “slow boot” behavior.
Therefore, we added the dependencies, so that the required services are started before the control service. This way we avoid the above problem.

source: http://www.toolzz.com

TMG 2010 on XenServer 6.0.2

This problem has been bugging me all day, Running Microsoft TMG 2010 on a virtual machine inside Citrix Xenserver.
Ok, time for some ninja system and network admin skills..

It turned out that TMG would run perfect if they where located on same host without XenServer tools installed,the downside with this was that the TMG array where not able to reach servers on other hosts.. If Installing XenServer Tools – The Array where not even able to comunicate with each other on the same host.

Turned out to be TCP Offloading in both the VM`s V`NIC and on the physical nic`s on the xenserver hosts.
The RX and TX can be turned off by running this script on all hosts in the pool.

#!/bin/bash

xe vif-list | grep “uuid ( RO) ” | cut -d “:” -f 2 | while read uuid
do
echo “Changing param for: $uuid”
xe vif-param-set uuid=$uuid other-config:ethtool-tx=”off”
xe vif-param-set uuid=$uuid other-config:ethtool-rx=”off”
done

Disable the Citrix Provisioning Services vDisk icon in the notification area

Now, you could just turn the option off in your base profile (if you have one), but sometimes you may get involved in a project after the base profile is already in place. So, to get rid of the vDisk icon for your users, set the following Registry value through the method of your choice (AppSense EM, Group Policy Preference, login script, etc.)

HKCU\Software\Citrix\ProvisioningServices\StatusTray
Value – ShowIcon
Type – REG_DWORD
Data – 0

Source: The AppSense Bigot : http://appsensebigot.blogspot.com/

Unable to Detach or Destroy SR on XenServer 6.0.2

If using XenCenter to detach or destroy a SR you might get an error

03.05.2012 12:41:18 Error: Detaching SR ‘LVMoHBA2_Raid5′ from ‘Skypoint.no’ – Internal error: Failure(“Storage_access failed with: SR_BACKEND_FAILURE: [ non-zero exit; ; Traceback (most recent call last):\n File \"/opt/xensource/sm/LVMoHBASR\", line 220, in ?\n SRCommand.run(LVHDoHBASR, DRIVER_INFO)\n File \"/opt/xensource/sm/SRCommand.py\", line 252, in run\n ret = cmd.run(sr)\n File \"/opt/xensource/sm/SRCommand.py\", line 94, in run\n return self._run_locked(sr)\n File \"/opt/xensource/sm/SRCommand.py\", line 131, in _run_locked\n return self._run(sr, target)\n File \"/opt/xensource/sm/SRCommand.py\", line 232, in _run\n return sr.detach(self.params['sr_uuid'])\n File \”/opt/xensource/sm/LVMoHBASR\”, line 164, in detach\n LVHDSR.LVHDSR.detach(self, sr_uuid)\n File \”/opt/xensource/sm/LVHDSR.py\”, line 584, in detach\n raise Exception(\”SR detach failed, please refer to the log \” \\\nException: SR detach failed, please refer to the log for details.\n ]”)

to use CLI to identify the host, first list all hosts that are connected to the SR.
xe pbd-list sr-uuid=your-SR-uuid

the output will look something like this:
uuid ( RO) : cb6fc953-b525-cb28-83b2-8de29e7355b8
host-uuid ( RO): 0a11177c-6dbc-41c5-9149-263fbf886024
sr-uuid ( RO): acde1cd9-ecb3-7a97-7106-1d4461ac4d6e
device-config (MRO): SCSIid: 36001438009b03dce0000500000520000
currently-attached ( RO): true

uuid ( RO) : 4fafd723-e948-ce6c-360c-da21e3d4f5b6
host-uuid ( RO): a7acb302-ff14-4c2b-a6d8-b1d4de52e9e6
sr-uuid ( RO): acde1cd9-ecb3-7a97-7106-1d4461ac4d6e
device-config (MRO): SCSIid: 36001438009b03dce0000500000520000
currently-attached ( RO): true

uuid ( RO) : e5bf135b-eb74-4b3d-2eb3-287e3be4af92
host-uuid ( RO): e2ad106c-40d2-452d-9033-f26276c9115b
sr-uuid ( RO): acde1cd9-ecb3-7a97-7106-1d4461ac4d6e
device-config (MRO): SCSIid: 36001438009b03dce0000500000520000
currently-attached ( RO): true

Then unplug the PBD by using this command: xe pbd-unplug uuid=your-PBD-uuid
xe pbd-unplug uuid=4fafd723-e948-ce6c-360c-da21e3d4f5b6
xe pbd-unplug uuid=e5bf135b-eb74-4b3d-2eb3-287e3be4af92

If all of the unbplugs are OK, you can destroy the SR
xe pbd-destroy uuid=your-PBD-uuid

Then delete the associatation of your SR and the PBD
xe sr-forget uuid=your-SR-uuid

In most cases the unplug will fail on one of the hosts, in my case it was the Second XenServer in the Pool that still had som asosiations with it.
Migrated all VMS`s off to another host in the pool and performed reboot on the XenServer hosts, now i could detach, destroy and forget :)

How to Reclaim Disk Space from Deleted XenServer Snapshots

When snapshots or clones are deleted on LVM-based, Local EXT and NFS SRs, XenServer automatically reclaims disk space by freeing unused data, a process known as coalescing. XenServer coalesces the disk space while Virtual Machines (VMs) remain running (online coalescing).

However, the XenServer 5.5 coalescing implementation has a known limitation. After deleting all the snapshots or clones of a VM’s disks, some disk space might remain allocated. Although space consumed by deleted snapshots or clones in NFS and ext-based SRs is minimal, for iSCSI FC and local SRs, the amount of space consumed might be more significant.

To address this limitation, XenServer 5.5 Update 1 and later includes an Off-line Coalesce tool. This tool can reclaim all disk space previously allocated to deleted snapshots or clones while the VM is temporarily shut down or suspended (offline). The Off-line Coalesce tool and its use are described below.

Caution!

Running this script will incur some downtime for the VM, due the suspend/resume operations performed.
Before running the tool, you must delete all snapshots and clones associated with the VM, using either XenCenter or the XenServer CLI. If you do not, the script fails, the tool displays an error message, and it exits.
All VM disks must be either on shared or local storage for a single host. VMs with disks in both types of storage cannot be coalesced.
Procedure – XenServer 5.5 Update 1 and later

Citrix has Described the procedure for doing this on XenServer 5.5 update 1 and XenServer 5.6 in arcticle CTX123400

XenApp 6.5 Mobility Pack ignores Group Policy and Shows Administrative tools on start menu

Late last year Citrix announced the availability of the XenApp 6.5 mobility pack. This mobility pack allows more native gestures to tablet users within their desktop session. Having testing it first hand, its really, really cool, but has a few issues to be aware of.

First there was the issue where Drives like C: and D: and others you had hided with GPO where shown for users on toutch devices,
Citrix Created a Private fix for this with refrence number: LA1302 – this is now included in public hotfix XA650W2K8R2X64025

Second, when users log on by using toutch devices the get access to the entire Start Menu – even if you`ve hidden it with the Group Policy “Remove common programs from start menu”.
Citrix is currently working to fix this but to overcome the issue you can remove the group “users” from %programdata%\microsoft\windows\Start Menu\Programs
If you only have one terminalserver it is quite fast just to logon and perform the customization, but if not you can do this with group policy.
To use Group Policy, simply follow my steps in the post on how to remove PowerShell and ServerManager from startmenu for all new users

Access Gateway Enterprise nCore 9.3 SSL Error 38

after implementing a new Netscaler MPX5500 licensed at Access Gateway Enterprice and the initial setup was finished we started to receive SSL errors and where not able to connect to Xenapp 6.5 servers.
It turned out that when you set the CAG`s hostname at the First Time Setup Wizzard it does not write the entry to /etc/hosts but keeps the default entry “127.0.0.0 ns” and therefore the License file does not match the hostname.

If you try to install your license file, the CAG does not give you any warning at all, and Access Gateway switches from a red X to a green V showing that is is licensed.
But if you look closer to the licensing, you will notice that “maximum ICA users allowed” shows 0

To fix this, SSH to the appliance, and run command SHELL to start the shell on the applicance.
run the command “hostname=yourhostname” and verifi that it has been written to the /etc/hosts file ..

After a reboot everything works like a charm, and shows 10.000 users allowed

Enhanced Access Gateway Enterprise Edition 9.x White Theme for Cleaner Integration with Web Interface 5.4 Design

Citrix Web Interface 5.4 was released in correlation with XenDesktop 5 back in December of 2010. This new release featured a newly redesigned white theme promoting Citrix’s any device campaign. With the switch from white (nFuse) to black (Carbon Fiber) and now back to white (5.4), there can be inconsistencies when used in conjunction with gateway products such as Access Gateway Standard/VPX or Access Gateway Enterprise Edition on Netscaler. This issue is perpetuated if you install as default, utilizing the black carbon fiber Access Gateway Enterprise Edition front-end (default theme on current Netscaler 9.x code) to pass authentication to a white Web Interface 5.4 portal. This blog post will show you how to enhance your access gateway logon screen to cleanly integrate with Web Interface 5.4, transforming the logon screen into this:

The procedures and photos to apply this can be found at youngtech’s blog

XenServer VM/VPS will not Shutdown – Force Shutdown

Sometimes when shutting down a VM via the standard XenCenter interface or command line (xe) on a Citrix XenServer machine the shutdown will not complete. The first thing to try is a force shutdown on the VM:

xe vm-shutdown –force vm=[vm name]

If this still doesn’t work you can try taking a look in the XenServer pending task queue:

xe task-list

And cancel the process’ that seem to be holding up the system:

xe task-cancel uuid=[task uuid]

If this still fails you can try the following:

xe-toolstack-restart

Removing A Lost Xen Server From Resource Pool

A xen server that was in a resource pool became corrupted and lost all nic/network information & needed to be reinstalled from scratch. After reinstalling it, there were duplicate server entries in the pool within the XenCenter console, one was the new server that was reinstalled and added back into the pool, the other was the old ghost entry that was no longer valid. There was no way to remove this lost entry from the XenCenter console. The way to do this is shown below using the xe command.

On the PC with XenCenter installed first get the uuid of the lost host by doing pool-sync-database. You need to connect to the pool master host ip:
C:\Program Files\Citrix\XenCenter>xe -s 10.10.1.25 -u root -pw PASSWORD pool-sync-database
You attempted an operation which involves a host which could not be contacted.
host: 1500f12e-54c6-4d23-b1ab-88ff5216c0d1 (ubcxensrv5)
Next, use the host-forget command to remove the host.
C:\Program Files\Citrix\XenCenter>xe -s 10.10.1.25 -u root -pw PASSWORD host-forget uuid=1500f12e-54c6-4d23-b1ab-88ff5216c0d1
WARNING: A host should only be forgotten if it is physically unrecoverable;
WARNING: if possible, Hosts should be ‘ejected’ from the Pool instead.
WARNING: Once a host has been forgotten it will have to be re-installed.
WARNING: This operation is irreversible.
Type ‘yes’ to continue
yes
Read yes
After that, the lost entry will disappear from the XenCenter console.

source: http://blog.vinteros.com/