Following on from my post on vCloud Director constantly syncing inventory I wanted to address a second point that could cause the underlying connection issue.

In the current revision of vCloud Director (5.1 and 5.1.1) there is an issue that may present itself as vCD disconnecting from vCenter at random times coupled with connection alerts from vCloud Director such as the email alert shown below.

Further information of the error can be seen in the log /opt/vmware/vcloud-director/logs/vcloud-container-info.log.
Look for the following error.  ORA-01013: user requested cancel of current operation.
You can do this as follows.
# less /opt/vmware/vcloud-director/logs/vcloud-container-info.log
Then press / and type in user requested cancel of current operation to go to the location in the log where this entry is recorded.

As detailed in my previous post you can change the connection time to get around it to allow vCD time to reconnect to the vCenter Server. However there is another work around available that involves modifying the vCloud Director SQL database to remove some null entries that keep on creeping up in value and trigger this disconnect in the first place.

To do this I suggest you stop your vCloud Director cells first and make sure to backup the SQL server database. These instructions are for Oracle.

1. Quiesce the services of the cells using the cell-management-tool and then stop the services with service vmware-vcd stop as described here.
2. Backup the Oracle server.
3. Open an SSH connection to the Oracle server and type sqlplus then provide the vcloud username and password.  (Hint username is vcloud)
4. Run the following commands at the SQL> prompt.

SQL> select count(*) from task_inv where (status = 2 OR status = 3) AND completion_date is null;

This will return a numerical value, probably in the tens or hundreds of thousands. What we need to do is to run a series of commands to reduce this number down. It is this number that is causing vCloud Director to time out during the synchronization process.
Run these commands to fix this.

A. Get list of all vc_ids in the setup.
SQL> select distinct vc_id from task_inv;

B. For each vCenter in the setup.
1. Get max managed object value for that vCenter. That is the vc_id obtained from the above query.
SQL> select substr(moref, 6) from (select * from task_inv where vc_id = vc_id order by to_number(substr(moref, 6)) desc) where rownum = 1;

This will give result_1. Next we need to do some basic maths. We will keep ‘top’ 1000 entries per VC and will delete rest of them.

2. result_1 minus 1000 = result_2

3. Using the above result_2, run the following.

SQL> delete from task_inv where (status = 2 OR status = 3) AND completion_date is null AND vc_id = vc_id AND to_number(substr(moref, 6)) < result_2;

4. Run a commit; command.

5. Finally run the original query to see if the number has gone down.
SQL> select count(*) from task_inv where (status = 2 OR status = 3) AND completion_date is null;

Don’t forget to restart the cell services on vCD.  service vmware-vmd start and tail the cell.log to watch the progress of restarting the cell service. /opt/vmware/vcloud director/logs/cell.log -f

It will continue to creep up until VMware fix this in an update currently due as release version 5.1.2 at the end of April 2013.