It's All Just Ones and Zeros

Class Diagrams

Over the years people have asked me for class diagrams of the Vault API.  I avoided doing them mainly because I don't have time.

Recently, I've started playing around with Visual Studio 2010, and I found that it can auto generate some pretty good diagrams.  So now I have no excuse.  Here are the class diagrams for the Vault API.  The only thing missing are the service classes from the Web Service API.  The diagram would have been incomprehensible if I left them in.

Web Service API diagram
Client API diagram
Job Processor API diagram

I don't like how enums get a self reference, but everything else looks accurate.

User Impersonation

 

Here is a common problem.  You have a custom command that you want people to use, but it does something that requires admin privileges.  Making everyone an administrator is a bad idea.  A better solution is for your command to perform an action as another user.

Vault doesn't support a true impersonation model, but there is a workaround that works pretty well.  Just have your program log in as another user to perform the special task.  Log out as soon as the task is done and let the rest of the operations be performed by the logged-in user.

Recycle Bin 2.0 makes liberal use of this technique.  The whole program rests on a special "Recycle Bin" folder where files go before they are deleted.  Obviously this folder needs to be locked down from normal use.  All interaction with the Recycle Bin folder needs to be through one of the custom commands.  To enforce this, the administrator needs to designate a special Recycle Bin User, which is the only one allowed to interact with the Recycle Bin folder.  Any time a Recycle Bin command is executed, the code logs in as the Recycle Bin user in order to interact with the folder.

Considerations:

First, your code needs a way to know how to log in as the impersonated user.  In practice the impersonated user is usually an administrator, so you have to keep the password hidden from the end user.  There are various ways to fix this.  You can hard-code the password, or you can encrypt the password and put it in a common location. 

Next, there is the issue of licensing.  If you log in as two different users, you will consume 2 licenses.  If licenses are tight, then impersonation might not be an option.  Logging out the original user is not a good idea because what if all the licenses are used up when the operation is over an they try to log back in? 
The best approach is to use the impersonated user only as long as needed.  When the operation is over, log out immediately to free up the license.  If all goes well, it shouldn't have a noticeable impact on the licenses being consumed.
Also, make sure to handle errors properly.  If errors come up during the operation, you still want to make sure the impersonated user get logged out at the end.

Lastly, impersonation has an impact on data history.  Let's say you have an operation that uses impersonation to move a file to the Released state.  Later in time you want to see who released a certain file.  That becomes hard to do since the history of the item will show the impersonated user as the one who made the change.  What you want to see is the actual user. 
This can be solved by storing the actual user as a piece of meta-data, such as a UDP value.  It's not a perfect workaround because the data you want is not in the place where you expect it to be. 

Spotlight: powerJobs

Welcome back to my Spotlight section, where I feature Vault application written by people outside of Autodesk. 

Today we have powerJobs, which is another program from coolOrange.

From the creators:
Create PDFs of your Vault documents via Jobserver and even better customize existing job scripts and create your own custom jobs.

Click here for screenshot

Product URL:  http://www.coolorange.net/html/products.php

About the Company:
coolOrange develops innovative software products, based on newest technologies, to squeeze the potential of Autodesk customers and helps resellers to succeed in their projects.

---

If you have a neat Vault application, let me know about it, and I'll feature it in the Spotlight section.  It doesn't have to be a released product either.  Internal utilities and consulting jobs are good too.  Just drop me a line in the comments section and I'll be in touch.

If I don't get any more entries soon, I may have to rename this section as the coolOrange section.

Replication - Testing on a Single Server

The recommended way to test your applications with the connected workgroups feature is to use 2 computers for testing.  If you only have 1 computer you can easily fix the problem by conjuring another one with your magic wand.

What's that you say?  You don't have a magic wand?  No problem.  Just go to your boss and ask for a new computer to be purchased from your unlimited budget.

Oh, I see.  You have a fixed budget and would rather spend money on other things.  Fine.  I figured out how to get 2 replicated Vaults running on the same server, and I wrote how to do it in a document.

Click here to download the document

Keep in mind, this technique is just for development and testing purposes.  Don't use it for production Vaults.  Although I can't think of why you would ever want to.

Vault Web View

Update:  The Vault 2012 version is available.

Here is a simple app which uses a custom tab view.  Vault Web View is an Internet Explorer control with a URL that is set on a folder by folder basis.  This is useful if you have data that is not in Vault, but is accessible through a browser.

(click image to see full size)

---

Requirements:
Vault Workgroup/Collaboration/Professional 2011

Click here to download the application
Click here to download the source code

---

To use:
Run the installer.
Right-click on a folder and run the "Set URL" command to set the URL for the folders you want an active Web View tab.

Notes:
You need to be an administrator to set the URL on a folder.
There are some special strings you can put in the URL which will get replaced at runtime:
    #FILENAME# - Gets replaced with the name of the currently selected file.
    #FOLDERID# - The ID of the current folder.
    #SERVER# - The server name.

Lesser Used Features

There are a couple of features on this blog that I want to make sure everybody knows about.

First, at the top and bottom is a sitemap link.  The site map is just a quick listing of all the major posts, grouped by category.  I find this to be better way of finding things than the date-based approach that blogs use.

Next, I have a Spotlight section, which has the intent of showcasing interesting things that people outside Autodesk are developing using the Vault API.  So far, there is only 1 entry, and it is from almost a year ago.  If I don't get more entries soon, I'll close the section down.
Your application doesn't need to be a for-sale product.  Maybe you created a tool for internal use, or you did a consulting job.  I still want to hear about it and feature it. 
If you are interested in sharing what you did, let me know in the comments section.  I will then get in touch with you over email.

Replication - Introduction

Replication.
Replication.
Also known as Connected Workgroups, Replication is the feature where you can create multiple, working copies of the Vault database.  The main reason you would want to do this is if you have multiple locations working on the same vault where your network latency is too large between locations.  Replication speeds things up by allowing users to work exclusively with their local location.  In the background, the data gets synchronized with the other site.  This article will go into more details on this process because it has an impact on application development.

To understand Vault replication, you need to first understand the SQL Server replication since that is the underlying technology.  As a rule, I try to avoid database details.  But for replication, that rule is trumped by a bigger rule: Give developers the information they need to write Vault applications. 

SQL Server Replication
In a nutshell, the concept is that you have separate databases on separate servers with the same data.  This sounds simple, but it's actually one of the most complex things in database programming. 

As you probably already know, most of the Vault data is in a database.  So when you are viewing the Where Used on a part file, for example, you are just seeing a representation of a bunch of database rows.  What SQL Server does is keep track of all the rows that change.  These changes then get transferred to the other SQL Servers via a background process, and that's how data stays in sync.

One important thing about this transfer: it is not instantaneous.  Even if you have a fast connection, you can expect there to be a delay of at least 1 minute before changes get transferred.  This delay can also be longer if you have a slow connection or the network is under heavy load. 

Here is where things start getting complex, because it's possible for data to be legal in one SQL Server but illegal when replicated.  For example, in Vault, two people on different workgroups add a file called A.dwg to the same folder at the same time.  Everything works fine until the data is replicated.  Now you have two files with the same name in the same folder, which violates the rules on how Vault data should be structured.  Problems like these are called conflicts. 

Ownership
There are various ways to deal with conflicts.  For Vault, we took the approach of preventing conflicts from happening in the first place.  We do this through the concept of ownership.  If you want to make any edit in Vault, your workgroup needs the proper ownership.  So my earlier example is not possible.  You can only add A.dwg to a folder if your workgroup (aka. SQL instance) has ownership of the folder.  The workgroup without ownership will get an error if the try to add a file to the folder (also, the folder will show up as locked).

The granularity of ownership varies depending on what you are working with.  There's the concept of Entity Ownership, where a workgroup has control of specific entities.  Files, folders, items and change orders are the only entities that can be owned in this way.  Everything else uses the concept of Database Ownership, which means that a workgroup owns all data in the database which is not controlled via entity ownership.  This covers things like property definitions, users, groups, categories, and so on.  Mostly administrator settings are controlled this way.

Transferring ownership between two workgroups involves a handshake between the two servers.

1. Workgroup A has ownership.
2. Workgroup B requests ownership from A.
3. Workgroup A changes ownership to B.  But this change is only recorded in A's database.
4. Data is replicated to B via the SQL Server background process.
5. Workgroup B has ownership.

Some things to note about this process:

• Ownership transfer is not instant.  The requesting workgroup needs to wait for a replication cycle before it has ownership.
• Between steps 3 and 4 things are in a strange state.  A thinks B has ownership and B thinks A has ownership.  Effectively, this means that nobody has ownership.

Leases
The last concept I want to talk about here is the lease concept.  Basically a lease prevents ownership from being transferred for a set length of time.  The rationale is that if you just grabbed ownership of something, you probably want to work on it for a while.  You don't another workgroup grabbing it away from you a few seconds later.  So when you transfer ownership, you set a lease period.  Until that lease expires, nobody can change the ownership.  Once the lease expires, it is still owned by the same workgroup.  Only when another workgroup requests ownership will the ownership change.

I think by default you have a 1 day lease on entities you create.  If you grab ownership from somewhere else, you get a 1 hour lease.  If you have ownership of something, you can fully edit the lease.  You can extend or shorten the lease time.  You can remove the lease on an item or you can add a lease on something that does not have one.  You can't edit the lease of something not owned by your workgroup.