A couple of people have been asking for this one. So here you go, Find Orphaned Files for Vault 2015.
Also, after my last app, I’ve been eager to post something that, you know, works.
Click here for a description of the app.
Requirements:
Vault Workgroup or Professional 2015
Click here to download the application
Click here to download the source code
Click here for the Vault 2014 version
As with all the samples on this site, the legal disclaimer applies.
Every now and then I write an app only to find out at the end that the app doesn’t work well. This is one of those cases. I’m going to post the app anyway because I believe the code may be of use to somebody. Splunk is a useful tool. I just wasn’t able to do what I wanted given the information that Vault is printing to the logs. Also, I spent a lot of time on it, so I should get a blog post out of it at least.
Here is the app. I was hoping for something that shows the download history of a file.
It would be pretty cool if it worked, but there are too many cases where downloads will not show up in the list. If your users download one file at a time, this will work great. But if they download multiple files at once (and they do), then this app completely falls apart. In my early tests, I only download one file at a time. Oops.
The underlying problem is that the Vault log files are structured around “operations” and not around the “objects” being acted on. Vault likes to operate on multiple objects at once for performance reasons. There is no clean way to log all the objects in one log entry, so that information just doesn’t get logged.
Requirements:
Click here for the application
Click here for the source code
As with all the samples on this site, the legal disclaimer applies.
Splunk Setup:
OK, there are a lot of steps here. But the added benefit is that you get Splunk up and running which provides a lot of ways for you to manage your Vault data.
2014-07-25 18:28:28 10.143.48.29 POST /AutodeskDM/Services/v19/PropertyService.svc op=GetProperties&uid=2&vaultName=Vault&sessID=143994478&app=VP 80 - 10.143.48.22 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+4.0.30319.18408) 200 0 0 9
Just because it’s in the Vault API, doesn’t mean you will be able to build anything cool with it. There is plenty of stuff in there that is so specific to Vault Explorer or so difficult to use that it’s basically useless to you as an outside developer. I’m feeling that this will probably be an entire series of articles... especially when I get to the Package Service.
So why am I bothering to write an article about something you can’t use? Because it’s good to know what you can’t do upfront. That way you don’t waste precious time on the impossible. GI Joe cartoons taught me that “Knowing is half the battle.” Well, following that logic, not-knowing is the other half. So let’s get started!!
Vault Professional has a cool feature where you can put a watermark on your DWF files. As a developer, you may be thinking that you can make use of this. Maybe you want to download these watermarked files or view them in you own app.
No. Sorry. You can’t do that.
Although the Vault Server keeps track of the watermark settings, it only includes things like color, text, and location. The server doesn’t actually generate a file with the watermark on it. The watermark is rendered client-side. It’s basically another layer drawn on top of the existing content in the DWF viewer.
Vault Explorer uses a special DWF viewer that is compiled just for Vault. And it doesn’t have an API. Basically, the watermark functions in the Vault API are specifically for the out-of-the-box Vault clients and no one else.
Useless functions:
All of these functions are in the Item Service.
Difficulty:
If you really want to watermark your own DWFs, here are the basic steps:
In the past, I posted the installer project I used to build Vault apps. However, that project was for Visual Studio 2010. In the 2012 version, Microsoft dropped the installer project type and replaced it with an Install Shield project type. So I’ll share with you my latest installer template. You do not need to purchase Install Shield.
Click here to download the project source
My recommendations on how to configure:
In the solution explorer, you will see that Install Shield sets things up in numbered steps, so I’ll just go through the relevant parts, from the top on down.
I usually just skip over the Getting Started and Project Assistant stuff. Personally, I found it more confusing than helpful.
Now you are ready to build. If all goes well, you will see your setup.exe file in your output folder, which is something like [projectFolder]\Express\SingleImage\DiskImages\DISK1
Now all you have to do is zip up the EXE, post it on your blog, and wait for the money to roll in.
One of the biggest confusions about the Vault/SharePoint integrations is how filtering and paging works. It confuses me from time to time, and I wrote the integration. In this article, I’ll try to describe what’s going on behind-the-scenes to give you a better picture of the integration and how to configure it correctly. Ahh, the joys of middleware.
Let’s start with a SharePoint list showing some Vault data.
(click image for full view)
Take a minute to guess how that data got from Vault to SharePoint. Whatever you came up with in your head is probably wrong. I’m not saying you are dumb or anything; I’m saying that the workflow is counter-intuitive.
Basically there are three components in play here: SharePoint, Vault and the integration between the two.
When the user goes to view the list, SharePoint invokes the Integration, the Integration invokes Vault. The data is then pass up through the Integration back to SharePoint. Pretty standard so far.
My example screenshot shows 10 items on the page. So you would think that 10 objects flowed through the Integration. But that’s not what happened. The fact that SharePoint is showing 10 rows/page is a view setting, which include page size, column layout, sorting, etc.
Here is the important thing that you need to understand:
| The integration does not know what the view settings are. |
For example, the Vault Integration does not know to ask Vault for 30 objects. It also doesn't know, which columns to sort by, which page the user is viewing or any of the other things in the view model.
So, what does the Integration use as criteria when it pulls data from the Vault? The answer is Data Source Filters. These are the only things that SharePoint communicates to the Integration. The filters themselves are defined by the integration and will be different depending on the Vault object being shown. In this example, the Integration defined four filters. The values are configurable, but you can't add or remove filters.
(click image for full view)
Basically, the integration only gets to see the part in red. The rest of the list settings are the SharePoint’s view model.
Let’s take a look at the Limit filter. This one is important because that’s how many objects the integration asks for. It has nothing to do with how much SharePoint shows on the screen. If you leave this field blank, the Integration uses the default value of 100. You can set the limit higher if you want, but that may slow things down.
The other filter values are pretty straightforward. Set the State to “Released” and the integration will ask for only Released objects in the Vault. Set the Vault Folder to “$/Projects” and only files under that folder will be returned from Vault.
When the Vault Objects are handed from the Integration over to SharePoint, the view settings are then applied. This includes sorting and filtering. If there are 100 Vault objects, but the view has only 10 items on a page, then the other 90 objects are discarded.
In the end, you have two levels of filtering:
Let’s walk through the whole process, just to drive the point home...
It’s been a while since I’ve highlighted the Vault API articles recently written on other blogs.
Vault API articles from the Autodesk Developer Network:
Data Standard articles from coolOrange:
Other:
Click here for Developer Diary #1
I’ve just been playing around with the out-of-the-box features in Splunk, and it’s quite impressive. Here is a video of what I’ve been able to do with the various Vault logs after only a short time. And I still haven’t downloaded their API.
Now that I have a feel of the product, I can sink my teeth into the API. Suggestions welcome.
Important News: Before I get into this article, there are some hotfixes you will want to get if you are going to use persist(ent/able) IDs. If you are already using these IDs and have replicated Vault database servers, you need to check those IDs to make sure they are valid.
Hotfix links:
Once again, the API contains two different things that are almost spelled the same (Naming scheme and numbering scheme, I’m looking at you). This time around it’s Persistent IDs and Persistable IDs. These two things are almost the same, but API’s don’t understand “almost”. If it’s not exactly the same, it’s invalid. There is no in-between.
I’ve already written about persistent IDs, so you can give that a read if you want to know about the concept. Persistable IDs are the same thing except those IDs have an extra character in from of them. No, I’m not making that up.
An “m” is put on the front if the Persistable ID is not version specific. A “v” is put on the front if the ID to a specific version. The “m” stands for Master, in case you are wondering. Like how a FileMasterId is the same for all versions of a file. Anyway, the rest of the ID is the same as the Persistent ID. So you can easily convert from one to the other by adding or removing the character at the front.
The other difference between the two ID types is where they are found at the API. Persistent IDs are at the web service layer, in the KnowledgeVaultService. Persistable IDs are in the VDF layer, in Connection.PersistableIdManager.
Sure, it would be nice if we could consolidate these two types, but that would require one of them to change. And the entire point of these IDs is that they can’t ever change. So were stuck with these two. If you use one to store data, just make sure you remember which type of ID you used.
To summarize:
| Persistent IDs | Persistable IDs | |
| Example ID | RkxEUjpMQVRFU1Q6Njc5NTg | mRkxEUjpMQVRFU1Q6Njc5NTg |
| Found in... | web service layer: KnowledgeVaultService |
VDF: Connection.PersistableIdManager |
| Is the ID version specific? | No way to tell. | Check the first character. m = not version specific v = version specific |
| Where do these IDs show up? | API | API Thin client URLs |
Usually I don’t post any information about my Vault apps until it’s completed. But this time I want to try something different. I’m going to post information on the app as it is being developed. That way people can provide early feedback and suggestions. If this format works out well I may use it for other projects in the future.
In this edition, I’ll just talk about the concept for the app. I haven’t started coding yet.
Coming Up with the App
I get my app ideas from many places. Sometimes it's from my own use of the product. Sometimes people make suggestions to me. Sometimes I just want to show off a cool API feature. Usually it comes from hearing about people’s usage of the product.
For example, AU. It’s hard to go to that event and not come away with a bunch of ideas on what can be improved. In addition to in-person events, there are plenty of online sources to go for ideas. There are discussion groups, beta forums, and the Idea Exchange.
This time around, I’m going to write an app to help manage the various log files that Vault spits out. Like most of my apps, this one has been in my head for a while. It’s a theme that keeps coming up when talking to customers, and I don’t think there are any Vault-specific solutions out there yet. In fact, the problem got worse in Vault 2014 with the AVFS, which has it’s own separate log file. This kind of thing that is perfect grounds for a custom app.
My Vision For App
I’m not big into problem statements, at least not when it comes to apps. I usuall just have an idea in my head of how it should work, and I try to make that vision a reality. Here is my vision for this app...
App Framework
“Don’t re-invent the wheel,” is something engineers like to say. Personally, I think it’s fun to re-invent the wheel. But in this case, I’m going to follow the popular advice. Splunk is a tool that I’ve seen in action and have been very impressed with. And Splunk has an API. So I’m sold at this point. At least sold enough to investigate further.
That’s all the blogging I'm goint to do for now. In my next Developer Diary, I’ll be installing Splunk, hooking it to Vault and seeing what I can do with their API. So keep a look out for that article. It will be the most fun you can have without a catapult and a truckload of fruit pies.
Subscribe
My Take on Vault Data Standard
Earlier this week I listened in on a webinar on Vault Data Standard and it got me to thinking. Coming from a Vault API background, I have a different view of Data Standard then what is usually presented. So I’d like to provide my thoughts since this is a blog and all....
APIs on top of APIs
Data Standard provides a lot of features you can get through the API. You can create custom commands, custom UI and make calls to the Vault server. What’s interesting is that Data Standard itself is a Vault Explorer plug-in. So the Data Standard API is basically an API on top of the Vault API.
It’s basically like the movie Inception, but with APIs instead of dreams.
Many pieces of the Data Standard just pass through to the Vault API. Adding custom commands is an example of this. The settings in the DS .mnu files are exactly the same as the properties on CommandItem from the Vault API. In this aspect, knowledge of the Vault API transfers directly over to Data Standard.
Another example is the $vault object that shows up in the .ps1 files. The purpose of the object is for making web service calls to the Vault server. $vault is a WebServiceManager object being passed in directly from the Vault SDK DLLs. If you want to do anything with $vault, you need the Vault API documentation on-hand. Again, if you are already familiar with the Vault API, than this is no problem.
What I’m more curious about is the people without a Vault API background. How well are they able to utilize Data Standard? Does DS ease them into the world of Vault programming, or does the Vault API hit them like an impassable brick wall?
Beyond the API
The stuff that interests me most is the stuff that can’t be done through the Vault API.
First and foremost, the CAD plug-ins are awesome. Data Standard is not just a plug-in to Vault; it plugs into AutoCAD and Inventor as well. That way you can easily create an Inventor dialog that is Vault-aware, for example. Going through the traditional APIs would be a daunting task. You would need to be an expert in both APIs and would have to figure out how to hook the two together. DS solves all that stuff for you in a way that makes it look easy.
Another aspect of DS are the template features. New CAD files can be copied from a template instead of staring from a blank file. DS uses Vault functionality to centralize the storage of the templates. This is less of an example of a generic API and more of an example of a focused solution. Data Standard is really a dual product: It’s an API, and it’s an end-user utility.
The two aspects are not at odds with each other, but I’m not sure if they blend well together either. It feels to me more like a Swiss Army knife. It’s a bunch of seemingly unrelated stuff packaged together. Maybe that’s why it’s a hard product to describe it to people.
Migration
One thing like like about compiled programming languages, such as C#, is that you get compile errors when something goes wrong. That way if an API changes, you know right away what broke. PowerShell, however, is a scripting language. So it’s harder to find the breakages. Usually, they show up at runtime and only if specific pieces of code are run.
If you have a lot of PowerShell code in your Data Standard implementation, you may find it hard to maintain. Everything will seem fine at first after an upgrade, but things start failing as people start using the custom functions. Even if you think you have everything updated, a CAD user somewhere may hit lesser-used code that calls an obsolete Vault API function.
Yes, Vault provides compatibility for older versions of the web service API, but in order to use them, you need to have the older SDK objects. If Data Standard is passing in the 2015 version of the WebServiceManager, then you can’t make use of the 2014 server APIs.
There are lots of ways to solve, or minimize these issues when they do come up. For now, Data Standard is new, so nobody has run into migration issues... yet.
Posted at 05:39 PM in Commentary, Vault | Permalink | Comments (8) | TrackBack (0)
Tweet This! |