Migrating documents to SharePoint the easy way

OK, easy is relative, but it if you don't mind making a couple XML configuration files, it's as easy as easy gets with Folders2SP PowerShell script.

Folders2SP Codeplex Project

Before writing the script I evaluated a number of products, but I found non offered the flexibility I wanted and at ~$2500 I reasoned I could do better; hence Folders2SP. Sure it doesn't have a UI and it doesn't support Permission migration... but it's free, 42kb, doesn't soil your production servers, no custom web services (for item level permissions) and you can extend it to your hearts content... so you can add a permissions method.

Description

---

Folder2SP.ps1 is a PowerShell script which helps you migrate documents from SharePoint and folder structures to SPS2010 while preserving versions, metadata and converting folder names to managed metadata terms.
Use case: You have documents in a MOSS library and in a network share folder structure which need to be migrated to a SPS 2010 library. You want to use the words in the folders as taxonomy terms in your SPS2010 library and perform regular expressions on the file name/path.

Case setup

---

In this example we're going to get some documents from MOSS document library "Documents", and along with the sample documents included in this package upload them to a "Codeplex" document library in a SPS2010 site.

1. Create a working directory either on the SPS2010 server or a network share Eg: C:\Folders2SP.
2. Extract the contents of this project to your working directory. You should have Folders2SP.ps1, run.ps1, codeplex.xml and a "files" directory with some more directories and text files.
3. Create a destination document library in you SPS2010 site eg: Codeplex.
4. Add the required fields to the document library eg: Year (choice with 2008-2011), Enterprise Metadata field (site settings > Enterprise Metadata and Keywords Settings), LibMeta1 (Managed Metadata single), LibMeta2(Metadata multi), LibMeta3(Metadata multi) and connect them to an appropriate termset or term.
5. Modify the example codeplex.xml. Use your own MOSS library (works with a SPS2010 document library too).
   • Lookup site: the site you want to get the documents from. The script uses web services. To interrogate the source site I recommend you try my FireWS script , cos it rocks.
     library: the library to get the files from.
     view: if you want specific files, enter the view Guid here.
     rootfolder: is the folder to look in... default is the name of the library.
   • Target site: is the only attribute you need to change for this demo.
   • The xml file is commented so you know the options.

Process overview

---

1. Run PowerShell as Administrator (make sure your execution policy is set to remote) from your SPS2010 server, run.ps1 includes Folder2SP.ps1 and calls the function Folder2SP passing Codeplex.xml as the configuration file.
2. Folder2SP loads the config file, assigns a bunch of variables and creates a log file.
3. Goes to http://spsdev/documents and starts traversing the library... if you get access denied, it's your fault, try running the scripts from C:\ or use a local MOSS library.
4. Downloads all files it finds to the "Folders" path
5. Loops through all the files in Folders path and (if replicatefolders=true) recreates the folder hierarchy on the SPS document library.
   • Connects the Lookup site
   • Gets Metadata
   • Downloads versions to the working directory
   • Uploads the file to the Target document library
   • Loops through each field and check if it's configured in the xml config file. This is an inefficient way of doing it, but the script started out as just a way to copy all field data from the Lookup to the Target.
   • If it's a taxonomy field, it looks up the value, if it can't find the value it creates it. There are a couple options here, so read the comments in the example xml config file.
   • It then loops through the "Field" "Value"s and performs logic associated with Value type. I've only created 3 Value types "FileDate" uses properties from the file, "RegEx" takes an expression and matches on the full file name, and "Lookup" uses a value from the Web Service call.
   • If there's a user field it will 1st attempt to find the user, if it can't find them it will attempt to add them, if that fails it uses the account who's running the script.
   • Sets the new item value.
6. At the end you should have a migration report, and all the files from the Lookup site and your Folder path in the SPS2010 library with the metadata.

To do

---

1. View Fields ($wsFields) for the web service call should be moved to the XML config file.
2. Permissions could be also migrated if a item level permissions web service were available on the Lookup server.
3. Improve error handling, and more consistent variable naming.
4. Improve xml config file
5. Add more field type handlers

SharePoint 2010 beta first impressions.

Maybe it's my scifi interest, but I think SPF 2010 & SPS 2010 roll off the tongue nicer than WSSv3 & MOSS 2007.

The email Microsoft sent me contained product keys for 2 versions:
1. SharePoint Server for Internet Sites 2010
2. SharePoint Server 2010

My setup is all x64:
1. VMWare Guess with 2.4GB RAM/ 80GB HDD
2. Win2008R2 SE
3. SQL 2008 + SP1
4. SharePoint Server for Internet Sites 2010

After installing Win2008 and SQL2008, I launched the SP installer.
The package extracts and you're presented with an SQL2005 type menu with Prepare/Install/Other Info. I skipped the reading and went directly to "Install SharePoint Server" and it failed because IIS 6 Compatibility Prerequisite was missing.
This time I clicked "Install Software Prerequisites" and it installed quite a few more things.

Back to "Install SharePoint Server" and after a while it failed again:


Google'd KB970315, downloaded and installed the Hotfix.
Away we go again.

A few new things...

• the setup asks for a pass phrase to secure the farm.
• there's now a Wizard option to guide you through the Services configuration.
  

I select all options bar "Lotus Notes Connector"... enter a new account, and after a couple minutes the first error:

Errors occured.

The service application proxy "User Profile Service Application" could not be provisioned because of the following error: Unrecognized attribute 'allowInsecureTransport'. Note that attribute names are case-sensitive. (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebClients\Profile\client.config line 56)

Corelation Id: 95684294-0888-46dd-8242-df5860c5188e

Central Administration, its all new and it has pretty icons like a Control Panel.

Let's add some user accounts and start using it... the Shared Service Provider doesn't seem to exist anymore and in its place is the much more logical Service Applications Management screen.


User Profile Service Application is my guess for adding users.

I guess this was the error being reported before... I google'd "95684294-0888-46dd-8242-df5860c5188e", nothing, then tried "allowInsecureTransport" which lead me to MSDN Forum after reading through it I tried replacing the attribute with enableUnsecuredResponse, that didn't work... I noticed other security nodes didn't have this attribute so I commented them all out and it worked after an IISReset. More new things...

• Review problems and solutions
  
  
• JQuery type popups/functions everywhere
  
  
• Ribbon menus everywhere
  
  
• Theme Gallery; Now themes are CAB files with an .thmx extension
  
  
  
• Solution Gallery with upload, woohoo... I guess you'll have to be more vigilant with assigning Site Collection Admins :)... browse Office.com uses Silverlight
  
• SharePoint Designer Settings, now I don't have to blanket disable SPD access!
  

All in all SharePoint 2010 is more logically laid out and has a refreshing dynamic look.
I'll delve in to SP2010 more in the future.

Event ID: 6398 6482 7076 SharePoint memory is corrupt.

Event ID: 6398 6482 7076 Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I read a lot of posts saying this was caused by SharePoint and fixed with SP1, and/or an issues with .net 2.0 fixed with .net 2.1. Both are incorrect for my scenario; my servers were completely up to date and all of a sudden I started getting these errors.
I also read that restarting the Services Timer would resolve the issue. All of these are incorrect. The issue is caused by IIS (Adsiis.dll & Wadsiis.dll) and >2 AD users trying to access the same process. None of my dev machines have exhibited these errors because none use AD accounts.

Fix: http://support.microsoft.com/kb/946517/en-us

High availability server diagram

MOSS Pricing

Source: http://office.microsoft.com/en-us/sharepointserver/FX102176831033.aspx

All prices reflect estimations for purchases within the United States and are representative of a company purchasing a small number of licenses through Microsoft Volume Licensing.

Note All prices are in U.S. dollars unless otherwise noted. For a customized price estimate, please visit the Microsoft Product Licensing Advisor.

Servers	Estimated Price
Office SharePoint Server 2007	$4424
Office SharePoint Server 2007 for Search Standard	$8213
Office SharePoint Server 2007 for Search Enterprise	$57,670
Office Forms Server 2007	$4424
Client Access Licenses	Estimated Price
Office SharePoint Server 2007 Standard CAL	$94
Office SharePoint Server 2007 Enterprise CAL1	$75
Office Forms Server 2007 CAL	$54
Office SharePoint Designer 2007	$187
Internet Facing Sites	Estimated Price
Office SharePoint Server 2007 for Internet sites	$40,943
Office Forms Server 2007 for Internet sites	$22,118

¹To be licensed for the Enterprise Edition functionality of Office SharePoint Server 2007, both the Standard and Enterprise client access licenses are required.

System Event Log: The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID

System Event log:

The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID
{61738644-F196-11D0-9953-00C04FD919C1}
 to the user <machine>\SPSharedServicesPool SID (S-1-5-21-974288755-3100551457-677853593-1023).  This security permission can be modified using the Component Services administrative tool.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

 

Cause:

This is caused by SPSharedServicesPool not having rights to start the application

 

Fix:

Give WSS_WPG or the specific user the necessary rights.

Copy the GUID following the CLSID above, and Start-->Run-->regedit

With the registry editor open, ensure that your cursor is on the computer at the beginning of the tree (make sure you are not in the middle of some previous edit session in the registry editor).

Edit-->Find and paste in the GUID. It'll stop at the application entry - and you will want to note the application name on the right side pane. In this example, it was the IIS WAMREG admin service that popped up.

Now, open Component Services (typically, from the server - Start-->Administrative Tools-->Component Services), expand Component Services, Computers, My Computer, DCOM Config. Scroll down and find the application (IIS WAMREG in this case). Right-Click-->Properties and select the Security tab. You'll have some options here - the first block Launch and Activation Permissions - ensure that the Customize radio button is selected, and click Edit. Now, add your service account - giving it launch and activate - and in some requirements - remote launch / activate permission.

Restart IIS and continue on.

Application Event Log: There is no administration site associated with the Shared Services Provider SharedServices1

Cause:

This is caused by there not being a web application specifically for the SSP.

 

Fix:

Click on Shared Services Administration > then edit SharedServices1 and click on the New Web App link.