As a small but significant follow up to my UPA session at the SharePoint Conference in Anaheim last month, is that yesterday we pushed out an update to the Database types and descriptions article on TechNet to finally detail that synchronous mirroring of the Social database IS 100% supported.
As already detailed in this document, the Profile database already supported synchronous mirroring. That leaves the Sync database, which if you were paying attention in the session we really don’t care about in terms of “HA” or “DR”.
Not perfect by a long shot but you can now officially do what many customers have already done successfully in production for very large deployments, and implement a mirroring solution for HA of the UPA. A very small change, but today (well yesterday) is a very good day indeed.
s.
As promised during my sessions at the SharePoint Conference in Anaheim last week, here are the Windows PowerShell scripts demonstrated.
Please note that these scripts are direct copies of those on my Virtual Machines. You *will* need to tweak them for use on your environments, and remember they are authored for the purposes of demonstration! If you wish to take pieces of them to use in a real deployment, they will need some work. The scripts are provided as is, without any warranties! You know the score.
SPC407: Enterprise Deployment Considerations for the User Profile Service Application.
These scripts create the UPA avoiding the Default Schema Issue even in UAC environments, and provision the UPS service instance. I will be posting a follow up to detail these in more depth in the next week or so.
UPADemos.zip
SPC370: Multi Tenancy with SharePoint 2010.
These scripts setup a multi-tenant environment from scratch. Before running them, you need a farm created, and the State Service and Usage and Health Data Collection Service Applications provisioned. The seventh part of my Multi Tenancy article series will be published soon.
MTDemos.zip
s.
SharePoint Conference 2011 is in full effect here in sunny Anaheim, CA. Following this morning’s keynote it is worth pointing you in the direction of a couple absolute must haves which were released today as part of the overall festivities.
First up is the Scale Test Report for Very Large Scale Document Repositories white paper which describes the design and implementation of a 120 million (count em) item, 30Tb farm – the same farm that was used in the keynote HA demo. The number of times I’ve had to discuss “scalability limits” of SharePoint with customers is frightening, and this is a first class resource. You must check this out. Also props to Paul Andrew for leading up this effort.
Next, and you are going to love this, is the SharePoint Network Topology Diagram Add-In, which is cooler than a cool thing, and will provide you with a fancy Visio Services representation of your SharePoint farm, think Services on Server, but with groovy 3D server icons. You know you want some! Just don’t be labelling anything a “WFE” or I'll be over quick sharp to correct that! :)
s.
This morning at the sold out SharePoint Conference 2011 we officially announced the new premier certification for SharePoint practitioners, the Microsoft Certified Architect (MCA) for SharePoint Server 2010.
During his keynote address, Jeff Teper, Corporate Vice President of the SharePoint Product Group discussed the key role the SharePoint eco system has and will continue to play in the run away success of Microsoft’s platform without peers in the marketplace. The MCA program helps the highest-achieving IT architecture professionals distinguish their expertise with Microsoft server technologies, including SharePoint solutions for enterprise customers. MCA will recognise the best of the best in the architecture field and engaging an MCA will help guarantee success with your SharePoint initiatives, on premise, in the cloud or a combination of both.
This new certification is intended to recognize and validate the expertise of the highest-achieving SharePoint architects. The MCA Program administers a unique, rigorous review board and case exam process to help enable the most experienced IT professionals in IT architecture to distinguish their expertise in architecting complex solutions using Microsoft server technologies. Designed specifically for seasoned, practicing SharePoint architects, this new MCA certification validates both the technical and leadership skills of those who deliver SharePoint solutions for enterprise customers.
Dovetailing with today’s announcement, Microsoft have also introduced the revised and restructured review board process, which applies across all of the MCA certifications. The goal of this redesign was to reinforce the distinction between the value delivered by technology-specific architect certifications and technology-agnostic certifications.
It is a great honour to be along with my good friend for many years, Kimmo Forss one of the “inaugural” SharePoint MCAs and we look forward to many more over the coming months.
I’ve already received a bunch of questions on the new certification, and will be posting a summary Q&A after the conference later this week. I will however mention here and now that yes, the Microsoft Certified Master (MCM) is a pre-requisite for MCA. MCM is our top tier *technical* certification for SharePoint. In our view you cannot be a real architect if you don’t have the technical chops. You can find out more details over on the MCA site.
During the conference I will be loitering around the SharePoint Readiness booth in the Microsoft Product Pavilion after my breakout sessions to take any questions you may have on either MCM or MCA.
Have a great conference, and don’t forget Every Vote Counts – Please help decide how to divide our $50,000 donation between NetHope’s member humanitarian organizations by voting using the SharePoint Conference web site.
s.
After Tech Ed New Zealand, I’ll be hanging back in Auckland with my good buddy Steve Smith to deliver the Combined Knowledge SharePoint 2010 Advanced Infrastructure Administrator course.
This is by no means your regular SharePoint admin training. According to the blurb, “This is a once in a lifetime opportunity to learn from two of the most experienced SharePoint experts in the World today and take your SharePoint knowledge to the next level!”
It’s always great fun to work with Steve, and I’m looking forward to delivering this material, which steps away from the usual SharePoint featurisms and delves into the real deal of the surrounding infrastructure you absolutely must have down for a successful SharePoint deployment in the real world.
You can find out more details over here, where you can also register, but hurry places are running out fast! I look forward to seeing you in Auckland!
.
I finally got around to getting myself a new network tune box. Those of you who know me will be aware I am somewhat of a muso and I have a distinctly uncompromising stance when it comes to “better sound”. For a while I was using a Squeezebox Duet, which is OK but leaves a lot to be desired. So the hunt for a proper network tune box commenced about this time last year. I’m not interested in a all in one media player, I want a dedicated device for tunes and that doesn’t need a television to control. Seriously, who listens properly to tunes with the television on? that’s just not right.
My tunes are all in FLAC stored on a Windows Home Server, which is also running the Asset UPnP Server from Illustrate. Asset UPnP is a dedicated music server and beats the usual suspects hands down in every key regard. Unfortunately WHS 2011 still doesn’t support streaming FLAC, but even if it did, I’d probably still use Asset. So I needed a box that could play that stuff, and met my other requirements.
Why not stick with the Squeezebox? Well it’s a nifty little device for sure but it has some key limitations. Unfortunately it only supports up to 48kHz and I am increasingly using higher bitrate source material. The new Touch can do better, but that’s no good - I wasn’t going to wait for them to refresh the Duet. Then there’s the DAC – it’s pretty lame when you hook it up to a system where you can hear the difference. So with the Duet I had the SPDIF hooked up to my amp to use it’s DAC, which is very good indeed, but of course that ties it to the amp. Then there’s the software, it needs it’s own server. It’s kinda OK, but when you have a large library it’s slooooow. Then there’s the controller software for it’s remote and iPad, PC etc. That’s actually kinda neat and one of the better ones, but I don’t want to compromise on sound just for this. So in the bin the Squeezebox goes. I coulda stepped up to the Transporter, but at over £2000 it’s just not worth it.
I lusted over the Linn Akurate and Klimax for a while and briefly considered the Sneaky Music, but the price is hard to justify, and frankly the controller software is very lame. Linn make great gear, but it just wasn’t up to scratch within a reasonable budget and a lot of hassle.
So in the end I plumped for the very affordable and excellently spec’d Cambridge Audio Sonata NP30. Here it is sat comfortably in the rack:
Looks wise it’s kinda nice, and as you’d expect follows the Cambridge Audio design, this one intended to be matched with their Sonata line. I have it hooked up to the amp (a Sony TRVA555ES, a kick ass piece of kit the like of which aren’t made any more) using analog outs – the NP30 doesn’t have “balanced” connectors which I’m not into anyway. I also have the SPDIF hooked up for comparisons when using crappy formats and also for the small number of surround sound recordings I have. It is also hooked up to the LAN and it has wireless as well.
It really doesn’t do much. It just plays tunes. But it plays them very well indeed. It can also stream internet services (radio and such) which I’m not that interested in, but useful for things like SomaFM. Those of course are all in MP3. For the real deal it will work with any UPnP server. It’s plays everything I can throw at it. The DAC is a nice one, the Wolfson WM8728 which can go up to 24-bit/96kHz. It sounds pretty warm, it’s very nice output especially at reasonable levels. Time will tell if it warms up more and beats the amp (unlikely), but there may come a time when the amp is replaced.
Overall sound quality is excellent, very good dynamic range. Sweet and rock solid, it’s what you could call an “in the pocket” player.
It’s basically silent, which is just the ticket. Streaming over the WLAN works great, but I’ll stick with the LAN I think! It has a mode where it will try LAN first then WLAN etc. Load times are extremely quick. There’s no lag waiting for anything thus far with a very large library of tunes! It of course grabs a bunch of data, pull the LAN out and it will continue to play for a few minutes before failing.
The NP30 comes with the standard Sonata remote, which is OK, but you are not really going to be using this to control the device as browsing the collection from the display is as awkward as it was on the very first network music players years ago. The front panel has a jog wheel which is much better. The display can be dimmed as well, but unfortunately it cannot be turned off.
Cambridge Audio ship a thing called uuVol, an iPhone/iPad remote. It’s not fantastic and it’s obviously early doors in terms of it’s development, but it’s perfectly adequate. Now an interesting bugette here is that if the NP30 is on the LAN, uuVol will struggle to connect. This doesn’t happen if both the NP30 and iPad are on the WLAN. Cambridge Audio are aware of the problem and working on a fix.
The only other gripe I have is it can’t do gapless playback. For concerts that is just crap. Cambridge Audio say they are working on it, but it’s “hard to do”. That’s kinda annoying as a £100 Squeezebox can do that no problem. But of course the squeezebox jitters like crazy and has a fetish for rebuffering so it’s not that easy!
So overall I'm very happy with it so far. It is a simple device, but that’s just how real audio gear should be. The three things I want fixed are the remote app LAN issue, gapless playback and the ability to turn off the display. But it’s a very nice tune box even with those issues, and so if you are into these sorts of things, I’d encourage you to take a listen to one.
It’s that time again folks, Conference craziness season is back with a vengeance after the summer. When is it *not* conference season I hear you chuckle! Anyways, I have the privilege of speaking at a number of events between now and the end of the year. It’s always a great deal of fun to meet folks at conferences and hear about their SharePoint experiences. I look forward to seeing you one of the following events.
TechReady 13
25-29 July - Seattle, WA
http://www.mytechready.com/
This event is for Microsoft employees only. I’ll be in Redmond for some other work and will be presenting a developer focused session.
SharePoint 2010 Site Provisioning Smack Down: Site Definitions vs. Web Templates, the What, When, How and Why.
Debate still rages around the “right way” to approach provisioning of SharePoint sites especially in relation to large scale deployments. This session will cover the various approaches to site provisioning available, looking at their pros and cons and associated considerations. Guidance around what approach to use when, and why will also be provided.
SharePoint User Group UK (SUGUK)
4th Annual Golf Day
11th August - Ullesthorpe, Leicestershire, UK
http://suguk.org/forums/thread/26940.aspx
The golf day is great fun, a round of golf and some user group sessions. My partner in crime Steve Smith will also be presenting.
The rational guide to Kerberos with SharePoint 2010
In this session Spence will drill into the cloudy often ignored world that is Kerberos and will show that Kerberos is not something to be scared off, but something to embrace providing you approach it right. Full scenario walkthroughs of Farm traffic, End User sign in and Service Application delegation to external services will be demonstrated live!
Tech Ed New Zealand
24-26 August - Auckland, New Zealand
http://newzealand.msteched.com/
I’ve always wanted to visit New Zealand but it’s been about four years before I’ve been able to make it happen. I’ll also be staying on for a week after the event to deliver Combined Knowledge’s Advanced Admin Boot Camp.
Real World Service Application federation with SharePoint 2010
SharePoint 2010 provides architects with a compelling new model for service publishing and federation, opening up exciting new approaches to farm design. This session will cover how Service Application Federation plays out in the real world, based upon early enterprise adopters. Learn how to approach the design of a enterprise services farms, provide true scalability and discover the constraints for each service application which can be published, including global deployment considerations. Related aspects such as Security, High Availability and performance will also be covered. This session will be split 70/30 between lecture and demonstrations.
Rational Guide to SharePoint 2010 User Profile Synchronization
The incredibly popular session with new content updated for SP1 and the latest CUs. Get the real deal on configuring User Profile Synchronization in SharePoint 2010 in this demo and best practices heavy session. This session will cover the architecture of the new User Profile Synchronization capability in SharePoint Server 2010 and provide a walkthrough of the configuration requirements and setup eccentricities. This session will be split 70/30 between demonstrations and lecture.
SharePoint Conference
3-6 October - Anaheim, CA
http://www.mssharepointconference.com/
The big daddy. It’s gonna be an awesome show. Be there!
Enterprise Deployment Considerations for the User Profile Service Application
Deploying the User Profile Service application presents unique design considerations for architects. Learn best practices from real enterprise deployments and understand the key architectural considerations in terms of high availability, scalability and geographic deployments. Also covered will be general UPA related best practices in terms of synchronization, policy and privacy and leveraging social features inside the enterprise.
Best Practices for Multi Tenancy
SharePoint 2010 delivers compelling new infrastructure features for those wishing to host multiple customers on a shared platform whilst retaining confidentiality, integrity and availability. This session will cover how multi-tenancy can benefit all sizes of deployment from a basic farm to the largest such as SharePoint Online. Learn how to approach the design of a multi-tenant deployment and to configure and operate multi-tenant infrastructure, create Member Sites, Subscriptions, Feature Packs, and Service Application Partitions. Understand the key design choices and development required. This session will be split 50/50 between lecture and demonstrations.
Capacity Planning your SharePoint 2010 deployment
One of the biggest challenges for architects is how to approach capacity planning and management for a SharePoint 2010 deployment. In this session we will cover approaches to the problem space and devise the appropriate capacity management strategy for SharePoint 2010 implementations. Learn how to manage capacity throughout the deployment lifecycle and adopt best practices from field experiences.
European SharePoint Conference
17-20 October - Berlin, Germany
http://sharepointeurope.com/
I’ll be co presenting one of the keynotes with Mirjam van Olst, and generally loitering around the rest of the conference.
Keynote 3: Successful Deployment: Lessons Learned From the Field
with Mirjam van Olst
Take a whirlwind tour of lessons from the field since the release of SharePoint Server 2010 to understand the key factors of a successful roll out in the enterprise across planning, architecture, implementation, deployment and operations. Based upon some of the most common pitfalls and worst practices of early adopters and the key challenges they have faced, the keynote will detail the lessons learnt alongside best practices to help ensure a successful deployment. Ideal for all disciplines, including Information Workers, Business Decision Makers, Developers and IT Professionals.
SharePoint & Exchange Forum 2011
14-15 November 2011 - Stockholm, Sweden
http://www.seforum.se/
My first visit to Sweden which I am very much looking forward to.
Sessions TBD but likely to include Kerberos, Sandbox for O365, UPA, Multi Tenancy. I may also co present with my buddy Steve Smith once again.
That’s my lot for the rest of the year! I look forward to seeing you at one of the above events.
.
One of the most common conversations I have with customers, partners and random SharePoint consultants is around the creation of SharePoint Server 2010 User Profile Synchronization Connections. These guys are the key link, or connection string if you will between the User Profile Service Application (UPA) and the connected directory services. A very common complaint is the inability to automate their creation using Windows PowerShell. The good news is that Service Pack 1 (SP1) introduces a couple of new cmdlets which help in this regard. This post looks at these cmdlets and also details why they might not be all you had hoped for…
[UPDATE 25/08/2011]
Please note that these cmdlets are only intended for use within SharePoint Online environments by SharePoint Online engineers. Their use in on premises deployments is NOT supported. You have been warned!
What is a Synchronization Connection?
A Sync Connection is the link between the User Profile Synchronization service instance (UPS) and the connected directory services. When they are created a Forefront Identity Manager (FIM) Management Agent is created behind the scenes and it’s these bad boys which do the work of getting data to and from the connected sources to SharePoint. Sync Connections can be created easily using the UPA’s Configure Synchronization Connections page. However this page suffers from numerous flaws, especially when working with enterprise directory services implementations.
Why do people want to automate them?
There are a number of reasons, but there are usually one of two key drivers. Firstly a lot of people promise customers a 100% automated SharePoint deployment. This of course is a nirvana that cannot be achieved with SharePoint 2010. Of course automation is a good thing, but it’s just a silly promise that cannot be met. It’s a bad driver on it’s own. Any SharePoint practitioner with any credibility would never make such a promise.
Secondly, and more importantly one of the key limitations of the Configure Synchronization Connections page is that it can’t really handle the directory services design of many large enterprise AD implementations. There are some timeouts that can be configured, but often these will not suffice and it becomes impossible using this page to create the connections in the manner desired. The page doesn’t scale basically. This leaves us with but one other approach, to use the Synchronization Services Manager (miisclient.exe) after creating the connection in UPA management to change the selected containers. Unfortunately such an approach is NOT supported.
Thus being able to create the connections using Windows PowerShell would avoid this problem.
The new Windows PowerShell cmdlets.
Service Pack 1 therefore introduces two new cmdlets for working with Sync Connections, Add-SPProfileSyncConnection and Remove-SPProfileSyncConnection. Luckily for us they decided not to spell out Synchronization in full :). These cmdlets do make it easier to manage Sync Connections, however there are some considerable limitations. Furthermore note that there is no Get-SPProfileSyncConnection or Update-SPProfileSyncConnection. Also rather annoyingly, like all of the new cmdlets in SP1 whilst of course get-help is implemented there are no examples.
Without further ado here is how to use Add-SPProfileSyncConnection:
$upa = Get-SPServiceApplication 972fe314-7eb9-47b7-a265-20ffbc94680b
$syncAccountPassword = convertto-securestring "Password1" -asplaintext -force
Add-SPProfileSyncConnection -ProfileServiceApplication $upa `
-ConnectionForestName "contoso.com" `
-ConnectionDomain "Contoso" `
-ConnectionUserName "spups" `
-ConnectionPassword $syncAccountPassword `
-ConnectionSynchronizationOU "OU=SharePoint Users,DC=contoso,DC=com"
Looks pretty straightforward right? Almost! But not quite!
There are some additional optional parameters. For reference the full params are:
|
Parameter name
|
Required
|
Description
|
|
ProfileServiceApplication
|
True
|
Service Application pipebind to the User Profile Service Application
|
|
ConnectionForestName
|
True
|
The FQDN of the forest you are connecting to
|
|
ConnectionDomain
|
True
|
The NETBIOS name of the domain you are connecting to
|
|
ConnectionUserName
|
True
|
Username used for the synchronization connection
|
|
ConnectionPassword
|
True
|
Secure string format of the password of the account used for directory connection
|
|
ConnectionSynchronizationOU
|
True
|
The top level OU that you would like to synchronize, this must be a DN and you can only include one container per command
|
|
ConnectionPort
|
False
|
The port used to connect to the directory service. Default port is 389.
|
|
ConnectionUseSSL
|
False
|
Boolean value if the connection to the directory service must be over SSL
|
|
ConnectionNamingContext
|
False
|
Naming Context of the Directory Information Tree to connect to
|
|
ConnectionServerName
|
False
|
Name of the Domain Controller to connect to
|
|
ConnectionClaimProviderTypeValue
|
False
|
|
|
ConnectionClaimProviderIdValue
|
False
|
|
|
ConnectionClaimIDMapAttribute
|
False
|
|
Now that’s all very nice. However there are some key limitations with the cmdlet which will impact how much you can use it in a real deployment.
- The account running the PowerShell host must be added as an administrator for the UPA
This isn’t really a limitation but it upsets purists. Instead of using the Proxy, we need the UPA itself, and this means we must have at least the Manage User Profiles administration rights on the UPA. If you don’t have this and attempt to run the cmdlet, you will receive the generic error from FIM, “MOSS MA Not Found”.
- There is no DisplayName parameter
The name of the connection will be the NETBIOS name of the domain, i.e. the ConnectionDomain parameter. This will also be used for the Description. This also means that you can add only one connection per domain. Now this is strong recommended practice, but it prevents some scenarios from being possible with this cmdlet and is a major oversight.
- There's no option to create more than one connection per forest
Since you have to specify the ConnectionDomain parameter. Again more than one connection per forest is strongly discouraged but there are numerous scenarios where this is needed. Again this cmdlet is no use to you if you are in that boat.
- If you specify the same ConnectionDomain parameter, the system will overwrite the ConnectionSynchronizationOU, ConnectionUserName and ConnectionPassword parameters.
- If the connection cannot be created due to a FIM error, the command completes
No errors are reported at all! We still need to use miisclient.exe to verify things have worked!
- Remove-SPProfileSyncConnection does not delete sync connections!
The Remove-SPProfileSyncConnection cmdlet only removes the ConnectionSynchronizationOU specified, will not delete the connection itself. There is no way to use these cmdlets to delete sync connections.
- These cmdlets only work for Active Directory Sync Connections
A couple of other things to understand about the cmdlets:
When creating the connection you may receive permission related errors, if so you need to use user@domain as the format for the ConnectionUserName parameter.
If you have an existing Sync Connection for the domain and omit the ConnectionSynchronizationOU parameter, it will update the credentials for the connection.
Conclusion
We have a couple of new cmdlets which do help the automation of sync connections. However they have significant limitations which restrict their value, and they could have been so much better. Ensure you are familiar with the limitations before diving in and attempting scripted UPA nirvana! :) We still have no cmdlets for working with connection filters.
[UPDATE 25/08/2011]
Please note that these cmdlets are only intended for use within SharePoint Online environments by SharePoint Online engineers. Their use in on premises deployments is NOT supported. You have been warned!
.
One of the most common complaints about the User Profile Synchronization service in SharePoint Server 2010 is the time it takes to perform synchronization runs or “sync” for short. This is due to a number of factors not least of which is that by leveraging Forefront Identity Manager (FIM) SharePoint now effectively includes a metadirectory. This is a good thing. However if you are just doing import then there is a huge increase in the time it takes over previous versions which were simply performing an ADSI query and inserting the results into a database.
It’s very important to note that there are many other factors which influence the time it takes to perform a sync, many of which are under your control as the farm administrator. I may do another post about these in the future.
Microsoft have heard this common complaint loud and clear and have been working hard to reduce the time taken to sync since RTM.
The December 2010 Cumulative Updates (CU) first introduced a significantly reduced sync time, due to the re-implementation of the SharePoint Management Agent that is created when you provision the UPS service instance. Depending upon environmental factors the December 2010 CU would decrease sync time by around 30-38%.
The really good news is that the June 2011 CU (*not* Service Pack 1) introduces some more changes that also help significantly reduce the sync time. This post takes a quick look at these.
A new build of Forefront Identity Manager (FIM)
The release of SharePoint Server 2010 includes a bundled version of FIM. However it wasn’t the RTM of FIM, but rather a stable build chosen to meet the release schedule of SharePoint. This by no means is the reason for all of the problems with UPS, but it is a factor.
The August 2010 CU included a minor build revision of the FIM bits, but since then the version has remained the same.
With the June 2011 CU, another build revision of the FIM bits has been included. This is one of the reasons why the UPS service instance must be re-provisioned after the installation of the June 2011 CU. This new build improves numerous aspects of the capability, including the sync time.
It’s important to note however that the bundled FIM is still not FIM RTM. Don’t get bogged down by this thou. Remember that UPS in SharePoint 2010 is not supposed to be full FIM, it’s FIM “Light”. The new build of FIM in the June 2011 CU is 4.0.2450.34. You can also see that the ‘Release Candidate 1’ text box on the about screen has been removed!
Here for your reference are the various build numbers:
| Product | Version | FIM Build |
| FIM 2010 | RC1 | 4.0.2560.0 |
| FIM 2010 | RC1 Update 1 | 4.0.2570.0 |
| FIM 2010 | RC1 Update 2 | 4.0.2574.0 |
| FIM 2010 | RTM | 4.0.2592.0 |
| SharePoint Server | RTM | 4.0.2450.5 |
| SharePoint Server | August 2010 CU + | 4.0.2450.11 |
| SharePoint Server | Service Pack 1 | 4.0.2450.11 |
| SharePoint Server | June 2011 CU | 4.0.2450.34 |
Changes to Synchronization Runs
Because FIM is a metadirectory, it works like one. :) One of the fundamental characteristics of a metadirectory is the requirement to perform delivery receipt and confirmation. This ensures the metadirectory is up to date and it also allows one to support a myriad of directory services, including those that may have intermittent network connectivity issues. All of this has been true since the very first true enterprise metadirectory (Zoomit Via in 1997).
This is the main driving force behind how directory synchronization works in SharePoint Server 2010. Even though the product can only do import or export it uses the synchronization engine provided by the metadirectory (FIM).
But if you are just doing an import from say Active Directory to SharePoint, that’s overkill. Take a look at the following screenshot of Synchronization Service Manager (miisclient.exe) which shows the sync runs for an incremental sync performed using SharePoint Server 2010 Service Pack 1, which is doing import only using the default property mappings:
Note that there are eight runs, and that the time taken is roughly six minutes. We of course here are only importing a couple of users, but it would still take six minutes if there were 50 adds. The number of users does influence sync time, but it is not a linear user/time relationship.
Now, take a look at the same thing this time with the June 2011 CU:
This time there are only five runs and the time is roughly 1.7 minutes. Nice! That’s another 33% time improvement (roughly) and you will experience the same improvements with more realistic numbers of users.
This improvement is due to the new version of FIM, and new versions of the AD, SharePoint and Metaverse management agents included in the June 2011 CU. I could bore you with the technical minutiae, but it’s not relevant. The thing you need to know is that sync is a lot quicker with the June 2011 CU!
Other Changes
With the June 2011 CU Profile synchronization now supports domain migration.
Synchronization Connections for SunOne (or later Oracle versions) LDAP are now possible regardless of the state of the nsslapd-return-exact-case base attribute. In previous builds creating the sync connection would fail.
The Profile Synchronization Status view now includes all stages and a better view of progress. (but miisclient.exe is still the way to go if you know what you are doing).
ULS reporting is vastly improved. Instead of just reporting a stage was started, we get the following detail:
This might seem pointless on face value, but it means that if there are errors during a run they are now bubbled up back to SharePoint, and can be recorded etc via ULS. It also avoids the need to use Synchronization Service Manager (miisclient.exe) to know there were problems. In the past even with errors, there would be nothing in the ULS.
Take this example:
The highlighted row is a problem with the export to AD. Of course to see the root cause of the problem we still need to dive into miisclient.exe. In this case, it’s the old chestnut – lack of permissions on the attribute in AD we are trying to write:
But the point is, in the past SharePoint would report success even if there were failures. Now it will report failures and that makes it much easier to manage the operational service of the farm with standard SharePoint tooling.
Synchronization Connections now run in parallel
If you have multiple Synchronization Connections their associated runs will now execute in parallel. Before the June 2011 CU, each run would run serially, thus increasing the overall time for sync to complete. Now, this is NOT a reason to have multiple connections. That remains as bad an idea as it always was, but there are some cases where it is the only way to achieve the end results desired. You should always try and avoid multiple connections, especially more than one for the same AD forest. However if you are using more than one, the sync will now be significantly quicker.
Check out the following, two Synchronization Connections:
And here are the sync runs, note that they run in parallel:
The IsSynchronizationRunning property
There is also now a property on the UPA (service application) that allows us to check if a profile sync is running before doing other operations such as adjusting timeouts or creating connections etc:
$upa = Get-SPServiceApplication 972fe314-7eb9-47b7-a265-20ffbc94680b
$upa.IsSynchronizationRunning
True
This is really important when developing custom code or Windows PowerShell automation solutions for UPA.
Conclusion
Some really great improvements to UPS are included in the June 2011 CU. Perhaps this is why Microsoft are so keen that you deploy! If you are dealing with a large synchronization requirement this package is definitely for you. But remember to test thoroughly before you deploy and watch out for the UPA gotchas with this update:
.
You may have noticed a few new Windows PowerShell cmdlets included with SharePoint Server 2010 Service Pack 1 (SP1), Get-SPProfileLeader, Add-SPProfileLeader & Remove-SPProfileLeader. These cmdlets are causing a little bit of confusion, so this short post explains them and the problem they are intended to address.
First up, they have absolutely nothing whatsoever to do with Organizational Profiles. The term “leader” here is somewhat misleading (no pun intended!).
What this is all about is User Profiles and People Search Relevance.
One of the key pieces of data that SharePoint Server Search uses to drive relevance in people search results is the user profile property Manager. This is sensible, social distance etc are great ways to deliver highly relevant search results. However, it all relies upon the quality of data. The old adage, “rubbish in, rubbish out”, was never more relevant.
The problem with the Manager property is that in most cases it will be coming from Active Directory via the User Profile Synchronization service. And this is where the problem lies. In most organizations, the Manager attribute in AD exhibits very poor data quality. It’s either out of date, or wrong. But more often than not it’s empty. If all your users have no manager, there is no way for people search to distinguish between company leaders, and everyone else.
The ideal implementation is that only the company leaders would have an empty (null) Manager attribute and therefore Manager user profile property, and everyone else would have correct and up to date managers. Telling customers that in order to improve people search relevancy there needs to be an exercise to fix up AD after the deployment of SharePoint is, as one of my good buddies says, a dog that won’t hunt.
And this is where these two new cmdlets come into play. In SP1 we can now specify the actual leaders of the company directly, regardless of the state of the AD attribute. This corrects the relevancy problem.
Add-SPProfileLeader
Unsurprisingly, this bad boy adds a leader. It checks to see if the account name specified exists (in the profile db). Then it checks the Manager profile property. If there is a manager, it will bail with an error. If there is no manager, it will write a row to a table someplace which identifies that user as a company leader.
$upaProxy = Get-SPServiceApplicationProxy 7643a5c2-a6ae-49eb-8ba0-de2f32a890ba
Add-SPProfileLeader -ProfileServiceApplicationProxy $upaProxy -Name "contoso\davism"
Add-SPProfileLeader -ProfileServiceApplicationProxy $upaProxy -Name "contoso\jonesq"
Get-SPProfileLeader -ProfileServiceApplicationProxy $upaProxy
Failed. User 'contoso\davism' has a manager.
User 'contoso\jonesq' added as a leader.
Once you’ve added a leader with Add-SPProfileLeader, you will need to perform a full crawl of your content sources for the changes to take effect.
Get-SPProfileLeader
Yup, you’ve guessed it. This guy returns the current leaders so you can verify it worked or see who they are.
Get-SPProfileLeader -ProfileServiceApplicationProxy $upaProxy
AccountName Valid ManagerAccountName ReportCount
----------- ----- ------------------ -----------
contoso\jonesq True 0
Remove-SPProfileLeader
Removes (or RIFs if you will :)) a leader.
Remove-SPProfileLeader -ProfileServiceApplicationProxy $upaProxy -Name "contoso\jonesq"
User 'contoso\jonesq' removed as a leader.
So there you go. A nice easy way to ensure that your CEO’s pages are high up the search results page in People Search.
.
One of the best new feature areas of SharePoint Server 2010 was the social computing capabilities delivered by the User Profile Service (UPA). Tags, Ratings, Activities as well as enhancements to the My Sites infrastructure allow enterprises to deliver rich “social” applications with the out of the box capabilities. Furthermore by using these features as building blocks a new class of composite social applications have become possible, enabling the enterprise to leverage social computing for both business benefit and end user happiness.
Of course, as with many aspects of SharePoint 2010, with great power comes the need for responsibility and appropriate planning. Some folks even call this stuff “governance”. Whilst the social computing features were exciting, carnage can quickly occur in your farm if you stick with all the default settings and controls.
With Service Pack 1 (SP1), Microsoft has refined the social computing features significantly. Whilst there are no whizz bang new things, a number of changes have been made which improve the overall operational service management of social data. This post takes a look at these changes.
1. Performance improvements to Social Data related processing
Not anything you can see (and certainly no Silverlight involved :)) but SP1 brings considerable performance improvements to social data related processing. Timer Jobs, and in particular, Activity Feed processing are measurably quicker and hog less resources than before SP1.
2. Activity Feed Timer Job is now enabled by default
Mainly due to privacy concerns (but also performance) in RTM, the Activity Feed Timer Job was disabled by default. This default led to many complaints such as “the activity feeds aren’t working” and “why isn’t such and such an activity showing up on my newsfeed”. Even though there is a Health Analyzer Rule Definition (named ‘Verify that the Activity Feed Timer Job is enabled’) many administrators simply overlooked the configuration of the timer job, thus missing activities for end users.
With SP1 the Activity Feed job is enabled by default. However this doesn’t mean that all hell will break loose! The Setup My Sites page of the UPA now includes the ability to enable or disable the newsfeed on My Sites:
This option is disabled (unchecked) by default. The My Newsfeed link remains in the top navigation of the My Site host, as does the display of the feed itself and the ATOM feed. (which will include changes to profile related activities (e.g. Manager change) but not social data related activities). Unfortunately this setting does not impact the Newsfeed settings page, which in reality is simply the last section of the Edit Profile page.
If you want to play around with this setting and toggle it on and off to see the behaviour, our old buddy IISRESET is needed after changing the setting (and don’t forget to run the timer jobs!).
OK, the ability for administrators to opt in or opt out of newsfeeds, but wouldn’t it be cool if you could do this based upon some value, such as group membership? Yup, but remember this is just a service pack. It is worth noting however that as this is a My Site setting, if you are in a partitioned UPA then this setting is configured on a per tenant basis.
3. Organization Browser fixes
The natty Silverlight Organization Browser is very sweet, especially when profiles are richly populated. However there were some user interface problems with this guy. If you used the browser back/forward buttons and then returned to the page which hosts the organization browser, the selected person was lost. Also if you attempted to host the organization browser in a different web app to that hosting the My Site host nasty authentication prompts would occur and sometimes it just plain didn’t work. Both of these issues have been fixed. Yay!
4. Customized Security Trimming
In RTM, there was no ability to change the behaviour of the Social Security Trimmer. Now we have three options.
1. Check all links for permissions
The same behaviour as RTM. Everything is checked, which of course incurs a penalty.
2. Check only specified links for permission
New. We can specify a list of URLs (one per line) for which we wish to perform permission checks.
We can also specify a list of URLs to display regardless of permissions. This is *really* nice, and allows us to specify when and when not to perform security trimming, in the example below we will do trimming on stuff under the corporate intranet, but not within the ‘socialcorp’ where all the fun stuff happens!
3. Show all links regardless of permission
New. Effectively turn off security trimming entirely.
5. Default Secondary My Site Owner
With RTM, the My Site Cleanup job would attempt to set the secondary site collection administrator when a user was deleted from the Profile DB. It would use the user’s manager. However if there is no manager, this operation would fail. Now we can specify a default secondary site collection administrator to be used by the My Site Cleanup job if the manager property is empty. Nice!
6. Manage Social Tags and Notes
A very small change that has a lot of value, this page now allows searching based on only one parameter, rather than at least two, as with RTM
7. The Move-SPSocialComments Windows PowerShell cmdlet
This is very handy indeed. It will take social comments (Noteboard entries) from one page and move them to another page. It does not move Tags or Ratings. This is very valuable for migration scenarios.
At the end of the day, this guy is updating the URL for each comment, and ensuring the new URL is in the table of URLs.
This is a Windows PowerShell wrapper for a new method, MergeSocialNotes of the SocialDataManager abstract class in Microsoft.Office.Server.SocialData.
Here’s how you use the cmdlet:
# Use Get-SPServiceApplicationProxy to find the UPA proxy identifier
$upaProxy = Get-SPServiceApplicationProxy 7643a5c2-a6ae-49eb-8ba0-de2f32a890ba
Move-SPSocialComments -ProfileServiceApplicationProxy $upaProxy '
-OldUrl "http://consps/Pages/mergetest.aspx" '
-NewUrl "http://consps/Pages/desttest.aspx"
Conclusion
There you have it. A brief round up of the interesting changes in social computing in SharePoint Server 2010 Service Pack 1. These improvements make it far easier to manage, and dare I say it, govern social data within the enterprise. Whilst the changes are small they will make a huge difference in terms of the operational service management cost of social implementations. It’s also a great example of how the SharePoint product group are listening to customers to deliver incremental improvements to this exciting feature capability.
As I’m sure you are all aware, a couple of days ago Microsoft released Service Pack 1 (SP1) for SharePoint 2010. On the same day the bi-monthly Cumulative Update (CU) – the June 2011 CU - was released. Service Pack 1 of course has been hotly anticipated both by the community and customers alike. Unfortunately these releases have caused mass confusion and much contradictory advice regarding the packages and their installation.
This post is simply an attempt to reduce the amount of questions I receive about this topic, or rather have something I can point people to when they ask. Now of course, I don’t work for the SharePoint product group, so none of this is “official”. If you want that then I recommend you head on over to the Office Sustained Engineering Team blog post, which is the best official resource right now. There are of course other Microsoft places where certain recommendations and so forth are made.
In this post I will answer the most common questions from the point of view of the field. I have the luxury of being able to call it as I see it, for real based on customer deployments, not how it “should” be. I’m sure there are other questions that I don’t answer here, but these again are the most common ones over the last couple of days.
[UPDATE 19/12/2011]
Partly due to this fiasco, the guidance from Microsoft has been fixed and made consistent. Since then this has proven to be the case. There have also been refinements made to the update packaging and application process. There are various updates throughout the rest of the post.
[/UPDATE]
Q: Got any advice for me?
A: First and foremost, step away from the keyboard!
Seriously, updates are a major lifecycle event in the farm. You should be properly planning for and testing any updates. This should be part of the operational service management approach for your SharePoint deployment. Period. If you are hacking around on a single box VM of course things are different, but the more you know, the more you know! If you deploy an update and it breaks something that’s bad, but if you didn’t test it first, that’s pretty much no one’s fault but your own.
Q: If I am running SharePoint Server 2010 (SPS), do I need to install both the SharePoint Foundation (SPF) and SharePoint Server updates?
A: There are separate downloads for SPF and SPS. This is true for all the CUs and the SP. TechNet and the SharePoint Team Blog recommend you install both the SPF package and then the SPS package.
For quite a long time after the RTM of SharePoint 2010 a big fuss was made of the new style of packaging. The main thing being that you only needed to install the SPS package, which contains the SPF bits. A good attempt to make what is known as “uber packages” which streamline the installation experience for customers.
A few months ago however the guidance was changed to say “the best practice is to install both”. This is not a copy and paste error from how it used to be with SharePoint 2007. It was changed for a reason.
In principle you can get away with just installing the SPS package. This is how it should work. You may have been doing this for previous CUs with no problems at all, many have including myself on my private VM environments.
HOWEVER, there are cases where this may not work properly. There have been numerous issues with this approach. And this is why the recommendation is to install both before running PSConfig. It’s a “safest” bet. I’m not a fan of the term best practice, but the recommendation is to do both to avoid potential issues. I agree with that recommendation based upon my experience patching customer farms.
If you are comfortable simply installing the SPS package, and you don’t have any problems then great. If you hit issues, don’t moan that the SharePoint PG or the likes of me didn’t warn you!
There is of course the argument that this is a complete waste of time. Well couple points here. Firstly, that’s the how it should be. Not the how it is. Secondly, this is only valid if you can be sure it works, 100% of the time. And thirdly, the installation of the bits is the least of your time concerns when patching a large deployment. Seriously, the time it takes is minimal, it’s the time it takes to run PSConfig you should be worried about, or the time it takes to run Content DB updates in a staggered fashion for real deployments. Saving a few minutes installing binaries really is not a consideration in the real world. You’ve likely got multiple packages anyway (Language Packs, OWA). So again, this is just small fry.
Again the SPS package includes the SPF bits, that is not the issue here, the issue is the application of those bits. Have I patched farms with the uber packages successfully? Yes. But I've also been in the situation where it didn’t work…
I know some very clever and knowledgeable people disagree here, but I’ve hit problems with customer farms where it simply doesn’t work. So until I am sure the so called “uber” packages work, every time, I am standing behind the recommendation from the product group.
Could this guidance change again in the future? Sure. No one would be happier than me to see “uber” packages become the reality.
[UPDATE 19/12/2011]
Now the guidance is to simply deploy the Server package. This works well now, much better than before and is what I do on all my deployments, customer and otherwise. There are additional considerations for language packs and OWA however.
[/UPDATE]
Q: Does the June 2011 CU include the SP?
A: No. It does not. If you install the June 2011 CU on RTM you will have different bits than if you install SP1 and then the June CU. Any statement you see that says “the June 2011 CU includes all updates in SP1” is just wrong.
SP1 is everything up to and including the April 2011 CU plus some other bits and bobs (the new features and Windows PowerShell cmdlets).
The June 2011 CU contains updates to some SP1 bits.
The packaging of updates for SharePoint is ingenious and very smart, but with that comes some complexity.
Q: Can I deploy the June 2011 CU and not the SP?
A: Yes, but you probably shouldn’t. SP1 is effectively the new baseline build in line with Microsoft’s service pack lifecycle. It all depends upon your scenario. You can later install SP1 and it’s smart enough to know which bits to update and which not to. Bear in mind again, that the June 2011 CU contains updates to some SP1 bits.
Q: Why are the CU packages so much bigger than the SP packages?
A: Because CU packages contain all languages, whereas the SP packages are single language installers.
Q: A Microsoft blog says to install the CU at the same time or immediately after the SP, should I?
A: The answer is NO. No, no, no!
This is frankly untenable advice. You should only deploy the CU if you are affected by an issue it fixes or are instructed to by a Microsoft engineer.
Remember that CUs are a collection of hotfixes. Service Packs on the other hand are much more broadly tested. Any update is major lifecycle event in the farm and should be thoroughly tested before deployment into production (and I know you all do that right? :)).
Applying a CU just to be “up to date” is madness. Again, unless you need a fix that is in the package, don’t deploy. At least not immediately upon release. There are some of us who have to do this for certain scenarios sure, but your customers farms are not among them!
Remember the October 2010 CU? That had regressions and was pulled and re-released later. Remember the December 2010 CU? That had regressions that broke stuff and we had to wait for the February 2011 CU to fix it. I think you see my point.
[UPDATE 01/07] Hey, whaddya know? There are already regressions with the June 2011 CU. Like this one over here on Todd Carter’s excellent blog.
[UPDATE 12/07] Today the June 2011 CU was “re-released” to avoid some of the issues with the previous release. Kinda proves the point doesn’t it!
You can have much more confidence in the Service Pack as it is a Service Pack. the packaging mechanism might be the same, but CUs and SPs are very different beasts. Regardless I cannot stress enough the importance of appropriate planning and testing for updates of any kind. You should be doing this. Remember, it’s not about build envy, and having the most up to date farm!
[UPDATE 20/07] More and more details about the June CU have been shaking loose. It now is becoming clear why the Product Group were recommending it’s immediate deployment. However the advice above still stands (you need to test!) but there are numerous improvements in the June CU:
There are also numerous issues with SAML claims with just SP1 which are resolved with the June CU.
Assuming you are using the re-released June CU package, I now recommend it’s deployment. Why the Product Group couldn’t have just detailed the why initially instead of completely ignoring the question and waiting for others to discover things is beyond me!
[UPDATE 19/12/2011]
The October 2011 CU is now my recommended baseline build.
[/UPDATE]
Q: OK, so what order should I install the stuff?
A: Because of the way the packages are engineered the order isn’t actually important, but it makes sense to apply them in a logical order. For no other reason than it keeps things simple, especially when working with large farms. I approach the installation like this:
- SPF SP1
- SPF language packs SP1 (as needed)
- SPS SP1
- SPS language packs SP1 (as needed)
- OWA SP1 (as needed)
If it asks you to reboot, reboot once after the installs, not each time it asks. Then of course you run PSConfig on each server in the farm.
As for the June CU, you could include that if you like living dangerously. Or you can wait a while to see what shakes loose over the next few weeks!
[UPDATE 19/12/2011]
As above it is no longer neccessary to deploy the Foundation packages, and you should consider the October 2011 CU as the baseline build. Why not the December? Well because it’s still December!
[/UPDATE]
Q: Why is it all so damn complicated?
A: SharePoint Server is a complex beast. Suck it up. The sustained engineering folks are doing a fantastic job pulling this off. If you think you can do a better job of delivering updates for a product this size, used by so many, in a gazillion languages, you could prove it over here.
.
Leading up to and since India’s thrilling victory in the 2011 World Cup, debate has intensified about the little master’s place in the game. This is really very silly. Can there be any debate? Surely not. Any serious and honest cricket lover’s view must be the same. There can be no denying that Sachin is the greatest of all cricketers. Here’s but just ten reasons, strictly in order of importance, why:
1. He is forever humble
2. He enjoys the game for what it is, his greatest privilege
3. He doesn't care about statistics, he is the ultimate team player
4. He is the ultimate batsman, whose tactical awareness is determined with strategic precision
5. He plays everywhere, not just in England and Australia
6. He faces Waquar, Ambrose, Slinger, Murali, Warne, Lee, McGrath
7. He excels in every form of the game, not just test matches
8. He carries the weight of a nation of a billion and then some, without remorse or desperation
9. He carries himself off the pitch with decency, respect and humility
10. He inspires a nation of greatness to become forever greater
10b. Check that ratty ole Newbury bat with sponsor’s dreds, you know that shows class!
He is quite simply, the master, Sachin Tendulkar, deservedly saluted as the ultimate idol of our times. May all the joy be yours for the entertainment you have bought, and the dreams you enthuse on a forever grateful public. He’s not even a left hander!
"I have seen God. He bats at no. 4 in India in Tests." – Matthew Hayden
I have the privilege of speaking at the following SharePoint events over the next couple of months. Looking forward to meeting everyone, if you are attending, drop by and say hello!
26th March 2011, Utrecht, Netherlands
I’ve done a couple of SharePoint Saturday events before, but only as a “virtual” presenter, this will be the first time I’ll actually be at the event. I’ll be presenting my increasingly popular Rational Guide to User Profile Synchronization session. More details.
11th – 13th April, London, United Kingdom
The best SharePoint Conference by a country mile is back. This year should be a little less of a workload than last! :) I’ll be presenting the following sessions, which will all include brand new content exclusive to this event:
Rational Guide to SharePoint 2010 User Profile Synchronization
Get the real deal on configuring User Profile Synchronization in SharePoint 2010 in this demo and best practices heavy session. This session will cover the architecture of the new User Profile Synchronization capability in SharePoint Server 2010 and provide a walkthrough of the configuration requirements and setup eccentricities. This session will be split 70/30 between demonstrations and lecture.
Best Practices for Multi Tenancy
SharePoint 2010 delivers compelling new infrastructure features for those wishing to host multiple customers on a shared platform whilst retaining confidentiality, integrity and availability. This session will cover how multi-tenancy can benefit all sizes of deployment from a basic farm to the largest such as SharePoint Online. Learn how to approach the design of a multi-tenant deployment and to configure and operate multi-tenant infrastructure, create Member Sites, Subscriptions, Feature Packs, and Service Application Partitions. Related features such as Host Named Site Collections and Claims Identity will also be covered. This session will be split 50/50 between lecture and demonstrations.
Real World Service Application federation with SharePoint 2010
SharePoint 2010 provides architects with a compelling new model for service publishing and federation, opening up exciting new approaches to farm design. This session will cover how Service Application Federation plays out in the real world, based upon early enterprise adopters. Learn how to approach the design of a enterprise services farms, provide true scalability and discover the constraints for each service application which can be published, including global deployment considerations. Related aspects such as Security, High Availability and performance will also be covered. This session will be split 70/30 between lecture and demonstrations.
I’ll also be participating in the Ask the Experts session on Wednesday afternoon. More details.