Configure FAST Search Server for KnowledgeLake Imaging 4.4

After a correct installation of FAST Search Server there are a few items to validate before moving forward (FAST Installation guide Please follow this guide and ensure that you have).
Before beginning the KnowledgeLake Imaging installation check that the Web Application is set to use the FAST Query SSA. To check this start at Central Administration -> Application Management -> Service Application Associations. Then select the Web Application that you want to use FAST Search on.
Rather than leaving the dropdown at default change it to custom. This will allow you to select only the exact Applications you want running on your Web Application.

Then select all the checkboxes appropriate for the Web Application but make sure to NOT check Search Service Application. And be sure to check FAST Query SSA. Additionally, click the ‘set as default’ link to set FAST as the default Search Service Application Proxy.

At this point it is important to note that the “normal” instructions for setting up Imaging 4.4.2 will be loosely followed. I will attempt to make this as thorough as I can to follow. Additionally the two Service Applications created for FAST will be referenced as either FAST Connect or FAST Query from now on. After the installation of FAST there are some pre-installation tasks for KnowledgeLake Imaging.
· From SharePoint Central Administration go to Manage Service Applications.

• Click on FAST Query SSA
• Click on FAST Search Administration
• This will probably be a new screen to users not familiar with FAST.

• Click on Managed Properties
• Click on Add Managed Property

Now the top section should look very familiar. So following the Imaging Admin Guide enter the Property name owsContentType

In the next section Type you can leave the default values

In the next section you need to add a mapping to he ows_ContentType. This is almost identical to the normal SharePoint Managed Properties mapping but has the ability to add more than one selected crawl properties at one time.

Click OK
Leave the Sort Property unchecked
On Query Property make sure to check the box. This insures that the KnowledgeLake Search Center can provide the property in the results column or as a search property.
Leave the Refiner property unchecked
On the Full-text Index Mapping section….It is very important to select a number between 1-7 as a priority mapping. If you leave the default value at zero it will not populate the Index. Considering this is a fairly important property in the KnowledgeLake Search Center I suggest using the priority of 6.

Then click OK
In the next section of the KnowledgeLake Imaging Admin guide it requires you to visit the Categories section and make a change to the SharePoint category so that all newly discovered properties are automatically added. This is set by default in FAST

Run a full crawl with the FAST Content Service Application.
After the crawl is complete run the following PowerShell in the Microsoft FAST PowerShell

Get-FASTSearchMetadataFullTextIndexMapping
The result should provide this in the resulting feed

SharePoint Tenant Creation with Change Request tracking in System Center Service Manager

One of the more troublesome issues in SharePoint is creating new site collections when not logged into Central Administration. Because this task always requires someone with farm admin access it tends to not get high priority in a normal administrators daily role. Additionally, Governance teams should validate when site collections are being created and monitored correctly versus creating a sub site. By combining the ease of submitting a request in a SharePoint list and the Best Practice of tracking changes in Systems Center Service Manager with Opalis there is a smooth, expedient, tracked manor to provision site collections. By combining SharePoint and SCSM a decision can be made as to where the approval workflow reside. For this demonstration the site collection will be automatically provisioned and a Change Request will automatically be created inside Systems Center Service Manager. Then the SharePoint list item that originally kicked off the process will be updated to show the request has been completed. Finally the SCSM ticket will be closed automatically so there is a record of the change.

Requirements

· Custom SharePoint List that includes

§ Title column to store the type of request. The default value is defined as Site Collection Request.

§ Site Collection URL column to the preferred url of the site.

§ Primary Site Collection Administrator column.

§ A Request column that is a calculated field that combines the Title and the Site Collection URL to create a unique name for reporting in SharePoint and in SCSM.

§ A status column that is a dropdown with the values of New & Complete available.

§ A customized All Items view that filters out all entries that do not have a status of New.

· Systems Center Service Manager Integration Pack for Opalis

· Systems Center Service Manager configured

· Execute PowerShell Script Integration Pack

· Global Variables

Start by configuring the SharePoint Integration Pack in Opalis. Under Options Select Microsoft SharePoint (see Figure 9-50). Then add in an a configuration to point to the new SharePoint list defined in the requirements (see Figure 9-51).

Figure 9-50 Navigating to the Microsoft Configuration section in Opalis.

Figure 9-51 Defining another configuration point.

Then add a configuration for the installed System Center Service Manager. Start by selecting Options and then select System Center Service Manager (see Figure 9-52). Then create entries to connect to the System Center instance. Finally, validate the settings by clicking the Test Connection (see Figure 9-52).

Figure 9-52 Navigating to the System Center Service Manager Configuration section in Opalis.

Figure 9-53 A defined configuration to connect to System Center.

Then expand Global Settings and right-click Variables select New then select Variable (see Figure 9-54). Rather than pass the variables from SharePoint the Global Variable functionality in Opalis will be used in case these values are required again. Create two variables (see Figure 9-55).

Figure 9-54 Creating a new Global Variable.

Figure 9-55 Defined Global Variables

Then create a New Policy inside the Opalis Client and add-in the following objects

· Microsoft SharePoint

o Monitor List Item

o Update List Item

· Execute PowerShell Script

o Execute PS Script

· System Center Service Manager

o Create Change with Template

o Get Activity

o Update Activity

The end result should end up looking like Figure 9-56.

Figure 9-56 A Policy setup in Opalis.

Configuration of the Policy objects

Monitor List Item

Under Configuration select the name by choosing the name of the configured SharePoint list for creating a site collection request (see Figure 9-57).

Figure 9-57 Monitor List Item object properties.

Create Change with Template

Under the Properties section set the Connection to the name of the System Center instance defined earlier. Then for Class choose Change Request and for Template choose Standard Change Request. The Fields need to be added to so click on Select optional Fields and choose Status, Impact, Title and Actual start date. After hitting OK individually set the values in these items (see Figure 9-58).

· Status = New

· Impact = Minor

Figure 9-58 Create Change with Template properties.

To set the Title field by right-clicking in the Title field, select Subscribe and select Published Data (see Figure 9-59). The only available option (after connecting the two objects) is Monitor list Item. Scroll through the columns and select the Request column from SharePoint. Actual start date is defined the same way but the value is from Created.

Figure 9-59 Setting the Title field by subscribing to Published Data.

Execute PS Script

Under the Input Properties right-click the PS Script 01 and select Expand. There are five variables in the PowerShell script and just a simple concatenation of the variables. Finally the script creates a new site collection with the url and primary site collection administrator passed in as dynamic variables (see Figure 9-60).

· #webUrl -This is subscribed to the Global Variable SharePoint Web Application URL

· $managedPath - This is subscribed to the Global Variable SharePoint Managed Path

· $scUrl - This is subscribed to from the Monitor List Item object.

· $ownerAlias – This is subscribed to from the Monitor List Item object

· $completeUrl – This concatenates $webUrl, $managedPath, and $scUrl

Figure 9-60 PS Script 1 properties to create a dynamic site collection

Update List Item

The objective of the Update List Item object in this scenario is to demonstrate automated completion of the operation without the complexity of creating a SharePoint workflow. This object takes in the ID of the list item from Monitor List Item and updates the Status column to Complete (see Figure 9-61).

Open the Configuration section of this object and select the SharePoint configuration that points the Site Collection Request list. There are two properties that are setup in this object.

· ID – This is subscribed to from the Monitor List Item

· Status – This is set to the value of Complete

Figure 9-61 Update List Item Properties.

Get Activity

The Get Activity object needs the Connection to point to the configured Systems Center in Opalis. The Activity Class is set to Manual Activity and Source Object Guid is subscribed to from the Create Change with Template object (see Figure 9-62).

Figure 9-62 Get Activity properties.

Update Activity

The final object in the policy updates SCSM and changes the status of the ticket to Completed. The Properties Connection needs to be set to the configured System Center in Opalis. The Activity Class is set to Manual Activity and the Object Guid is subscribed to from Get Activity. The only field utilized is the Status field and it is set to Completed (see Figure 9-63).

Figure 9-63 Update Activity properties connected to the System Center connection, set to a Manual Activity and receiving the Object Guid from Get Activity.

Testing

Place the Policy in Policy Test Console and click the Run button. The Monitor List Item object will be waiting for a new SharePoint list item to be created (see Figure 9-64). After the new item is detected a ticket is created (see Figure 9-65).

Figure 9-64 Site Collection provisioning Policy in debug mode.

Figure 9-65 Site collection ticket in System Center Service Manager showing a status of completed.

Solving real world SharePoint problems with Opalis

Monitoring Content DB Size and provisioning a new Content DB

By taking the previous example a little farther it would be possible to monitor the content databases of a SharePoint environment to prevent content databases from becoming too large. This is one of the main advantages of Opalis is that it assists in maintaining Best Practices in the enterprise (see Figure 9-49).

Requirements

· Initial Global scheduler to analyze the environment on a daily basis

· PowerShell script that finds all the web application url’s, then finds the content databases that make up the site collections

· Database query that passes in the content database name and returns the size

· Logic that defines the maximum size of the content database

· Logic that calculates 90% and 70% and provides alerts

· PowerShell script that automatically provisions a new content database to the appropriate web application when the database size hits 70%. The content database should initially be offline to prevent use initially.

· At 90% of the defined Best Practice size use a PowerShell script that automatically put the initial content database in offline mode and puts the new database in Ready mode.

Figure 9-49 Policy to monitor content databases in SharePoint.

SharePoint Integration Kit

Opalis has built-in Integration Packs and the ability to utilized custom Integration Packs. An Integration Pack has been created for SharePoint and is available at http://opalis.codeplex.com. After installing the Integration Pack using the steps provided previously it is necessary to configure the SharePoint Integration Pack. Start by selection Options from the top menu and select Microsoft SharePoint (see Figure 9-40). After selecting that option the Configuration Window opens (see Figure 9-41). Click Add to enter the specifics about what SharePoint environment that this configuration should connect to. Every configuration can be different logins, different lists or document libraries, or different web url’s (see Figure 9-42).

Figure 9-40 Microsoft SharePoint configuration selection for the Microsoft SharePoint Integration Pack.

Figure 9-41 Microsoft SharePoint configurations defined in Opalis.

Figure 9-42 An individual Microsoft SharePoint configuration for Opalis.

Database sizes into a SharePoint List

Over the years I have received many requests from different levels of management and normally the more difficult ones have the biggest priority. I recall an issue with another group’s database that was consistently causing issues because they were getting too large and were not being monitored correctly. So the question came up as to how do we get all the databases and their sizes provided to management on a daily basis? Normally this requires some manual intervention in Excel or providing access to the administration tools in the datacenter or the DBA’s tools. Providing ODBC access in Excel or Access has the potential to expose the username and password. What if on a daily basis, or even hourly, Opalis queried all the databases and populated a SharePoint list that could be provided to anyone? This would remove any security concerns and alleviates another request.

Start by creating a new Policy and drag a Query Database object from the utilities section. Then enter a simple query to get the top 100 databases from the master databases and get the size into Megabytes (see Figure 9-42). Finally, define the connection and security credentials.

/** Select the databases from the master database**/

SELECT TOP (100) name, (size*8)/1024 SizeMB

FROM sys.master_files

Figure 9-42 Query Database object properties displaying the SQL statement.

Now drag a Create List Item object from the Microsoft SharePoint section of objects and drag a link from the Query Database object to the Create List Item object (see Figure 9-43).

Figure 9-43 Connecting objects in Opalis.

Double-click the Create List Item object to open the properties dialog. Under Configuration set the name to SharePoint Database List. This queries SharePoint and provides the Title field. Click Optional Properties to add-in the Size MB field and select to move to the Selected column (see Figure 9-44).

Figure 9-44 Adding properties available to a SharePoint Create List Item.

To configure the field mapping it takes a little more configuring so since it known that the SQL Query is going to return two fields separated by a semicolon it is possible to use the following function (see Figure 9-45).

Figure 9-45 Data manipulation function inside Opalis.

The function passes in the familiar Full line as a string with fields separated by ‘;’ and passes in a delimiter and requests the first value from the pair. The field for the Size MB is similar but requests the second value. The final result looks like Figure 9-46.

Figure 9-46 Create List Item properties showing two fields populated.

From this point it is possible to run a Test on this policy and populate the database. Figure 9-47 shows the Policy running. Figure 9-48 shows the SharePoint list being populated.

Figure 9-47 Policy populating a SharePoint list from the Policy Testing Console.

Figure 9-48 SharePoint list populated with values from Opalis.

Creating a basic Policy

Figure 9-28 The Opalis Integration Server Interface.

By opening the Opalis Integration Server and right-clicking on the Policies tree select Policy to create a new Policy (see Figure 9-29). This will create a policy in the main section of the interface (see Figure 9-28 B). For the scenario with the new Policy we need a process to run on a schedule. Expand the section called Global Settings and right-click Schedule and select New Schedule (See Figure 9-30). For this SharePoint specific example the Schedule name is Content Databases (see Figure 9-31). By updating the Detail section this will check on the size of the content databases in a SharePoint environment once a day (see Figure 9-32).

Figure 9-29 Creating a new Policy in Opalis.

Figure 9-30 Creating a new Schedule in Opalis.

Figure 9-31 General Scheduling information in Opalis

Figure 9-32 Setting a Schedule in Opalis.

Testing

Opalis has the ability to run tests on the Policies created. The following example has a Custom Start object that runs a Database Query (see Figure 9-33). In the Task bar select Test to perform a test on the Policy. This brings up a new application window that allows the user Run, set Breakpoints and debug objects (see Figure 9-34).

Figure 9-33 Simple Policy with two connected objects.

Figure 9-34 Policy Testing Console

In the Testing Console if an individual object is selected the properties are displayed in the Design Time Properties window (see Figure 9-35). There is a simple SQL statement that runs a built in stored procedure that returns all the databases on the SQL Database that the Query Database object is setup to connect to. By clicking the Run button the two objects will execute and provide a log of their actions, properties, and return values (if available) (see Figure 9-36).

Figure 9-35 Design Time Properties for the Query Database object.

Figure 9-36 Policy Testing Console displaying the populated logs after running a policy.

The Query Database stores all its returned values in the Full lines as a string with fields separated by a ‘;’. By highlighting that line in the logs the interface adds a ellipsis button. But clicking on the ellipsis button a small modal window provides the data returned but the query (see Figure 9-37). While the results are interesting the name of the modal is very important. The data is designated as Published Data. So when objects are connected in the Policy and there is a desire to interact with the data, this is the actual data that is published from the Query Database object that was defined.

Figure 9-37 Published Data Browser displaying the results of the Query Database object.

By exiting the Test client and returning to the Integration Server Client we can show how it is possible process this Published Data some more. By selecting and adding a Run .Net Script object to the Policy the capacity to run C#, Jscript, PowerShell or VB.Net code is available (see Figure 9-38).

Figure 9-38 The Run .Net Script Object displaying the types of script it can run.

After selecting PowerShell as the script type it is necessary to right-click in the Script section to open a context menu. This allows the ability to select the Published Data from the Query Database object (or from a Global Variable). Select Published Data and then the field Full line as string with fields separated by ‘;’ the script section will display a hyperlinked section of code that can be assigned to a variable in PowerShell. By adding a few more lines of PowerShell for processing and putting the same Policy in the Test Console again it is possible to see the first value passed in from the Query Database object into the PowerShell (when set as a breakpoint) script (see Figure 9-39).

Figure 9-39 Policy Testing Console in debug mode with a breakpoint hit at the Run .Net Script object.

Automating SharePoint in a Private Cloud using Opalis

• Define what Opalis can provide in an organization
• How to install Opalis
• Opalis Integration Packs
• Creating a basic Policy
• Testing Opalis Policies
• Opalis SharePoint Integration Kit
• Solving real world SharePoint problems with Opalis

 Now let us focus on a specific solution that offers a unique toolset to the current mindset of datacenter administration.
Opalis is an automation platform that improves the reliability of IT processes by orchestrating and integrating IT tools to control the cost of datacenter operations. By enabling an organization to automate best practices through workflow processes that coordinate’s System Center & management/monitoring tools to automate incident response, change & compliance, and service-lifecycle management processes.
Opalis offers a designer for custom workflow processes, through Integration Packs, and automatically shares data and initiates tasks in System Center Operations Manager, System Center Configuration Manager, System Center Service Manager, Virtual Machine Manager, Active Directory and many other 3^rd party toolsets. I have seen some impressive implementations by organizations where they can script out a remediation’s but consistently the one missing piece is the ability to track that the incident, what did it take to resolve it, and close the ticket without delayed manual intervention. Opalis offers the ability to not only configure the scripting portion but also try multiple variations of scripted solutions till a remedy is found, offer escalations through email or System Center, track/monitor incident and change requests and much, much more. Opalis offers the “Last Mile” piece many IT organizations have been trying to accomplish. By combining System Center and Opalis organizations open the doors to increased integration, efficiency and business alignment of the datacenter IT services by:

• Automating cross-silo processes and enforcing best practices for incident, change and service lifecycle management.
• Reduce unanticipated errors and service delivery time by automating tasks cross vendor and organization silos.
• Integrating System Center with non-Microsoft tools to enable interoperability across the datacenter.
• Orchestrating tasks systems for consistent, documented, compliant activity.

Many organizations have successfully jumped on the PowerShell bandwagon and accomplished many automation techniques. There are now many books that have sections dedicated to PowerShell automation within the Microsoft solution space. But normally this requires multiple systems that are running timer jobs across the organization. This tends to get so overwhelming for organizations that they are unable to define where all the jobs are running or when they are running because they become so disparate and have no ability to roll up an accomplished task to a central repository (e.g. System Center) the jobs end up being lost in the organization. Consider another scenario where management starts seeing random issues with a particular job and the only fallback an administrator can provide is to attempt to find logs for a particular script and hope the logs actually caught the issue.
Opalis Components
Opalis has the following main components
Integration Server Client This is the application that creates, modifies, deploys, tests and manages policies.
Action Server Policies are executed on the Action Servers and more than one Action Server can be created. It is possible to dedicate Action Servers for policies requiring a lot of disk space.
Policy Testing Console - Application that allows for testing of policies configured in the Integration Server Client before being deployed to the Action Server. It also has the ability to debug and step through policies.
Operator Console Web-Based application that allows real-time information about metrics such as policy state, duration, and capacity utilization.
Management Server Server that provides connections from the Integration Server Clients to the Opalis Datastore for building and testing policies.
Opalis Datastore A Oracle or Microsoft SQL Server database where configuration information, policies and logs are stored.
Deployment Server Tool that offers a view of the entire Opalis Integration Server infrastructure and deploy Action Servers, Integration Clients, Integration Packs and hotfixes from one place.

SharePoint 2010

I came across some videos regarding SharePoint 2010

SharePoint 2010 Demo Video Downloads

If you are having trouble viewing the SharePoint 2010 preview demos on the mini-site, or would like to host them internally in your environment, here are the links to the downloadable wmv's:

Overview: http://msstudios.vo.llnwd.net/o21/msstudios/0906/1001251/1001251_SharePoint2010_Overview_SneakPeek_Rizzo_mobile.wmv

For Developers: http://msstudios.vo.llnwd.net/o21/msstudios/0906/1001251/1001251_SharePoint_SneakPeek_For_Developers_Andrew_mobile.wmv

For IT Pro's: http://msstudios.vo.llnwd.net/o21/msstudios/0906/1001251/1001251_SharePoint_SneakPeek_For_IT_Professionals_Riley_mobile.wmv

Plus the Sneak Peak page http://sharepoint.microsoft.com/2010/Sneak_Peek/Pages/default.aspx