Sites that require forms-based authentication or cookie-based authentication are not crawled in SharePoint Server 2007

Microsoft Office SharePoint Server 2007 or Microsoft Office SharePoint Server 2007 for Search is directed to crawl content that is saved on sites that require forms-based authentication or cookie-based authentication. However, only the logon page of a site is crawled.

To enable the crawling of sites that require forms-based authentication or cookie-based authentication, use the AddRule.exe command-line tool after you apply this hotfix. To obtain the AddRule.exe command-line tool, visit the following Microsoft Web site:

http://www.microsoft.com/downloads/details.aspx?FamilyId=D5090BC4-5B4F-411B-8CDE-E37D33F7EFDF

Command-line use

AddRule.exe This command displays the following help text:

Usage: AddRule.exe <xml file>

The structure of the input file is specified in the instructions provided with this hotfix.

AddRule.exe input xml file This command will add the crawl rule based on the XML file. These rules are added to the end of their current set of crawl rules. The administrator can later change the order by using the user interface.If the XML file is malformed, you may receive an error message.

You may receive the following error messages if the XML file is malformed.

  • If there is no <Rules> tag, you receive the following error message:

    Syntax error: [rules] element not found as the only node at the root.

  • If a required node is missing in the XML file, you receive the following error message:

    Syntax error: <missing node> element unexpected.

  • If a node in the XML file is incorrectly duplicated, you receive the following error message:

    Syntax error: <node name> element already exists for the current rule

  • If the type is not "FORM" or "COOKIE," you receive the following error message:

    Syntax error: unrecognized value for the <type> element

  • If the login_type is not "POST," you receive the following error message:

    Syntax error: unrecognized value for the <login_type> element

    Note If the administrator reruns this command by using another input file and then finds that the path is identical to an existing rule, the command will modify the rule.


Crawl rules object model

The CrawlRuleAuthenticationType enumeration includes the following new values:

  • FormsRuleAccess = 4
  • CookieRuleAccess = 5

The SetCredentials method in the crawl rules object model is overloaded with two new implementations.

The forms-based authentication rule takes the following input parameters in the following order:

  • type::CrawlRuleAuthenticationType: This will be FormsRuleAccess.
  • AuthSubmissionMethod::String: This will be "POST."
  • AuthSubmissionPath::String: This is the URL in which the parameters should be posted.
  • authData::NameValueCollection: This is where the hidden name value pairs are stored.
  • privateAuthData:: NameValueCollection: This is where the encrypted name value pairs such as user names and passwords are stored.
  • errorPages::StringCollection: This will store the various error pages that would indicate to the crawler to refetch a cookie or to fail the URL with an "Access Denied" error message.

The cookie-based authentication rule takes the following input parameters in the following order:

  • type::CrawlRuleAuthenticationType: This will be CookieRuleAccess.
  • cookies::StringCollection: This will store the cookies that the crawler should use.
  • errorPages::StringCollection: This will store the various error pages that would indicate to the crawler to fail the URL with an "Access Denied" error message.

Note The encryption of the name value pairs and of the cookies is performed by using the same mechanism that is currently available.

Reference: http://support.microsoft.com/kb/934577

Leave a Reply