Indexing Limits with Sitecore & Azure Search

In the recent past i have had an opportunity to work on a green field Sitecore PaaS + SXA implementation for one of our clients . The client being a Microsoft shop preferred the idea of Azure Search over Solr as their Search provider

Sitecore has been all-in on native Azure PaaS support since Sitecore 9.0 was released in October of 2017. Unfortunately one major pieces does not fully come together with the rest: Azure Search

While there are a few smaller issues, the first issue you would face and most common issue with Azure Search is the 1,000 field limit per index , This is an Azure Search limitation — not a Sitecore issue , Out of the box, Sitecore’s master index already contains approximately 600 fields. This severely limits the amount of new fields that can be added. A moderately-sized website can probably consume those remaining 450 fields, thus hitting the Azure Search limit.

Exception: Sitecore.ContentSearch.Azure.Http.Exceptions.AzureSearchServiceRESTCallException Message: {"error":{"code":"","message":"The request is invalid. Details: definition : Invalid index: The index contains 1076 leaf fields (fields of a non-complex type). An index can have at most 1000 leaf fields.\r\n"}}

The index rebuild will not respond

There are two ways to work around this limitation , In the \App_Config\Sitecore\ContentSearch.Azure\ Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config configuration file (or create a patch file),

Set <indexAllFields> to false, then include fields which are necessary in the <include hint=”list:AddIncludedField”> section.

OR

Set <indexAllFields> to true , then exclude fields which are unnecessary in the <exclude hint=”list:AddExcludedField”> section.

we chose the first approach ,

• Create a patch file to override Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config
• Set <indexAllFields> to false
• While tried to manually add all the fields that would be required for our Search services , this became very tedious and error prone , i was looking for an automated error prone way to do this , Thanks to Gabe Streza for creating this script, Change the path in the script to your Custom template root folder and copy paste the output to your patch file

The Output can be copy pasted to your patch file under AddIncludedField section, the indexes should rebuild successfully after all the custom fields are added

Happy Sitecoring :)