SharePoint Lookup Field Throttling Causes Missing Fields in CSOM Query

A very annoying bug appeared few weeks ago in one of our production environments with SharePoint 2013.

SYMPTOMS

You have a custom list with some lookup columns that refer to other lists. In our case the main list contained news and the lookup columns contained the classification of the news.

You add a new lookup column to the list, due to customer feedback.

Suddenly, you can't get the list results any more by code. When you do a CSOM query, the lookup fields are lost. Only the non-lookup fields are retrieved.

CAUSE

SharePoint throttling also includes list item maximum lookup references. It is set to eight (8) lookup fields per list, by default.

This particular configuration is set on the web application throttling page in Central Administration, under the heading "List View Lookup Threshold".

SOLUTION

If you don't need all of the lookups at the same time, you can still make the query by choosing the fields you want to retrieve. However, in our case we needed all of the classification columns.

In this case you have two choices:

• Increase the list lookup threshold limit to more than 8 (that's what we did)
• Establish a large query window, an interval during the day during which you can perform the queries