Diffuse radiation generally increases photosynthetic rates if total radiation is kept constant. Different hypotheses have been proposed to explain this enhancement of photosynthesis, but conclusive results over a wide range of diffuse conditions or about the effect of canopy architecture are lacking. Here, we show the response of canopy photosynthesis to different fractions of diffuse light conditions for five major arable crops (pea, potato, wheat, barley, rapeseed) and cover crops characterized by different canopy architecture. We used 13 years of flux and microclimate measurements over a field with a typical 4 year crop rotation scheme in Switzerland. We investigated the effect of diffuse light on photosynthesis over a gradient of diffuse light fractions ranging from 100% diffuse (overcast sky) to 11% diffuse light (clear-sky conditions). Gross primary productivity (GPP) increased with diffuse fraction and thus was greater under diffuse than direct light conditions if the absolute photon flux density per unit surface area was kept constant. Mean leaf tilt angle (MTA) and canopy height were found to be the best predictors of the diffuse versus direct radiation effect on photosynthesis. Climatic factors, such as the drought index and growing degree days (GDD), had a significant influence on initial quantum yield under direct but not diffuse light conditions, which depended primarily on MTA. The maximum photosynthetic rate at 2,000 µmol m^-2 s^-1 photosynthetically active radiation under direct conditions strongly depended on GDD, MTA, leaf area index (LAI) and the interaction between MTA and LAI, while under diffuse conditions, this parameter depended mostly on MTA and only to a minor extent on canopy height and their interaction. The strongest photosynthesis enhancement under diffuse light was found for wheat, barley and rapeseed, whereas the lowest was for pea. Thus, we suggest that measuring canopy architecture and diffuse radiation will greatly improve GPP estimates of global cropping systems.