All of these include a check for whether or not the column HAS a point/band/interval in it, and perhaps the additional logic there is slowing things down.

It's okay to modify these levels to ALWAYS store the feature they describe. (i.e. not just at most one point in a column, but exactly one point in a column). This should simplify the logic in these levels quite a bit.