PropertyName Not Showing Up In ModelState.errors

Imagine, a model like this:

[AddressValidation]
public AddressType[] Address { get; set; }
internal class AddressValidation : ValidationAttribute
{
public override bool IsValid(object value)
{
//Assume we are valid
var isValid = true;
//Cast to something useful
var addresses = (AddressType[])value;
var defaultAddresses = addresses.Count(a => a.AddressCode == AddressCodeEnum.@default);
if (defaultAddresses == 0)
{
ErrorMessage = "One address must be the default address";
isValid = false;
}
else if (defaultAddresses > 1)
{
ErrorMessage = "Only one address can be the default address";
isValid = false;
}
//Return the result
return isValid;
}
}

When the model is validated by the controller, any of the subordinate addresses are properly validated and any errors are returned as modelstate errors. However, the custom attribute's error is never added to modelstate, even though it validates false. It seems as if this should work, the validation is called and I can step through it - it just never gets added to modelstate.