Glad you were able to resolve this. As for the behavior - this is enforced in Policy under Passcode:
Lockout features are built in to the OS many times, even if MDM enforces nothing. If they've put themselves in a situation where the device is locked out and MDM can not be disabled due to policy, there isn't much we can do as our actions (and policy changes) get locked out as well.
They may have to flash back to factory settings similar to what was described above, but in worst case scenarios the devices need to be sent back to the manufacturer for recovery.
We strongly suggest that the features that harden device management be used sparingly, it's not an everyday use feature because of the kind of behavior seen in this post. Clients can leverage Factory Reset Protection features as a theft deterrent and zero touch programs to enforce enrollment. These features provide many of the protections they want with far fewer risks.