AllowAll is the last check before the parent window is checked. This allows
to override a DenyAll in the parent window in a simpler way than before (the
previous method required all devices to be in the permit list).
We shouldn't be able to restrict events like Expose, etc. with device based
ACLs. So we just ignore all non-input events when checking for permissions.
