You can download a sample project demonstrating the issue below here:
When creating a keyboard extension and not specifying a concrete height for its components but instead anchoring them to the view/inputView so that in theory the system will determine their height based on environment and orientation, in some situations that height instead turns into 0 and the keyboard is crushed (with the exception of anything that has a concrete height such as a self sized label or button).
This only seems to occur on iOS 10. On iOS 9, the child views resized correctly to fit the default automatic keyboard height.
There are several scenarios this can manifest and this project demonstrates a basic one. It starts with the basic keyboard extension template with the default "next keyboard" button and the 2 size constraints it comes with:
self.nextKeyboardButton.leftAnchor.constraint(equalTo: self.view.leftAnchor).isActive = true
self.nextKeyboardButton.bottomAnchor.constraint(equalTo: self.view.bottomAnchor).isActive = true
let anotherView = UIView()
anotherView.backgroundColor = UIColor.red
anotherView.translatesAutoresizingMaskIntoConstraints = false
anotherView.leftAnchor.constraint(equalTo: self.view.leftAnchor).isActive = true
anotherView.rightAnchor.constraint(equalTo: self.view.rightAnchor).isActive = true
anotherView.topAnchor.constraint(equalTo: self.view.topAnchor).isActive = true
anotherView.bottomAnchor.constraint(equalTo: self.view.bottomAnchor).isActive = true
anotherView.bottomAnchor.constraint(equalTo: self.nextKeyboardButton.topAnchor).isActive = true
Apple has responded to my DTS ticket and told me to file a bug report, so this is actually an iOS 10 bug. I have filed a radar (#28532959) and will update this answer if I ever get a response. If someone else comes up with a concrete solution that allows me to still use autolayout to achieve an automatic height, answers are still accepted.