We are splitting our monolith API into micro services.
We do not need rate-limiting, auth, caching or any other gateway like abilities.
Would it be a valid approach to use very simple stateless Nginx containers that route to the underlying services?
Yes, Nginx can be a deployment and a service (of loadbalancer or externalIP type) and can forward to upstream services.
You might have to frequently change the nginx.conf though (when you add/remove services), so I would recommend using a ConfigMap to keep your nginx.conf and mounting that as a volume in your deployment. Refer: http://kubernetes.io/docs/user-guide/configmap/ and scroll down to consume configmap via volumes.
Another thing to keep in mind is that if you delete and create a service that is referred to in nginx.conf as an upstream service, you'll have to restart your deployment because nginx resolves all service DNS labels when nginx starts.