I've fallen for this several times and every time I forget about it, so time to put it in a blog post, next time I will google for this issue I might find my own post.
Enable Serve Static Files
All you have to do is to define the ENV variable
RAILS_SERVE_STATIC_FILES to true to make rails server static files in production. Normally it would assume the web server will take over this job but since it's running out of a container the web server can't do that.
To workaround this just define this in your docker-compose.yml or however you define your ENV settings.
Thanks to cloud66.com for the post Deploying Rails 6 Assets with Docker and Kubernetes