Before we get into the implementation details, I would like to introduce the topic of designing 3D user interfaces and VR experiences. A lot of work has been done over the past few decades, and more so in the past few years.
With consumer VR devices so readily available, and powerful development tools like Unity, it's not surprising there are many people inventing and trying new things, innovating continuously, and producing really excellent VR experiences. You are probably one of them. But the context of today's VR is not a vacuum. There is a history of research and development that feeds into present-day work. The book 3D User Interfaces: Theory and Practice (Bowman et al), for example, is a classic academic survey of 3D user interaction for consumer, industrial, and scientific applications and research. Originally published in 2004,...