Thanks. i figured out. The issue here is that the hotbar is not getting any touch event when the inventory is closed due to the logic in the InventoryInputManager. I have to override the InventoryInputManager to not disable Raycaster for my hotbar - as I want to allow my player to select the weapon without having to open up the inventory. I want to create a scheme that's similar to minecraft where we have an inventory bar at the bottom where all the items will be filled in there first and then the rest will be put to the main inventory.