You can connect your Xbox to your Palworld server. However, Xbox clients do not support Direct Connect via IP and Port. The server must be configured as a public Community Server.
1. How to Setup a Public Community Server:
Files Tab: Edit the configuration file located at Pal/Saved/Config/LinuxServer/PalWorldSettings.ini and set the platform variable to AllowConnectPlatform=Xbox.
Console Tab: Save your changes and Restart the server.
If you need help with this, please submit a ticket.