Silvereen I think it has something to do with the server version, he is talking about a custom domain to his free Minecraft server. I could not get a SRV record to work on his server, but when I switched to someone else's server I could join through the domain. I believe this has to do something with spigot 1.21 as I see it reported a lot on reddit.
@TheRiniki Could you try another Minecraft version or server type like paper? I know its the version type of the server because you even deleted it and restarted it and it was still broken.