The idea of JWT is that the message cannot be changed without invalidating the token. The tokens in the database are tied to the specific user as well. You can’t use a random refresh token to get a new access token.
After usage, the refresh token gets deleted, so the token can never be used twice. This ensures that an attacker cannot use an expired access token to get a new access token if the refresh token has already been used.
If you use a whitelist on the server, using the refresh token in the client is a good solution as well.