Soak the salt cod in plenty of cold water for 24 hours, changing the water at least four times. Drain before using.
Boil the potatoes in salted water until soft, then drain.
Place the drained salt cod in a pan of cold water and bring to a simmer, then remove the pan from the heat and let it stand for 15 minutes. Drain, then carefully remove any skin and bones (there are usually quite a few) and flake into small pieces with your fingers.
Put the fish in a food processor and blend to a paste. Then pour in the cream or milk and olive oil, a little at a time, alternating them, and pulse to a creamy paste.
Add the garlic to the cod and season with pepper, then add the boiled potatoes and blend to a purée that retains a little texture. If you have oversoaked the fish and desalted it too much, you may need to add a little salt. You may also like to add a little lemon juice.
If serving cold, spoon into a bowl and sprinkle with the chives (if using). If you want to serve it hot, preheat the oven to 350°F. Spoon the mixture into a baking dish and bake for 10 minutes.