Oxygen and water are different questions
Many fish obtain dissolved oxygen through their gills. They do not split water molecules to breathe. Water passing through the mouth and over the gills does not necessarily mean it has been swallowed into the intestine.
In freshwater
Many freshwater bony fish gain water osmotically, mainly across the gill surface. They excrete the excess as large quantities of dilute urine and regulate salts through specialised mechanisms. Their need to drink is therefore generally small. An explanation based on supposedly oversized salt particles is incorrect: membrane properties and ion transport matter.
In seawater
Many marine bony fish lose water to the saltier surroundings and replace it by drinking. Water is absorbed in the intestine, while excess salts are removed through mechanisms including the gills. Kidney size alone does not explain the difference. Sharks and other groups have their own adaptations, so the rule cannot be applied to every fish without exception.
The short answer
Yes, many fish drink, but the amount and means of maintaining balance depend on species and environment. This is not simply human thirst in miniature. Fish moving between fresh and salt water adjust the relevant systems.