Yes and no.
Restaurants offer tap water (free) as well as bottled water. If you’re really concerned, you could ask if they have a water purifier like my work does.
Visit London suggest it’s safe to drink tap water too, apparently the tap water is rated among the best around the world.
If you’re out and about in London, use Find a Fountain’s map to find your nearest source of water to fill up your bottle. Mayor Sadiq Khan has been introducing more public water drinking fountains like in train stations. Perfect when summer hits in London and marathon training around Hyde Park!
Elsewhere, you’ll see ‘don’t drink water from the taps’ in public toilets in London. I’d follow that advice, they know something we don’t!
At home, I think it’s safe to drink in small doses too. I used to drink tap water as my main source, but developed a sore throat after a while. The kettle would also develop a lot of limescale quickly.
I bought a cheap purifier which does make the water taste different and I’ve noticed a reduced build up of limescale in my kettle. You can get bottles with water purifiers too, but I haven’t tested these yet.