I am not in any way frugal, and spend money if I think it would be worth it and/ or enjoyable. Sometimes I choose a hostel with the sink next to the pillow and shared bathroom, - I was taking part in a dinner in the evening and left early in the morning other times I like a comfortable hotel room. For instance travelling with children one would often spend more time in the room and it would be more important to have some space, a kettle and a nice tv.
I will do an intercontinental sit next month, and have booked a nice hotel the first two nights before sit to get accustomed to time zone and be a tourist, for instance. So a central and comfy room is more important at that occasion.
Travelling witout petsitting I also did as @PVGemini suggests - searched Free things to do in… It has been so rewarding - not only saving money to spend on something else, but it has opened up so many exciting experiences I wouldn’t otherwise find. Some examples - when we went to New York, we visited Brooklyn Bridge, Bryant Park, China town - but we also got free tickets to be audience in a tv show and we had a free guided tour in the New York Federal Reserve Bank incl. a visit to the gold vault 10 storeys underground.
Museums often has a day (a week or a month) with free admission - at least in off season. So look for those. For those big, costly museums - a hack could be to visit the museum store, which would often have a selection of what is shown in the museum. In my opinion it is in no way a good substitute for the museum itself! But sometimes it is «enough» for that trip. We did that at MoMa - when we came, we were really tired, and as the tickets were expensive we just did the museum shop before going back to our hotel.
Check out where the public transport can take you. In NYC you could take the Long Island ferry for free, in several coastal cities ferries are part of the public transport covered by your buscard, or buslines could be a great way for sightseeing for a small cost. Whether you take the bus round Manhattan, a doubledecker in London or the tram from Benidorm to Alicante. Many of those have great views and can well be taken just for the sightseeing.
I think using cafés and restaurants can be a great way to explore the local community and culture. Having aperitivo in Italy in a small cafe with the locals is a special experience. One time in Italy we signed up to barista classes at a coffee supplyer (not cheap, but worth it), and at lunch time they took us to a local workers café. As a tourist we would never have known that the dull store front was in fact a lunch place with good and affordable lunch, with simple furniture, a couple of dishes to choose and no-fuss serving with local workers. A real treat, both food and culture.