There are a couple of possible candidates in our own backyard like Europa or Encaladus. We struggle to even test those, albeit, they do have possibilities. But you are correct, For any exoplanet the best we can do so far is claim that the evidence merely "suggests" the presence of liquid water. Sometimes I wonder if some people think Star Trek tricorders actually exist.
If we are able to observe exoplanets using spectroscopy in the future, we may be able to determine their atmospheric composition. There are certain elements which, if found in an exoplanet's atmosphere would be strong indicators for life.
Photosynthesis is the primary reason the Earth has such high oxygen levels as well as the ozone layer. This would make O2 and O3 atmospheric biomarkers; indicators that oxygen-producing biological activities may be present. Some other biomarkers are methane and nitrous oxide (emitted by microbes) and chloromethane (given off by seaweeds). Detecting the presence of all these chemicals in the atmosphere, in the right quantities, could indicate that life may be present on the planet.
Examining the atmospheres of exoplanets is something that has only been done rarely in a very limited way. It is hoped that the James Webb Space Telescope will allow astronomers to do so much more easily.
ETA: The observable universe is estimated to contain 200 billion to 2 trillion galaxies. Based
Kepler data regarding the number of exoplanets discovered in our immediate vicinity of our galaxy astronomers reported that there could be as many as 40 billion Earth-sized planets orbiting in the habitable zones of Sun-like stars and red dwarfs in the Milky Way.
Even a conservative estimate makes that 200 billion x 40 billion Earth-like planets in their star's habitable zones. Thats 8e+21 or eight sextillion of these planets in the universe... 8,000,000,000,000,000,000,000 of them. I find it very difficult to believe that in all those potential opportunities for life (as we know it) it only happened once.