So, after watching blinry's talk about Timelens at MRMCD 2018, Felix made a userscript for YouTube, which integrates into browsers really easily!
It makes use of the thumbnails which YouTube generates for most videos. Because there are not many thumbnails for short videos, the visual timeline will look blurry for those. Also, some videos do not have thumbnails at all, so this userscript won't work there.
Here's a screenshot:
This is a first step in doing a "proper" Timelens integration, but this is so much better than nothing! And if you run your own website, you can properly integrate it with Timelens!