What is the best all-around prime lens for Olympus M4/3 cameras?

I have used the E-M10 series for a long time and I definitely think your issue is the glass, not the body. To add to the point above: if you really want to keep that setup, you need a lens with a high-end MSC drive system to keep up with your kids. The Olympus M.Zuiko Digital 17mm f1.8 has been one of my most used lenses and I am very satisfied with its performance. It basically solves every complaint you have about that plastic 25mm lens.

• The autofocus is silent and nearly instantaneous. It doesnt hunt like the entry-level primes do, even in tricky indoor light.
• The build quality is excellent. It is a full metal housing with a manual focus clutch that feels very premium and durable.
• The 35mm equivalent focal length is much more versatile for street and indoor family shots than the 25mm. If you are okay with a much heavier lens, the Sigma 16mm f1.4 DC DN Contemporary for Micro Four Thirds is another great option. It is technically sharper than the Olympus 17mm wide open and is a low-light beast, though it is quite large for an E-M10 body. You can easily find both of these used from places like MPB within your 500 dollar budget. Honestly, getting the 17mm and maybe a used Olympus M.Zuiko Digital 45mm f1.8 for portraits would be a perfect kit for your Vancouver trip. That 17mm stays on my camera 90% of the time and I have zero complaints.

I had the exact same frustration when I first started shooting M4/3. Buying the Panasonic Lumix G 20mm f1.7 II ASPH seemed like a good idea, but unfortunately, the autofocus on Olympus bodies is just painful. Missed so many shots of my dog and kids because it kept hunting. Then I tried the Olympus M.Zuiko Digital ED 17mm f1.8. Honestly, it was a bit disappointing. While the build was nice and metal, the sharpness wide open was not as good as expected, especially in the corners. For your budget, you should really look at the Panasonic Leica DG Summilux 15mm f1.7 ASPH. Picking one up used on MPB for about $300 a while back saved me a ton of cash and totally changed my view of the system. The autofocus is near-instant, even in gloomy weather, and it has a physical aperture ring that feels great. The metal build is super solid. It is slightly wider than a 17mm, but it gives you that extra bit of breathing room indoors. Dont waste your money on the cheap stuff anymore, just grab a used copy of the 15mm Leica and you will actually want to keep your camera.

Ugh, I totally get the frustration. I almost threw my gear into the sound last year because the AF kept missing focus on simple street scenes. It is actually embarrassing when you have a real camera but your phone gets a better shot because the lens is hunting in the shadows. I had such high hopes for the compact setup, but unfortunately, that cheap glass is just a bottleneck for the sensor. The 25mm you have is notorious for that toy-like feel and the kit lens is basically useless once the sun goes down. It's just... soul-crushing when you miss a moment of your kids because the gear couldn't keep up. If you want to give the system one last shot before jumping ship to Fuji, look at the Sigma 16mm f1.4 DC DN Contemporary. It is a bit of a beast size-wise, which kind of ruins the tiny camera vibe, but the performance is on another level:

• The f1.4 is a total lifesaver for those gloomy PNW afternoons.
• Autofocus is actually reliable and doesn't do that frantic hunting dance in low light.
• It's sharp right from f1.4, so you dont have to stop down and lose precious light. You can usually find them used for around $350, which fits your budget perfectly. It is a shame the smaller lenses are so hit-or-miss with the AF speed on Olympus bodies, but this Sigma is a tank. Just keep in mind it'll make your camera a bit front-heavy... kinda annoying, but at least the photos will actually be in focus for once.