Why is the riad so hard to find?

The short answer

The door is designed to blend into the wall.

Same colour, no sign, no shop front. This is deliberate: in medina architecture, the exterior reveals nothing about what's inside. Most riads send someone to meet you at a recognisable drop-off point, or guide you in by WhatsApp voice or video call.