lechleiter, I just looked and on Amazon, USA you can order a 3mm x 5mm coupler and other sizes as well. I would suggest that you find a Flexible Coupling as it's the same design that ZWO is using and would give much more than a straight coupler.
For my Tak FS-102 I found a 8mm (Tak shaft size ) to 5mm (ZWO EAF) Flexible Coupler and works great! Sorry to say that I did not see a 3mm to 5mm Flexible Coupler?
If you use your 4mm to 5mm Flexible Coupler what will happen is when the EAF motor shaft rotates it will be off center and cause a possible issue with the stepper motor or your focus shaft but with the Flexible Coupler is will flex and reduce the possible issues? Ideally find a flex coupler that's the correct size.
Last you could maybe find at a hardware store an 4mm insert that would get the hold diam closer to the 3mm?