After put my flic aside for a few days. It magically becomes much more stable now. It has been connected to my phone for two days in a row.
I am not sure what I did. I did installed a few BLE scanner on my device. Maybe the software got updated. I will report if anything strange happens or I found out the reason.
Nice job, Flic! I really like the design and feeling in hand.