Palm and olive trees could be at risk of extinction on British shores post-Brexit, claims a leading online garden centre.