ASIO4All is not a true ASIO driver, it is actually a ‘wrapper’ for one of the Windows drivers.
You will get much better performance with the ASIO driver that was designed for your interface.
Have you tried downloading the current driver from Focusrite?