I have again gone through it and the main problem is the ajax.
Here is post on another page that has the same issue:
https://support.woothemes.com/hc/communities/public/questions/201220087-Woocommerce-Causes-Slow-admin-ajax-php-Load-Times
So, do you have any idea on that?
Thanks!