The ajax pagination on the Products Page does not remember the browser history. If the user is on the second page of the products page and clicks through to a single product and then clicks the back button, the first page of the products page is loaded instead of the page the user was at.
Return to previous page uses back() method and loads the previous URL in the history list. But if you enables Ajax filter for shop page it means that page is not reloaded. So, either disable Ajax Filter (Theme Options > Shop > Enable Ajax Filter > Off) or hide Return to previous page link.