Hello,
Check how it works with s default WC theme – Storefront. If the situation is independent of the theme, contact WooCommerce plugin support with this question.
WooCommerce > Settings > Products > Inventory > Out of stock visibility > Hide out of stock items from the catalog – try this option.
Regards