Hello,
Are you talking about categories in the Product categories list widget http://prntscr.com/q5640m
Go to Appearance > Widgets > add the Product Categories widget to Shop sidebar widget area and make sure that you enabled “Hide empty categories” options in the widget settings.
Regards