Do I need to manually create products for each variation?

No. Once the plugin is enabled, it automatically fetches all variations of variable products and displays them individually, based on your global settings. You do not need to create separate products for each variant.