The impact of the pandemic could cause a structural downshift in the potential output of Indian economy, the RBI said on Tuesday in its 2019-20 annual report.