We haven't seen it drop either. Japan, with no immigration, has seen that happen.
GDP per capita has continued to rise in Japan even as the population declines:
https://tradingeconomics.com/japan/gdp-per-capita-pppMore importantly, Japan used to have out of control property prices. For a generation they have stabilized. There is nothing that impacts quality of life more than the cost of roof over your head. If we did a comparison of millennial living in Japan to one living in Canada your would have a tough time determining who is actually better off even though Japan had to deal with deflation and tepid aggregate GDP growth.
Lastly, Canada does not need to deal with a declining population because we can use immigration to keep it stable. But we should be keeping it stable because, in the long term, more people is not a benefit (look at China and India).