Visualforce is not meant to display million(s) of records. Depending on your use case you could implement a pagination solution or a filtering solution to filter VF to return a few records at a time, but in general VF doesn't handle that amount of data very well. Not only do you run into SOQL governore limits, but view state and CPU limits as well.
If you're doing this as purposes for a CSV or PDF generation mechanism, then you should probably look at a more comprehensive, integrated solution or package, or simply use data loader as an alternative.
If you're doing this as purposes for a CSV or PDF generation mechanism, then you should probably look at a more comprehensive, integrated solution or package, or simply use data loader as an alternative.