要用python从几百万行数据的文件一行行读数据并计算,如何处理最快,哪位大牛可以指点一下
要用python从几百万行数据的文件一行行读数据并计算,如何处理最快,哪位大牛可以指点一下
[解决办法]
1. 使用readlines(buffer)
f.readlines() returns a list containing all the lines of data in the file. If given an optional parameter sizehint, it reads that many bytes from the file and enough more to complete a line, and returns the lines from that. This is often used to allow efficient reading of a large file by lines, but without having to load the entire file in memory. Only complete lines will be returned.
2. 试试fileinput模块