用 pandas 读取大型 CSV
Reading Large CSVs in pandas

中文

读取大文件时，提前指定列类型并按块处理，可以显著降低内存占用：

import pandas as pd
for chunk in pd.read_csv("big.csv", dtype={"id": "int32"}, chunksize=100_000):
    process(chunk)

English

For big files, declare column dtypes up front and process in chunks to cut memory use:

import pandas as pd
for chunk in pd.read_csv("big.csv", dtype={"id": "int32"}, chunksize=100_000):
    process(chunk)