读取8x8整数矩阵csv文件Python

Question

我从另一个程序生成了一个csv文件，如下所示：

45, 133, 148, 213,  65,  26,  22,  73
 84,  51,  41, 249,  25, 167, 102,  72
217, 198, 117, 123, 160,   9, 210, 211
230,  64,  37, 215,  91,  76, 240, 163
123, 169, 197,  16, 225, 160,  68,  65
 89, 247, 170,  88, 173, 206, 158, 235
144, 138, 188, 164,  84,  38,  67,  29
 98,  23, 106, 159,  96,   7,  77,  67
 
142, 140, 240,  56, 176,   0, 131, 160
241, 199,  96, 245, 213, 218,  51,  75
 22, 226,  81, 106,  94, 252, 252, 110
  0,  96, 132,  38, 189, 150, 162, 177
 95, 252, 107, 181,  72,   7,   0, 247
228, 207, 203, 128,  91, 158, 164, 116
 70, 124,  20,  37, 225, 169, 245, 103
103, 229, 186, 108, 151, 170,  18, 168

 52,  86, 244, 244, 150, 181,   9, 146
115,  60,  50, 162,  70, 253,  43,  94
201,  72, 132, 207, 181, 106, 136,  70
 92,   7,  97, 222, 149, 145, 155, 255
 55, 188,  90,  58, 124, 230, 215, 229
231,  60,  48, 150, 179, 247, 104, 162
 45, 241, 178, 122, 149, 243, 236,  92
186, 252, 165, 162, 176,  87, 238,  29

在每个8x8整数矩阵后面总是有一个空格。

我需要将每个8x8矩阵读入Python程序，在其上生成一个操作，然后编写格式相同的结果。结果将是8x8浮子矩阵，空间跟随每一个8x8矩阵。

我如何在Python3.x中完成这两件事？我可以一点一点地读取文件，但是Python可能有一种健壮的方法，可以使用少量的代码来实现这一点。

Answer 1

也许Python有一种健壮的方法，可以使用少量的代码来完成这一任务。

实际上是这样的。作为一个选项，您可以使用熊猫模块。以下是一个例子：

import pandas as pd

df = pd.read_csv('mtrx.csv', header=None, chunksize=9)
for i, matrix in enumerate(df):
    matrix.mul(10**i).fillna('').to_csv('mtrx1.csv', index=False, header=False, mode='a')

此代码将每个矩阵乘以10乘以i，结果文件如下所示：

45,133.0,148.0,213.0,65.0,26.0,22.0,73.0
84,51.0,41.0,249.0,25.0,167.0,102.0,72.0
217,198.0,117.0,123.0,160.0,9.0,210.0,211.0
230,64.0,37.0,215.0,91.0,76.0,240.0,163.0
123,169.0,197.0,16.0,225.0,160.0,68.0,65.0
89,247.0,170.0,88.0,173.0,206.0,158.0,235.0
144,138.0,188.0,164.0,84.0,38.0,67.0,29.0
98,23.0,106.0,159.0,96.0,7.0,77.0,67.0
 ,,,,,,,
1420.0,1400.0,2400.0,560.0,1760.0,0.0,1310.0,1600.0
2410.0,1990.0,960.0,2450.0,2130.0,2180.0,510.0,750.0
220.0,2260.0,810.0,1060.0,940.0,2520.0,2520.0,1100.0
0.0,960.0,1320.0,380.0,1890.0,1500.0,1620.0,1770.0
950.0,2520.0,1070.0,1810.0,720.0,70.0,0.0,2470.0
2280.0,2070.0,2030.0,1280.0,910.0,1580.0,1640.0,1160.0
700.0,1240.0,200.0,370.0,2250.0,1690.0,2450.0,1030.0
1030.0,2290.0,1860.0,1080.0,1510.0,1700.0,180.0,1680.0
,,,,,,,
5200,8600,24400,24400,15000,18100,900,14600
11500,6000,5000,16200,7000,25300,4300,9400
20100,7200,13200,20700,18100,10600,13600,7000
9200,700,9700,22200,14900,14500,15500,25500
5500,18800,9000,5800,12400,23000,21500,22900
23100,6000,4800,15000,17900,24700,10400,16200
4500,24100,17800,12200,14900,24300,23600,9200
18600,25200,16500,16200,17600,8700,23800,2900

upd

至于带有逗号的行，这意味着csv文件中的那些行没有数据，即空行。

Answer 2

如果您已经知道矩阵中有8行，则可以使用pandas.read_csv加载numpy数组中的所有数据，然后再对其进行整形。

如果事先不知道每个矩阵的行数，pandas.read_csv将为空行生成所有NaN的行，这将允许您推断每个矩阵的行数，并进行整形：

import numpy as np
import pandas as pd

def read_csv(file, num_rows=None):
    if num_rows is not None:
        df = pd.read_csv(file, header=None, skip_blank_lines=True)
        arr = df.values
    else:
        df = pd.read_csv(file, header=None, skip_blank_lines=False)
        num_rows = extract_matrices_num_rows(df)
        valid_idxs = np.delete(
            np.arange(len(df)), np.arange(num_rows, len(df), num_rows + 1)
        )
        arr = df.iloc[valid_idxs].values

    return arr.reshape(-1, num_rows, arr.shape[-1])

def extract_matrices_num_rows(df):
    blank_lines_indices = all_nans_indices(df)
    blank_lines_indices = [-1, *blank_lines_indices, len(df)]
    num_rows = np.diff(blank_lines_indices) - 1
    num_rows = set(num_rows)
    if len(num_rows) > 1:
        raise ValueError(
            f"Matrices detected to have various number of rows: {num_rows}"
        )
    return num_rows.pop()

def all_nans_indices(df):
    return list(df[df.isnull().all(axis=1)].index)

快速检查它在这两种情况下是否都同样有效：

file = "data.csv"

assert np.array_equal(read_csv(file), read_csv(file, num_rows=8))

Answer 3

下面的解决方案使用Pandas & Numpy。例如，下面将[df.valuesi:i+8+2. 2添加到矩阵的每个值中。输出将与输入格式CSV相同，包括空行。

import pandas as pd
import numpy as np
df = pd.read_csv('Book2.csv', skip_blank_lines=False, header=None)

updated_metrcies = [np.vstack([df.values[i:i+8]+2,np.repeat(np.nan, df.shape[1])]) for i in range(0, df.shape[0], 9) if i < df.shape[0]]

pd.DataFrame(np.vstack(updated_metrcies)[:-1]).to_csv('Book4.csv', index=False, header=None)