如何将pip命令的结果存储到Pandas Datafarme中

Question

为了获得已安装库的列表，我在朱庇特笔记本中运行以下命令：

!pip list

我得到的输出如下：

Package                            Version
---------------------------------- ---------
alabaster                          0.7.12
altair                             4.1.0
anaconda-client                    1.7.2
anaconda-navigator                 1.9.7
anaconda-project                   0.8.3
asn1crypto                         0.24.0
astroid                            2.2.5
astropy                            3.2.1
atomicwrites                       1.3.0
attrs                              20.3.0
alabaster                          0.7.12
altair                             4.1.0
anaconda-client                    1.7.2
anaconda-navigator                 1.9.7
anaconda-project                   0.8.3
asn1crypto                         0.24.0
astroid                            2.2.5
astropy                            3.2.1
atomicwrites                       1.3.0
attrs                              20.3.0
Automat                            20.2.0
azureml                            0.2.7
Babel                              2.7.0
backcall                           0.1.0
backports.functools-lru-cache      1.5
backports.os                       0.1.1
backports.shutil-get-terminal-size 1.0.0
backports.tempfile                 1.0
backports.weakref                  1.0.post1
beautifulsoup4                     4.7.1
bitarray                           0.9.3
bkcharts                           0.2
bleach                             3.1.0
bokeh                              1.2.0
boto                               2.49.0
boto3                              1.14.61
botocore                           1.17.61
Bottleneck                         1.2.1
branca                             0.4.1
category-encoders                  2.2.2
certifi                            2019.6.16
cffi                               1.12.3
chardet                            3.0.4
Click                              7.0
click-spinner                      0.1.10
cloudpickle                        1.2.1
clyent                             1.2.2
colorama                           0.4.1
colorlog                           4.2.1
comtypes                           1.1.7
conda                              4.7.10
conda-build                        3.18.8
conda-package-handling             1.3.11
conda-verify                       3.4.2
configparser                       5.0.1
constantly                         15.1.0
contextlib2                        0.5.5
coverage                           5.3
coveralls                          2.1.2
cryptography                       2.7
cycler                             0.10.0
Cython                             0.29.14
cytoolz                            0.10.0
dacite                             1.5.1
dask                               2.1.0
decorator                          4.4.0
defusedxml                         0.6.0
distributed                        2.1.0
docopt                             0.6.2
docutils                           0.14
entrypoints                        0.3
et-xmlfile                         1.0.1
fastcache                          1.1.0
feature-engine                     0.6.1
filelock                           3.0.12
Flask                              1.1.1
flit-core                          2.3.0
folium                             0.11.0
furl                               2.1.0
future                             0.17.1
gensim                             3.8.3
genson                             1.2.2
gevent                             1.4.0
glob2                              0.7
greenlet                           0.4.15
h5py                               2.9.0
heapdict                           1.0.0
html5lib                           1.0.1
hyperlink                          20.0.1
hypothesis                         5.41.2
idna                               2.8
imageio                            2.5.0
imagesize                          1.1.0
imbalanced-learn                   0.7.0
importlib-metadata                 0.17
incremental                        17.5.0
ipykernel                          5.1.1
ipython                            7.6.1
ipython-genutils                   0.2.0
ipywidgets                         7.5.0
isort                              4.3.21
itsdangerous                       1.1.0
jdcal                              1.4.1
jedi                               0.13.3
Jinja2                             2.11.2
jmespath                           0.10.0
joblib                             0.13.2
json5                              0.8.4
jsonschema                         3.2.0
jupyter                            1.0.0
jupyter-client                     5.3.1
jupyter-console                    6.0.0
jupyter-core                       4.5.0
jupyterlab                         1.0.2
jupyterlab-server                  1.0.0
keyring                            18.0.0
kiwisolver                         1.1.0
lazy-object-proxy                  1.4.1
libarchive-c                       2.8
llvmlite                           0.29.0
locket                             0.2.0
lxml                               4.5.2
MarkupSafe                         1.1.1
matplotlib                         3.1.0
mccabe                             0.6.1
menuinst                           1.4.16
mistune                            0.8.4
mkl-fft                            1.0.12
mkl-random                         1.0.2
mkl-service                        2.0.2
mock                               3.0.5
more-itertools                     7.0.0
mpmath                             1.1.0
msgpack                            0.6.1
multipledispatch                   0.6.0
munch                              2.5.0
navigator-updater                  0.2.1
nbconvert                          5.6.1
nbformat                           4.4.0
networkx                           2.3
nibabel                            3.1.1
nltk                               3.4.4
nose                               1.3.7
notebook                           6.0.0
numba                              0.44.1
numexpr                            2.6.9
numpy                              1.19.2
numpydoc                           0.9.1
olefile                            0.46
openpyxl                           2.6.2
orderedmultidict                   1.0.1
packaging                          20.4
pandas                             1.1.3
pandocfilters                      1.4.2
parso                              0.5.0
partd                              1.0.0
path.py                            12.0.1
pathlib2                           2.3.4
patsy                              0.5.1
pep517                             0.8.2
pep8                               1.7.1
pickleshare                        0.7.5
Pillow                             6.1.0
pip                                20.2.4
pkginfo                            1.5.0.1
plotly                             4.9.0
pluggy                             0.12.0
ply                                3.11
pmdarima                           1.7.1
prometheus-client                  0.7.1
prompt-toolkit                     2.0.9
psutil                             5.6.3
psycopg2                           2.8.6
py                                 1.8.0
pycodestyle                        2.5.0
pycosat                            0.6.3
pycparser                          2.19
pycrypto                           2.6.1
pycurl                             7.43.0.3
pyflakes                           2.1.1
Pygments                           2.4.2
PyHamcrest                         2.0.2
pylint                             2.3.1
pyodbc                             4.0.26
pyOpenSSL                          19.0.0
pyparsing                          2.4.0
pyreadline                         2.1
pyrsistent                         0.14.11
PySocks                            1.7.0
pytest                             5.0.1
pytest-arraydiff                   0.3
pytest-astropy                     0.5.0
pytest-cov                         2.10.1
pytest-doctestplus                 0.3.0
pytest-openfiles                   0.3.2
pytest-remotedata                  0.3.1
python-dateutil                    2.8.0
pytoml                             0.1.21
pytz                               2019.1
PyWavelets                         1.0.3
pywin32                            223
pywinpty                           0.5.5
PyYAML                             5.3.1
pyzmq                              18.0.0
QtAwesome                          0.5.7
qtconsole                          4.5.1
QtPy                               1.8.0
requests                           2.22.0
requests-toolbelt                  0.9.1
retrying                           1.3.3
rope                               0.14.0
ruamel-yaml                        0.15.46
ruamel.yaml                        0.16.12
ruamel.yaml.clib                   0.2.2
s3transfer                         0.3.3
scikit-image                       0.15.0
scikit-learn                       0.23.2
scipy                              1.5.3
seaborn                            0.10.1
Send2Trash                         1.5.0
setuptools                         41.0.1
simplegeneric                      0.8.1
simplejson                         3.17.2
singledispatch                     3.4.0.3
six                                1.12.0
sklearn                            0.0
smart-open                         2.1.1
snowballstemmer                    1.9.0
sortedcollections                  1.1.2
sortedcontainers                   2.1.0
soupsieve                          1.8
Sphinx                             2.1.2
sphinxcontrib-applehelp            1.0.1
sphinxcontrib-devhelp              1.0.1
sphinxcontrib-htmlhelp             1.0.2
sphinxcontrib-jsmath               1.0.1
sphinxcontrib-qthelp               1.0.2
sphinxcontrib-serializinghtml      1.1.3
sphinxcontrib-websupport           1.1.2
spyder                             3.3.6
spyder-kernels                     0.5.1
SQLAlchemy                         1.3.5
statsmodels                        0.11.1
stumpy                             1.5.0
sympy                              1.4
tables                             3.5.2
tabpy                              2.3.1
tabulate                           0.8.7
tblib                              1.4.0
terminado                          0.8.2
testpath                           0.4.2
textblob                           0.15.3
threadpoolctl                      2.1.0
toml                               0.10.1
toolz                              0.10.0
tornado                            6.0.3
tqdm                               4.32.1
traitlets                          4.3.2
Twisted                            20.3.0
unicodecsv                         0.14.1
urllib3                            1.24.2
validators                         0.16.0
wcwidth                            0.1.7
webencodings                       0.5.1
Werkzeug                           0.15.4
wheel                              0.33.4
widgetsnbextension                 3.5.0
win-inet-pton                      1.1.0
win-unicode-console                0.5
wincertstore                       0.2
wrapt                              1.11.2
xlrd                               1.2.0
XlsxWriter                         1.1.8
xlwings                            0.15.8
xlwt                               1.3.0
zict                               1.0.0
zipp                               0.5.1
zope.interface                     5.2.0

如何将此结果存储到Pandas Dataframe中？仅仅复制数据并粘贴在两个列表中就可以完成这项工作，但是是否有任何基于代码的替代方案来实现这一点呢？简而言之，如何将pip命令的结果存储到Pandas中？

P.S:我提到了here，但没有任何帮助。

Answer 1

我们可以使用os模块创建pip列表，然后使用pandas.read_csv和\s+作为分隔器，将pip列表读入数据文件：

import os
import pandas as pd

# create pip list txt
os.system('pip list > pip_list.txt')

# read content into pandas df
df = pd.read_csv('pip_list.txt', sep=r'\s+', skiprows=[1])

# clean up
os.remove('pip_list.txt')

             Package  Version
0            appnope    0.1.0
1        argon2-cffi   20.1.0
2            astroid    2.4.2
3    async-generator     1.10
4              attrs   20.2.0
..               ...      ...
100          urllib3  1.25.10
101          wcwidth    0.2.5
102     webencodings    0.5.1
103            wrapt   1.12.1
104             xlrd    1.2.0

[105 rows x 2 columns]

Answer 2

您可以使用subprocess在内存中获得结果，而不是创建新文件。

import subprocess
from io import StringIO
command = "pip list"
process = subprocess.Popen(command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()
pd.read_csv(StringIO(output.decode("utf-8")),  sep=r"\s*", skiprows=[1])