gcc/gfortran 11无法在默认位置找到标头/库

Question

在使用自制将python3.8.6升级到3.9.10之后，如果不显式地将/usr/local/include (针对我的Intel MacBook)或/opt/homebrew/include添加到扩展的include_dirs中，我的Cython扩展就不再存在了。

我的setup.py.in：

import os, sys
from numpy.distutils.core import setup, Extension
from Cython.Build import cythonize

link_arguments = []
extra_include_dirs = []
if (sys.platform == 'darwin'):
    link_arguments.append("-Wl,-rpath")
    link_arguments.append("-Wl,@loader_path/")
    if os.path.exists('/opt/homebrew/'):
        extra_include_dirs.append("/opt/homebrew/include/")
    else:
        extra_include_dirs.append("/usr/local/include/")
else:
    link_arguments.append("-Wl,-rpath=${CMAKE_SOURCE_DIR}/lib/")

pynwp_extension = Extension(
    name="pynwp",
    sources=["${CMAKE_CURRENT_SOURCE_DIR}/lambert.f90", "${CMAKE_CURRENT_SOURCE_DIR}/pynwp.f90", "${CMAKE_CURRENT_SOURCE_DIR}/readAtmosphereGen.f90", "${CMAKE_CURRENT_SOURCE_DIR}/ptogrot.f", "${CMAKE_CURRENT_SOURCE_DIR}/bilin1.f", "${CMAKE_CURRENT_SOURCE_DIR}/fl2pres_f.f","${CMAKE_CURRENT_SOURCE_DIR}/message.c","${CMAKE_CURRENT_SOURCE_DIR}/gridWindDirCorrection.F"],
    libraries=["HirlamUtils_fPIC", "eccodes_f90", "jasper"],
    library_dirs=["${PROJECT_BINARY_DIR}", "${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}/src/libHirlamUtils/", "/opt/homebrew/lib/"],
    extra_link_args = link_arguments,
    include_dirs=["${CMAKE_SOURCE_DIR}/include", "/usr/lib64/gfortran/modules/",
                "${CMAKE_SOURCE_DIR}/build${CMAKE_BUILD_TYPE}"] + extra_include_dirs,
    extra_f90_compile_args=["-DLINUX", "-DIS_LITTLE_ENDIAN", "-DUSEWALLTIME", "-DHAS_BLAS", "-DHAS_LAPACK", "-DGRIB32", "-DTIMING", "-DPREC32", "-fno-whole-file", "-g", "-fbounds-check"]
    #compiler_directives={'language_level' : "3"}
)

setup(name="pynwp",
    author="me",
    author_email="me!me.com",
    version="1.0.1",
    description="Python wrapper for pynwp",
    package_dir={"": "${CMAKE_CURRENT_SOURCE_DIR}"},
    url="http://emaddc.eu",
    license="MIT License",
    ext_modules=[pynwp_extension]
    )

在上面的文件中，我硬编码了自制库dir的位置(暂时)，并添加了一些基于/opt/homebrew的包含dir的功能。如果我将其从文件中删除，编译就会失败，因为无法找到eccodes.mod，当我手动运行python/CMake生成的命令时，请参阅输出：

buildDebug git:(master) ✗ /opt/homebrew/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -I<project_dir>//include -I/usr/lib64/gfortran/modules/ -I<project_dir>//buildDebug -Ibuild/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9 -I/opt/homebrew/lib/python3.9/site-packages/numpy/core/include -Ibuild/src.macosx-12-arm64-3.9/numpy/distutils/include -I/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c -c <project_dir>//src/pynwp/readAtmosphereGen.f90 -o build/temp.macosx-12-arm64-3.9<project_dir>//src/pynwp/readAtmosphereGen.o -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -DPREC32 -fno-whole-file -g -fbounds-check
    f951: Warning: Nonexistent include directory '/usr/lib64/gfortran/modules/' [-Wmissing-include-dirs]
    f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/build/src.macosx-12-arm64-3.9' [-Wmissing-include-dirs]
    f951: Warning: Nonexistent include directory 'build/src.macosx-12-arm64-3.9/numpy/distutils/include' [-Wmissing-include-dirs]
    <project_dir>//src/pynwp/readAtmosphereGen.f90:3:7:
    
        3 |   use eccodes
          |       1
    Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
    compilation terminated.

此扩展是使用CMake构建的大型项目的一部分。当手动运行gfortran命令时，同样的错误指示找不到eccodes.mod。但是，该文件位于gfortran/gcc搜索路径上的默认位置：

locate eccodes.mod
/opt/homebrew/Cellar/eccodes/2.24.2/include/eccodes.mod
/opt/homebrew/include/eccodes.mod

以及gfortran的搜索路径：

gfortran -E -Wp,-v -
#include <...> search starts here:
 /opt/homebrew/include
 /opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include
 /opt/homebrew/Cellar/gcc/11.2.0_3/bin/../lib/gcc/11/gcc/aarch64-apple-darwin21/11/include-fixed
 /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks
End of search list.

另一个项目使用gcc/ld，也有类似的问题。我需要将LINK_DIRECTORIES(/opt/homebrew/lib)显式地添加到CMakeList.txt中，这样gcc才能找到电子代码库。如果没有这些，我会得到：

gcc-11: warning: this compiler does not support X86 (arch flags ignored)
ld: library not found for -leccodes
collect2: error: ld returned 1 exit status
make[2]: *** [src/smoothModeS-v51/smoothModeS-v51.x] Error 1
make[1]: *** [src/smoothModeS-v51/CMakeFiles/smoothModeS-v51.x.dir/all] Error 2
make: *** [all] Error 2

在CPATH和LIBRARY_PATH中添加路径没有影响。

这种方法很有效，但似乎很难维护。我缺少了什么，什么改变了gfortran/gcc不再在默认路径中搜索，而是在库和头/模块中搜索？

编辑

刚刚发现，使用与python扩展相似的代码并同时使用eccodes的独立可执行文件也存在类似的问题。如果我没有将INCLUDE_DIRECTORIES(/opt/homebrew/include)包括在CMakeLists.txt中，我会得到：

cd <project_dir>/buildDebug/src/collocEHS && /opt/homebrew/bin/gfortran  -I<project_dir>/buildDebug -I<project_dir>/include -I<project_dir>/src/readASTERIX2 -I<project_dir>/src/geomag70_linux -I<project_dir>/src/libDTG -fallow-argument-mismatch -ffpe-trap=invalid,zero,overflow -DPREC32 -DLINUX -DIS_LITTLE_ENDIAN -DUSEWALLTIME -DHAS_BLAS -DHAS_LAPACK -DGRIB32 -DTIMING -ffixed-line-length-none  -g -fcheck=all -Wall -fcheck=bounds -O0 -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -c <project_dir>/src/pynwp/readAtmosphereGen.f90 -o CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o
<project_dir>/src/pynwp/readAtmosphereGen.f90:3:7:

    3 |   use eccodes
      |       1
Fatal Error: Cannot open module file 'eccodes.mod' for reading at (1): No such file or directory
compilation terminated.
make[2]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/__/pynwp/readAtmosphereGen.f90.o] Error 1
make[1]: *** [src/collocEHS/CMakeFiles/collocEHSv2.dir/all] Error 2
make: *** [all] Error 2

因此，我想这个问题与python无关，但更多的是gcc/gfortran (gcc版本11.2.0 ) (Homebrew 11.2.0_3)。

编辑2

笔记本电脑的重新启动解决了python扩展构建和setup.py需要附加目录的问题。对于使用CMake的常规构建，我仍然需要为gcc/gfortran提供额外的INCLUDE_DIRECTORIES和LINK_DIRECTORIES命令，以查找brew在/opt/homebrew (或/usr/local )中安装的库。

Answer 1

按照家用开发人员的要求，这是需要的行为：/opt/homebrew和/usr/local是要手动添加到CMake项目中的“特殊”目录。这一点在我在家庭酿造的GitHub上的bug报告中已经解释过了，参见https://github.com/Homebrew/homebrew-core/issues/95561。

我还不能用文件来证实这一点。