如何让Postgres容器与amazon EFS共享数据？

Question

我有一个集群，它有2个instances.Both实例，有Postgres容器，它的卷链接到弹性文件系统accesspoint作为卷。

我将两个实例的卷映射到/var/lib/postgresql/ data，但是容器没有共享它的数据。

这是我的配置

docker-compose.yml

version: "3.0"
services:
    front:
        image: 540744822643.dkr.ecr.ap-southeast-1.amazonaws.com/front:latest
        links:
            - app:app.plasgate.com
        networks:
            - app
        container_name: front
        environment:
            - NODE_OPTIONS=--max-old-space-size=2048
        ports:
            - "8080:80"
        logging:
            driver: awslogs
            options:
                awslogs-group: sms-gateway
                awslogs-region: ap-southeast-1
                awslogs-stream-prefix: "front"
    app:
        image: 540744822643.dkr.ecr.ap-southeast-1.amazonaws.com/plasgate:latest
        links:
            - jasmin:jasmin
            - db:db
        networks:
            - app
        container_name: app
        environment:
            - PYTHONUNBUFFERED=1
            - PYTHONIOENCODING=UTF-8
        restart: on-failure:10
        ports:
            - "5000:5000"
        logging:
            driver: awslogs
            options:
                awslogs-group: sms-gateway
                awslogs-region: ap-southeast-1
                awslogs-stream-prefix: "app"
    nginx:
        image: 540744822643.dkr.ecr.ap-southeast-1.amazonaws.com/nginx:latest
        links:
            - app:app
            - front:front
        container_name: nginx
        networks:
            - app
        environment:
            API_HOST: "service.wpdevelop.xyz"
            API_PORT: 5000
            FRONT_HOST: "customer.wpdevelop.xyz"
            FRONT_PORT: 8080

        ports:
            - "80:80"
            - "443:443"
        logging:
            driver: awslogs
            options:
                awslogs-group: sms-gateway
                awslogs-region: ap-southeast-1
                awslogs-stream-prefix: "nginx"
    db:
        image: 540744822643.dkr.ecr.ap-southeast-1.amazonaws.com/postgres:latest
        volumes:
            - postgres:/var/lib/postgresql/data:rw
        restart: on-failure:10
        networks:
            - app
        environment:
            POSTGRES_PASSWORD: "xxx@2020"
            POSTGRES_USER: webadmin
            POSTGRES_DB: smsgwdev
        ports:
            - "5432:5432"
        logging:
            driver: awslogs
            options:
                awslogs-group: sms-gateway
                awslogs-region: ap-southeast-1
                awslogs-stream-prefix: "db"
    redis:
        image: 540744822643.dkr.ecr.ap-southeast-1.amazonaws.com/radis:latest
        container_name: redis
        restart: on-failure:10
        networks:
            - app
        ports:
            - "6379:6379"
        logging:
            driver: awslogs
            options:
                awslogs-group: sms-gateway
                awslogs-region: ap-southeast-1
                awslogs-stream-prefix: "redis"
volumes:
    postgres:
networks:
    app:
        driver: bridge

ecs-params.yml

version: 1
task_definition:
    family: sms-gateway
    ecs_network_mode: bridge
    services:
        front:
            essential: true
            cpu_shares: 100
            mem_limit: 2147483648
            healthcheck:
                test: ["CMD-SHELL", "curl -f http://localhost || exit 1"]
                interval: 5s
                timeout: 10s
                retries: 3
                start_period: 30s
        app:
            essential: false
            cpu_shares: 100
            mem_limit: 2147483648
            healthcheck:
                test: ["CMD-SHELL", "curl -f http://localhost:5000 || exit 1"]
                interval: 5s
                timeout: 10s
                retries: 3
                start_period: 30s
            depends_on:
                - container_name: db
                  condition: HEALTHY
        nginx:
            essential: false
            cpu_shares: 100
            mem_limit: 2147483648
            healthcheck:
                test: ["CMD-SHELL", "curl -f http://localhost || exit 1"]
                interval: 5s
                timeout: 10s
                retries: 3
                start_period: 30s
        db:
            essential: false
            cpu_shares: 100
            mem_limit: 2147483648
            healthcheck:
                test: pg_isready -U webadmin -d smsgwdev
                interval: 5s
                timeout: 10s
                retries: 2
                start_period: 30s
        redis:
            essential: false
            cpu_shares: 100
            mem_limit: 2147483648
            healthcheck:
                test: ["CMD-SHELL", "redis-cli", "ping"]
                interval: 5s
                timeout: 10s
                retries: 2
                start_period: 30s

    efs_volumes:
        - name: postgres
          filesystem_id: fs-a4aa73e4
          transit_encryption: ENABLED
          access_point: fsap-007405b3e9bc7bc2f

如何让两个Postgres容器使用相同的pgdata？

Answer 1

首先，在EFS上运行Postgres不是一个好主意。我认为如果您在测试环境中需要一些快速和非常低负载的东西，那么EFS是很好的，但是EFS不是数据库引擎的正确后端。其次，在两个容器之间共享EFS是一个更糟糕的想法。这种设置是一个禁忌，因为每个数据库将同时获得对相同数据文件的非仲裁访问，而这不是Postgres应该如何工作的。

其次，您没有显式地调用它，但是您是否使用ecs-cli来部署它？如果是这样的话，我的建议是看看我们(AWS)与Docker一起引入的另一种机制，它依赖于new Docker Compose capabilities to deploy to the Cloud (e.g. ECS)。新版本的ecs-cli被称为Docker，它不再支持Copilot了。注意:新的Docker Compose集成目前不需要单独的ecs-params文件(尽管有讨论要引入一个)，并依赖于docker compose文件本身中的x-aws-扩展。

第三，不管这是不是一个好主意(不是！)受this示例的启发，下面这个简单的组合允许您部署2个共享相同数据目录的Postgres容器：

version: '3.4'
services:
  db1:
    container_name: db1
    image: postgres:latest
    environment:
      - POSTGRES_USER=me
      - POSTGRES_PASSWORD=mypassword
    volumes:
      - my-vol:/var/lib/postgresql/data
  db2:
    depends_on:
      - db1
    container_name: db2
    image: postgres:latest
    environment:
      - POSTGRES_USER=me
      - POSTGRES_PASSWORD=mypassword
    volumes:
      - my-vol:/var/lib/postgresql/data
  app:
    container_name: app
    image: nginx
volumes:
  my-vol:

如果您在ECS上下文(see the blog for more details)中使用docker compose up，您将获得3个ECS服务(1个app/nginx和2个DB服务)，其中2个DB服务坚持使用相同的EFS访问点。同样，这只是一个证明docker组合文件工作的学术示例。我不建议在任何有意义的部署中使用它。

更新:我刚刚注意到你在上面的作文中只有1个postgres。因此，我假设您有两个单独的组合文件，其中一个postgres DB坚持使用相同的文件系统。我上面所说的一切仍然适用，但请注意，这里有a limitation that will prevent you to even technically deploy this scenario。