blocks|key|2299134|text|最后，Google发布了云批处理(目前在Beta中)，这完全是你想要的。您推送作业(容器或脚本)，然后它运行。就这么简单。https://cloud.google.com/batch/docs/get-started|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2299135|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/batch/docs/get-started^0|1P|1B|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

Finally Google released (in Beta for the moment) Cloud Batch which does exactly what you want.
You push jobs (containers or scripts) and it runs. Simple as that.
<a href="https://cloud.google.com/batch/docs/get-started" rel="nofollow noreferrer">https://cloud.google.com/batch/docs/get-started</a>

blocks|key|2298897|text|注意:云运行和云功能现在可以持续60分钟。如果你有一份多小时的工作，下面的答案仍然是一个可行的方法。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2298898|顶点人工智能训练是无服务器和长寿的。将批处理代码包装到Docker容器中，按gcr.io，然后执行以下操作：|2298899|gcloud+ai+custom-jobs+create+\
++--region=LOCATION+\
++--display-name=JOB_NAME+\
++--worker-pool-spec=machine-type=MACHINE_TYPE,replica-count=REPLICA_COUNT,executor-image-uri=EXECUTOR_IMAGE_URI,local-package-path=WORKING_DIRECTORY,script=SCRIPT_PATH|code-block|syntax|javascript|2298900|您可以运行任何任意的Docker容器-它不一定是一个机器学习的工作。详情见：|2298901|job-gcloud|offset|length|2298902|今天，您还可以使用Cloud：https://cloud.google.com/batch/docs/get-started#create-basic-job|2298903|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/vertex-ai/docs/training/create-custom-job#create_custom_job-gcloud|1|https://cloud.google.com/batch/docs/get-started#create-basic-job^0|0|0|0|0|0|A|0|0|F|1S|1|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|11|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|12|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|13|8|@]|9|@]|A|$]]|$1|K|3|L|5|6|7|14|8|@]|9|@$M|15|N|16|1|17]]|A|$]]|$1|O|3|P|5|6|7|18|8|@]|9|@$M|19|N|1A|1|1B]]|A|$]]|$1|Q|3|-4|5|6|7|1C|8|@]|9|@]|A|$]]]|R|$S|$5|T|U|V|A|$W|X]]|Y|$5|T|U|V|A|$W|Z]]]]

Note: Cloud Run and Cloud Functions can now last up to 60 minutes. The answer below remains a viable approach if you have a multi-hour job.
Vertex AI Training is serverless and long-lived. Wrap your batch processing code in a Docker container, push to gcr.io and then do:
<pre><code>gcloud ai custom-jobs create \
 --region=LOCATION \
 --display-name=JOB_NAME \
 --worker-pool-spec=machine-type=MACHINE_TYPE,replica-count=REPLICA_COUNT,executor-image-uri=EXECUTOR_IMAGE_URI,local-package-path=WORKING_DIRECTORY,script=SCRIPT_PATH
</code></pre>
You can run any arbitrary Docker container — it doesn’t have to be a machine learning job. For details, see:
<a href="https://cloud.google.com/vertex-ai/docs/training/create-custom-job#create_custom_job-gcloud" rel="nofollow noreferrer">https://cloud.google.com/vertex-ai/docs/training/create-custom-job#create_custom_job-gcloud</a>
Today you can also use Cloud Batch: <a href="https://cloud.google.com/batch/docs/get-started#create-basic-job" rel="nofollow noreferrer">https://cloud.google.com/batch/docs/get-started#create-basic-job</a>

blocks|key|278930|text|Google没有提供与AWS批处理类似的产品(参见比较)。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|278931|相反，您需要使用云任务或Pub/Sub将工作委托给另一个产品，如Compute+Engine，但这缺乏以“无服务器”的方式进行此操作的能力。|278932|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/docs/compare/aws/#service_comparisons^0|P|2|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

Google Cloud does not offer a comparable product to AWS Batch (see <a href="https://cloud.google.com/docs/compare/aws/#service_comparisons" rel="noreferrer">https://cloud.google.com/docs/compare/aws/#service_comparisons</a>).

Instead you'll need to use Cloud Tasks or Pub/Sub to delegate the work to another product, such as Compute Engine, but this lacks the ability to do this in a "serverless" way.

blocks|key|279066|text|这个回答到一个如何使GCE实例在其已部署的容器完成后停止？也适用于您：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|279067|简言之：|279068|首先，对批处理过程进行修改。|unordered-list-item|279069|然后，创建一个实例：|279070|使用容器视觉化图像|279071|并使用启动脚本来提取您的对接映像，运行它，并关闭机器的最后。|279072|279073|entityMap|0|LINK|mutability|MUTABLE|url|https://stackoverflow.com/a/58215421/6760372|1|https://stackoverflow.com/questions/52748332/how-to-make-gce-instance-stop-when-its-deployed-container-finishes^0|2|2|0|7|M|1|0|0|0|1|1|1|0^^$0|@$1|2|3|4|5|6|7|Z|8|@]|9|@$A|10|B|11|1|12]|$A|13|B|14|1|15]]|C|$]]|$1|D|3|E|5|6|7|16|8|@]|9|@]|C|$]]|$1|F|3|G|5|H|7|17|8|@]|9|@]|C|$]]|$1|I|3|J|5|H|7|18|8|@]|9|@]|C|$]]|$1|K|3|L|5|H|7|19|8|@]|9|@]|C|$]]|$1|M|3|N|5|H|7|1A|8|@]|9|@]|C|$]]|$1|O|3|-4|5|6|7|1B|8|@]|9|@]|C|$]]|$1|P|3|-4|5|6|7|1C|8|@]|9|@]|C|$]]]|Q|$R|$5|S|T|U|C|$V|W]]|X|$5|S|T|U|C|$V|Y]]]]

This <a href="https://stackoverflow.com/a/58215421/6760372">answer</a> to a <a href="https://stackoverflow.com/questions/52748332/how-to-make-gce-instance-stop-when-its-deployed-container-finishes">How to make GCE instance stop when its deployed container finishes?</a> will work for you as well:
In short:
<ul>
<li>First dockerize your batch process.</li>
<li>Then, create an instance:
<ul>
<li>Using a container-optmized image</li>
<li>And using a Startup script that pulls your docker image, runs it, and shutdown the machine at the end.</li>
</ul>
</li>
</ul>

blocks|key|2298920|text|我也面临过同样的问题。就我而言，我的目标是：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2298921|云计划程序通过推到Pub/Sub开始这项工作。|ordered-list-item|2298922|Pub/Sub触发云函数。|2298923|云功能挂载计算引擎实例。|2298924|计算引擎运行批处理工作负载，并在完成后自动关闭实例。您可以在媒体：https://link.medium.com/1K3NsElGYZ上通过帖子阅读|offset|length|2298925|也许能帮你开始工作。还有一个后续文章展示了如何在Compute实例中使用Docker容器：https://medium.com/google-cloud/serverless-batch-workload-on-gcp-adding-docker-and-container-registry-to-the-mix-558f925e1de1|2298926|entityMap|0|LINK|mutability|MUTABLE|url|https://link.medium.com/1K3NsElGYZ|1|https://medium.com/google-cloud/serverless-batch-workload-on-gcp-adding-docker-and-container-registry-to-the-mix-558f925e1de1^0|0|0|0|0|X|Y|0|0|19|3H|1|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|Z|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|10|8|@]|9|@]|A|$]]|$1|G|3|H|5|D|7|11|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|12|8|@]|9|@$K|13|L|14|1|15]]|A|$]]|$1|M|3|N|5|6|7|16|8|@]|9|@$K|17|L|18|1|19]]|A|$]]|$1|O|3|-4|5|6|7|1A|8|@]|9|@]|A|$]]]|P|$Q|$5|R|S|T|A|$U|V]]|W|$5|R|S|T|A|$U|X]]]]

I have faced the same problem. in my case I went for:

<ol>
<li>Cloud Scheduler to start the job by pushing to Pub/Sub.</li>
<li>Pub/Sub triggers Cloud Functions.</li>
<li>Cloud Functions mounting a Compute Engine instance.</li>
<li>Compute Engine runs the batch workload and auto kills the instance
once it’s done. You can read by post on medium:
<a href="https://link.medium.com/1K3NsElGYZ" rel="nofollow noreferrer">https://link.medium.com/1K3NsElGYZ</a></li>
</ol>

It might help you get started. There's also a follow up post showing how to use a Docker container inside the Compute Engine instance: <a href="https://medium.com/google-cloud/serverless-batch-workload-on-gcp-adding-docker-and-container-registry-to-the-mix-558f925e1de1" rel="nofollow noreferrer">https://medium.com/google-cloud/serverless-batch-workload-on-gcp-adding-docker-and-container-registry-to-the-mix-558f925e1de1</a>

blocks|key|1663573|text|您可以使用云运行。在撰写这篇文章时，云运行(完全管理)的超时时间增加到60分钟，但在beta版。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1663574|https://cloud.google.com/run/docs/configuring/request-timeout|offset|length|1663575|重要:尽管云运行(完全管理)的最大超时时间为60分钟，但通常只有15分钟或更短的超时可用:设置大于15分钟的超时是Beta特性。|blockquote|1663576|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|1P|0|0|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Q|8|@]|9|@$D|R|E|S|1|T]]|A|$]]|$1|F|3|G|5|H|7|U|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|V|8|@]|9|@]|A|$]]]|J|$K|$5|L|M|N|A|$O|C]]]]

You can use Cloud Run. At the time of writing this, the timeout of Cloud Run (fully managed) is increased to 60 minutes, but in beta.
<a href="https://cloud.google.com/run/docs/configuring/request-timeout" rel="nofollow noreferrer">https://cloud.google.com/run/docs/configuring/request-timeout</a>
<blockquote>
Important: Although Cloud Run (fully managed) has a maximum timeout of 60 minutes, only timeouts of 15 minutes or less are generally available: setting timeouts greater than 15 minutes is a Beta feature.
</blockquote>

blocks|key|279099|text|批量计算的另一种选择是使用谷歌云生命科学。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|279100|使用Cloud的一个示例应用程序是dsub。|279101|或者去看云生命科学Quickstart文档。|279102|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/life-sciences|1|https://github.com/DataBiosphere/dsub|2|https://cloud.google.com/life-sciences/docs/quickstart^0|D|7|0|0|H|4|1|0|4|H|2|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@$A|U|B|V|1|W]]|C|$]]|$1|D|3|E|5|6|7|X|8|@]|9|@$A|Y|B|Z|1|10]]|C|$]]|$1|F|3|G|5|6|7|11|8|@]|9|@$A|12|B|13|1|14]]|C|$]]|$1|H|3|-4|5|6|7|15|8|@]|9|@]|C|$]]]|I|$J|$5|K|L|M|C|$N|O]]|P|$5|K|L|M|C|$N|Q]]|R|$5|K|L|M|C|$N|S]]]]

Another alternative for batch computing is using <a href="https://cloud.google.com/life-sciences" rel="nofollow noreferrer">Google Cloud Lifesciences</a>.
An example application using Cloud Life Sciences is <a href="https://github.com/DataBiosphere/dsub" rel="nofollow noreferrer">dsub</a>.
Or see the <a href="https://cloud.google.com/life-sciences/docs/quickstart" rel="nofollow noreferrer">Cloud Life Sciences Quickstart documentation</a>.

blocks|key|2299121|text|我发现自己正在寻找解决这个问题的方法，并以名为gcp-运行批处理的可重用工具的形式，构建了与mesmacosta在另一个答案中描述的类似的东西。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2299122|如果您可以将工作负载打包到Docker映像中，那么可以使用GCP-run批处理运行它。一旦触发，它将执行以下操作：|2299123|创建一个新的VM|ordered-list-item|2299124|在VM启动时，docker+run指定的映像|style|CODE|2299125|当docker+run退出时，VM将被删除|2299126|支持的一些特性：|2299127|从命令行调用批处理工作负载，或作为云函数部署并以这种方式调用(例如，通过云调度程序触发批处理工作负载)|unordered-list-item|2299128|stdout和stderr将通过管道传输到云日志记录。|2299129|环境变量可以由调用方指定，也可以从秘密管理器中提取|2299130|下面是一个命令行调用示例：|2299131|$+gcp-runbatch+\
++--project-id=long-octane-350517+\
++--zone=us-central1-a+\
++--service-account=1234567890-compute@developer.gserviceaccount.com+\
++hello-world
Successfully+started+instance+runbatch-38408320.+To+tail+batch+logs+run:
CLOUDSDK_PYTHON_SITEPACKAGES=1+gcloud+beta+--project=long-octane-350517
logging+tail+'logName="projects/long-octane-350517/logs/runbatch"+AND
resource.labels.instance_id="runbatch-38408320"'+--format='get(text_payload)'|code-block|syntax|javascript|2299132|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/bitcomplete/gcp-runbatch^0|N|9|0|0|0|0|7|A|0|1|A|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1C|8|@]|9|@$A|1D|B|1E|1|1F]]|C|$]]|$1|D|3|E|5|6|7|1G|8|@]|9|@]|C|$]]|$1|F|3|G|5|H|7|1H|8|@]|9|@]|C|$]]|$1|I|3|J|5|H|7|1I|8|@$A|1J|B|1K|K|L]]|9|@]|C|$]]|$1|M|3|N|5|H|7|1L|8|@$A|1M|B|1N|K|L]]|9|@]|C|$]]|$1|O|3|P|5|6|7|1O|8|@]|9|@]|C|$]]|$1|Q|3|R|5|S|7|1P|8|@]|9|@]|C|$]]|$1|T|3|U|5|S|7|1Q|8|@]|9|@]|C|$]]|$1|V|3|W|5|S|7|1R|8|@]|9|@]|C|$]]|$1|X|3|Y|5|6|7|1S|8|@]|9|@]|C|$]]|$1|Z|3|10|5|11|7|1T|8|@]|9|@]|C|$12|13]]|$1|14|3|-4|5|6|7|1U|8|@]|9|@]|C|$]]]|15|$16|$5|17|18|19|C|$1A|1B]]]]

I found myself looking for a solution to this problem and built something similar to what mesmacosta has described in a different answer, in the form of a reusable tool called <a href="https://github.com/bitcomplete/gcp-runbatch" rel="nofollow noreferrer">gcp-runbatch</a>.
If you can package your workload into a Docker image then you can run it using gcp-runbatch. When triggered, it will do the following:
<ol>
<li>Create a new VM</li>
<li>On VM startup, <code>docker run</code> the specified image</li>
<li>When the <code>docker run</code> exits, the VM will be deleted</li>
</ol>
Some features that are supported:
<ul>
<li>Invoke batch workload from the command line, or deploy as a Cloud Function and invoke that way (e.g. to trigger batch workloads via Cloud Scheduler)</li>
<li>stdout and stderr will be piped to Cloud Logging</li>
<li>Environment variables can be specified by the invoker, or pulled from Secret Manager</li>
</ul>
Here's an example command line invocation:
<pre><code>$ gcp-runbatch \
 --project-id=long-octane-350517 \
 --zone=us-central1-a \
 --service-account=1234567890-compute@developer.gserviceaccount.com \
 hello-world
Successfully started instance runbatch-38408320. To tail batch logs run:
CLOUDSDK_PYTHON_SITEPACKAGES=1 gcloud beta --project=long-octane-350517
logging tail 'logName=&quot;projects/long-octane-350517/logs/runbatch&quot; AND
resource.labels.instance_id=&quot;runbatch-38408320&quot;' --format='get(text_payload)'
</code></pre>

blocks|key|279209|text|GCP于22年7月推出了新的"批处理“服务，它基本上是与一些实用程序一起打包的计算引擎，可以轻松地生产批处理作业--包括定义所需的资源、可执行文件(基于脚本或容器)，以及定义运行计划。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|279210|还没有使用它，但似乎是一个非常适合的批处理工作，需要超过1小时。|279211|entityMap|0|LINK|mutability|MUTABLE|url|https://cloud.google.com/blog/products/compute/new-batch-service-processes-batch-jobs-on-google-cloud^0|F|3|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]]|C|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|C|$]]]|G|$H|$5|I|J|K|C|$L|M]]]]

GCP launched their new &quot;<a href="https://cloud.google.com/blog/products/compute/new-batch-service-processes-batch-jobs-on-google-cloud" rel="nofollow noreferrer">Batch</a>&quot; service in July '22. It basically Compute Engine packaged with some utilities to easily productionize a batch job -- including defining required resources, executables (script or container-based), and define a run schedule.
Haven't used it yet, but seems like a great fit for batch jobs that take over 1 hr.

I have a batch job that takes a couple of hours to run. How can I run this in a serverless way on Google Cloud?

AppEngine, Cloud Functions, and Cloud Run are limited to 10-15 minutes. I don't want to rewrite my code in Apache Beam.

Is there an equivalent to AWS Batch on Google Cloud?

How do I run a serverless batch job in Google Cloud

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云AI代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

功能1上新10个字符

功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符。

功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符。

功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符

功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符

功能4上新

文章&问答评论现已支持表情

全新交互，全新视觉，新增快捷键、悬浮工具栏、高亮块等功能并同时优化现有功能，全面提升创作效率和体验

社区富文本编辑器全新改版！诚邀体验～ 

精选全网热门MCP server，让你的AI更好用 🚀

💥开发者 MCP广场重磅上线！

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

我有一个批处理任务，需要几个小时才能运行。我怎样才能在Google上以一种无服务器的方式运行？AppEngine、云函数和云运行仅限10-15分钟.我不想在Apache中重写我的代码。谷歌云上有相当于AWS批处理的吗？

问如何在Google中运行无服务器批处理作业
EN

回答 9

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在Google中运行无服务器批处理作业EN

回答 9

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在Google中运行无服务器批处理作业
EN