文档建议反馈控制台
首页
学习
活动
专区
圈层
工具
MCP广场
文章/答案/技术大牛
发布
社区首页 >问答首页 >在Windows上使用"storm“命令远程部署Storm拓扑

问在Windows上使用"storm“命令远程部署Storm拓扑
EN

Stack Overflow用户
提问于 2015-10-06 17:40:11
回答 1查看 5.2K关注 0票数 2

我试图使用Windows上的storm命令将风暴拓扑部署到远程计算机上

代码语言：javascript
复制
storm jar jarName.jar mainClass

下面是我的storm.yaml文件中的内容：

代码语言：javascript
复制
storm.zookeeper.servers:
     - "HostName"

nimbus.host: "HostName"
nimbus.thrift.port: 6627

但是，每当我尝试部署时，它总是尝试部署到本地机器上。

编辑

我尝试过放置以下配置，但它不起作用：

代码语言：javascript
复制
conf.put(Config.NIMBUS_HOST, "HostName");
conf.put(Config.NIMBUS_THRIFT_PORT,6627);
conf.put(Config.STORM_ZOOKEEPER_SERVERS,Arrays.asList(new String[]{"HostName"}));
conf.put(Config.STORM_ZOOKEEPER_PORT,2181);

我还在命令行上尝试了-c选项，如：

代码语言：javascript
复制
storm -c nimbus.host=HostName jar jarName.jar mainClass

但它不起作用；我得到：

代码语言：javascript
复制
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

我试着将-c部件放置在不同的位置，但似乎没有什么效果。

另外，我得到

代码语言：javascript
复制
The field Config.NIMBUS_HOST is deprecated

当我设定：

代码语言：javascript
复制
conf.put(Config.NIMBUS_HOST, "HostName");

我使用的是Storm版本的0.10.0-beta

EDIT2

在我使用IDE之后，提交工作非常好

代码语言：javascript
复制
System.setProperty("storm.jar", "path to jar");

但是，它不能从命令行工作。

Edit3

谢谢马特的帮助。我能够使用eclipse执行的java命令部署拓扑。不过，我仍然无法使用storm命令进行部署。它抱怨拓扑已经存在，即使我在Storm上看不到任何东西。此外，当我使用Eclipse或命令行中的Java命令进行部署时，拓扑将成功部署。我应该如何比较执行java命令和difference命令之间的区别呢？我试图回显cmd文件这一节中的变量。

代码语言：javascript
复制
 if %storm-command% == jar (
    set STORM_OPTS=%STORM_CLIENT_OPTS% %STORM_OPTS% -Dstorm.jar=%2
    echo %STORM_OPTS%
    set CLASSPATH=%CLASSPATH%;%2

    echo %CLASSPATH%

    set CLASS=%3
    set args=%4
    goto start
    :start
    shift
    if [%4] == [] goto done
    set args=%args% %4

    echo args: %args%

    goto start

    :done
    echo %args%
    set storm-command-arguments=%args%

  )

但我没看到什么不寻常的东西。此外，我还在堆栈跟踪中看到以下异常

代码语言：javascript
复制
    Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])

通知

代码语言：javascript
复制
 topologies:[]

然后，它会抱怨拓扑名称存在。

代码语言：javascript
复制
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
        at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)

        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)

        at mainClass.submitTopology(mainClass.java:127)
        at mainClass.main(mainClass.java:57)
Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
        at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
        at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
read(ClusterSummary.java:613)
        at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
read(ClusterSummary.java:549)
        at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)

        at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
resultStandardScheme.read(Nimbus.java:16546)
        at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
resultStandardScheme.read(Nimbus.java:16531)
        at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
        at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)

        at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
        at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
        ... 5 more

最后，这里是整个堆栈跟踪。

代码语言：javascript
复制
 X:\abc\xyz\target>ERROR StatusLogger Invalid URL C:/asd/ewt/apache-storm 0.10.0-beta/log4j2/cluster.xml java.net.MalformedURLException: unknown protocol: c
            at java.net.URL.<init>(URL.java:593)
            at java.net.URL.<init>(URL.java:483)
            at java.net.URL.<init>(URL.java:432)
            at java.net.URI.toURL(URI.java:1089)
            at org.apache.logging.log4j.core.config.ConfigurationFactory.getInputFromUri(ConfigurationFactory.java:296)
            at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:395)
            at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:254)
            at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:419)
            at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:138)
            at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:147)
            at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
            at org.apache.logging.log4j.LogManager.getContext(LogManager.java:175)
            at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:102)
            at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
            at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
            at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
            at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:277)
            at org.apache.log4j.Category.<init>(Category.java:56)
            at org.apache.log4j.Logger.<init>(Logger.java:35)
            at org.apache.log4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:59)
            at org.apache.log4j.Logger.getLogger(Logger.java:39)
            at mainClass.<clinit>(mainClass.java:32)
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field nimbus_uptime_secs' is unset!Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
                at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
                at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
                at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)

                at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)

                at mainClass.submitTopology(mainClass.java:127)
                at mainClass.main(mainClass.java:57)
        Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
                at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
                at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
        read(ClusterSummary.java:613)
                at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
        read(ClusterSummary.java:549)
                at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)

                at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
        resultStandardScheme.read(Nimbus.java:16546)
                at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
        resultStandardScheme.read(Nimbus.java:16531)
                at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
                at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)

                at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
                at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
                at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
                ... 5 more

顺便问一下，堆栈跟踪顶部的奇怪日志异常/错误是什么？这是否与暴风jar命令的怪异行为有关？

如果我做了

代码语言：javascript
复制
java -cp jarName.jar;C:\abc\asdasd\apache-storm-0.10.0-beta\lib\* mainClass

我得到了

代码语言：javascript
复制
513  [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
582  [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
590  [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for MD
5-digest: -6100635753051151148:-8346710213172614614
591  [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
599  [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeM
s [2000] the maxSleepTimeMs [60000] the maxRetries [5]
626  [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeM
s [2000] the maxSleepTimeMs [60000] the maxRetries [5]
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protoc
ol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName, uptime_secs:83775, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
        at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)

        at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)

        at com.rbccm.th2015.storm.SampleTopologyX.submitTopology(SampleTopologyX.java:127)
        at com.rbccm.th2015.storm.SampleTopologyX.main(SampleTopologyX.java:57)
Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName,uptime_secs:83775, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
        at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
        at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.read(ClusterSummary.java:613)
        at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.read(ClusterSummary.java:549)
        at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)

        at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_resultStandardScheme.read(Nimbus.java:16546)
        at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_resultStandardScheme.read(Nimbus.java:16531)
        at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
        at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)

        at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
        at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
        at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
        ... 5 more

所以，它抱怨同样的事情，但通过不同的途径。这跟我的YAML有关吗？nimbus_uptime_secs的业务是什么？

最终编辑

好了，是时候放下这个了。至少现在看来，sotrm命令不适合用于windows。我可以清楚地看到，storm.py试图在Windows中执行的命令与在Linux中尝试执行的命令有区别。基本上，这些路径在窗口中被搞砸了，这一定是导致所有这些奇怪异常的原因。

谢谢马特所有的帮助和支持。我真的学到了很多。

java
apache-storm
EN

回答 1

Stack Overflow用户

回答已采纳

发布于 2015-10-06 21:24:06

通过storm jar运行拓扑只在Linux上工作(而不是在Windows上)。"storm“命令是一个Python脚本，它假定在Linux上运行(Storm官方不支持窗口)。

但是，“风暴”命令只在适当的设置下启动常规的java命令。如果一个拓扑是通过IDE提交的，IDE会做同样的事情。如果使用Eclipse，则可以提取Eclipse: export running configuration中描述的已使用配置

注意，您需要设置JVM参数-Dstorm.jar=path\to\myJar.jar；否则，StormSubmitter将找不到应该提交到Storm集群的jar。

提取java命令之后，它应该允许您通过Windows上的命令行提交拓扑。如果IDE不支持提取命令，则仍然可以手动将命令放在一起(即使这是一个麻烦的解决方案)。

它将类似于：

代码语言：javascript
复制
java -Dstorm.jar=yourJarFile.jar
     -classpath yourJarFile.jar:
                /home/mjsax/.m2/repository/org/apache/storm/storm-core/0.11.0-SNAPSHOT/storm-core-0.11.0-SNAPSHOT.jar:
                /home/mjsax/.m2/repository/com/esotericsoftware/kryo/kryo/2.21/kryo-2.21.jar:
                /home/mjsax/.m2/repository/com/esotericsoftware/reflectasm/reflectasm/1.07/reflectasm-1.07-shaded.jar:
                /home/mjsax/.m2/repository/org/ow2/asm/asm/4.0/asm-4.0.jar:
                /home/mjsax/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:
                /home/mjsax/.m2/repository/org/objenesis/objenesis/1.2/objenesis-1.2.jar:
                /home/mjsax/.m2/repository/org/clojure/clojure/1.6.0/clojure-1.6.0.jar:
                /home/mjsax/.m2/repository/clj-time/clj-time/0.8.0/clj-time-0.8.0.jar:
                /home/mjsax/.m2/repository/joda-time/joda-time/2.3/joda-time-2.3.jar:
                /home/mjsax/.m2/repository/compojure/compojure/1.1.3/compojure-1.1.3.jar:
                /home/mjsax/.m2/repository/org/clojure/core.incubator/0.1.0/core.incubator-0.1.0.jar:
                /home/mjsax/.m2/repository/org/clojure/tools.macro/0.1.0/tools.macro-0.1.0.jar:
                /home/mjsax/.m2/repository/clout/clout/1.0.1/clout-1.0.1.jar:
                /home/mjsax/.m2/repository/ring/ring-core/1.1.5/ring-core-1.1.5.jar:
                /home/mjsax/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar:
                /home/mjsax/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:
                /home/mjsax/.m2/repository/commons-fileupload/commons-fileupload/1.2.1/commons-fileupload-1.2.1.jar:
                /home/mjsax/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:
                /home/mjsax/.m2/repository/hiccup/hiccup/0.3.6/hiccup-0.3.6.jar:
                /home/mjsax/.m2/repository/ring/ring-devel/1.3.0/ring-devel-1.3.0.jar:
                /home/mjsax/.m2/repository/clj-stacktrace/clj-stacktrace/0.2.7/clj-stacktrace-0.2.7.jar:
                /home/mjsax/.m2/repository/ns-tracker/ns-tracker/0.2.2/ns-tracker-0.2.2.jar:
                /home/mjsax/.m2/repository/org/clojure/tools.namespace/0.2.4/tools.namespace-0.2.4.jar:
                /home/mjsax/.m2/repository/org/clojure/java.classpath/0.2.2/java.classpath-0.2.2.jar:
                /home/mjsax/.m2/repository/ring/ring-jetty-adapter/1.3.0/ring-jetty-adapter-1.3.0.jar:
                /home/mjsax/.m2/repository/ring/ring-servlet/1.3.0/ring-servlet-1.3.0.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/jetty-server/7.6.13.v20130916/jetty-server-7.6.13.v20130916.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/2.5.0.v201103041518/javax.servlet-2.5.0.v201103041518.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/jetty-continuation/7.6.13.v20130916/jetty-continuation-7.6.13.v20130916.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/jetty-http/7.6.13.v20130916/jetty-http-7.6.13.v20130916.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/jetty-io/7.6.13.v20130916/jetty-io-7.6.13.v20130916.jar:
                /home/mjsax/.m2/repository/org/eclipse/jetty/jetty-util/7.6.13.v20130916/jetty-util-7.6.13.v20130916.jar:
                /home/mjsax/.m2/repository/ring/ring-json/0.3.1/ring-json-0.3.1.jar:
                /home/mjsax/.m2/repository/cheshire/cheshire/5.3.1/cheshire-5.3.1.jar:
                /home/mjsax/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.3.1/jackson-core-2.3.1.jar:
                /home/mjsax/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.3.1/jackson-dataformat-smile-2.3.1.jar:
                /home/mjsax/.m2/repository/tigris/tigris/0.1.1/tigris-0.1.1.jar:
                /home/mjsax/.m2/repository/org/clojure/tools.logging/0.2.3/tools.logging-0.2.3.jar:
                /home/mjsax/.m2/repository/com/googlecode/disruptor/disruptor/2.10.4/disruptor-2.10.4.jar:
                /home/mjsax/.m2/repository/org/apache/logging/log4j/log4j-api/2.1/log4j-api-2.1.jar:
                /home/mjsax/.m2/repository/org/apache/logging/log4j/log4j-core/2.1/log4j-core-2.1.jar:
                /home/mjsax/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.1/log4j-slf4j-impl-2.1.jar:
                /home/mjsax/.m2/repository/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.jar:
                /home/mjsax/.m2/repository/org/slf4j/log4j-over-slf4j/1.6.6/log4j-over-slf4j-1.6.6.jar:
                /home/mjsax/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:
                /home/mjsax/.m2/repository/args4j/args4j/2.0.16/args4j-2.0.16.jar
    package.to.your.MainClass
票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：

https://stackoverflow.com/questions/32976198

复制
相关文章

相似问题

领券

腾讯云开发者

扫码关注腾讯云开发者

扫码关注腾讯云开发者

领取腾讯云代金券

热门产品

热门推荐

更多推荐

Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有 

深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 粤公网安备44030502008569号

腾讯云计算（北京）有限责任公司 京ICP证150476号 |  京ICP备11018762号

问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档