blocks|key|2294672|text|这真的取决于你想要做什么。我喜欢阿帕奇星火，但斯托姆有一些历史。我相信，随着星火的流媒体功能的建立，它将成为一个有竞争力的解决方案。然而，在星火有一些重击用户(流)之前，仍然会有未知的bug。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2294673|你也可以考虑社区。星火有一个伟大的社区。我不确定暴风社区的水平，因为我通常是接收数据的人，而不是吞食者。我可以说，我们在项目中使用了Storm，我对实时分析和大量的流数据印象深刻。|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|E|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|F|8|@]|9|@]|A|$]]]|D|$]]

It really depends on what you are looking to do. I love Apache Spark, but Storm has some history. I am sure as the streaming capability in Spark is built out that it will become a competitive solution. However, until Spark has some heavy hitting users (for streaming) there will remain unknown bugs. 

You can also consider the community. Spark has a great community. I am not sure the level of the Storm community as I am usually the one receiving the data not handling the ingest. I can say we have used Storm on projects and I have been impressed with the real-time analysis and volumes of streaming data.

blocks|key|1149544|text|您可能还想看看Apache+Flink。它是Apache的一个新的竞争者，因为它可以同时进行实时分析和批处理分析。它还声称比Apache更快，但我认为主要取决于您的usecase。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1149545|好的是:+Apache项目在其开发版本中有一些Storm兼容层。通过这种方式，您可以使用现有的Storm项目，只需在Flink中运行它。对我来说，在一个非常简单的拓扑结构中，它至少比Storm快2倍。所以试试看！:)|entityMap|0|LINK|mutability|MUTABLE|url|https://flink.apache.org/^0|7|C|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@$A|N|B|O|1|P]]|C|$]]|$1|D|3|E|5|6|7|Q|8|@]|9|@]|C|$]]]|F|$G|$5|H|I|J|C|$K|L]]]]

You might also want to take a look at <a href="https://flink.apache.org/">Apache Flink</a>. It is a new contender to Apache Spark as it can do both, realtime and batch analysis. It also claims to be faster than Apache Spark, but I think mostly in depends on your usecase.

Good thing is: Apache Flink project has some Storm compatibility layer in its development version. In this way you can take your existing Storm project and simply run it inside Flink. For me it was at least 2 times faster than Storm in a very simple topology. So give it a try! :)

blocks|key|2294717|text|Apache和Apache比其他的更受欢迎，已经有很多关于Quora(风暴与火花，用例进行比较)的讨论。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2294718|就我个人而言，我认为火花是一个更好的选择。|entityMap|0|LINK|mutability|MUTABLE|url|http://www.quora.com/What-is-the-difference-between-Apache-Storm-and-Apache-Spark|1|http://www.quora.com/Are-there-any-use-cases-for-a-comparison-between-Storm-and-Spark-Streaming^0|Z|5|0|15|6|1|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@$A|P|B|Q|1|R]|$A|S|B|T|1|U]]|C|$]]|$1|D|3|E|5|6|7|V|8|@]|9|@]|C|$]]]|F|$G|$5|H|I|J|C|$K|L]]|M|$5|H|I|J|C|$K|N]]]]

Apache Storm and Apache Spark are more popular than the other ones, there are already many discussions on Quora(<a href="http://www.quora.com/What-is-the-difference-between-Apache-Storm-and-Apache-Spark" rel="nofollow">Storm vs Spark</a>, <a href="http://www.quora.com/Are-there-any-use-cases-for-a-comparison-between-Storm-and-Spark-Streaming" rel="nofollow">Use cases for comparison</a>).

Personally, I think Spark is a better choice.

blocks|key|1149449|text|如果您不想自己设置集群，则Amazon可能是流处理的另一个选择。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|entityMap^0^^$0|@$1|2|3|4|5|6|7|C|8|@]|9|@]|A|$]]]|B|$]]

Amazon Kinesis might be another choice for stream processing, if you don't want to set up the clusters by yourself.

blocks|key|1149510|text|“风暴”和“火花”都是很棒的工具。这取决于你的用例。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1149511|您想要快速解析巨大的数据流并将其存储到数据库中吗？使用Storm+(例如，计数tweet)。|unordered-list-item|1149512|对数据流进行分类器训练将是一项适合于星火的任务。有一个数据窗口，您正在工作，这将需要一段时间。|1149513|我还没有试过Flink，但它看起来更像火花。Spark有着RDD+(弹性分布式数据集)的一般概念，它也可以用于图、大型矩阵等。|1149514|如果你想写一个单词计数，你任何一个。但是谁想要一个单词计数(除了你好世界教程)？|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|M|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|N|8|@]|9|@]|A|$]]|$1|G|3|H|5|6|7|O|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|P|8|@]|9|@]|A|$]]]|K|$]]

Both Storm and Spark are great tools. It depends on your use-case. 

<ul>
<li>Do you want to quickly parse huge stream of data and store it into a database? Use Storm (e.g. counting tweets).</li>
<li>Training a classifier on a stream of data would be a task suitable for Spark. There's a data window on which you're working and it will take a while.</li>
</ul>

I haven't tried Flink, but it looks more similar to Spark. Spark has general concept of RDD (Resilient Distributed Datasets) that can be used also for graphs, huge matrices etc.

If you want to write a word count, you any of them. But who wants a word count (except from hello world tutorials)?

blocks|key|2294871|text|只是为了加入一个额外的选项，微软Azure的流分析。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2294872|就不同技术及其相对性能的比较而言，这些统计数据似乎很难得到，但是在这篇文章中有一些技术进行了有用的比较，尽管没有提到Analytics。我认为困难之处在于架构非常不同，这使得简单的度量难以生成。需要的是一些更健壮的度量工具，可以应用这些工具为各种操作提供一些度量。|2294873|我认为高德纳会有话要说，但我能找到的最接近的是他们的BI与解析象限，它只会倾斜地寻址流分析。|entityMap|0|LINK|mutability|MUTABLE|url|https://azure.microsoft.com/en-gb/services/stream-analytics/|1|https://www.infoq.com/articles/stream-processing-hadoop|2|http://www.kdnuggets.com/2016/02/gartner-2016-mq-analytics-platforms-gainers-losers.html^0|E|B|0|0|X|4|1|0|Q|7|2^^$0|@$1|2|3|4|5|6|7|S|8|@]|9|@$A|T|B|U|1|V]]|C|$]]|$1|D|3|E|5|6|7|W|8|@]|9|@$A|X|B|Y|1|Z]]|C|$]]|$1|F|3|G|5|6|7|10|8|@]|9|@$A|11|B|12|1|13]]|C|$]]]|H|$I|$5|J|K|L|C|$M|N]]|O|$5|J|K|L|C|$M|P]]|Q|$5|J|K|L|C|$M|R]]]]

Just to throw an extra option in, <a href="https://azure.microsoft.com/en-gb/services/stream-analytics/" rel="nofollow">Microsoft Azure's Stream Analytics</a>. 

As far as a comparison of the different technologies and their relative performance, those stats seem difficult to get, but there is a useful comparison of a few technologies in <a href="https://www.infoq.com/articles/stream-processing-hadoop" rel="nofollow">this article</a>, though doesn't mention Azure Stream Analytics. I think the difficulty is that the architectures are so different, that is makes simple metrics hard to produce. What is needed is some more robust measurement tools that can be applied to give a few metrics for various operations.

I would think that Gartner would have something to say, but the nearest I could find was their <a href="http://www.kdnuggets.com/2016/02/gartner-2016-mq-analytics-platforms-gainers-losers.html" rel="nofollow">BI and Analytics quadrant</a>, which only obliquely address stream analytics.

blocks|key|1149757|text|有很多选择是基于你想要做的事情。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1149758|你想怎么写逻辑？|header-two|1149759|如果您希望为您的逻辑编写代码(而不是寻找类似SQL的声明性语言)，那么您可以使用一个流处理引擎:+Apache、Apache、Apache、Apache或Kafka流。好消息是它们都是开源的。|1149760|其中，我打赌是芬克。它有一个相当好的架构，它可以处理其他人遇到的许多陷阱。特别是，如果你想做精确的一次处理，Fink是最好的选择。|1149761|但是，如果您只做简单的计数，编写java代码是可以的，但是如果您想要做一些严肃的事情，比如时间窗口、联接和时态事件序列模式等，则是一场噩梦(有关此参数的详细信息，请参见流式实时分析模式)。如果您想使用声明性语言来表示逻辑，那么您将需要使用复杂的事件处理，它允许您编写类似SQL的查询并执行复杂的操作。具体来说，这类查询支持时态操作符，如滑动窗口、连接、时序模式等。|1149762|如果您正在寻找开放源码解决方案，其中的选择是WSO2数据分析服务器，正式称为WSO2+CEP+(+Apache+)和Esper+(GPL)。(免责声明我为WSO2工作)。|offset|length|1149763|如果您可以支付，那么就有、和SQLStreams。如果需要的话，WSO2+DAS和Esper都有商业支持。保罗文森特是保持CEP市场调查，这提供了几乎所有CEP引擎的历史概况。|1149764|最近的Forrester报告调查了这些选择，您可以从15个实时流媒体分析平台中找到详细信息。|1149765|上面的流处理引擎增加了对SQL类查询语言的支持。然而，它们仍然落后于特征(例如时间序列模式)。正如我在https://softwareengineeringdaily.com/2016/02/04/stream-processing-vs-complex-event-processing/中所讨论的，CEP和流处理引擎正在合并。|1149766|可伸缩性|1149767|如果计划每秒处理数百万个事件，则需要设置一个复杂的分布式处理图。在这个主题上，流处理在历史上被构建为可扩展的地方有一个优势。然而，正如我提到的，这两个模型正在合并。|1149768|IoT+Analytics|1149769|如果您正在进行IoT分析，那么数据将创建一个时间序列。在这种情况下，通常需要执行诸如滑动窗口之类的时态操作。在这个用例中，您最好使用类似SQL的语言，因此我建议使用CEP。|1149770|流%2B批处理|1149771|最后一个问题是您是否想要进行批处理(+MapReduce样式处理)。如果是，请阅读关于Lambda架构和Kappa体系结构的文章。Apache和WSO2+DAS支持此场景。(可能还有其他人)。|entityMap|0|LINK|mutability|MUTABLE|url|http://wso2.com/smart-analytics|1|https://softwareengineeringdaily.com/2016/02/04/stream-processing-vs-complex-event-processing/^0|0|0|0|0|0|M|B|0|0|0|0|1F|2M|1|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1F|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|1G|8|@]|9|@]|A|$]]|$1|E|3|F|5|6|7|1H|8|@]|9|@]|A|$]]|$1|G|3|H|5|6|7|1I|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|1J|8|@]|9|@]|A|$]]|$1|K|3|L|5|6|7|1K|8|@]|9|@$M|1L|N|1M|1|1N]]|A|$]]|$1|O|3|P|5|6|7|1O|8|@]|9|@]|A|$]]|$1|Q|3|R|5|6|7|1P|8|@]|9|@]|A|$]]|$1|S|3|T|5|6|7|1Q|8|@]|9|@$M|1R|N|1S|1|1T]]|A|$]]|$1|U|3|V|5|D|7|1U|8|@]|9|@]|A|$]]|$1|W|3|X|5|6|7|1V|8|@]|9|@]|A|$]]|$1|Y|3|Z|5|D|7|1W|8|@]|9|@]|A|$]]|$1|10|3|11|5|6|7|1X|8|@]|9|@]|A|$]]|$1|12|3|13|5|D|7|1Y|8|@]|9|@]|A|$]]|$1|14|3|15|5|6|7|1Z|8|@]|9|@]|A|$]]]|16|$17|$5|18|19|1A|A|$1B|1C]]|1D|$5|18|19|1A|A|$1B|1E]]]]

There are a lot of choices based on what you are looking to do.

How do you want to write the logic?

If you are looking to write code for your logic ( and not looking for a SQL like declarative language) then you can use one of Stream Processing Engines: Apache Storm, Apache Spark, Apache Fink, Apache Samza or Kafka Streams. The good news is all of them are open source.

Among them, my bet is Fink. It has a pretty nice architecture which handles many pitfalls others stepped into. Especially, if you want to do exactly once processing, Fink is the best bet.

However, writing java code to do this is fine if you do simple counting, but a nightmare if you want to do serious things like time windows, join and temporal event sequence patterns etc. ( see Patterns for Streaming Realtime Analytics for details on this argument). If you want to express the logic using a declarative language, then you will want to use Complex Event Processing, which let you write SQL-like queries and do complex things. Specifically, these kinds of queries support temporal operators like sliding windows, joins, temporal sequence patterns etc.

If you are looking for an Open source solution, the among the choices are <a href="http://wso2.com/smart-analytics" rel="nofollow noreferrer">WSO2 DAS (Data Analytics Server)</a> formally known as WSO2 CEP ( Apache License) and Esper (GPL). ( Disclaimer I work for WSO2).

If you are OK to pay, then there is Tibco Stream Base, IBM Infosphere Streams, and SQLStreams. Both WSO2 DAS and Esper have commercial support if you need it. Paul Vincent is maintaining CEP market Survey which provides historical overview of almost all CEP engines.

Recent Forrester report surveyed the choices and you can find the details from 15 "True" Streaming Analytics Platforms For Real-Time Everything.

Above stream processing engines are adding support for SQL like query languages. However, they still lag behind with the features ( e.g. temporal sequence patterns). As I discussed in <a href="https://softwareengineeringdaily.com/2016/02/04/stream-processing-vs-complex-event-processing/" rel="nofollow noreferrer">https://softwareengineeringdaily.com/2016/02/04/stream-processing-vs-complex-event-processing/</a>, CEP and Stream processing engines are merging.

Scalability

If you plan to process millions of events per second, then you need to setup a complex distributed processing graph. On this topic, stream processing has an advantage where they are historically built to scale. However, as I mentioned, these two models are merging.

IoT Analytics

If you are doing an IoT analytics, then the data creates a time series. In that case, often you need to do temporal operations such as sliding windows. On that use case, you are better off with an SQL like language hence I recommend to go for CEP.

Streaming + Batch

The final twist is whether you want to do batch ( MapReduce style processing) as well. If yes, read about Lambda Architecture and Kappa architecture. Apache Fink and WSO2 DAS supports this scenario. ( and there may be others).

blocks|key|2294765|text|Flink是一个即将到来的好，相当于火花(甚至更好)，因为它处理每个元组，不像微批概念火花流。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|entityMap^0^^$0|@$1|2|3|4|5|6|7|C|8|@]|9|@]|A|$]]]|B|$]]

Flink is an upcoming good equivalent to Spark (or even better) as it process each tuples unlike micro batch concept of spark streaming.

I found that Apache-Storm, Apache-Spark, Apache-Flink and TIBCO StreamBase are some powerful frameworks for stream processing. but I don't know which one of them has the best performance and capabilities.

I know Apache-Spark and Apache-Flink are two general purpose frameworks that support stream processing. but Apache-Storm and TIBCO StreamBase are built for stream processing specially. Is there any considerable advantage between these frameworks?

Thanks

What is the best Big-Data framework for stream processing?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云AI代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

功能1上新10个字符

功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符。

功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符。

功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符

功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符

功能4上新

文章&问答评论现已支持表情

全新交互，全新视觉，新增快捷键、悬浮工具栏、高亮块等功能并同时优化现有功能，全面提升创作效率和体验

社区富文本编辑器全新改版！诚邀体验～ 

精选全网热门MCP server，让你的AI更好用 🚀

💥开发者 MCP广场重磅上线！

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

我发现Apache、Apache、Apache和TIBCO StreamBase是一些强大的流处理框架.但我不知道哪一个有最好的性能和能力。我知道Apache和Apache是两个支持流处理的通用框架。但是Apache和TIBCO StreamBase是专门为流处理而建的.这些框架之间有什么相当大的优势吗？谢谢

问用于流处理的最佳大数据框架是什么？
EN

回答 8

Data Science用户

Data Science用户

Data Science用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问用于流处理的最佳大数据框架是什么？EN

回答 8

Data Science用户

Data Science用户

Data Science用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问用于流处理的最佳大数据框架是什么？
EN