Microservice CQRS独立构建(写)查询模型和读取模型

Question

以下情景

每分钟从队列中接收60000条消息。REST每分钟10次为这些消息提供数据服务。

我有一个带有事件源和CQRS的微服务架构。所以我的命令已经与查询部分分开了。问题在于同步查询和查询查询，而不是命令部分。

每隔几分钟就会发送大约60000条命令，并使用event-sourcing模式将其存储为事件。通过CQRS，实际数据(而不是事件)被同步到另一个服务，后者将其存储在数据库中。同时，每隔几分钟只读取一次数据。

换句话说。这个服务接收60000个写操作，但只有十几个读操作。

我真的很想坚持微服务的设计模式，也就是one database per service，但是由于缩放的原因，我认为在我的场景中这是不可行的。写入数据库比读取数据库要大得多。

我看到了一个similar question，但答案是建议使用CQRS，这一点我已经实现了。以前有人告诉我要删除事件源，但这仍然给我留下了6万条文字和10条读物。

我应该用什么架构来独立地缩放读写呢？我正在考虑创建两个独立的服务，但这将违背one database per service模式。

Answer 1

假设

据我所知，问题是您的写入模型需要尽快反映读取模型状态，而且由于您每分钟只有10次读取，所以它们需要实时地或接近实时地反映真实状态。

问题

基于这一假设，将该域划分为2个微服务并使用CQRS并不能解决这个问题。为什么？

因为如果您有一个写微服务和读微服务，并且使用从写微服务发布的事件更新读取的微服务，那么就会出现延迟问题。这意味着，在读取微服务db与写入微服务db同步之前，您的最小延迟将在50-100毫秒左右(大多数情况下延迟会更大)。这是正常的，在使用队列处理分布式系统时需要考虑到这一点。

基于此，将域的这一部分分割成2个微服务并不是最好的方法(同样，这是基于我的假设，即您几乎需要实时读取微服务数据)。

可能的解决方案

你能做什么？在这里你可以做几件事：

1. 选项-数据库复制和CQRS

- The database part Again if you need the data up to date you could use something like Read-only replication of your Write database. SQL Server provides something like this out of the box. I was using it on one of my projects. This would mean that you would have another database where SQL server would replicate your data to another database on a database level (this would be much faster then doing the whole, publish to a queue and consume the message from another micro-service). This database would an exact copy of your Write database and it would be read-only. This way your 10 Read operations would almost always be up to date with a very very low latency. You can read about this feature from SQL Server here.
- The CQRS part for backend  When it comes to CQRS you would still continue using it having your 2 micro-services (Write and Read). The Write micro-service would use your Primary SQL Server instance and your Read micro-service would use the Read only replica of the Primary database. Keep in mind that this read only replica is a separate database running on a separate machine. So based on this you would satisfy the rule: "1 database per one micro-service". Of course this options is possible if your are using Microsoft Sql Server.

​

1. 选项--使用事件源生成物化视图和服务器端CQRS

- The database part In this approach you would use a materialized Views which would serve for reads on the same database as your main or Write micro-service database. For example if you use PostgreSQL you could use Marten([https://jasperfx.github.io/marten/](https://jasperfx.github.io/marten/)) for event sourcing and storing the events. It is for .NET but I guess there are other solutions for other languages as well. The good thing about Marten is that you can generate Materialized Views (called Projection-views) which are generated as soon as the Aggregate/Model changes. Means if you have for example some Customer object changed using Event sourcing you would publish an event which would be stored to the database(using Marten). After that marten will update your Projection-view (which would be a Table in your db like CustomersProjection) applying just the last event. This is very performant and your view would be up to date as soon as the event is published. This way you would make use of your already existing Event Sourcing implementation.
- The back-end CQRS part server/back-end side would be splitted to 2 micro-services as in the previous approach. Unlike the other approach here everything would be in one Physical database. When it comes to CQRS you would still continue using it but only on your  server/back-end level. When it comes to database level you would physically access the same database from both micro-services. There would be just a logical split and using the same db for both has some drawbacks as well. Even though you would have everything in one database you would access only the Projection-views from the Read micro-service and all the other tables from the Write micro-service. There are ways how you can get around this to add restriction on Code level not to access specific tables. For example if you use some ORM with .NET or Java you could do this easily. For other technologies there are similar solutions. 
- The problem with this is that you would use one database for both micro-services.

​

1. 选项-在域的这一部分中根本不使用CQRS

- If you have your Application/Domain split into micro-services for some parts of the domain it makes sense to use CQRS to separate the reads and writes not only from the Database point of view but also from the server scaling point of view. From database point of view the good thing if you use 2 databases you can even pick different technologies for your Write and Read database. From serve side the benefit is that you can scale the read part independently from the Write part and vice versa.  In your case if your have only 10 reads per 1 minute means that you don't have a big load on your data using a separate database and micro-service is not necessary(and I would even say an overkill in this case).  10 loads more on the same server per minute where the Writes would be handled together with the  reads makes almost no difference. But I don't know if this is only  for now or the requirements are going to change and the reading requests are going to increase. For now there is no need for it and I would put everything in one micro-service and database.  In one of my previous project we had exact that secnario where we had a huge micro-service based architecture and in parts we where using CQRS splitting the specific Sub-Domain in 2 micro-services having their dedicated databases, but in other parts of domain we did not use CQRS as it simply did not make any sense for us in that part of the Domain.

​

请记住，在某些情况下，这些建议是特定于某些数据库技术的，如Server、PostgreSQL或类似的，但对您来说重要的是其思想和方法。无论您使用哪个数据库，都可以完成这些操作。

一般情况下：

打破“每个服务的一个数据库”是一种罪过，还是可以将一个服务以书面形式(到数据库)和读取/呈现(从同一个数据库)作为一个服务来考虑。

我想说的是，对于每条规则，都有例外，如果使用带有2 db的CQRS会使您的生活变得很困难，并且由于使用系统或域而有问题，那么这就意味着您没有正确地使用模式/实践，或者您在使用错误的模式。请记住，模式的存在是为了解决常见的问题，如果它们不能解决特定的情况，就不要使用它们。在微观服务的意义上，事情变得更加复杂，因为很多事情都适合你的业务需求。这是好的，因为目标是为客户提供最好的解决方案。即使您和您的团队自己发现您可以使用2个微服务，并使用1个数据库作为最佳解决方案，也可以使用它。不要将它作为整个体系结构的规则，因为它不是微服务世界中的实践，但与往常一样，如果您有很好的理由支持它，您可以打破规则。

写入数据库比读取数据库要大得多。

从服务器/后端的角度来看，这根本不是一个问题，因为您可以进行水平扩展，并且可以根据需要运行这个微服务的实例。例如，让其中的10个同时运行就可以了。10读，除了这个每分钟是在这个规模，没有什么可担心的(在意义上说，所有是在一个微服务，读和写)。当涉及到扩展数据库的点时。这是另一个话题。将读取分离到专用数据库无助于扩展写入数据库中的数据。为了解决这一问题，需要考虑的问题还有:查询优化、添加合适的索引、数据分割、数据历史化等。但这是另一个话题。

摘要：

我的建议是，只有在使用Server的情况下，才能使用1.1.Options。另一方面，如果您发现当前的数据库技术提供了类似的功能，那么您也可以使用它来实现它。如果这对你不起作用，我建议与3。选项和放弃CQRS的这一部分(或域)完全。在我看来你不需要这个案子。

Answer 2

为了找到你的问题的答案，你可能需要结合Microservices和CQRS的思维过程。

有两种观点可以帮助你思考这个问题：

1. 在CQRS的许多优点中，其中一个优点是将数据的读端和写端分离开来，与您的需求完全相同，在性能需求方面有许多数量级的差异。您需要能够以不同的方式扩展这两个部分，这意味着在Microservices世界中有两个不同的服务。
2. 对于真正的CQRS，您倾向于在数据的写和读部分之间没有任何关系/链接。这方面的一个完美示例是使用关系数据库管理系统来存储写入端，并在查询端使用文档存储(比如MongoDB，甚至是索引ElasticSearch)，因为它支持聚合数据结构。

因此，如果您希望能够单独扩展，并且不希望在写和读之间发生任何争用，那么两个独立的服务是可行的，IMHO。