This article mainly introduces the evolutionary journey of distributed transactions from internal development to commercialization and open source, as well as the current progress and future planning of the Seata community. Seata is an open-source distributed transaction solution designed to provide a comprehensive solution for distributed transactions under modern microservices architecture. Seata offers complete distributed transaction solutions, including AT, TCC, Saga, and XA transaction modes, supporting various programming languages and data storage schemes. Seata also provides easy-to-use APIs, extensive documentation, and examples to facilitate quick development and deployment for enterprises applying Seata. Seata's advantages lie in its high availability, high performance, and high scalability, and it does not require extra complex operations for horizontal scaling. Seata is currently used in thousands of customer business systems on Alibaba Cloud, and its reliability has been recognized and applied by major industry manufacturers. As an open-source project, the Seata community is also expanding continuously, becoming an important platform for developers to exchange, share, and learn, attracting more and more attention and support from enterprises. Today, I will primarily share about Seata on the following three topics:

• From TXC/GTS to Seata
• Latest developments in the Seata community
• Future planning for the Seata community
  

From TXC/GTS to Seata​

The Origin of Distributed Transactions​

Seata is internally codenamed TXC (taobao transaction constructor) within Alibaba, a name with a strong organizational structure flavor. TXC originated from Alibaba's Wushi (Five Color Stones) project, which in ancient mythology were the stones used by the goddess Nüwa to mend the heavens, symbolizing Alibaba's important milestone in the evolution from monolithic architecture to distributed architecture. During this project, a batch of epoch-making Internet middleware was developed, including the well-known "Big Three":

• HSF service invocation framework Solves service communication issues after the transition from monolithic applications to service-oriented architectures.
• TDDL database sharding framework Addresses storage capacity and connection count issues of databases at scale.
• MetaQ messaging framework Addresses asynchronous invocation issues. The birth of the Big Three satisfied the basic requirements of microservices-based business development, but the data consistency issues that arose after microservices were not properly addressed, lacking a unified solution. The likelihood of data consistency issues in microservices is much higher than in monolithic applications, and the increased complexity of moving from in-process calls to network calls exacerbates the production of exceptional scenarios. The increase in service hops also makes it impossible for upstream and downstream services to coordinate data rollback in the event of a business processing exception. TXC was born to address the pain points of data consistency at the application architecture layer, and the core data consistency scenarios it aimed to address included:
• Consistency across services. Coordinates rollback of upstream and downstream service nodes in the event of system exceptions such as call timeouts and business exceptions.
• Data consistency in database sharding. Ensures internal transactions during logical SQL operations on business layers are consistent across different data shards.
• Data consistency in message sending. Addresses the inconsistency between data operations and successful message sending. To overcome the common scenarios encountered, TXC was seamlessly integrated with the Big Three. When businesses use the Big Three for development, they are completely unaware of TXC's presence in the background, do not have to consider the design of data consistency, and leave it to the framework to ensure, allowing businesses to focus more on their own development, greatly improving development efficiency.
  TXC has been widely used within Alibaba Group for many years and has been baptized by the surging traffic of large-scale events like Singles' Day, significantly improving business development efficiency and ensuring data accuracy, eliminating financial and reputational issues caused by data inconsistencies. With the continuous evolution of the architecture, a standard three-node cluster can now handle peak values of nearly 100K TPS and millisecond-level transaction processing. In terms of availability and performance, it has reached a four-nines SLA guarantee, ensuring no failures throughout the year even in unattended conditions.
  

The Evolution of Distributed Transactions​

The birth of new things is always accompanied by doubts. Is middleware capable of ensuring data consistency reliable? The initial birth of TXC was just a vague theory, lacking theoretical models and engineering practice. After we conducted MVP (Minimum Viable Product) model testing and promoted business deployment, we often encountered faults and frequently had to wake up in the middle of the night to deal with issues, wearing wristbands to sleep to cope with emergency responses. These were the most painful years I went through technically after taking over the team. Subsequently, we had extensive discussions and systematic reviews. We first needed to define the consistency problem. Were we to achieve majority consensus consistency like RAFT, solve database consistency issues like Google Spanner, or something else? Looking at the top-down layered structure from the application node, it mainly includes development frameworks, service invocation frameworks, data middleware, database drivers, and databases. We had to decide at which layer to solve the data consistency problem. We compared the consistency requirements, universality, implementation complexity, and business integration costs faced when solving data consistency issues at different levels. In the end, we weighed the pros and cons, decided to keep the implementation complexity to ourselves, and adopted the AT mode initially as a consistency component. We needed to ensure high consistency, but not be locked into specific database implementations, ensuring the generality of scenarios and the business integration costs were low enough to be easily implemented. This is also why TXC initially adopted the AT mode. A distributed transaction is not just a framework; it's a system. We defined the consistency problem in theory, abstractly conceptualized modes, roles, actions, and isolation, etc. From an engineering practice perspective, we defined the programming model, including low-intrusion annotations, simple method templates, and flexible APIs, and defined basic and enhanced transaction capabilities (e.g., how to support a large number of activities at low cost), as well as capabilities in operations, security, performance, observability, and high availability. What problems do distributed transactions solve? A classic and tangible example is the money transfer scenario. The transfer process includes subtracting balance and adding balance, how do we ensure the atomicity of the operation? Without any intervention, these two steps may encounter various problems, such as account B being canceled or service call timeouts, etc. Timeout issues have always been a difficult problem to solve in distributed applications; we cannot accurately know whether service B has executed and in what order. From a data perspective, this means the money in account B may not be successfully added. After the service-oriented transformation, each node only has partial information, while the transaction itself requires global coordination of all nodes, thus requiring a centralized role with a god's-eye view, capable of obtaining all information, which is the TC (transaction coordinator), used to globally coordinate the transaction state. The TM (Transaction Manager) is the role that drives the generation of transaction proposals. However, even gods nod off, and their judgments are not always correct, so we need an RM (resource manager) role to verify the authenticity of the transaction as a representative of the soul. This is TXC's most basic philosophical model. We have methodologically verified that its data consistency is very complete, of course, our cognition is bounded. Perhaps the future will prove we were turkey engineers, but under current circumstances, its model is already sufficient to solve most existing problems. After years of architectural evolution, from the perspective of transaction single-link latency, TXC takes an average of about 0.2 milliseconds to process at the start of the transaction and about 0.4 milliseconds for branch registration, with the entire transaction's additional latency within the millisecond range. This is also the theoretical limit value we have calculated. In terms of throughput, the TPS of a single node reaches 30,000 times/second, and the TPS of a standard cluster is close to 100,000 times/second.

Seata Open Source​

Why go open source? This is a question many people have asked me. In 2017, we commercialized the GTS (Global Transaction Service) product sold on Alibaba Cloud, with both public and private cloud forms. At this time, the internal group developed smoothly, but we encountered various problems in the process of commercialization. The problems can be summed up in two main categories: First, developers are quite lacking in the theory of distributed transactions, most people do not even understand what local transactions are, let alone distributed transactions. Second, there are problems with product maturity, often encountering various strange scenario issues, leading to a sharp rise in support and delivery costs, and R&D turning into after-sales customer service. We reflected on why we encountered so many problems. The main issue here is that Alibaba Group internally has a unified language stack and unified technology stack, and our polishing of specific scenarios is very mature. Serving Alibaba, one company, and serving thousands of enterprises on the cloud is fundamentally different, which also made us realize that our product's scenario ecology was not well developed. On GitHub, more than 80% of open-source software is basic software, and basic software primarily solves the problem of scenario universality, so it cannot be locked in by a single enterprise, like Linux, which has a large number of community distributions. Therefore, in order to make our product better, we chose to open source and co-build with developers to popularize more enterprise users. Alibaba's open-source journey has gone through three main stages. The first stage is the stage where Dubbo is located, where developers contribute out of love, Dubbo has been open sourced for over 10 years, and time has fully proven that Dubbo is an excellent open-source software, and its microkernel plugin extensibility design is an important reference for me when I initially open sourced Seata. When designing software, we need to consider which is more important between extensibility and performance, whether we are doing a three-year design, a five-year design, or a ten-year design that meets business development. While solving the 0-1 service call problem, can we predict the governance problems after the 1-100 scale-up? The second stage is the closed loop of open source and commercialization, where commercialization feeds back into the open-source community, promoting the development of the open-source community. I think cloud manufacturers are more likely to do open source well for the following reasons:

• First, the cloud is a scaled economy, which must be established on a stable and mature kernel foundation, packaging its product capabilities including high availability, maintenance-free, and elasticity on top of it. An unstable kernel will inevitably lead to excessive delivery and support costs, and high penetration of the R&D team's support Q&A will prevent large-scale replication, and high penetration rates will prevent rapid evolution and iteration of products.
• Second, commercial products know business needs better. Our internal technical teams often YY requirements from a development perspective, and what they make is not used by anyone, and thus does not form a value conversion. The business requirements collected through commercialization are all real, so its open source kernel must also evolve in this direction. Failure to evolve in this direction will inevitably lead to architectural splits on both sides, increasing the team's maintenance costs.
• Finally, the closed loop of open source and commercialization can promote better development of both parties. If the open-source kernel often has various problems, would you believe that its commercial product is good enough? The third stage is systematization and standardization. First, systematization is the basis of open-source solutions. Alibaba's open-source projects are mostly born out of internal e-commerce scenario practices. For example, Higress is used to connect Ant Group's gateways; Nacos carries services with millions of instances and tens of millions of connections; Sentinel provides degradation and throttling capabilities for high availability during major promotions; and Seata ensures transaction data consistency. This set of systematized open-source solutions is designed based on the best practices of Alibaba's e-commerce ecosystem. Second, standardization is another important feature. Taking OpenSergo as an example, it is both a standard and an implementation. In the past few years, the number of domestic open-source projects has exploded. However, the capabilities of various open-source products vary greatly, and many compatibility issues arise when integrating with each other. Therefore, open-source projects like OpenSergo can define some standardized capabilities and interfaces and provide some implementations, which will greatly help the development of the entire open-source ecosystem.
  

Latest Developments in the Seata Community​

Introduction to the Seata Community​

At present, Seata has open-sourced 4 transaction modes, including AT, TCC, Saga, and XA, and is actively exploring other viable transaction solutions. Seata has integrated with more than 10 mainstream RPC frameworks and relational databases, and has integrated or been integrated relationships with more than 20 communities. In addition, we are also exploring languages other than Java in the multi-language system, such as Golang, PHP, Python, and JS. Seata has been applied to business systems by thousands of customers. Seata applications have become more mature, with successful cooperation with the community in the financial business scenarios of CITIC Bank and Everbright Bank, and successfully adopted into core accounting systems. The landing of microservices systems in financial scenarios is very stringent, which also marks a new level of maturity for Seata's kernel.

Seata Ecosystem Expansion​

Seata adopts a microkernel and plugin architecture design, exposing rich extension points in APIs, registry configuration centers, storage modes, lock control, SQL parsers, load balancing, transport, protocol encoding and decoding, observability, and more. This allows businesses to easily perform flexible extensions and select technical components.

Seata Application Cases​

Case 1: China Aviation Information's Air Travel Project The China Aviation Information Air Travel project introduced Seata in the 0.2 version to solve the data consistency problem of ticket and coupon business, greatly improving development efficiency, reducing asset losses caused by data inconsistency, and enhancing user interaction experience. Case 2: Didi Chuxing's Two-Wheeler Business Unit Didi Chuxing's Two-Wheeler Business Unit introduced Seata in version 0.6.1, solving the data consistency problem of business processes such as blue bicycles, electric vehicles, and assets, optimizing the user experience, and reducing asset loss. Case 3: Meituan's Infrastructure Meituan's infrastructure team developed the internal distributed transaction solution Swan based on the open-source Seata project, which is used to solve distributed transaction problems within Meituan's various businesses. Case 4: Hema Town Hema Town uses Seata to control the flower-stealing process in game interactions, significantly shortening the development cycle from 20 days to 5 days, effectively reducing development costs.

Evolution of Seata Transaction Modes​

Current Progress of Seata​

• Support for Oracle and PostgreSQL multi-primary keys.
• Support for Dubbo3.
• Support for Spring Boot3.
• Support for JDK 17.
• Support for ARM64 images.
• Support for multiple registration models.
• Extended support for various SQL syntaxes.
• Support for GraalVM Native Image.
• Support for Redis lua storage mode.
  

Seata 2.x Development Planning​

Mainly includes the following aspects:

• Storage/Protocol/Features Explore storage and computing separation in Raft cluster mode; better experience, unify the current 4 transaction mode APIs; compatible with GTS protocol; support Saga annotations; support distributed lock control; support data perspective insight and governance.
• Ecosystem Support more databases, more service frameworks, while exploring support for the domestic trust creation ecosystem; support the MQ ecosystem; further enhance APM support.
• Solutions In addition to supporting microservices ecosystems, explore multi-cloud solutions; closer to cloud-native solutions; add security and traffic protection capabilities; achieve self-convergence of core components in the architecture.
• Multi-Language Ecosystem Java is the most mature in the multi-language ecosystem, continue to improve other supported programming languages, while exploring Transaction Mesh solutions that are independent of languages.
• R&D Efficiency/Experience Improve test coverage, prioritize quality, compatibility, and stability; restructure the official website's documentation to improve the hit rate of document searches; simplify operations and deployment on the experience side, achieve one-click installation and metadata simplification; console supports transaction control and online analysis capabilities.

In one sentence, the 2.x plan is summarized as: Bigger scenarios, bigger ecosystems, from usable to user-friendly.

Contact Information for the Seata Community​

Introduction to Seata​

Seata is the predecessor of Alibaba Group's massively used middleware for ensuring consistency of distributed transactions, and Seata is its open source product, maintained by the community. Before introducing Seata, let's discuss with you some problematic scenarios that we often encounter in the course of our business development.

Business Scenarios​

Our business in the development process, basically from a simple application, and gradually transitioned to a huge scale, complex business applications. These complex scenarios inevitably encounter distributed transaction management problems, Seata's emergence is to solve these distributed scenarios of transaction management problems. Introduce a few of the classic scenarios:

Scenario 1: distributed transactions in the scenario of split library and split table​

Initially, our business was small and lightweight, and a single database was able to secure our data links. However, as the business scale continues to grow and the business continues to become more complex, usually a single database encounters bottlenecks in terms of capacity and performance. The usual solution is to evolve to a split-database, split-table architecture. At this point, that is, the introduction of **split library and split table scenario **distributed transaction scenarios.

Scenario 2: Distributed Transactions in a Cross-Service Scenario​

A solution to reduce the complexity of a monolithic application: application microservice splitting. After splitting, our product consists of multiple microservice components with different functions, each of which uses independent database resources. When it comes to data consistency scenarios involving cross-service calls, distributed transactions are introduced in cross-service scenarios.

Seata architecture​

Its core components are mainly as follows:

• Transaction Coordinator (TC).

Transaction coordinator that maintains the running state of the global transaction and is responsible for coordinating and driving the commit or rollback of the global transaction.

• Transaction Manager (TM)

Controls global transaction boundaries, responsible for opening a global transaction and ultimately initiating a global commit or global rollback resolution, TM defines the boundaries of a global transaction.

• Resource Manager (RM)

Controls branch transactions and is responsible for branch registration, status reporting, and receiving commands from the Transaction Coordinator to drive the commit and rollback of branch (local) transactions.RM is responsible for defining the boundaries and behaviour of branch transactions.

Seata's observable practices​

Why do we need observables?​

• *# Distributed transaction message links are more complex *#

Seata, while solving these problems of user ease of use and distributed transaction consistency, requires multiple interactions between TC and TM, RM, especially when the links of microservices become complex, the interaction links of Seata will also increase in positive correlation. In this case, we actually need to introduce observable capabilities to observe and analyse things links.

• Anomalous links, hard to locate for troubleshooting, no way to optimise performance*

When troubleshooting Seata's abnormal transaction links, the traditional approach requires looking at logs, which is troublesome to retrieve. With the introduction of observable capabilities, it helps us to visually analyse links and quickly locate problems; providing a basis for optimising time-consuming transaction links.

• Visualisation, Data Quantification

The visualisation capability allows users to have an intuitive feeling of transaction execution; with quantifiable data, it helps users to assess resource consumption and plan budgets.

Overview of observable capabilities​

Metrics dimension​

Design Ideas​

1. Seata as an integrated data consistency framework, the Metrics module will use as few third-party dependencies as possible to reduce the risk of conflicts.
2. the Metrics module will strive for higher metrics performance and lower resource overhead to minimise the side effects of turning it on.
3. When configured, whether Metrics is activated and how the data is published depends on the corresponding configuration; if the configuration is turned on, Metrics is automatically activated and the metrics data will be published via prometheusexporter by default.
4. do not use Spring, use SPI (Service Provider Interface) to load extensions

module design​

• seata-metrics-core: Metrics core module, organises (loads) 1 Registry and N Exporters according to configuration;
• seata-metrics-api: defines the Meter metrics interface, the Registry metrics registry interface;
• seata-metrics-exporter-prometheus: built-in implementation of prometheus-exporter;
• seata-metrics-registry-compact: built-in Registry implementation and lightweight implementation of Gauge, Counter, Summay, Timer metrics;

metrics module workflow​

The above figure shows the workflow of the metrics module, which works as follows:

1. load Exporter and Registry implementation classes based on configuration using SPI mechanism;
2. based on the message subscription and notification mechanism, listen for state change events for all global transactions and publish to EventBus;
3. event subscribers consume events and write the generated metrics to Registry;
4. the monitoring system (e.g. prometheus) pulls data from the Exporter.

TC Core Metrics​

TM Core Metrics​

RM Core Metrics​

Large Cap Showcase​

Tracing dimensions​

Why does Seata need tracing?​

1. for the business side, how much of a drain on business performance will the introduction of Seata cause? Where is the main time consumption? How to optimise the business logic? These are all unknown.
2. All the message records of Seata are persisted through the log to fall the disc, but the log is very unfriendly to users who do not understand Seata. Can we improve the efficiency of transaction link scheduling by accessing Tracing?
3. for novice users, can through Tracing records, quickly understand the working principle of seata, reduce the threshold of seata use.

Seata's tracing solution​

• Seata defines Header information in a custom RPC message protocol;
• SkyWalking intercepts the specified RPC message and injects tracing related span information;
• The lifecycle scope of the span is defined with the RPC message sending & receiving as the threshold.

Based on the above approach, Seata implements transaction-wide tracing, please refer to Accessing Skywalking for [Seata application | Seata-server] for specific access.

tracing effects​

• Based on the demo scenario:

1. user requests transaction service
2. transaction service locks inventory
3. the transaction service creates a bill
4. The billing service performs a debit

• Transaction link for GlobalCommit success (example)

Logging dimension​

Design Ideas​

Logging is the bottom of the observable dimensions. Placed at the bottom, in fact, is the design of our log format, only a good log format, we can make it a better collection, modular storage and display. On top of it, is the log collection, storage, monitoring, alarms, data visualisation, these modules are more ready-made tools, such as Ali's SLS logging service, and ELK's set of technology stack, we are more overhead costs, access complexity, ecological prosperity, etc. as a consideration.

Log format design​

Here we take a log format of Seata-Server as a case study:

• Thread pool canonical naming: When there are more thread pools and threads, canonical thread naming can clearly show the execution order of the threads that are executed in an unordered way.
• Traceability of method class names: Quickly locate specific code blocks.
• Key Runtime Information Transparency: Focus on highlighting key logs and not printing non-critical logs to reduce log redundancy.
• Extensible message format: Reduce the amount of code modification by extending the output format of the message class.

Summary & Outlook​

Metrics​

Summary: basically achieve quantifiable and observable distributed transactions. Prospect: more granular metrics, broader ecological compatibility.

Tracing​

Summary: Traceability of the whole chain of distributed transactions. Prospect: trace transaction links according to xid, and quickly locate the root cause of abnormal links.

Logging​

Summary: Structured log format. Prospect: Evolution of log observable system.

Production-ready seata-go 1.2.0 is here!​

Seata is an open source distributed transaction solution that provides high-performance and easy-to-use distributed transaction services.

Release overview​

Seata-go version 1.2.0 supports the XA schema, a distributed transaction processing specification proposed by the X/Open organisation, which has the advantage of being non-intrusive to business code. Currently, Seata-go's XA mode supports MySQL databases. So far, seata-go has gathered three transaction modes: AT, TCC and XA. Main features of XA mode.

• Supported XA data source proxy https://github.com/apache/incubator-seata-go-samples/tree/main/xa
• XA transaction mode is supported. XA related sampes can be found in the following example: https://github.com/apache/incubator-seata-go-samples/tree/main/xa

feature​

• [#467] implements XA schema support for MySQL.
• [#534] Load balancing for session support.

bugfix​

• [#540] Fix a bug in initialising xa mode.
• [#545] Fix a bug in getting db version number in xa mode.
• [#548] Fix bug where starting xa fails.
• [#556] Fix bug in xa datasource.
• [#562] Fix bug with committing xa global transaction
• [#564] Fix bug committing xa branching transactions
• [#566] Fix bug with local transactions using xa data source.

optimise​

• [#523] Optimise CI process
• [#525] rename jackson serialisation to json
• [#532] Remove duplicate code
• [#536] optimise go import code formatting
• [#554] optimise xa mode performance
• [#561] Optimise xa mode logging output

test​

• [#535] Add integration tests.

doc​

• [#550] Added changelog for version 1.2.0.

contributors​

Thanks to these contributors for their code commits. Please report an unintended omission.

• georgehao
• luky116
• jasondeng1997
• 106umao
• wang1309
• iSuperCoder
• Charlie17Li
• Code-Fight
• Kirhaku
• Vaderkai

Link​

• https://github.com/apache/incubator-seata
• https://github.com/apache/incubator-seata-go
• https://github.com/apache/incubator-seata-samples
• https://github.com/apache/incubator-seata-go-samples
• https://seata.apache.org

Welcome everyone to register for Seata Open Source Summer 2023 topics​

The registration period for Open Source Summer 2023 runs from April 29th to June 4th, and we welcome registration for Seata 2023 topics! Here, you will have the opportunity to delve into the theory and application of distributed transactions, and collaborate with students from different backgrounds to complete practical projects. We look forward to your active participation and contribution, jointly promoting the development of the distributed transaction field.

Seata Open Source Summer 2023​

Open Source Summer is a summer open source activity initiated and long-term supported by the Institute of Software, Chinese Academy of Sciences, as part of the "Open Source Software Supply Chain Lighting Program", aimed at encouraging students to actively participate in the development and maintenance of open source software, cultivating and discovering more outstanding developers, promoting the vigorous development of excellent open source software communities, and assisting in the construction of the open source software supply chain.

Participating students will collaborate remotely online with senior mentors to participate in the development of various organizational projects in the open source community and receive bonuses, gifts, and certificates. These gains are not only a highlight on future graduation resumes but also a brilliant starting point towards becoming top developers. Each project is divided into two difficulty levels: basic and advanced, with corresponding project completion bonuses of RMB 8,000 and RMB 12,000 before tax, respectively.

Introduction to the Seata Community​

Seata is an open-source distributed transaction solution, with over 23K+ stars on GitHub, dedicated to providing high-performance and easy-to-use distributed transaction services under the microservices architecture. Before being open-sourced, Seata played a role as a middleware for distributed data consistency within Alibaba, where almost every transaction needed to use Seata. After undergoing the baptism of Double 11's massive traffic, it provided strong technical support for business.

Summary of Seata Community Open Source Summer 2023 Project Topics​

The Seata community recommends 6 selected project topics for the Open Source Summer 2023 organizing committee. You can visit the following link to make your selection:
https://summer-ospp.ac.cn/org/orgdetail/064c15df-705c-483a-8fc8-02831370db14?lang=zh
Please communicate promptly with the respective mentors and prepare project application materials, and register through the official channels (the following topics are not listed in any particular order):

Project One: Implementation of NamingServer for Service Discovery and Registration​

Difficulty: Advanced

Project Community Mentor: Chen Jianbin

Mentor's Contact Email: 364176773@qq.com

Project Description:
Currently, Seata's service exposure and discovery mainly rely on third-party registration centers. With the evolution and development of the project, it brings additional learning and usage costs. Most mainstream middleware with servers have begun to evolve their own service self-loop and control and provide components or functions with higher compatibility and reliability to the server, such as Kafka's KRaft, RocketMQ's NameServer, ClickHouse's ClickHouse Keeper, etc. To address the above problems and architectural evolution requirements, Seata needs to build its own NamingServer to ensure more stability and reliability.

Project Link: https://summer-ospp.ac.cn/org/prodetail/230640380?list=org&navpage=org

...

(Projects Two to Six translated in the same manner)

...

How to Participate in Seata Open Source Summer 2023 and Quickly Select a Project?​

Welcome to communicate with each mentor through the above contact information and prepare project application materials.

During the project participation period, students can work online from anywhere in the world. The completion of Seata-related projects needs to be submitted to the Seata community repository as a PR by September 30th, so please prepare early.

To obtain information about mentors and other information during the project, you can scan the QR code below to enter the DingTalk group for communication —— Understand various projects in the Seata community, meet Seata community open source mentors, and help with subsequent applications.

Reference Materials:​

Seata Website: https://seata.apache.org/

Seata GitHub: https://github.com/seata

Open Source Summer Official Website: https://summer-ospp.ac.cn/org/orgdetail/ab188e59-fab8-468f-bc89-bdc2bd8b5e64?lang=zh

If students are interested in other areas of microservices projects, they can also try to apply, such as:

• For students interested in microservice configuration registration centers, they can try to apply for Nacos Open Source Summer;
• For students interested in microservice frameworks and RPC frameworks, they can try to apply for Spring Cloud Alibaba Open Source Summer and Dubbo Open Source Summer;
• For students interested in cloud-native gateways, they can try to apply for Higress Open Source Summer;
• For students interested in distributed high-availability protection, they can try to apply for [Sentinel Open Source Summer](https://summer-ospp.ac. cn/org/orgdetail/5e879522-bd90-4a8b-bf8b-b11aea48626b?lang=zh);
• For students interested in microservices governance, they can try to apply for [OpenSergo Open Source Summer](https://summer-ospp.ac. cn/org/orgdetail/aaff4eec-11b1-4375-997d-5eea8f51762b?lang=zh).

Seata 1.6.0 Released with Significant Performance Improvement​

Seata is an open-source distributed transaction solution that provides high performance and easy-to-use distributed transaction services.

Download Links for seata-server:

source | binary

Updates in this version:

feature：​

• [#4863] Support for multiple primary keys in Oracle and PostgreSQL
• [#4649] Support for multiple registry centers in seata-server
• [#4779] Support for Apache Dubbo3
• [#4479] TCC annotations can now be added to interfaces and implementation classes
• [#4877] Client SDK supports JDK17
• [#4914] Support for update join syntax for MySQL
• [#4542] Support for Oracle timestamp type
• [#5111] Support for Nacos contextPath configuration
• [#4802] Dockerfile supports arm64

Bug Fixes:​

• [#4780] Fixed the issue where TimeoutRollbacked event wasn't sent after a successful timeout rollback.
• [#4954] Fixed NullPointerException when the output expression was incorrect.
• [#4817] Fixed the problem with non-standard configuration in higher versions of Spring Boot.
• [#4838] Fixed the issue where undo log wasn't generated when using Statement.executeBatch().
• [#4533] Fixed inaccurate metric data caused by duplicate events handling for handleRetryRollbacking.
• [#4912] Fixed the issue where mysql InsertOnDuplicateUpdate couldn't correctly match column names due to inconsistent case.
• [#4543] Fixed support for Oracle nclob data type.
• [#4915] Fixed the problem of not obtaining ServerRecoveryProperties attributes.
• [#4919] Fixed the issue where XID's port and address appeared as null:0.
• [#4928] Fixed NPE issue in rpcContext.getClientRMHolderMap.
• [#4953] Fixed the issue where InsertOnDuplicateUpdate could bypass primary key modification.
• [#4978] Fixed kryo support for cyclic dependencies.
• [#4985] Fixed the issue of duplicate undo_log id.
• [#4874] Fixed startup failure with OpenJDK 11.
• [#5018] Fixed server startup failure issue due to loader path using relative path in startup script.
• [#5004] Fixed the issue of duplicate row data in mysql update join.
• [#5032] Fixed the abnormal SQL statement in mysql InsertOnDuplicateUpdate due to incorrect calculation of condition parameter fill position.
• [#5033] Fixed NullPointerException issue in SQL statement of InsertOnDuplicateUpdate due to missing insert column field.
• [#5038] Fixed SagaAsyncThreadPoolProperties conflict issue.
• [#5050] Fixed the issue where global status under Saga mode wasn't correctly changed to Committed.
• [#5052] Fixed placeholder parameter issue in update join condition.
• [#5031] Fixed the issue of using null value index as query condition in InsertOnDuplicateUpdate.
• [#5075] Fixed the inability to intercept SQL statements with no primary key and unique index in InsertOnDuplicateUpdate.
• [#5093] Fixed accessKey loss issue after seata server restart.
• [#5092] Fixed the issue of incorrect AutoConfiguration order when seata and jpa are used together.
• [#5109] Fixed NPE issue when @GlobalTransactional is not applied on RM side.
• [#5098] Disabled oracle implicit cache for Druid.
• [#4860] Fixed metrics tag override issue.
• [#5028] Fixed null value issue in insert on duplicate SQL.
• [#5078] Fixed interception issue for SQL statements without primary keys and unique keys.
• [#5097] Fixed accessKey loss issue when Server restarts.
• [#5131] Fixed issue where XAConn cannot rollback when in active state.
• [#5134] Fixed issue where hikariDataSource auto proxy fails in some cases.
• [#5163] Fixed compilation failure in higher versions of JDK.

Optimization:​

• [#4681] Optimized the process of competing locks.
• [#4774] Optimized mysql8 dependency in seataio/seata-server image.
• [#4750] Optimized the release of global locks in AT branch to not use xid.
• [#4790] Added automatic OSSRH github action publishing.
• [#4765] XA mode in mysql8.0.29 and above no longer holds connection to the second phase.
• [#4797] Optimized all github actions scripts.
• [#4800] Added NOTICE file.
• [#4761] Used hget instead of hmget in RedisLocker.
• [#4414] Removed log4j dependency.
• [#4836] Improved readability of BaseTransactionalExecutor#buildLockKey(TableRecords rowsIncludingPK) method.
• [#4865] Fixed security vulnerabilities in Saga visualization designer GGEditor.
• [#4590] Dynamic configuration support for automatic degradation switch.
• [#4490] Optimized tccfence record table to delete by index.
• [#4911] Added header and license checks.
• [#4917] Upgraded package-lock.json to fix vulnerabilities.
• [#4924] Optimized pom dependencies.
• [#4932] Extracted default values for some configurations.
• [#4925] Optimized javadoc comments.
• [#4921] Fixed security vulnerabilities in console module and upgraded skywalking-eyes version.
• [#4936] Optimized storage configuration reading.
• [#4946] Passed SQL exceptions encountered when acquiring locks to the client.
• [#4962] Optimized build configuration and corrected base image of docker image.
• [#4974] Removed limitation on querying globalStatus quantity under redis mode.
• [#4981] Improved error message when tcc fence record cannot be found.
• [#4995] Fixed duplicate primary key query conditions in the SQL statement after mysql InsertOnDuplicateUpdate.
• [#5047] Removed unused code.
• [#5051] When undolog generates dirty write during rollback, throw exception BranchRollbackFailed_Unretriable.
• [#5075] Intercept insert on duplicate update statements without primary keys and unique indexes.
• [#5104] ConnectionProxy is no longer dependent on druid.
• [#5124] Support deleting TCC fence record table for oracle.
• [#4468] Support kryo 5.3.0.
• [#4807] Optimized image and OSS repository publishing pipelines.
• [#4445] Optimized transaction timeout judgment.
• [#4958] Optimized execution of triggerAfterCommit() for timeout transactions.
• [#4582] Optimized transaction sorting in redis storage mode.
• [#4963] Added ARM64 pipeline CI testing.
• [#4434] Removed seata-server CMS GC parameters.

Testing:​

• [#4411] Tested Oracle database AT mode type support.
• [#4794] Refactored code and attempted to fix unit test DataSourceProxyTest.getResourceIdTest().
• [#5101] Fixed ClassNotFoundException issue in zk registration and configuration center DataSourceProxyTest.getResourceIdTest().

Special thanks to the following contributors for their code contributions. If there are any unintentional omissions, please report.

• slievrly
• renliangyu857
• wangliang181230
• funky-eyes
• tuwenlin
• conghuhu
• a1104321118
• duanqiaoyanyu
• robynron
• lcmvs
• github-ganyu
• 1181954449
• zw201913
• wingchi-leung
• AlexStocks
• liujunlin5168
• pengten
• liuqiufeng
• yujianfei1986
• Bughue
• AlbumenJ
• doubleDimple
• jsbxyyx
• tuwenlin
• CrazyLionLi
• whxxxxx
• neillee95
• crazy-sheep
• zhangzq7
• l81893521
• zhuyoufeng
• xingfudeshi
• odidev
• miaoxueyu

At the same time, we have received many valuable issues and suggestions from the community, and we are very grateful to everyone.

Link​

• Seata: https://github.com/apache/incubator-seata
• Seata-Samples: https://github.com/apache/incubator-seata-samples
• Release: https://github.com/apache/incubator-seata/releases
• WebSite: https://seata.apache.org

Seata 1.5.2 Released with XID Load Balancing Support​

Seata is an open-source distributed transaction solution that provides high-performance and easy-to-use distributed transaction services.

seata-server Download Links:

source | binary

The key updates in this version include:

Features:​

• [#4661] Added support for XID load balancing algorithm.
• [#4676] Added support for Seata server to expose services through SLB when using Nacos as the registry center.
• [#4642] Added support for parallel processing of client batch requests.
• [#4567] Added support for the find_in_set function in the WHERE condition.

Bug Fixes:​

• [#4515] Fixed an issue where SeataTCCFenceAutoConfiguration on the develop branch throws a ClassNotFoundException when the client does not use a DB.
• [#4661] Fixed SQL exceptions when using PostgreSQL in the console.
• [#4667] Fixed an exception when updating the map in RedisTransactionStoreManager on the develop branch.
• [#4678] Fixed the issue of cache penetration when the property transport.enableRmClientBatchSendRequest is not configured.
• [#4701] Fixed the issue of missing command line parameters.
• [#4607] Fixed a defect in skipping global lock verification.
• [#4696] Fixed the insertion issue when using the Oracle storage mode.
• [#4726] Fixed a possible NPE issue when sending messages in batches.
• [#4729] Fixed the issue of incorrect setting of AspectTransactional.rollbackForClassName.
• [#4653] Fixed the exception of non-numeric primary key in INSERT_ON_DUPLICATE.

Optimizations:​

• [#4650] Fixed a security vulnerability.
• [#4670] Optimized the number of threads in the branchResultMessageExecutor thread pool.
• [#4662] Optimized the monitoring metrics for rolling back transactions.
• [#4693] Optimized the console navigation bar.
• [#4700] Fixed failures in the execution of maven-compiler-plugin and maven-resources-plugin.
• [#4711] Separated the lib dependency during deployment.
• [#4720] Optimized pom descriptions.
• [#4728] Upgraded the logback version dependency to 1.2.9.
• [#4745] Added support for mysql8 driver in the distribution package.
• [#4626] Used easyj-maven-plugin plugin instead of flatten-maven-plugin to fix compatibility issues between shade plugin and flatten plugin.
• [#4629] Checked the constraint relationship before and after updating the globalSession status.
• [#4662] Optimized the readability of EnhancedServiceLoader.

Tests:​

• [#4544] Optimized the jackson package dependency issue in TransactionContextFilterTest.
• [#4731] Fixed unit test issues in AsyncWorkerTest and LockManagerTest.

A big thanks to the contributors for their valuable code contributions. If inadvertently omitted, please report.

• slievrly
• pengten
• YSF-A
• tuwenlin
• 2129zxl
• Ifdevil
• wingchi-leung
• liurong
• opelok-z
• funky-eyes
• Smery-lxm
• lvekee
• doubleDimple
• wangliang181230
• Bughue
• AYue-94
• lingxiao-wu
• caohdgege

At the same time, we have received many valuable issues and suggestions from the community, thank you very much.

Link​

• Seata: https://github.com/apache/incubator-seata
• Seata-Samples: https://github.com/apache/incubator-seata-samples
• Release: https://github.com/apache/incubator-seata/releases
• WebSite: https://seata.io

Today, let's talk about how the new version (1.5.1) of Alibaba's Seata resolves the issues of idempotence, dangling, and empty rollback in TCC mode.

1 TCC​

TCC mode is the most classic solution for distributed transactions. It divides the distributed transaction into two phases. In the try phase, resources are reserved for each branch transaction. If all branch transactions successfully reserve resources, the global transaction proceeds to the commit phase for committing the transaction globally. However, if any node fails to reserve resources, the global transaction enters the cancel phase to rollback the transaction globally.

Taking traditional order, inventory, and account services as an example, in the try phase, resources are attempted to be reserved by inserting orders, deducting inventory, and deducting amounts. These three services require local transaction commits, and the resources can be transferred to an intermediate table. In the commit phase, the resources reserved in the try phase are transferred to the final table. In the cancel phase, the resources reserved in the try phase are released, such as returning the account amount to the customer's account.

Note: The try phase must involve committing local transactions. For example, when deducting the order amount, the money must be deducted from the customer's account. If it is not deducted, there will be a problem in the commit phase if the customer's account does not have enough money.

1.1 try-commit​

In the try phase, resources are first reserved, and then they are deducted in the commit phase. The diagram below illustrates this process:

1.2 try-cancel​

In the try phase, resources are first reserved. If the deduction of inventory fails, leading to a rollback of the global transaction, the resources are released in the cancel phase. The diagram below illustrates this process:

2 TCC Advantages​

The biggest advantage of TCC mode is its high efficiency. In the try phase, the resource locking in TCC mode is not a true lock, but rather a real local transaction submission that reserves resources in an intermediate state without the need for blocking and waiting. Therefore, it is more efficient than other modes.

Additionally, the TCC mode can be optimized as follows:

2.1 Asynchronous Commit​

After the try phase succeeds, instead of immediately entering the confirm/cancel phase, it is considered that the global transaction has already ended. A scheduled task is started to asynchronously execute the confirm/cancel phase, which involves deducting or releasing resources. This approach can greatly improve performance.

2.2 Same-Database Mode​

In the TCC mode, there are three roles:

• TM: Manages the global transaction, including starting the global transaction and committing/rolling back the global transaction.
• RM: Manages the branch transaction.
• TC: Manages the state of the global transaction and branch transactions.

The diagram below is from the Seata official website:

When TM starts a global transaction, RM needs to send a registration message to TC, and TC saves the state of the branch transaction. When TM requests a commit or rollback, TC needs to send commit or rollback messages to RM. In this way, in a distributed transaction with two branch transactions, there are four RPCs between TC and RM.

After optimization, the process is as shown in the diagram below:

TC saves the state of the global transaction. When TM starts a global transaction, RM no longer needs to send a registration message to TC. Instead, it saves the state of the branch transaction locally. After TM sends a commit or rollback message to TC, the asynchronous thread in RM first retrieves the uncommitted branch transactions saved locally, and then sends a message to TC to obtain the state of the global transaction in which the local branch transaction is located, in order to determine whether to commit or rollback the local transaction.

With this optimization, the number of RPCs is reduced by 50%, resulting in a significant performance improvement.

3 RM Code Example​

Taking the inventory service as an example, the RM inventory service interface code is as follows:

@LocalTCC
public interface StorageService {

    /**
     * decrease
     * @param xid 
     * @param productId 
     * @param count 
     * @return
     */
    @TwoPhaseBusinessAction(name = "storageApi", commitMethod = "commit", rollbackMethod = "rollback", useTCCFence = true)
    boolean decrease(String xid, Long productId, Integer count);

    /**
     * commit
     * @param actionContext
     * @return
     */
    boolean commit(BusinessActionContext actionContext);

    /**
     * rollback
     * @param actionContext
     * @return
     */
    boolean rollback(BusinessActionContext actionContext);
}

By using the @LocalTCC annotation, when the RM is initialized, it registers a branch transaction with the TC. The try phase method (e.g., decrease method) is annotated with @TwoPhaseBusinessAction, which defines the branch transaction's resourceId, commit method, cancel method, and the useTCCFence property, which will be explained in the next section.

4 Issues with TCC​

There are three major issues with the TCC pattern: idempotence, suspension, and empty rollback. In version 1.5.1 of Seata, a transaction control table named tcc_fence_log is introduced to address these issues. The useTCCFence property mentioned in the previous @TwoPhaseBusinessAction annotation is used to enable or disable this mechanism, with a default value of false.

The creation SQL statement for the tcc_fence_log table (in MySQL syntax) is as follows:

CREATE TABLE IF NOT EXISTS `tcc_fence_log`
(
    `xid`           VARCHAR(128)  NOT NULL COMMENT 'global id',
    `branch_id`     BIGINT        NOT NULL COMMENT 'branch id',
    `action_name`   VARCHAR(64)   NOT NULL COMMENT 'action name',
    `status`        TINYINT       NOT NULL COMMENT 'status(tried:1;committed:2;rollbacked:3;suspended:4)',
    `gmt_create`    DATETIME(3)   NOT NULL COMMENT 'create time',
    `gmt_modified`  DATETIME(3)   NOT NULL COMMENT 'update time',
    PRIMARY KEY (`xid`, `branch_id`),
    KEY `idx_gmt_modified` (`gmt_modified`),
    KEY `idx_status` (`status`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

4.1 Idempotence​

During the commit/cancel phase, if the TC does not receive a response from the branch transaction, it needs to retry the operation. Therefore, it is necessary for the branch transaction to support idempotence.

Let's take a look at how this is addressed in the new version. The following code is from the TCCResourceManager class:

@Override
public BranchStatus branchCommit(BranchType branchType, String xid, long branchId, String resourceId,
								 String applicationData) throws TransactionException {
	TCCResource tccResource = (TCCResource)tccResourceCache.get(resourceId);
	Object targetTCCBean = tccResource.getTargetBean();
	Method commitMethod = tccResource.getCommitMethod();
	try {
		//BusinessActionContext
		BusinessActionContext businessActionContext = getBusinessActionContext(xid, branchId, resourceId,
			applicationData);
		Object[] args = this.getTwoPhaseCommitArgs(tccResource, businessActionContext);
		Object ret;
		boolean result;
		//whether the useTCCFence property is set to true
		if (Boolean.TRUE.equals(businessActionContext.getActionContext(Constants.USE_TCC_FENCE))) {
			try {
				result = TCCFenceHandler.commitFence(commitMethod, targetTCCBean, xid, branchId, args);
			} catch (SkipCallbackWrapperException | UndeclaredThrowableException e) {
				throw e.getCause();
			}
		} else {
		}
		LOGGER.info("TCC resource commit result : {}, xid: {}, branchId: {}, resourceId: {}", result, xid, branchId, resourceId);
		return result ? BranchStatus.PhaseTwo_Committed : BranchStatus.PhaseTwo_CommitFailed_Retryable;
	} catch (Throwable t) {
		return BranchStatus.PhaseTwo_CommitFailed_Retryable;
	}
}

The above code shows that when executing the commit method of the branch transaction, it first checks if the useTCCFence property is true. If it is true, it follows the commitFence logic in the TCCFenceHandler class; otherwise, it follows the normal commit logic.

The commitFence method in the TCCFenceHandler class calls the commitFence method of the same class. The code is as follows:

public static boolean commitFence(Method commitMethod, Object targetTCCBean,
								  String xid, Long branchId, Object[] args) {
	return transactionTemplate.execute(status -> {
		try {
			Connection conn = DataSourceUtils.getConnection(dataSource);
			TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
			if (tccFenceDO == null) {
				throw new TCCFenceException(String.format("TCC fence record not exists, commit fence method failed. xid= %s, branchId= %s", xid, branchId),
						FrameworkErrorCode.RecordAlreadyExists);
			}
			if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
				LOGGER.info("Branch transaction has already committed before. idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
				return true;
			}
			if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
				if (LOGGER.isWarnEnabled()) {
					LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
				}
				return false;
			}
			return updateStatusAndInvokeTargetMethod(conn, commitMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_COMMITTED, status, args);
		} catch (Throwable t) {
			status.setRollbackOnly();
			throw new SkipCallbackWrapperException(t);
		}
	});
}

From the code, we can see that when committing the transaction, it first checks if there is a record in the tcc_fence_log table. If a record exists, it checks the transaction execution status and returns. This ensures idempotence by avoiding duplicate commits if the transaction status is already STATUS_COMMITTED. If there is no record in the tcc_fence_log table, a new record is inserted for later retry detection.

The rollback logic is similar to the commit logic and is implemented in the rollbackFence method of the TCCFenceHandler class.

4.2 Empty Rollback​

In the scenario shown in the following diagram, the account service consists of a cluster of two nodes. During the try phase, the account service on Node 1 encounters a failure. Without considering retries, the global transaction must reach the end state, requiring a cancel operation to be performed on the account service.

Seata's solution is to insert a record into the tcc_fence_log table during the try phase, with the status field set to STATUS_TRIED. During the rollback phase, it checks if the record exists, and if it doesn't, the rollback operation is not executed. The code is as follows:

//TCCFenceHandler 
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
	return transactionTemplate.execute(status -> {
		try {
			Connection conn = DataSourceUtils.getConnection(dataSource);
			boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
			LOGGER.info("TCC fence prepare result: {}. xid: {}, branchId: {}", result, xid, branchId);
			if (result) {
				return targetCallback.execute();
			} else {
				throw new TCCFenceException(String.format("Insert tcc fence record error, prepare fence failed. xid= %s, branchId= %s", xid, branchId),
						FrameworkErrorCode.InsertRecordError);
			}
		} catch (TCCFenceException e) {
		} catch (Throwable t) {
		}
	});
}

The processing logic in the Rollback phase is as follows:

//TCCFenceHandler 
public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
									String xid, Long branchId, Object[] args, String actionName) {
	return transactionTemplate.execute(status -> {
		try {
			Connection conn = DataSourceUtils.getConnection(dataSource);
			TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
			// non_rollback
			if (tccFenceDO == null) {
				//The rollback logic is not executed
				return true;
			} else {
				if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
					LOGGER.info("Branch transaction had already rollbacked before, idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
					return true;
				}
				if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
					if (LOGGER.isWarnEnabled()) {
						LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
					}
					return false;
				}
			}
			return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
		} catch (Throwable t) {
			status.setRollbackOnly();
			throw new SkipCallbackWrapperException(t);
		}
	});
}

updateStatusAndInvokeTargetMethod method executes the following SQL:

update tcc_fence_log set status = ?, gmt_modified = ?
    where xid = ? and  branch_id = ? and status = ? ;

As we can see, it updates the value of the status field in the tcc_fence_log table from STATUS_TRIED to STATUS_ROLLBACKED. If the update is successful, the rollback logic is executed.

4.3 Hanging​

Hanging refers to a situation where, due to network issues, the RM did not receive the try instruction initially, but after executing the rollback, the RM receives the try instruction and successfully reserves resources. This leads to the inability to release the reserved resources, as shown in the following diagram:

Seata solves this problem by checking if there is a record for the current xid in the tcc_fence_log table before executing the rollback method. If there is no record, it inserts a new record into the tcc_fence_log table with the status STATUS_SUSPENDED and does not perform the rollback operation. The code is as follows:

public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
									String xid, Long branchId, Object[] args, String actionName) {
	return transactionTemplate.execute(status -> {
		try {
			Connection conn = DataSourceUtils.getConnection(dataSource);
			TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
			// non_rollback
			if (tccFenceDO == null) {
				boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_SUSPENDED);
				return true;
			} else {
			}
			return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
		} catch (Throwable t) {
		}
	});
}

When executing the try phase method, a record for the current xid is first inserted into the tcc_fence_log table, which causes a primary key conflict. The code is as follows:

//TCCFenceHandler 
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
	return transactionTemplate.execute(status -> {
		try {
			Connection conn = DataSourceUtils.getConnection(dataSource);
			boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
		} catch (TCCFenceException e) {
			if (e.getErrcode() == FrameworkErrorCode.DuplicateKeyException) {
				LOGGER.error("Branch transaction has already rollbacked before,prepare fence failed. xid= {},branchId = {}", xid, branchId);
				addToLogCleanQueue(xid, branchId);
			}
			status.setRollbackOnly();
			throw new SkipCallbackWrapperException(e);
		} catch (Throwable t) {
		}
	});
}

Note: The queryTCCFenceDO method in the SQL statement uses for update, so there is no need to worry about not being able to determine the execution result of the local transaction in the rollback method due to the inability to obtain records from the tcc_fence_log table.

5 Summary​

TCC mode is a very important transaction mode in distributed transactions. However, idempotence, hanging, and empty rollback have always been issues that need to be considered in TCC mode. The Seata framework perfectly solves these problems in version 1.5.1. The operations on the tcc_fence_log table also need to consider transaction control. Seata uses a proxy data source to execute the operations on the tcc_fence_log table and the RM business operations in the same local transaction. This ensures that the local operations and the operations on the tcc_fence_log table succeed or fail together.

Seata currently supports AT mode, XA mode, TCC mode, and SAGA mode. Previous articles have talked more about non-intrusive AT mode. Today, we will introduce TCC mode, which is also a two-phase commit.

What is TCC

TCC is a two-phase commit protocol in distributed transactions. Its full name is Try-Confirm-Cancel. Their specific meanings are as follows:

1. Try: Check and reserve business resources;
2. Confirm: Commit the business transaction, i.e., the commit operation. If Try is successful, this step will definitely be successful;
3. Cancel: Cancel the business transaction, i.e., the rollback operation. This step will release the resources reserved in Try.

TCC is an intrusive distributed transaction solution. All three operations need to be implemented by the business system itself, which has a significant impact on the business system. The design is relatively complex, but the advantage is that TCC does not rely on the database. It can manage resources across databases and applications, and can implement an atomic operation for different data access through intrusive coding, better solving the distributed transaction problems in various complex business scenarios.

Seata TCC mode

Seata TCC mode follows the same principle as the general TCC mode. Let's first use Seata TCC mode to implement a distributed transaction:

Suppose there is a business that needs to use service A and service B to complete a transaction operation. We define a TCC interface for this service in service A:

public interface TccActionOne {
    @TwoPhaseBusinessAction(name = "DubboTccActionOne", commitMethod = "commit", rollbackMethod = "rollback")
    public boolean prepare(BusinessActionContext actionContext, @BusinessActionContextParameter(paramName = "a") String a);

    public boolean commit(BusinessActionContext actionContext);

    public boolean rollback(BusinessActionContext actionContext);
}

Similarly, we define a TCC interface for this service in service B:

public interface TccActionTwo {
    @TwoPhaseBusinessAction(name = "DubboTccActionTwo", commitMethod = "commit", rollbackMethod = "rollback")
    public void prepare(BusinessActionContext actionContext, @BusinessActionContextParameter(paramName = "b") String b);

    public void commit(BusinessActionContext actionContext);

    public void rollback(BusinessActionContext actionContext);
}

In the business system, we start a global transaction and execute the TCC reserve resource methods for service A and service B:

@GlobalTransactional
public String doTransactionCommit(){
    // Service A transaction participant
    tccActionOne.prepare(null,"one");
    // Service B transaction participant
    tccActionTwo.prepare(null,"two");
}

The example above demonstrates the implementation of a global transaction using Seata TCC mode. It can be seen that the TCC mode also uses the @GlobalTransactional annotation to initiate a global transaction, while the TCC interfaces of Service A and Service B are transaction participants. Seata treats a TCC interface as a Resource, also known as a TCC Resource.

TCC interfaces can be RPC or internal JVM calls, meaning that a TCC interface has both a sender and a caller identity. In the example above, the TCC interface is the sender in Service A and Service B, and the caller in the business system. If the TCC interface is a Dubbo RPC, the caller is a dubbo:reference and the sender is a dubbo:service.

When Seata starts, it scans and parses the TCC interfaces. If a TCC interface is a sender, Seata registers the TCC Resource with the TC during startup, and each TCC Resource has a resource ID. If a TCC interface is a caller, Seata proxies the caller and intercepts the TCC interface calls. Similar to the AT mode, the proxy intercepts the call to the Try method, registers a branch transaction with the TC, and then executes the original RPC call.

When the global transaction decides to commit/rollback, the TC will callback to the corresponding participant service to execute the Confirm/Cancel method of the TCC Resource using the resource ID registered by the branch.

How Seata Implements TCC Mode

From the above Seata TCC model, it can be seen that the TCC mode in Seata also follows the TC, TM, RM three-role model. How to implement TCC mode in these three-role models? I mainly summarize the implementation as resource parsing, resource management, and transaction processing.

Resource Parsing​

Resource parsing is the process of parsing and registering TCC interfaces. As mentioned earlier, TCC interfaces can be RPC or internal JVM calls. In the Seata TCC module, there is a remoting module that is specifically used to parse TCC interfaces with the TwoPhaseBusinessAction annotation:

The RemotingParser interface mainly has methods such as isRemoting, isReference, isService, getServiceDesc, etc. The default implementation is DefaultRemotingParser, and the parsing of various RPC protocols is executed in DefaultRemotingParser. Seata has already implemented parsing of Dubbo, HSF, SofaRpc, and LocalTCC RPC protocols while also providing SPI extensibility for additional RPC protocol parsing classes.

During the Seata startup process, the GlobalTransactionScanner annotation is used for scanning and executes the following method:

io.seata.spring.util.TCCBeanParserUtils#isTccAutoProxy

The purpose of this method is to determine if the bean has been TCC proxied. In the process, it first checks if the bean is a Remoting bean. If it is, it calls the getServiceDesc method to parse the remoting bean, and if it is a sender, it registers the resource:

io.seata.rm.tcc.remoting.parser.DefaultRemotingParser#parserRemotingServiceInfo

public RemotingDesc parserRemotingServiceInfo(Object bean, String beanName, RemotingParser remotingParser){
    RemotingDesc remotingBeanDesc = remotingParser.getServiceDesc(bean, beanName);
    if(remotingBeanDesc == null){
    return null;
    }
    remotingServiceMap.put(beanName, remotingBeanDesc);

    Class<?> interfaceClass = remotingBeanDesc.getInterfaceClass();
    Method[] methods = interfaceClass.getMethods();
    if (remotingParser.isService(bean, beanName)) {
        try {
            //service bean, registry resource
            Object targetBean = remotingBeanDesc.getTargetBean();
            for (Method m : methods) {
                TwoPhaseBusinessAction twoPhaseBusinessAction = m.getAnnotation(TwoPhaseBusinessAction.class);
                if (twoPhaseBusinessAction != null) {
                    TCCResource tccResource = new TCCResource();
                    tccResource.setActionName(twoPhaseBusinessAction.name());
                    tccResource.setTargetBean(targetBean);
                    tccResource.setPrepareMethod(m);
                    tccResource.setCommitMethodName(twoPhaseBusinessAction.commitMethod());
                    tccResource.setCommitMethod(interfaceClass.getMethod(twoPhaseBusinessAction.commitMethod(),
                    twoPhaseBusinessAction.commitArgsClasses()));
                    tccResource.setRollbackMethodName(twoPhaseBusinessAction.rollbackMethod());
                    tccResource.setRollbackMethod(interfaceClass.getMethod(twoPhaseBusinessAction.rollbackMethod(),
                    twoPhaseBusinessAction.rollbackArgsClasses()));
                    // set argsClasses
                    tccResource.setCommitArgsClasses(twoPhaseBusinessAction.commitArgsClasses());
                    tccResource.setRollbackArgsClasses(twoPhaseBusinessAction.rollbackArgsClasses());
                    // set phase two method's keys
                    tccResource.setPhaseTwoCommitKeys(this.getTwoPhaseArgs(tccResource.getCommitMethod(),
                    twoPhaseBusinessAction.commitArgsClasses()));
                    tccResource.setPhaseTwoRollbackKeys(this.getTwoPhaseArgs(tccResource.getRollbackMethod(),
                    twoPhaseBusinessAction.rollbackArgsClasses()));
                    // registry tcc resource
                    DefaultResourceManager.get().registerResource(tccResource);
                }
            }
        } catch (Throwable t) {
            throw new FrameworkException(t, "parser remoting service error");
        }
    }
    if (remotingParser.isReference(bean, beanName)) {
        // reference bean, TCC proxy
        remotingBeanDesc.setReference(true);
    }
    return remotingBeanDesc;
    }

The above method first calls the parsing class getServiceDesc method to parse the remoting bean and puts the parsed remotingBeanDesc into the local cache remotingServiceMap. At the same time, it calls the parsing class isService method to determine if it is the initiator. If it is the initiator, it parses the content of the TwoPhaseBusinessAction annotation to generate a TCCResource and registers it as a resource.

Resource Management​

1. Resource Registration

The resource for Seata TCC mode is called TCCResource, and its resource manager is called TCCResourceManager. As mentioned earlier, after parsing the TCC interface RPC resource, if it is the initiator, it will be registered as a resource:

io.seata.rm.tcc.TCCResourceManager#registerResource

public void registerResource(Resource resource){
    TCCResource tccResource=(TCCResource)resource;
    tccResourceCache.put(tccResource.getResourceId(),tccResource);
    super.registerResource(tccResource);
    }

TCCResource contains the relevant information of the TCC interface and is cached locally. It continues to call the parent class registerResource method (which encapsulates communication methods) to register with the TC. The TCC resource's resourceId is the actionName, and the actionName is the name in the @TwoParseBusinessAction annotation.

2. Resource Commit/Rollback

io.seata.rm.tcc.TCCResourceManager#branchCommit

public BranchStatus branchCommit(BranchType branchType,String xid,long branchId,String resourceId,
    String applicationData)throws TransactionException{
    TCCResource tccResource=(TCCResource)tccResourceCache.get(resourceId);
    if(tccResource==null){
    throw new ShouldNeverHappenException(String.format("TCC resource is not exist, resourceId: %s",resourceId));
    }
    Object targetTCCBean=tccResource.getTargetBean();
    Method commitMethod=tccResource.getCommitMethod();
    if(targetTCCBean==null||commitMethod==null){
    throw new ShouldNeverHappenException(String.format("TCC resource is not available, resourceId: %s",resourceId));
    }
    try{
    //BusinessActionContext
    BusinessActionContext businessActionContext=getBusinessActionContext(xid,branchId,resourceId,
    applicationData);
    // ... ...
    ret=commitMethod.invoke(targetTCCBean,args);
    // ... ...
    return result?BranchStatus.PhaseTwo_Committed:BranchStatus.PhaseTwo_CommitFailed_Retryable;
    }catch(Throwable t){
    String msg=String.format("commit TCC resource error, resourceId: %s, xid: %s.",resourceId,xid);
    LOGGER.error(msg,t);
    return BranchStatus.PhaseTwo_CommitFailed_Retryable;
    }
    }

When the TM resolves the phase two commit, the TC will callback to the corresponding participant (i.e., TCC interface initiator) service to execute the Confirm/Cancel method of the TCC Resource registered by the branch.

In the resource manager, the corresponding TCCResource will be found in the local cache based on the resourceId, and the corresponding BusinessActionContext will be found based on xid, branchId, resourceId, and applicationData, and the parameters to be executed are in the context. Finally, the commit method of the TCCResource is executed to perform the phase two commit.

The phase two rollback is similar.

Transaction Processing​

As mentioned earlier, if the TCC interface is a caller, the Seata TCC proxy will be used to intercept the caller and register the branch before processing the actual RPC method call.

The method io.seata.spring.util.TCCBeanParserUtils#isTccAutoProxy not only parses the TCC interface resources, but also determines whether the TCC interface is a caller. If it is a caller, it returns true:

io.seata.spring.annotation.GlobalTransactionScanner#wrapIfNecessary

As shown in the figure, when GlobalTransactionalScanner scans the TCC interface caller (Reference), it will proxy and intercept it with TccActionInterceptor, which implements MethodInterceptor.

In TccActionInterceptor, it will also call ActionInterceptorHandler to execute the interception logic, and the transaction-related processing is in the ActionInterceptorHandler#proceed method:

public Object proceed(Method method, Object[] arguments, String xid, TwoPhaseBusinessAction businessAction,
    Callback<Object> targetCallback) throws Throwable {
    //Get action context from arguments, or create a new one and then reset to arguments
    BusinessActionContext actionContext = getOrCreateActionContextAndResetToArguments(method.getParameterTypes(), arguments);
    //Creating Branch Record
    String branchId = doTccActionLogStore(method, arguments, businessAction, actionContext);
    // ... ...
    try {
    // ... ...
    return targetCallback.execute();
    } finally {
    try {
    //to report business action context finally if the actionContext.getUpdated() is true
    BusinessActionContextUtil.reportContext(actionContext);
    } finally {
    // ... ...
    }
    }
}

In the process of executing the first phase of the TCC interface, the doTccActionLogStore method is called for branch registration, and the TCC-related information such as parameters is placed in the context. This context will be used for resource submission/rollback as mentioned above.

How to control exceptions

In the process of executing the TCC model, various exceptions may occur, the most common of which are empty rollback, idempotence, and suspense. Here I will explain how Seata handles these three types of exceptions.

How to handle empty rollback​

What is an empty rollback?

An empty rollback refers to a situation in a distributed transaction where the TM drives the second-phase rollback of the participant's Cancel method without calling the participant's Try method.

How does an empty rollback occur?

As shown in the above figure, after the global transaction is opened, participant A will execute the first-phase RPC method after completing branch registration. If the machine where participant A is located crashes or there is a network anomaly at this time, the RPC call will fail, meaning that participant A's first-phase method did not execute successfully. However, the global transaction has already been opened, so Seata must progress to the final state. When the global transaction is rolled back, participant A's Cancel method will be called, resulting in an empty rollback.

To prevent empty rollback, it is necessary to identify it in the Cancel method. How does Seata do this?

Seata's approach is to add a TCC transaction control table, which contains the XID and BranchID information of the transaction. A record is inserted when the Try method is executed, indicating that phase one has been executed. When the Cancel method is executed, this record is read. If the record does not exist, it means that the Try method was not executed.

How to Handle Idempotent Operations​

Idempotent operation refers to TC repeating the two-phase commit, so the Confirm/Cancel interface needs to support idempotent processing, which means that it will not cause duplicate resource submission or release.

So how does idempotent operation arise?

As shown in the above figure, after participant A completes the two phases, network jitter or machine failure may cause TC not to receive the return result of participant A's execution of the two phases. TC will continue to make repeated calls until the two-phase execution result is successful.

How does Seata handle idempotent operations?

Similarly, a status field is added to the TCC transaction control table. This field has 3 values:

1. tried: 1
2. committed: 2
3. rollbacked: 3

After the execution of the two-phase Confirm/Cancel method, the status is changed to committed or rollbacked. When the two-phase Confirm/Cancel method is called repeatedly, checking the transaction status can solve the idempotent problem.

How to Handle Suspend​

Suspension refers to the two-phase Cancel method being executed before the phase Try method, because empty rollback is allowed. After the execution of the two-phase Cancel method, directly returning success, the global transaction has ended. However, because the Try method is executed later, this will cause the resources reserved by the phase Try method to never be committed or released.

So how does suspension arise?

As shown in the above figure, when participant A's phase Try method is executed, network congestion occurs, and due to Seata's global transaction timeout limit, after the Try method times out, TM resolves to roll back the global transaction. After the rollback is completed, if the RPC request arrives at participant A at this time and the Try method is executed to reserve resources, it will cause suspension.

How does Seata handle suspension?

Add a status to the TCC transaction control table:

1. suspended: 4

When the two-phase Cancel method is executed, if it is found that there is no related record in the TCC transaction control table, it means that the two-phase Cancel method is executed before the phase Try method. Therefore, a record with status=4 is inserted. Then, when the phase Try method is executed, if status=4 is encountered, it means that the two-phase Cancel has been executed, and false is returned to prevent the phase Try method from succeeding.

Author Introduction

Zhang Chenghui, currently working at Ant Group, loves to share technology. He is the author of the WeChat public account "Advanced Backend," the author of the technical blog (https://objcoding.com/), and his GitHub ID is: objcoding.

Seata AT mode is a non-intrusive distributed transaction solution. Seata internally implements a proxy layer for database operations. When using Seata AT mode, we actually use the built-in data source proxy DataSourceProxy provided by Seata. Seata adds a lot of logic in this proxy layer, such as inserting rollback undo_log records and checking global locks.

Why check global locks? This is because the transaction isolation of Seata AT mode is based on the local isolation level of supporting transactions. Under the premise of database local isolation level of read committed or above, Seata designs a global write exclusive lock maintained by the transaction coordinator to ensure write isolation between transactions. At the same time, global transactions are by default defined at the read uncommitted isolation level.

Understanding Seata Transaction Isolation Levels

Before discussing Seata transaction isolation levels, let's review the isolation levels of database transactions. Currently, there are four types of database transaction isolation levels, from lowest to highest:

1. Read uncommitted
2. Read committed
3. Repeatable read
4. Serializable

The default isolation level for databases is usually read committed, such as Oracle, while some databases default to repeatable read, such as MySQL. Generally, the read committed isolation level of databases can satisfy the majority of business scenarios.

We know that a Seata transaction is a global transaction, which includes several local transaction branches. During the execution of a global transaction (before the global transaction is completed), if a local transaction commits and Seata does not take any measures, it may lead to reading of committed local transactions, causing dirty reads. If a local transaction that has been committed before the global transaction commits is modified, it may cause dirty writes.

From this, we can see that traditional dirty reads involve reading uncommitted data, while Seata's dirty reads involve reading data that has not been committed under the global transaction, where the global transaction may include multiple local transactions. The fact that one local transaction commits does not mean that the global transaction commits.

Working under the read committed isolation level is fine for the vast majority of applications. In fact, the majority of scenarios that work under the read uncommitted isolation level also work fine.

In extreme scenarios, if an application needs to achieve global read committed, Seata also provides a global lock mechanism to implement global transaction read committed. However, by default, Seata's global transactions work under the read uncommitted isolation level to ensure efficiency in the majority of scenarios.

Implementation of Global Locks

In AT mode, Seata uses the internal data source proxy DataSourceProxy, and the implementation of global locks is hidden within this proxy. Let's see what happens during the execution and submission processes.

1. Execution Process​

The execution process is in the StatementProxy class. During execution, if the executed SQL is select for update, the SelectForUpdateExecutor class is used. If the executed method is annotated with @GlobalTransactional or @GlobalLock, it checks if there is a global lock. If a global lock exists, it rolls back the local transaction and continuously competes to obtain local and global locks through a while loop.

io.seata.rm.datasource.exec.SelectForUpdateExecutor#doExecute

public T doExecute(Object... args) throws Throwable {
    Connection conn = statementProxy.getConnection();
    // ... ...
    try {
        // ... ...
        while (true) {
            try {
                // ... ...
                if (RootContext.inGlobalTransaction() || RootContext.requireGlobalLock()) {
                    // Do the same thing under either @GlobalTransactional or @GlobalLock, 
                    // that only check the global lock  here.
                    statementProxy.getConnectionProxy().checkLock(lockKeys);
                } else {
                    throw new RuntimeException("Unknown situation!");
                }
                break;
            } catch (LockConflictException lce) {
                if (sp != null) {
                    conn.rollback(sp);
                } else {
                    conn.rollback();
                }
                // trigger retry
                lockRetryController.sleep(lce);
            }
        }
    } finally {
        // ...
    }

2. Submission Process​

The submission process occurs in the doCommit method of ConnectionProxy.

1. If the executed method is annotated with @GlobalTransactional, it will acquire the global lock during branch registration:

• Requesting TC to register a branch

io.seata.rm.datasource.ConnectionProxy#register

private void register() throws TransactionException {
    if (!context.hasUndoLog() || !context.hasLockKey()) {
        return;
    }
    Long branchId = DefaultResourceManager.get().branchRegister(BranchType.AT, getDataSourceProxy().getResourceId(),
                                                                null, context.getXid(), null, context.buildLockKeys());
    context.setBranchId(branchId);
}

• When a TC registers a branch, it obtains a global lock

io.seata.server.transaction.at.ATCore#branchSessionLock

protected void branchSessionLock(GlobalSession globalSession, BranchSession branchSession) throws TransactionException {
    if (!branchSession.lock()) {
        throw new BranchTransactionException(LockKeyConflict, String
                                             .format("Global lock acquire failed xid = %s branchId = %s", globalSession.getXid(),
                                                     branchSession.getBranchId()));
    }
}

2）If the execution method has a '@GlobalLock' annotation, the global lock is checked for existence before committing, and if it does, an exception is thrown:

io.seata.rm.datasource.ConnectionProxy#processLocalCommitWithGlobalLocks

private void processLocalCommitWithGlobalLocks() throws SQLException {
    checkLock(context.buildLockKeys());
    try {
        targetConnection.commit();
    } catch (Throwable ex) {
        throw new SQLException(ex);
    }
    context.reset();
}

GlobalLock Annotation Explanation​

From the execution process and submission process, it can be seen that since opening a global transaction with the @GlobalTransactional annotation can check if the global lock exists before transaction submission, why does Seata still provide a @GlobalLock annotation?

This is because not all database operations require opening a global transaction, and opening a global transaction is a relatively heavy operation that involves initiating RPC processes to TC. The @GlobalLock annotation only checks the existence of the global lock during the execution process and does not initiate a global transaction. Therefore, when there is no need for a global transaction but the global lock needs to be checked to avoid dirty reads and writes, using the @GlobalLock annotation is a lighter operation.

How to Prevent Dirty Writes

Let's first understand how dirty writes occur when using Seata AT mode:

Note: Other processes in the branch transaction execution are omitted.

When Business One starts a global transaction containing branch transaction A (modifying A) and branch transaction B (modifying B), Business Two modifies A. Business One's branch transaction A obtains a local lock before Business Two, waiting for Business One to complete the execution of branch transaction A. Business Two then obtains the local lock, modifies A, and commits it to the database. However, Business One encounters an exception during the execution of branch transaction A. Since the data of branch transaction A has been modified by Business Two, Business One's global transaction cannot be rolled back.

How to prevent dirty writes?

1. Business Two uses @GlobalTransactional annotation:

Note: Other processes in the branch transaction execution are omitted.

During the execution of the global transaction by Business Two, when registering the branch transaction before the submission of branch transaction A and acquiring the global lock, it finds that Business One's global lock has not been released yet. Therefore, Business Two cannot commit and throws an exception to roll back, thus preventing dirty writes.

2. Business Two uses @GlobalLock annotation:

Note: Other processes in the branch transaction execution are omitted.

Similar to the effect of @GlobalTransactional annotation, but without the need to open a global transaction, it only checks the existence of the global lock before local transaction submission.

3. Business Two uses @GlobalLock annotation + select for update statement:

If a select for update statement is added, it checks the existence of the global lock before the update operation. Business Two can only execute the updateA operation after the global lock is released.

If only @Transactional is used, there is a possibility of dirty writes. The fundamental reason is that without the GlobalLock annotation, the global lock is not checked, which may lead to another global transaction finding that a branch transaction has been modified when rolling back. Therefore, adding select for update also has a benefit, which is that it allows for retries.

How to Prevent Dirty Reads

Dirty reads in Seata AT mode refer to the scenario where data from a branch transaction that has been committed is read by another business before the global transaction is committed. Essentially, this is because Seata's default global transaction isolation level is read uncommitted.

So how to prevent dirty reads?

Business Two queries A with @GlobalLock annotation + select for update statement:

Adding the select for update statement checks the existence of the global lock before executing the SQL. The SQL can only be executed after the global lock is acquired, thus preventing dirty reads.

Author Bio:

Zhang Chenghui currently works at Ant Group and is passionate about sharing technology. He is the author of the WeChat public account "后端进阶" (Backend Advancement) and the owner of the technical blog (https://objcoding.com/). He is also a Seata Contributor with GitHub ID: objcoding.

In the previous analysis of the new version of the Snowflake algorithm, we mentioned two changes made in the new version:

1. The timestamp no longer constantly follows the system clock.
2. The exchange of positions between node ID and timestamp. From the original: to:

A careful student raised a question: In the new version, the algorithm is indeed monotonically increasing within a single node, but in a multi-instance deployment, it is no longer globally monotonically increasing! Because it is obvious that the node ID is in the high bits, so the generated ID with a larger node ID will definitely be greater than the ID with a smaller node ID, regardless of the chronological order. In contrast, the original algorithm, with the timestamp in the high bits and always following the system clock, can ensure that IDs generated earlier are smaller than those generated later. Only when two nodes happen to generate IDs at the same timestamp, the order of the two IDs is determined by the node ID. So, does it mean that the new version of the algorithm is wrong?

This is a great question! The fact that students can raise this question indicates a deep understanding of the essential differences between the standard Snowflake algorithm and the new version. This is commendable! Here, let's first state the conclusion: indeed, the new version of the algorithm does not possess global monotonicity, but this does not affect our original intention (to reduce database page splits). This conclusion may seem counterintuitive but can be proven.

Before providing the proof, let's briefly review some knowledge about page splits in databases. Taking the classic MySQL InnoDB as an example, InnoDB uses a B+ tree index where the leaf nodes of the primary key index also store the complete records of data rows. The leaf nodes are linked together in the form of a doubly linked list. The physical storage form of the leaf nodes is a data page, and each data page can store up to N rows of records (where N is inversely proportional to the size of each row). As shown in the diagram: The characteristics of the B+ tree require that the left node should be smaller than the right node. What happens if we want to insert a record with an ID of 25 at this point (assuming each data page can only hold 4 records)? The answer is that it will cause a page split, as shown in the diagram: Page splits are unfriendly to I/O, requiring the creation of new data pages, copying and transferring part of the records from the old data page, etc., and should be avoided as much as possible.

Ideally, the primary key ID should be sequentially increasing (for example, setting the primary key as auto_increment). This way, a new page will only be needed when the current data page is full, and the doubly linked list will always grow sequentially at the tail, avoiding any mid-node splits.

In the worst-case scenario, if the primary key ID is randomly generated and unordered (for example, a UUID string in Java), new records will be randomly assigned to any data page. If the page is already full, it will trigger a page split.

If the primary key ID is generated by the standard Snowflake algorithm, in the best-case scenario, only one node is generating IDs within each timestamp. In this case, the algorithm's effect is equivalent to the ideal situation of sequential incrementation, similar to auto_increment. In the worst-case scenario, all nodes within each timestamp are generating IDs, and the algorithm's effect is close to unordered (but still much better than completely unordered UUIDs, as the workerId with only 10 bits limits the nodes to a maximum of 1024). In actual production, the algorithm's effectiveness depends on business traffic, and the lower the concurrency, the closer the algorithm is to the ideal scenario.

So, how does it fare with the new version of the algorithm?

The new version of the algorithm, from a global perspective, produces IDs in an unordered manner. However, for each workerId, the generated IDs are strictly monotonically increasing. Additionally, since workerId is finite, it can divide into a maximum of 1024 subsequences, each of which is monotonically increasing.

For a database, initially, the received IDs may be unordered, coming from various subsequences, as illustrated here:

If, at this point, a worker1-seq2 arrives, it will clearly cause a page split:

However, after the split, interesting things happen. For worker1, subsequent seq3, seq4 will not cause page splits anymore (because there is still space), and seq5 only needs to link to a new page for sequential growth (the difference is that this new page is not at the tail of the doubly linked list). Note that the subsequent IDs of worker1 will not be placed after any nodes from worker2 or beyond (thus avoiding page splits for later nodes) because they are always smaller than the IDs of worker2; nor will they be placed before the current node of worker1 (thus avoiding page splits for previous nodes) because the subsequences of worker1 are always monotonically increasing. Here, we refer to such subsequences as reaching a steady state, meaning that the subsequence has "stabilized," and its subsequent growth will only occur at the end of the subsequence without causing page splits for other nodes.

The same principle can be extended to all subsequences. Regardless of how chaotic the IDs received by the database are initially, after a finite number of page splits, the doubly linked list can always reach a stable state:

After reaching the steady state, subsequent IDs will only grow sequentially within their respective subsequences, without causing page splits. The difference between this sequential growth and the sequential growth of auto_increment is that the former has 1024 growth points (the ends of various subsequences), while the latter only has one at the end.

At this point, we can answer the question posed at the beginning: indeed, the new algorithm is not globally monotonically increasing, but the algorithm converges. After reaching a steady state, the new algorithm can achieve the same effect as global sequential incrementation.

Further Considerations​

The discussion so far has focused on the continuous growth of sequences. However, in practical production, there is not only the insertion of new data but also the deletion of old data. Data deletion may lead to page merging (InnoDB, if it finds that the space utilization of two adjacent data pages is both less than 50%, it will merge them). How does this affect the new algorithm?

As we have seen in the above process, the essence of the new algorithm is to utilize early page splits to gradually separate different subsequences, allowing the algorithm to continuously converge to a steady state. Page merging, on the other hand, may reverse this process by merging different subsequences back into the same data page, hindering the convergence of the algorithm. Especially in the early stages of convergence, frequent page merging may even prevent the algorithm from converging forever (I just separated them, and now I'm merging them back together, back to square one~)! However, after convergence, only page merging at the end nodes of each subsequence has the potential to disrupt the steady state (merging the end node of one subsequence with the head node of the next subsequence). Merging on the remaining nodes of the subsequence does not affect the steady state because the subsequence remains ordered, albeit with a shorter length.

Taking Seata's server as an example, the data in the three tables of the server has a relatively short lifecycle. After a global transaction ends, the data is cleared. This is not friendly to the new algorithm, as it does not provide enough time for convergence. However, there is already a pull request (PR) for delayed deletion in the review process, and with this PR, the effect will be much better. For example, periodic weekly cleanup allows sufficient time for the algorithm to converge in the early stages, and for most of the time, the database can benefit from it. At the time of cleanup, the worst-case result is that the table is cleared, and the algorithm starts from scratch.

If you wish to apply the new algorithm to a business system, make sure to ensure that the algorithm has time to converge. For example, for user tables or similar, where data is intended to be stored for a long time, the algorithm can naturally converge. Alternatively, implement a mechanism for delayed deletion, providing enough time for the algorithm to converge.

If you have better opinions and suggestions, feel free to contact the Seata community!