The Engineer Banker

🔒 Account lifecycle

TEB — Mon, 25 Mar 2024 13:29:38 GMT

Want free access to the article?

Summary

In the ecosystem of banking and finance, the lifecycles of accounts and payments are intertwined, each influencing the other in profound ways. Understanding the dynamics of these interactions is crucial for financial institutions to ensure smooth operations, compliance with regulations, and the delivery of efficient services to customers. In this article, we delve into the relationship between account lifecycles and payment lifecycles, exploring how changes in account statuses impact payment processing and clearing. From account creation to closure, each stage in the account lifecycle plays a pivotal role in shaping the trajectory of payments, affecting everything from transaction routing to risk management.

Welcome to another episode of The Engineer Banker, today we will delve into the interactions between the account lifecycle and the payment lifecycle. In the realm of banking, accounts undergo a lifecycle that dictates their status and accessibility. Understanding this lifecycle is crucial, as it directly impacts payments processing and clearing. Today, we will explore the key statuses in the lifecycle of bank accounts, including creation, opening, closure, seizure, locking, and dormancy. We'll delve into each status, their implications on payments processing, and examples to illustrate their significance. An account lifecycle refers to the different stages an account goes through during its existence with a bank or financial institution. Think of it like the life of a plant, which starts as a seed, grows, blooms, and eventually may wither away. Similarly, an account begins when it's created, then goes through various stages depending on how it's used and managed.

The account lifecycle and payment lifecycle are closely interconnected in both directions.

Each stage of the account's journey has a significant influence on payment processing and clearing. At the onset, during the account creation phase, essential customer information is collected and verified as part of the due diligence process. This information forms the foundation for subsequent payment transactions, serving as a reference point for identity verification and risk assessment. As the account matures, its status may evolve, transitioning through various states such as active, dormant, locked, or closed.

Account and Payment lifecycles influence each other

For instance, an account flagged as dormant may have restrictions and limits on outgoing transactions, affecting payment initiation. Similarly, a locked or seized account due to suspected fraudulent activity can disrupt payment flows, triggering alerts and additional scrutiny from financial institutions. Conversely, when an account is opened, it signals potential for increased payment activity, prompting enhanced monitoring and risk assessment measures. Moreover, account statuses may also impact payment routing decisions, with certain status indicators influencing the choice of payment channels or processing timelines. Overall, the account lifecycle serves as a critical framework for managing risk, ensuring compliance with regulatory requirements, and facilitating seamless payment experiences for both customers and financial institutions.

Payment trajectory depends on account state

In the example provided below, we observe an asynchronous method of altering the trajectory or lifecycle of a transaction. This adjustment is based on the synchronized account state stored within a local cache at the payment processor.

Payment trajectory depends on account state

Also, the payment activity of an account holder plays a significant role in shaping the account lifecycle within banking institutions. A pattern of regular, consistent payments and transactions reflects positively on the account, indicating financial stability and responsible account management. This can contribute to the account remaining in an active or open state, as it demonstrates the account holder's ongoing engagement and utilization of banking services. Conversely, irregular or suspicious payment activity, such as large, unexplained transactions, frequent overdrafts, or involvement in fraudulent schemes, may trigger closer scrutiny from the bank. Such activities can lead to adverse effects on the account lifecycle, potentially resulting in the account being flagged for review, restricted, or even closed due to compliance or security concerns.

Therefore, the payment activity directly influences the health and status of the account, impacting its progression through various lifecycle stages

The account lifecycle, delineating the various stages an account progresses through from creation to closure, can vary significantly from one financial institution to another. Different banks may opt to model their account lifecycles with varying numbers of statuses, tailoring them to suit their specific needs, account types, and financial products. Some institutions might choose to have more nuanced and detailed lifecycle models, incorporating additional statuses to reflect the intricacies of their operations and customer relationships. Factors such as regulatory requirements, risk management strategies, and the complexity of banking services offered can also influence the design and implementation of account lifecycle frameworks. Thus, the configuration and structure of account lifecycles can exhibit considerable diversity across different banks and financial organizations.

Below is an illustrative example showcasing the various states an account can assume. We will subsequently delve into an in-depth explanation of each of these states, discussing their significance and objectives.

Example of an account lifecycle

Enjoyed this content?

Throughout the remainder of this article, we will delve into a comprehensive exploration of how these various states within the account lifecycle can significantly influence the trajectory of payments within a payments processor. By examining the impact of these account states on payment processing, we can gain a deeper understanding of the challenges, opportunities, and potential risks involved in facilitating seamless and efficient payment flows.

The role of suspense accounts

TEB — Tue, 06 Feb 2024 09:36:37 GMT

Welcome to another installment of Payment Bites, in this edition, we're shining a spotlight on the often-mysterious world of suspense accounts. As part of our ongoing series, we aim to demystify the complex mechanisms that underpin the financial industry, providing you with insights into how transactions are managed and processed behind the scenes.

Payments processing entails numerous dependencies within a bank's infrastructure. Today, we delve into some of the core system dependencies involved. We'll explore how the accounting system and its various account types contribute to transaction clearing. For those seeking a refresher on the various dependencies inherent in a bank's payments infrastructure, feel free to read the following article.

In the context of banking, a suspense account serves as a temporary holding account used to manage discrepancies or uncertainties in financial transactions. It acts as a buffer for transactions with incomplete information or under investigation, allowing banks to maintain accurate records while resolving discrepancies.

The purpose of this account is to hold customer funds that have been intercepted for different reasons and are to be temporarily held at this account until routed to their final destination. One of the advantages of using suspense accounts is that they ensure proper bookkeeping even while transactions are in a temporary state. When a transaction encounters an issue or discrepancy, it may not be immediately clear where the funds should be allocated. By temporarily placing them in a suspense account, the transaction can still be recorded in the accounting system, maintaining the accuracy and integrity of financial records. This prevents the funds from being lost or unaccounted for during the resolution process, providing transparency and accountability in financial management

Happy and unhappy case of a transaction monitoring outcome

One common scenario where a suspense account is utilized is when a transaction requires manual review. During this process, the transaction is temporarily posted in the suspense account as you can see in the previous diagram. In general, funds may end up in the suspense account due to various reasons:

Closure of the main account with remaining balance

When a main account is closed but there's a remaining balance, the funds are typically transferred to a suspense account temporarily before the closure. This ensures that the funds are properly accounted for while the closure process is completed. Once the closure process is finalized, any remaining balance in the suspense account can be reconciled and appropriately disbursed according to the bank's policies and procedures.

Fraudulent Transactions: Manual review on compliance check

In cases where payments are suspected to be fraudulent or involve suspicious activity, banks may place the funds in a suspense account while conducting investigations. When the transaction is flagged as high-risk by the monitoring system, the funds intended for that account are typically further scrutinized. Once the compliance issue is resolved or clarified, the funds can be either credited to the correct account if possible or returned to the sender depending on the outcome of the investigation.

Beneficiary Account is non-creditable

When the beneficiary account is non-creditable, meaning it cannot receive funds and the transaction cannot be returned within the same payment scheme the transaction is credited to a separate suspense account because the account is longer able to receive funds. Typically, this triggers a notification in payments operations teams to reach out to the beneficiary to explore options for returning the funds.

A return is initiated for a past transaction that occurred during a period when the account was closed in the interim.

Duplicate transaction detection

When a duplicate transaction is detected, meaning that the same transaction is processed more than once, the duplicate is routed to a suspense account. This ensures that the duplicate transaction does not affect the recipient's account balance or create confusion in the reconciliation process. By crediting the duplicate transaction to a suspense account, it can be properly investigated and resolved without impacting the normal flow of funds. Additionally, this process allows for accurate tracking of duplicate transactions and facilitates their swift resolution by the appropriate teams

Inability to locate the original transaction

When there is an inability to locate the original transaction, perhaps due to technical issues or data discrepancies, the subsequent transaction is directed to a suspense account

It’s a wrap, suspense accounts play a crucial role in payments processing by temporarily holding funds for transactions that for one reason or another cannot be fully cleared on the beneficiary side. They enable banks to maintain accurate records, prevent errors, and ensure that payments are properly allocated to the intended recipients.

🔒 Understanding account proxies and aliases

TEB — Mon, 29 Jan 2024 07:01:28 GMT

Summary

The article explores the innovative concept of account proxies, offering simplified identifiers like phone numbers and email addresses for financial transactions. It delves into the two implementation approaches: centralized, where a single entity manages the mapping of proxies to bank accounts, and distributed, where multiple entities maintain their own mapping systems. The piece further investigates examples like UPI (Unified Payments Interface) in India, Zelle in the US and Bizum in Spain that showcase the practicality and efficiency of these systems.

Welcome to another episode of Payment Fundamentals!. In today's digital age, where convenience and efficiency are paramount, payment systems have undergone significant evolution. Bank account proxies or aliases, such as phone numbers and email addresses, are revolutionizing the way we conduct financial transactions.

Account aliases serve as intermediaries between users and their bank accounts, simplifying the process of making payments and facilitating seamless transactions across various platforms. They act as virtual representations of a user's bank account, allowing them to make payments without directly exposing account information.

Instead of cumbersome account numbers, these proxies serve as user-friendly identifiers, simplifying payments and transfers. With the rise of digital banking and mobile payments, linking bank accounts to familiar contact details offers convenience and accessibility.

For instance, individuals can seamlessly send money using just a recipient's phone number or email address, eliminating the need to remember complex account information. This shift towards account proxies heralds a new era of effortless and secure financial interactions in the digital age.

To ensure the functionality of an alias, all participating banks need to share this information within a directory. This directory serves as a distinct infrastructure that maintains the link between the bank account identifier (e.g., IBAN) and the account alias (e.g., email, phone number, national ID). The exchange of this pair, consisting of the account identifier and alias, typically occurs during the customer's Know Your Customer (KYC) process or when the customer agrees to the terms and conditions of the product offering alias functionality. Once this information exchange takes place, the alias can be used during transaction initiation, being resolved beforehand for the final payment to be directed to the appropriate account number within the banking system.

Exchange and initiation with Alias/Proxy

Banks synchronize their account identifiers, such as IBAN and BIC, at specific intervals. Subsequently, they can resolve these proxies at a later stage to obtain the actual account identifier of the beneficiary.

The use of account proxies not only facilitates push payments but also plays a crucial role in pull mechanisms like request-to-pay, streamlining the payment process and enhancing convenience for users. By associating a proxy, such as a phone number or email, with a bank account, request-to-pay mechanisms simplify transaction initiation. For a refresher on how request-to-pay and proxies interact, we recommend reading the following article.

In the realm of payment infrastructure implementations, there exists a spectrum of approaches, with some systems opting for a centralized directory model, while others favor a more distributed architecture. The choice between these two models often hinges on the need for flexibility in design and the desire to leverage existing directories within the region. A centralized directory offers streamlined management and uniformity but may lack the adaptability required for certain use cases. Conversely, a distributed approach provides greater flexibility and resilience, allowing for tailored solutions that can integrate seamlessly with diverse existing systems and directories across the region.

In this model there is one unique repository of account aliases

In the distributed scenario of account proxy resolution, multiple directories collaborate to achieve the ultimate resolution of the proxy.

Sweden RIX-INST

TEB — Mon, 08 Jan 2024 07:01:06 GMT

Welcome to a new episode of Payment Bites! In our previous episode on SEPA cross-border transactions, we introduced TIPS, a critical infrastructure in Europe for handling instant payments. Today, we delve deeper into its two distinct settlement methods. If you're just joining us, you can catch up by reading our two earlier articles where we provided an introduction to TIPS.

TIPS, TARGET Instant Payment Settlement System, offers the unique ability to settle transactions in central bank money. TIPS also boasts multi-currency settlement capabilities, making it a versatile solution. With TIPS, transactions are not only lightning-fast but also flexible, catering to a wide range of currencies, and backed by the security of central bank money settlement.

With the inclusion of Riksbank, central bank of Sweden, into TIPS service, a novel settlement model called Single Instructing Party (SIP) was introduced. This SIP model enables immediate settlement without the requirement for reserving funds in TIPS, following successful validations and checks by the SIP. Initially designed for the SEK market, the SIP model may also interest the EUR community in the future.

The necessity for this model arose during the preparations for incorporating the Swedish krona into TIPS. In Sweden, instant payments presently settle via a domestic technical infrastructure. To adapt to this setup and streamline settlement in central bank money, RIX-INST will assume the role of the instructing party for both originator and beneficiary banks, differing from the standard TIPS settlement model.

Swish also stands out as a key player in the Swedish market. Swish launched in 2012, is a leading mobile payment application in Sweden, created by major Swedish banks. It simplifies real-time money transfers by allowing transactions through just phone numbers. Its ease of use and integration with the Swedish banking system have made it a key player in Sweden's shift towards a cashless society. Efforts are in progress to transition Swish payments clearing and settlement to RIX-INST, with completion anticipated in early 2024. Upon completion, Swish will become the inaugural service to utilize RIX-INST.

As highlighted earlier in the article, TIPS incorporates two settlement models, standard and single instructing party or SIP. For an in-depth review of the standard settlement model in SICT, please refer to the article linked here.

As a summary to the previous article, in the conventional SCT Inst settlement model, upon successful validation of the payment transaction, TIPS reserves funds to guarantee the availability of resources for settlement once the reply from the Beneficiary Participant is received. This ensures that the transaction won't be declined due to insufficient funds. Should the Beneficiary Participant or Instructing Party reject the payment, TIPS will reverse the fund reservation and reject the transaction.

The single instructing party settlement model comes into play when both the debtor and creditor PSPs are connected to TIPS via the same instructing entity. In such cases, the majority of the pacs.008 and pacs.002 transmissions occur within the instructing party's domain. TIPS intervenes only in the final stage, executing the ultimate settlement in the Dedicated Cash Accounts (DCAs).

RIX-INST is acting as instructing party for both debtor and creditor PSP

TIPS does not require fund reservation at this stage, as it is already confirmed that the beneficiary has accepted the transaction. Before executing the settlement, TIPS conducts several validations. If these validations are successful, the accounts are updated, and a confirmation message is sent to the instructing party's infrastructure. However, if the validations fail due to insufficient balance for the operation, a rejection message is dispatched back to the instructing party. TIPS is capable of settling payments only when the currency of the account being debited aligns with both the currency of the account being credited and the currency used in the payment transaction.

See you until next time!, for a deeper understanding of TIPS, feel free to explore our comprehensive article on SEPA Instant Cross-Border.

SEPA Instant CSMs interoperability

TEB — Sun, 31 Dec 2023 13:59:27 GMT

Welcome back to another exciting episode of Payment Bites. In today's session, we're building upon our previous article. In our previous article about SEPA Instant cross-border, we introduced the concepts of instant transactions between borders.

In today's discussion, we will delve into some insights concerning the diagram we introduced, using it as a practical exercise. Before we dive in, I encourage you to revisit the diagram, reviewing the article where we initially introduced the concepts and taking some time to formulate your own conclusions. Once you've done that, we can delve into a deeper analysis together.

SICT Architectural Example

🎯 Insights and conclusions

Domestic CSM1, domestic CSM2 and RT1 are acting as TIPS instructing parties.
Transactions between PSP1, PSP2 and PSP3 are cleared and settled within the domestic CSM1 technical accounts except for those transactions initiated by PSP3 directed to to either PSP1 or PSP2 that will settle in TIPS accounts.
Transactions between any participant connected to CSM1 and the rest of the community (PSP4, PSP5, PSP6, PSP7) will get routed to TIPS and will settle in TIPS accounts.
Transactions between PSP4 and PSP5 will clear and settle locally in CMS2.
Any payment initiation from PSP4 or PSP5 will get routed by CSM2 to either RT1 or TIPS depending on the reachability of the beneficiary BIC.
Transactions involving PSP6, PSP4, and PSP5 will default to settlement through RT1. However, if any of these parties explicitly express a preference for TIPS settlement, that choice will be honored.
Any payment involving PSP7 as instructing or instructed agent will settle in TIPS.
The treasury teams of PSP4 and PSP5 need to manage three separate settlement accounts.

Your turn – any additional insights or conclusions to share?

🔒 SEPA Instant cross-border

TEB — Sat, 23 Dec 2023 22:07:57 GMT

Summary

This article provides a comprehensive overview of the evolution of SEPA Instant infrastructures across Europe. It delves into the details of the formation process, shedding light on how these infrastructures have developed over time. Additionally, the article explores the nuances of routing within both domestic and pan-European infrastructures, offering valuable insights into the path that instant transactions follow. Finally, it offers a detailed explanation of the TIPS settlement model, providing a holistic view of the entire SEPA Instant landscape.

Welcome to another episode of Realtime Infrastructures, where we take a deep dive into the intricate international routing mechanisms of SEPA Instant payments and explore the fascinating journey of an instant transaction as it traverses the diverse landscape of Europe's financial ecosystem. To comprehend this journey, we must first grasp the ever-evolving landscape of SEPA Instant in Europe today.

The inception of the SEPA Instant Credit Transfer (SCT Inst) scheme dates back to November 21, 2016, when the European Payments Council (EPC) officially released its pioneering rulebook. This momentous event heralded the birth of a groundbreaking system designed to seamlessly facilitate real-time euro (EUR) payments across the Single Euro Payments Area (SEPA). This foundational rulebook laid down the structural groundwork, offering a robust framework that allowed participating financial institutions to embrace and operationalize instant credit transfers within the SEPA zone.

The first SEPA Instant transaction was conducted by CaixaBank, a Spanish financial institution. CaixaBank made history by initiating the first SEPA Instant Credit Transfer (SCT Inst) transaction, marking a significant milestone in the adoption of real-time euro payments across the Single Euro Payments Area (SEPA).

Since its initial debut, the SCT Inst rulebook has undergone a series of meticulous updates and revisions. These ongoing refinements have been instrumental in elevating the scheme's capabilities, expanding its reach, and ensuring that it continues to meet the evolving needs of Europe's dynamic payments landscape.

In the wake of this landmark release, numerous nations and regions across Europe embarked on their individual implementations of the SCT Inst rulebook. Each implementation brought its unique nuances and adaptations to the framework, aligning it with the specific requirements and practices of the local financial industry. SEPA Instant Credit Transfer schemes were gradually being implemented by various countries within the Single Euro Payments Area (SEPA).

This is just a glimpse of the domestic implementations of SICT since its launch:

Spain: In Spain, the Spanish Payment System Operator (Iberpay) launched its SCT Inst scheme in December 2017.
France: France implemented its SCT Inst scheme in November 2018, led by the French Prudential Supervision and Resolution Authority (ACPR).
Lithuania: Lithuania's SCT Inst scheme was introduced in May 2018, coordinated by the Bank of Lithuania.
Slovenia: Slovenia adopted its SCT Inst scheme in March 2019, managed by the Bank of Slovenia.
Portugal: Portugal introduced its SCT Inst scheme in September 2018. The scheme was managed by the Portuguese Interbank Company (SIBS).

Simultaneously, the European Banking Association (EBA) initiated the development of a pan-European implementation in line with the rulebook. The RT1 (Real-Time 1) platform, operated by EBA CLEARING, was officially launched and made available to the public on November 21, 2017. RT1 is a pan-European infrastructure for processing SEPA Instant Credit Transfers (SCT Inst) in real-time, enabling financial institutions to offer instant payment services to their customers across the SEPA zone.

The European Central Bank embarked on the process of implementing its own version of the rulebook, adopting a pan-European approach as well. TIPS (TARGET Instant Payment Settlement) was officially launched and made available to the public by the European Central Bank (ECB) on November 30, 2018. TIPS is a real-time payment settlement system for instant euro (EUR) transactions, designed to enable financial institutions to settle payments in real-time, 24/7, across the Single Euro Payments Area (SEPA). TIPS was introduced to support the growing demand for real-time payments and to provide a pan-European solution for instant payments.

In the beginning, these domestic infrastructures ushered in real-time domestic transactions. Furthermore, they paved the way for the emergence of real-time person-to-person (P2P) payment methods, giving rise to innovative solutions such as Bizum in Spain and similar offerings in other jurisdictions. However, on the downside, this expansion led to a fragmented landscape, with numerous domestic implementations making the achievement of seamless European connectivity increasingly challenging. Those domestic implementations could only settle transactions between banks connected to the same CSM.

In one of our previous articles, we delved into the details of liquidity management within Ancillary System 6, particularly focusing on RT1. If you're looking to gain a deeper understanding of the liquidity management aspect within the SEPA Instant Credit Transfer (SICT) framework, it is recommended giving that article a read.

If you need a refresher on how the pacs.008 message is transported and utilized, consider reading these two informative pieces:

🎄 Unwrap the Holiday Magic! 🎁

TEB — Sat, 23 Dec 2023 21:39:01 GMT

Get into the festive spirit with our special Christmas offer! 🎁 Sign up for our newsletter today and enjoy a 25% discount on your paid subscription. Don't miss out on this exclusive deal to make your holiday season even brighter ✨.

Hurry, this merry offer won't last forever! 🎅

Get 25% off forever

ACH Crime Prevention: PayGuard and FPAD

TEB — Fri, 08 Dec 2023 20:11:18 GMT

Welcome to another insightful episode of The Engineer Banker! In today's discussion, we delve into the dynamic world of ACH systems, exploring their evolving capabilities in the ongoing battle against fraud within connected institutions. This article will be followed by another in which we explore how banks can effectively employ and integrate these crime prevention capabilities.

Automatic Clearing Houses (ACH) and Clearing and Settlement Mechanisms (CSM) around the world have undergone a substantial transformation, adapting to the evolving landscape of electronic payments. This evolution extends well beyond the traditional clearing process, incorporating new capabilities and infrastructure enhancements.

One of the most significant developments in the ACH realm is the introduction of "Request to Pay" (RTP). This feature empowers payees to send payment requests to payers, adding flexibility and interactivity to the payment process. RTP enables payers to exercise greater control over the timing and nature of their payments.

Furthermore, some ACH systems have embraced the concept of aliases and proxies for account numbers and International Bank Account Numbers (IBANs). This innovation enhances convenience by allowing users to employ user-friendly identifiers instead of sharing account details.

Real-time payments have also emerged as a core feature, enabling immediate fund transfers between accounts. This functionality proves invaluable for urgent or time-sensitive transactions.

Lately, security has been a paramount concern in this evolution. ACH networks have implemented advanced security modules, leveraging technologies like machine learning and artificial intelligence. These modules operate in real time, swiftly identifying and preventing suspicious transactions to safeguard against financial crimes like fraud and money laundering.

Until the advent of instant payments, the primary focus was on fraud in card payments, as it is the most widely used payment method in both physical and online commerce. In the case of card fraud, unlike account-to-account payments, the bank or card issuer has complete autonomy to investigate and monitor their customer's payment transactions. For each transaction a customer makes, the issuing bank pre-approves the transaction based on the cardholder's balances, credit, or risks, as well as the characteristics of the merchant. Therefore, in the event of fraud, the bank has the most relevant data and information about the card's transactions, enabling them to analyze and investigate the entire transaction process.

This is not the case with account-to-account payment transactions, where the issuing bank of the transfer only has data about its own customer, and funds are transferred to an unknown beneficiary account, unknown to both the issuing bank and the customer. This complicates autonomous fraud analysis and investigation by the issuing bank. Furthermore, in case of fraud, funds can be transferred to various different accounts and subsequently move to new accounts within the banking system, making the information available to the issuing bank very limited and hindering proper monitoring of the transactions.

Analyzing a money laundering cycle that involves multiple banks can be a daunting task. Each bank typically possesses only a partial view of the overall transaction flow, making it challenging to identify suspicious activities in isolation. However, the central ACH infrastructure holds a unique vantage point in this regard. With access to a comprehensive overview of transaction data across connected institutions, the ACH is ideally positioned to detect and analyze intricate money laundering schemes, particularly those involving mule accounts.

In the illustration below, we present a visual representation of a bank's transaction graph visibility. The nodes represent the accounts and the edges represent the transactions. The bank can only see its internal accounts and external accounts that make transactions with its internal accounts. Every red account and red transaction are invisible to the bank.

For all these reasons, sectorial collaboration among banks is crucial to prevent fraud in account-to-account payments, protect customers, and ensure the security of their transactions. Banks need to share information and data to combat fraud, and this sharing cannot be partial or bilateral; it must be approached collaboratively, securely, and at a neutral point by all banks involved.

The recent announcement of Iberpay's Payguard functionality is a testament to the ongoing evolution of payment systems. Just as ACH networks have been adapting and enhancing their capabilities, Iberpay's Payguard represents a significant stride in fortifying payment security within the financial sector. Payguard, developed by the Spanish payment system operator Iberpay, exemplifies the industry's commitment to bolstering cybersecurity and fraud prevention efforts.

Payguard employs advanced fraud detection and prevention measures, safeguarding both financial institutions and consumers against a spectrum of cyber threats and serves as a sentinel against fraudulent transactions, offering real-time monitoring and risk assessment capabilities. For an introduction to Mule accounts you can read our introduction in the following article:

Another project with shared goals is EBA’s FPAD. The European Banking Authority's (EBA) Fraud Prevention and Awareness Division (FPAD) is a pivotal initiative designed to fortify cybersecurity and combat fraud within the European financial landscape. It operates as a multifaceted entity focused on various facets of fraud prevention, detection, and response.

The European banking landscape is in the midst of a substantial transformation, largely influenced by the recently published preliminary perspectives on the scope and requirements under PSD3. This development is shaping the direction of the market, emphasizing the need to enhance and refine the payment ecosystem. While PSD3 garners significant attention, the introduction of the Fraud Pattern and Anomaly Detection (FPAD) functionality, spearheaded by EBA CLEARING, stands out as a critical component in the ongoing efforts of Financial Institutions to manage and mitigate payment fraud risks.

FPAD, short for Fraud Pattern and Anomaly Detection, represents a pan-European initiative aimed at delivering tools for fraud prevention and detection. It comprises two key components: the implementation of an IBAN verification check, akin to the Confirmation of Payee (CoP) scheme in the UK, and the creation of a fraud detection model. These components work together to assess the probability of fraud risk within a payment based on performance and attributes within the payment message.

FPAD's overarching objective is to empower users of STEP2 and RT1 systems, allowing them to supplement their individual risk assessments with insights into patterns and anomalies at a central infrastructure level.

EBA CLEARING, at the forefront of this initiative, has recently issued specifications for the FPAD functionality. They have also launched a developer portal equipped with a sandbox, providing valuable support for users in developing and testing FPAD's application programming interfaces (APIs). This journey commenced in early 2023, following the release of a blueprint and subsequent user consultations. STEP2 and RT1, both pan-European retail payment systems processing SEPA Credit Transfers, Direct Debits (STEP2), and SEPA Instant Credit Transfers (RT1), are operated by EBA CLEARING. FPAD collects and analyzes vast amounts of STEP2 and RT1 data related to fraudulent activities, including patterns, trends, and modi operandi.

As preparations for the FPAD functionality's go-live intensify, nine banks from six countries actively contribute to the data model training during the analytical pilot phase. This phase aims to develop models for identifying fraud patterns and qualifying anomalies in collaboration with users' feedback.

The introduction of FPAD marks a significant milestone for European banks. It addresses the growing need for heightened security and more robust tools to combat fraud, particularly in the context of the rapid growth of instant payments. FPAD represents a unified approach where banks across Europe unite to bolster fraud-fighting capabilities within the SEPA perimeter.

FPAD, powered by APIs, represents a remarkable advancement for European banks. Beyond fraud detection, it fosters an integrated, efficient, and secure banking ecosystem, vital as instant payments become the norm. However, successful implementation relies on how banks connect to this system. Transitioning to an API-driven approach is crucial for seamless integration with advanced systems like FPAD, especially for institutions with legacy payment architectures.

The introduction of FPAD exemplifies the collaborative spirit of European banks, setting a benchmark for the global banking community in combating fraud and ensuring customer safety and security.

Similar initiatives to Payguard and FPAD have been emerging in various parts of the world, especially within Automated Clearing Houses (ACHs) and financial institutions. Here are a few examples from different regions:

NACHA's Phixius: NACHA, the organization responsible for managing the ACH network in the United States, has introduced Phixius. It's a platform designed to facilitate secure data sharing between financial institutions and other entities to streamline the verification of account information, reducing fraud and errors in ACH payments.
SWIFT's Payment Control Service: SWIFT, a global provider of secure financial messaging services, offers a Payment Control Service that enables financial institutions to screen and control their payment messages in real-time. It helps identify and prevent fraudulent or non-compliant transactions.
RBI's Bharat Bill Payment System (BBPS): In India, the Reserve Bank of India (RBI) introduced BBPS to facilitate the bill payment ecosystem. BBPS includes fraud prevention measures and standardized protocols to secure bill payments made through various channels.
Payments Canada's Real-Time Rail (RTR): Payments Canada is developing a Real-Time Rail system in Canada, which is expected to include advanced fraud detection and prevention features to safeguard real-time payments.

In the ongoing fight against financial crime, initiatives like FPAD (Fraud Pattern and Anomaly Detection) and Payguard are instrumental. They disrupt and deter money laundering schemes, especially those involving mule account chains. As the financial landscape evolves, these initiatives, driven by technology and collaboration, promise a more secure future for the global banking community, ensuring the integrity of financial systems worldwide. Stay tuned for the next episode where we will see how banks can use these new ACH fraud functionalities.

Stay tuned for the upcoming episode, where we'll explore how banks can leverage these cutting-edge anti-fraud features.

The Power of ISO 20022 IDs and References

TEB — Sun, 03 Dec 2023 17:46:25 GMT

Welcome to another episode of Payment Bites. Today, we'll delve into the diverse world of identifiers and references commonly encountered in ISO 20022 credit transfers. At its core, ISO 20022 is designed to facilitate precision, consistency, and interoperability in financial communication. Identifiers, as integral components of this standard, come in various forms, each with its own unique purpose. These identifiers encompass everything from globally unique message IDs, payment instruction identifiers, and transaction-specific codes to references for remittance information and local instruments.

For an in-depth exploration of ISO 20022, we invite you to delve into our comprehensive introduction featured in the following article.

Let's explore two fundamental concepts in financial transactions: point-to-point and end-to-end identifiers. Think of them as labels on a journey. Point-to-point identifiers are like tags specific to certain stops, while end-to-end identifiers are like a unique tracking number that stays the same throughout the entire trip. These concepts simplify data flow, ensuring financial information reaches its destination accurately and securely.

Point-to-point data is essential in payment messaging, pertaining exclusively to a specific payment stage. For instance, in a multi-bank cross-border payment, details like intermediary bank information and fees are considered point-to-point data, vital for that specific stage but not needed in later messages.

Conversely, end-to-end data exchange entails information that must traverse the entire payment chain unaltered. This data maintains its integrity from initiation through to its final destination, ensuring a seamless and coherent transaction process.

End-to-end data is vital in payment messaging, maintaining its significance across the entire transaction journey. For instance, a unique transaction reference code serves as end-to-end data, ensuring consistency and reliability from initiation to the payment's final destination, regardless of how many stages it passes through. It remains constant and unmodified as the payment moves from originator to beneficiary, offering a consistent reference point for all stakeholders involved.

In the diagram above, we present an illustrative example of a correspondent banking payment execution, showcasing a chain of three interconnected banks. Here are some key points to take note of regarding the preceding diagram:

Bank B generate their own point to point identifications
Bank C does not know point to point identifications of Bank A.
Intermediary parties like Bank B do not generate end to end references.

The following diagram offers a detailed view of the identifiers integrated into a credit transfer under ISO 20022.

To delve deeper into the Business Application Header, we invite you to explore our earlier post dedicated to this topic.

Message Identification (MsgId)

Message identification is the reference that CSM will check for uniqueness during the business day. A Message Identification (MsgId) is a globally unique identifier assigned to a specific message. It uniquely identifies the entire message, which may contain one or more payment instructions. It applies to the entire message, encompassing all the payment instructions it contains. In a batch of multiple payment instructions within a single message, the MsgId identifies the entire message itself.

Instruction Identification (InstrId)

An Instruction Identification (InstrId) is an identifier assigned to a specific payment instruction within a message. It helps distinguish and manage individual payment instructions within the message. In a message with multiple payment instructions, each payment instruction would have its own InstrId to uniquely identify it.

Transaction Identification (TxId)

A Transaction Identification (TxId) is an identifier assigned to an individual financial transaction. It is used to uniquely identify each transaction, whether it's a payment, trade, or another financial operation. It applies to a single transaction, which may or may not be part of a larger payment instruction or message. In a payment instruction to transfer funds to multiple beneficiaries, each beneficiary's transaction within that instruction would have its own TxId.

Unique End-to-End Reference (UETR)

This is a tracking reference that allows to track where the payment is. It is becoming a standard for cross border payments. You can use the transaction identification or the UETR. ISO requires either transaction identification or UETR which is an end to end reference in the interbank space. In ISO 20022 messages, the "Unique End-to-End Reference" (UETR) is a key field that can be used to trace and link the entire transaction from the initiating party to the final recipient. The UETR is designed to be globally unique and remains unchanged throughout the entire lifecycle of the transaction, ensuring that it can be tracked and reconciled accurately.

The UETR plays a crucial role in enhancing transparency, traceability, and security in financial transactions, making it easier to identify, monitor, and resolve issues or discrepancies that may arise during the transaction process. It is an important element of SWIFT gpi as we discussed in our SWIFT Overview article.

End-to-End Identification (ETEI)

Reference that the debtor wants to communicate all the way to the creditor. It serves as a unique end-to-end identification traveling between parties, primarily pertinent to customers, rather than financial institutions. Unlike references that financial institutions use for their internal processes, this identifier is customer-centric therefore this is not a reference of interest for the financial institution. End to end between the customers.

🔒 Payments Threat Modeling

TEB — Sun, 26 Nov 2023 20:47:16 GMT

Summary

Threat modeling in payment systems is crucial for identifying and addressing security threats in the rapidly evolving digital payments landscape. This article emphasizes the importance of cybersecurity, heightened by increased regulatory scrutiny and the severe consequences of data breaches. It details a structured approach to threat modeling, involving collaborative meetings where teams scrutinize their system's architecture against potential threats. Key areas of focus include authentication, authorization, data validation, and configuration management. This proactive process is vital in ensuring robust security measures, safeguarding consumer data, and maintaining trust in payment systems amidst a broad spectrum of risks.

Greetings and welcome to another installment of our security series! Threat modeling in the realm of payment systems is a crucial process that involves identifying, assessing, and addressing potential security threats. This proactive approach is vital in the constantly evolving landscape of digital payments, where new technologies and methods are perpetually introduced, bringing along their own set of vulnerabilities and risks.

In the early days of electronic payments, the primary concern was the physical security of transaction data. However, as technology advanced and digital transactions became more prevalent, the focus shifted towards cybersecurity. Today, threat modeling in payments encompasses a broad spectrum of potential risks, from data breaches and fraud to system outages and compliance violations.

The importance of threat modeling in payments is further highlighted by the increasing regulatory scrutiny in this sector. Data protection regulations like GDPR (General Data Protection Regulation) in Europe and various state-level laws in the United States impose strict penalties for data breaches. This regulatory environment makes it imperative for payment providers to adopt robust security measures to protect consumer data.

Moreover, the reputational damage from security incidents can be severe. Consumers trust payment providers with their sensitive financial information, and a breach of that trust can lead to loss of business and long-term damage to a company's reputation. In our comprehensive article linked below, we delve into essential security properties that are imperative to validate as you approach the launch of your product. You can utilize this article as a foundational reference for the current one.

Also, explore our comprehensive article on SIEM, learn how Security Information and Event Management systems empower proactive threat detection and response, forming a vital part of a robust cybersecurity strategy:

Threat modeling consists of a series of structured meetings where teams systematically analyze and contrast their system's architecture and design against potential security threats. This process is not merely a one-off exercise but an ongoing practice integral to the lifecycle of a system, especially in high-stakes environments like payment processing. During these meetings, participants from various disciplines – including software development, network engineering, security, and often representatives from compliance and risk management – collaborate to scrutinize every aspect of the system's architecture. The objective is to ensure that the system is not only robust in its current state but also prepared to withstand evolving security threats and challenges. Engaging in these discussions will provide you with a critical perspective, allowing you to examine and evaluate your product exclusively through the lens of security.

Below, you'll find a possible guide to structure your threat modeling meetings and the topics to be addressed within each. Often, teams are unable to complete the entire threat modeling process in a single session, so additional sessions may need to be scheduled.

1. Identify security objectives

Black Swan Incident in FPS

TEB — Sun, 19 Nov 2023 09:46:33 GMT

At precisely 3 p.m., as the London skyline was bathed in the dull gray of an overcast afternoon, Michael, a seasoned engineer at UK's Faster Payments infrastructure, sat hunched over his computer, a lukewarm cup of tea forgotten at his side. His eyes, usually calm and measured, widened in alarm. The central transaction view, the heart of their operation, was showing inconsistencies he had never seen before. Data that should have been flowing like the Thames was now a jumbled mess, impossible to reconcile.

"This can't be right," he muttered, his fingers flying over the keyboard in a frantic dance of keystrokes. He refreshed screens, ran diagnostic tools, but the results were the same - an inexplicable anomaly in the transaction data. A black swan incident, unforeseen and potentially catastrophic, was unfolding.

With a sense of foreboding, Michael grabbed his phone to alert the Incident Commander. The weight of responsibility pressed down on him like the heavy London fog. Within minutes, a flurry of activity erupted. The incident response team, a seasoned group of crisis managers and tech gurus, assembled in the war room.

“Everyone, we’ve lost our eyes on the transactions. The central view is compromised,” Michael announced, his voice steady despite the chaos swirling in his mind.

The room, filled with the best minds in the industry, fell into a tense silence. This was not a routine glitch. This was a threat to the financial lifeline of the nation, where millions of transactions were processed every hour.

“Initiate the backup systems,” commanded the Incident Commander, a steely edge to her voice. “I want eyes on every transaction. Manually, if we have to.”

With the clock ticking, and the UK's financial stability hanging in the balance, Michael and his team embarked on a treacherous journey through the labyrinth of their advanced payment system.

As the team dove into the manual reconciliation process, the war room transformed into a hive of intense focus and rapid communication. Teams were quickly assigned to different segments of the transaction data, each meticulously combing through records to identify discrepancies and validate transactions. Michael coordinated with the data analysis team, cross-referencing their findings against backup logs and historical patterns. The task was Herculean - reconciling millions of transactions manually was like finding a needle in a haystack, except the haystack was constantly growing.

In another corner of the room, a subgroup focused on constructing a real-time tracking dashboard, piecing together fragments of data to restore some semblance of visibility. Meanwhile, liaisons from customer service and public relations were crafting messages to stakeholders and the public, preparing for the inevitable inquiries and concerns.

Michael, amidst the controlled chaos, couldn't help but feel a sense of awe at the team's synergy and expertise. Despite the daunting task, there was a shared determination to restore order, to piece together the puzzle that the missing central view had scattered. As they worked tirelessly into the evening, the first glimmers of success began to emerge, with critical transactions being identified and verified. The path to full recovery was still long and uncertain, but the team had taken its crucial first steps…

This previous text mirrors the real-life crisis that struck the UK's Faster Payments Service (FPS) back in July 2018. In that instance, pay.uk's FPS infrastructure experienced a significant disruption, affecting countless transactions and sending shockwaves across the financial landscape. A black swan event or incident refers to an unpredictable, rare occurrence that has significant and widespread ramifications. Coined by Nassim Nicholas Taleb in his 2007 book, "The Black Swan," the term originally described financial market surprises but has since been applied more broadly.

These events are characterized by their extreme rarity, severe impact, and the widespread insistence they were obvious in hindsight. In the context of technology and business, a black swan incident could be an unforeseen catastrophic system failure, a major security breach, or any other event that falls outside the realm of regular expectations and preparedness. Because of their nature, these incidents are nearly impossible to predict and can cause a significant upheaval in the functioning of systems, economies, or societies. The concept emphasizes the need for robustness in systems and the ability to respond flexibly to the unknown, as traditional risk management strategies often fall short in preparing for such exceptional events.

For an insightful overview of incident management, consider reading the following introductory piece.

On that fateful July afternoon, the UK's Faster Payments Service (FPS) faced an unprecedented crisis – the loss of its central view of transactions. This central view provides a real-time overview of all transactional data flowing through the system. Without this critical visibility, the FPS could not accurately track, process, or reconcile transactions. This sudden and unexpected failure left the system operators scrambling to understand the magnitude of the problem. Banks, businesses, and individual customers were left in limbo, unable to execute or confirm transactions, triggering a ripple effect of uncertainty and concern across the nation's financial landscape.

This incident underscored the fragility of even the most advanced financial systems and highlighted the need for robust contingency plans to manage such critical failures and the Major Incident Data Exchange Process to prevent central failure was born.

Following the significant incident that occurred in July 2018, the Faster Payments Service (FPS) has implemented retrospective measures to streamline the process of incident reconciliation for any future occurrences. As a result, in the event of a scheme-wide incident, it is now mandatory for all participating entities to generate a data extract of payments pertaining to a specified timeframe or settlement cycle. This proactive approach is designed to facilitate a more efficient and effective response to incidents, ensuring that any disruptions can be resolved with greater speed and accuracy.

The MIDEP is a procedure focused on restoring the central infrastructure's comprehensive view of transactions. This is achieved by amalgamating and reconciling the fragmented and partial views from all participating entities. In the event of a major disruption where the central system's ability to monitor and manage transactions is compromised, this process becomes crucial. It involves gathering transactional data from all participants, each holding a piece of the overall puzzle, and methodically piecing them together to reconstruct a holistic picture. This collaborative effort is vital for ensuring the integrity and continuity of the transaction process.

Banks act as the distributed backup of the central infrastructure

When the central infrastructure recognizes the loss of its integral view of transactions, it promptly activates the Major Incident Data Exchange Process (MIDEP) across all participating entities. This invocation includes specifying the exact time range that requires data replenishment.

Upon receiving the MIDEP notification, each participant is then responsible for retrieving transactions from their own database that fall within the defined timeframe. They compile these transactions into a detailed file, which reflects their individual, partial view of the transactional state. This file is then submitted back to the central system. This process is crucial for piecing together a complete and accurate picture of the transactions that occurred during the outage, ensuring that the integrity of the transactional data is maintained and that any discrepancies or anomalies can be swiftly addressed.

Upon receiving all the partial transaction views from every participant, these individual data sets are meticulously brought together in a comprehensive reconciliation process. This crucial step involves carefully comparing and matching each piece of data against the others to reconstruct a unified and accurate representation of the transactional activities. The aim of this process is to potentially restore the central view of transactions, effectively piecing back together the complete picture that was lost due to the incident.

Any discrepancies found between the partial views are thoroughly investigated and resolved. This detailed procedure not only helps in restoring the central transaction view but also plays a pivotal role in identifying the root cause of the discrepancy, be it a technical glitch, a processing error, or a systemic failure.

The end goal of this reconciliation process is to reestablish the seamless operation of the payments system, ensuring that all transactions are accurately accounted for and reflected in the central system. This not only helps in maintaining the reliability and trust in the payment infrastructure but also provides valuable insights for preventing similar incidents in the future.

See you next time!

ACH participation modes

TEB — Sun, 12 Nov 2023 11:20:04 GMT

Welcome to a new installment of Payment Bites. The dynamic landscape of payment systems has evolved to become a complex network of financial institutions, payment providers, and other entities. To manage this complexity and create an efficient, inclusive environment for secure transactions, payment schemes often offer various participation modes to accommodate the diverse needs and capabilities of financial institutions, intermediaries, and other entities involved in payment processing. Here are some of the common participation modes:

Direct Participants: These are usually large financial institutions that have a direct relationship with the payment scheme. They are responsible for clearing and settlement activities and can offer payment services to other institutions as well. They are ideal for high-volume, high-value transactions. Direct participants enjoy greater control over transactions, quicker settlements, and typically lower transaction fees. However, the financial and operational costs, including the requirements for liquidity and compliance, can be substantial. There are instances where our choice between direct or indirect participation in a scheme is constrained by specific conditions imposed by the scheme itself, such as minimum transaction volume requirements for direct joining

Indirect or Sponsored Participants: These institutions do not have a direct relationship with the payment scheme but access the system through a direct participant. These are often smaller financial institutions that have an agreement with a direct participant. They rely on direct participants for clearing and settlement activities. They are suitable for institutions that don't have the resources to manage direct participation but still want to offer certain payment services. Lower upfront costs and reduced compliance and operational burdens. Indirect participants have less control over transactions and might face higher transaction fees (scheme plus direct participant fees).

Indirect participants in the payment system are relieved of the responsibility to manage their own liquidity within the system, shifting this complex task to their sponsor banks. However, as a trade-off for this convenience, they are typically obligated to maintain an account with their sponsor bank. This arrangement facilitates the bilateral settlement of transactions, where the sponsor bank acts as an intermediary, handling the funds and ensuring the smooth processing of payments on behalf of the indirect participant. This structure allows smaller institutions or those with limited resources to participate in the payment system without the need for extensive infrastructure or liquidity management capabilities.

For a comprehensive overview and deeper insights into liquidity management within the RT1 framework, highly recommended exploring the detailed analysis provided in this article:

Technical Participants: These are entities like payment processors or third-party service providers who participate for the purpose of facilitating payments but do not engage in clearing or settlement. They are usually contracted by direct or indirect participants to provide specific services. Technical participants focus on technology and services, usually working in symbiosis with direct or indirect participants.

Observer Participants: These entities are involved in the scheme for monitoring or regulatory purposes but do not partake in the clearing and settlement activities. This category can include regulatory bodies, central banks, or even potential future participants assessing the scheme.

See you next time!

🔒 Incident management in payments

TEB — Mon, 30 Oct 2023 07:00:43 GMT

Summary

This article delves into the critical importance of a well-defined Incident Management Framework in payment infrastructures. Given the intricate and interconnected nature of payment systems, incidents can have a domino effect, causing disruptions that range from minor inconveniences to significant financial losses and reputational damage. The article provides a comprehensive overview of essential factors used to gauge the severity of incidents within a payment infrastructure. It delineates the specific roles and responsibilities that team members assume during an incident, and explains how each role fits into the broader incident management framework. This structured approach enables a coordinated and efficient response to emergencies, ensuring swift resolution and minimal impact. The piece serves as a comprehensive guide for payment service providers aiming to fortify their incident response strategies.

It was 2:47 a.m. when Sarah's phone buzzed violently on her nightstand, pulling her from a dreamless sleep into immediate wakefulness. She was the on-call engineer for one of the largest payments infrastructures in the world, and a nighttime page was the omen of something critically wrong. Her eyes scanned the message: "URGENT: Payment Processing Failure - SEPA Instant Network Down."

Adrenaline coursing through her veins, she bolted out of bed, stumbling over the tangled blanket as she reached her home office. Her fingers flew over the keyboard, pulling up dashboards and logs. Every second counted; thousands of transactions were failing, leaving a trail of havoc across the financial ecosystem.

The first Slack message popped up: "Incident room created: #SEPA-incident-5028. All hands on deck." Her teammates, scattered across different time zones, began to join the channel. The chat was filled with a flurry of technical jargon as they assessed the situation.

"Initial diagnostics show an unhandled exception in the validation module," Sarah typed, her hands almost trembling as she realized the magnitude of the issue.

"Has anyone alerted the CTO and Compliance teams?" asked Mike, her counterpart in London.

"Doing it now," Sarah replied, drafting a high-priority email to the upper echelons of the company. There was no room for error; one wrong move could lead to regulatory fines or, worse, irreparable reputational damage.

As Sarah awaited their response, she found her mind racing through the possible scenarios. Was it a malicious attack? A bug in the latest code push? She knew they needed to 'stop the bleeding' before diving into the root cause.

Just then, a message from the CTO appeared on Slack: "Authorize the failover to backup systems. Prepare for a manual review of the failed transactions. And Sarah, keep me posted every 15 minutes. We can't afford to lose any more time."

Sarah's fingers returned to the keyboard, ready to initiate the first critical steps of the incident response plan. But as she looked at her screen, a new alert popped up — something about data integrity. Her heart sank; this was more convoluted than she'd initially thought.

The clock was ticking, and every second felt like an eternity. With a deep breath, she braced herself for the turbulent hours that lay ahead…

In the complex ecosystem of payment systems, incidents are not just undesirable events; they are critical disruptions that can have far-reaching implications. Payment systems are the backbone of any financial infrastructure, facilitating transactions between multiple parties across diverse geographies. They are designed to be robust, secure, and highly available. However, even with advanced technologies and rigorous operational protocols, they are not completely immune to incidents. These unexpected occurrences can range from technical glitches and security breaches to large-scale outages, each with its own unique set of challenges and implications.

An incident management framework helps in effective allocation and utilization of resources, be they human or technical, in resolving the incident.

Having a well-defined incident management framework is critical for organizations, especially those operating in fields requiring high levels of reliability and security, such as payment systems. Incidents can disrupt operations and lead to loss of service. A structured response plan ensures that the organization can recover quickly and continue providing services to its clients. Efficient incident response reduces downtime and the associated costs, whether they are contractual penalties or lost revenue opportunities. Clients are more likely to stick with service providers who can demonstrate competency in managing and resolving issues swiftly. A proactive response can mitigate this risk and may even demonstrate the organization's commitment to reliability and customer care.

Having a set framework helps in setting clear roles and responsibilities, ensuring that all team members know what to do, thereby reducing chaos and confusion during high-pressure incidents. When frontline teams are well-drilled in incident management, senior management can focus on strategic priorities rather than getting caught up in operational issues. A well defined incident management framework is a strategic necessity for modern financial institutions. Consistent and effective incident response maintains and even boosts customer trust.

To begin, let's clearly establish what constitutes an incident in this context:

Such incidents necessitate an immediate emergency response to restore normalcy and mitigate any adverse effects. To comprehensively assess the severity level of a payments incident, multiple variables should be taken into consideration:

Financial Impact: This measures the direct monetary loss or cost incurred due to the incident, which could involve additional operational expenses, fines, or penalties.
Economic Impact: This broader view considers the potential ripple effects on business operations or market conditions, such as lost sales or reduced productivity, which may indirectly influence financial health.
Reputational Impact: This gauges the extent to which the incident affects the public perception and trust in the payment service, which can have long-term consequences for customer retention and acquisition.
Service Disruption or Downtime Period: This variable accounts for the duration for which the payment service is unavailable or operating at reduced efficiency, affecting both providers and users.
Number of Customers Affected: A crucial metric that quantifies the scale of the incident, identifying how many end-users are impacted directly.
Number of Transactions Affected: This provides an insight into the volume of payment transactions that were either failed, delayed, or otherwise compromised due to the incident.

Roles and responsibilities during an incident

Unhappy SEPA Instant settlement

TEB — Tue, 24 Oct 2023 09:20:34 GMT

Welcome to a new installment of Payment Bites. Today, we're building on our prior discussions about SICT settlement by focusing on the less-than-ideal scenarios—what we like to call the 'unhappy cases.' For a look at the happy scenario, you may refer to our previous article:

In a distributed system like RT1, errors can occur at multiple layers. These could range from network latencies and timeouts to issues related to data consistency. For example, a transaction might get delayed due to network congestion or fail altogether due to an unresponsive node. Such anomalies are much harder to predict, isolate, and resolve in a distributed environment compared to a monolithic one.

In distributed systems, the concept of partial errors presents a unique set of challenges. Unlike in a monolithic system, where a failure is usually total and clear-cut, distributed systems may experience partial errors where some nodes or components fail while others continue to operate. This ambiguous state complicates error detection and recovery strategies, as the system has to decide how to proceed when only a part of it is malfunctioning.

The architecture must also account for eventual consistency. In a distributed system, especially one that deals with financial transactions, ensuring that all parties have the same view of the transaction is crucial. This is even more vital in the context of RT1, where even minor discrepancies can have a large impact, given the high volume of transactions and the real-time nature of the platform.

Before diving into the diagram to explore possible business and technical pitfalls, let's first revisit the common misconceptions about distributed systems and the fallacies of distributed systems:

The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
Topology doesn't change
There is one administrator
Transport cost is zero
The network is homogeneous

This list of distributed systems fallacies is intended to serve as a critical guide for developers and architects working within the context of a distributed system, encouraging them to take these potential pitfalls into account during the design and programming stages.

In the diagram above, we provide a comprehensive overview of potential error cases, which are categorized into business and technical failures. Starting with the first set of errors, you'll find an assortment of validation issues that could arise within RT1. These range from structural inconsistencies in pacs.008 messages to routing errors, and even liquidity checks that could potentially halt transaction processing.

The second set focuses on business-related errors that may occur on the beneficiary side. These include a wide array of challenges such as accounts being closed or seized, the triggering of sanctions screenings, and the activation of fraud detection mechanisms. It's important to note that the SICT rulebook does not accommodate scenarios where transactions may be accepted without posting, even in cases where a manual review could be deemed necessary.

For an explanation of accept without posting functionality you can read our article about instant credit transfers in ISO20022 infrastructures.

Last but not least, the diagram delves into the realm of technical errors. These types of errors encompass scenarios where different types of messages are lost during transmission, along with the ramifications of such losses. Finally, the diagram illustrates the implications of losing a settlement confirmation on the originating side. Specifically, it shows how such a loss can activate an automatic transaction status request via a built-in recovery mechanism. This feature is critical for ensuring the robustness of the system.

Liquidity management in RT1

TEB — Sun, 22 Oct 2023 18:37:53 GMT

Welcome to the latest installment of Payment Bites! This week, we delve into the intricacies of liquidity management with a comprehensive new diagram focused on the RT1 platform.

Imagine the RT1 system as the powerful engine that drives real-time payments across Europe, efficiently processing and settling transactions. Just as an engine relies on gasoline to run smoothly, RT1 depends on adequate liquidity to function optimally. Liquidity serves as the fuel that ensures the continuous, seamless execution of transactions. Without sufficient liquidity, the robust infrastructure that RT1 provides would be akin to a high-performance engine running on fumes, unable to fulfill its potential.

To ensure uninterrupted payment settlement, treasury teams must expertly manage liquidity, a process that generally entails three key actions: monitoring, funding, and defunding. Vigilant monitoring allows for real-time visibility into liquidity levels, enabling timely decisions. Funding involves injecting additional liquidity into the system to meet transactional demands, while defunding is the strategic withdrawal of excess liquidity. Together, these actions create a well-balanced liquidity framework that underpins seamless payment settlement.

Today, we are exploring diverse strategies that treasury operations teams can employ to manage liquidity within the RT1 system—insights that are readily applicable to general real-time infrastructures as well.

SEPA Instant settlement in one diagram

TEB — Mon, 16 Oct 2023 15:00:18 GMT

Welcome to a new installment of Payments Bites—your go-to source for compact, easily digestible, and visually engaging articles on payments technology. In today's feature, we will jump into the mechanics of SEPA Instant clearing and settlement, illustrated through a comprehensive diagram.

SEPA Instant is a payment mechanism that enables real-time money transfers within the Single Euro Payments Area (SEPA), facilitating instant clearing and settlement of transactions. SICT is specified in one rulebook however has multiple implementations across Europe.

As we've discussed in prior episodes of The Engineer Banker, Europe hosts two prominent pan-European SICT infrastructures: RT1 and TIPS. Additionally, certain countries have their own domestic SICT implementations; for instance, Spain has Iberpay’s implementation supporting Bizum. In this article, we will depict how RT1 handles transactions and explore the standard clearing actions carried out on the beneficiary's end.

Exceptionally, the comments have been left open for questions from all subscribers.

🔒 The Power of Commands in Event-Driven Architectures

TEB — Sun, 15 Oct 2023 09:15:35 GMT

Want free access to paid posts like this one?

Summary

This comprehensive article from The Engineer Banker delves into the nuances of Event-Driven Architectures, focusing on the key distinctions between commands, queries, and events in a payments system context. It explores how large-scale, real-world systems often employ a hybrid approach to cater to different needs, utilizing either a Fan-Out or Competing Consumers pattern based on propagation expectations. The article also discusses the intricacies of a payments automation engine, emphasizing how commands and events function within this specialized component of a payments infrastructure. Finally, it explores the necessity of temporal coupling between components and how to guide the choice between synchronous HTTP requests and asynchronous commands.

Event-driven architectures (EDAs) have become pivotal in modern software systems, offering dynamic, responsive, and scalable solutions for complex applications. Within this paradigm, the distinction between commands and events is fundamental, and understanding their semantics is crucial for effective design and implementation.

In our initial exploration of Event-Driven Architectures, we delved into the details of events. If you haven't caught up on that, I'd recommend giving it a read. Today, we'll dive deeper into commands and highlight the distinctions between commands and events.

Commands, queries and events

Before we go any further, consider that there are three distinct ways that programs can interact over a network: commands, events, and queries.

Commands are actions, requests for something to happen. They may include a response.
Events are both a fact and a trigger. Never include a response.
Queries are a request to look something up. Unlike events or commands, queries are free of side effects. Always include a response.

In the request-driven approach, components are in a constant dialogue, making specific requests for information or actions whenever a need arises. One component, recognizing that it needs something—be it data or a particular action to be taken—actively sends out a request to another component that can fulfill this need. This form of collaboration is proactive and often synchronous, as one component waits for the response from another.

Conversely, in the event-driven model, also known as Event Collaboration, the dynamics of interaction are fundamentally different. Components in this model do not explicitly request what they need. Instead, they emit events whenever a change in their state occurs. These events serve as signals or notifications to the rest of the system. Other components are designed to listen for these specific events and react accordingly, be it by updating their own state, executing a particular function, or triggering additional events in turn.

The key difference lies in how components consider their relationships with each other. In the request-driven approach, the focus is on direct and immediate need—'I need this, so I'll ask for it now.' In contrast, the event-driven approach engenders a more decentralized and reactive form of interaction. Components in this architecture are more self-contained; they announce changes and trust that other parts of the system will listen and respond if they have a vested interest in the event.

While request-driven collaboration is characterized by active, direct communication to fulfill immediate needs, event-driven collaboration promotes a more indirect, reactive interaction based on state changes signaled by events. Each approach leads to different architectural decisions and ways of thinking about component interactions.

An event essentially is a testament that something of relevance has happened and are inherently immutable: Once an event occurs, it represents an irreversible fact

20 Software Properties to Check for Your Next Payments Project

TEB — Sun, 08 Oct 2023 06:00:47 GMT

Refer a friend

Welcome to another installment of our Payments Bites series. In today's article, we're providing you with an exhaustive checklist of key properties that you should evaluate and discuss during the discovery and final delivery phase of your upcoming payments initiative. Whether you are a Product Manager, Engineer, or another stakeholder involved in payments, this comprehensive checklist is designed to ensure you cover all essential aspects and conversations necessary for the successful rollout of your project. It's a tool to help you identify potential challenges, align objectives, and build a strong foundation for your payments endeavor.

You can utilize this checklist as a structured guide for facilitating a trade-offs discussion session with your team and key stakeholders. In this session, you can collaboratively explore and decide which components of the system need optimization and what essential features should be included in your Minimum Viable Product (MVP). For example, you might discuss whether to prioritize speed over security in certain workflows or perhaps evaluate the trade-offs between developing an in-house solution versus leveraging third-party services. This conversation allows everyone involved to align their expectations and make informed decisions about the project's direction.

Reconciliation: Are your systems capable of matching and reconciling transactions consistently? What level of reconciliation do you consider for the project? all messages? only financial movements? nostro account entries? How often will you reconcile? Who is responsible of this business process?
Rollout Strategy: Friends and Family? Silent rollout? Which markets we will launch first? What is in scope for MVP? post MVP? How many rollouts to release the functionality in full?
Technical Readiness: Do we possess all the necessary skills within the team to successfully complete the project? Is the operations team trained to support this new feature? Is there any external system to the project that can not support the use-case?
Data privacy: Should some data be encrypted? at rest? in transit? Can staging have copies of production data? Do we need anonymization tools? What is the protocol for engineers to obtain privileged access to customer data for troubleshooting issues?
Performance: Peak number of users? peak time of the year? SLAs? What is the estimated transactions-per-second throughput? Which are the response times defined by the business? Is there a need for parallel or asynchronous processing?
Leverageability and reuse: Can we reuse existing platform components? Is the new functionality going to be reused across multiple products?
Resilence and fault tolerance: Can the system auto-heal from a failure? Is it possible to implement a service degradation strategy allowing us to continue offering service with reduced functionality? Is the system designed with high availability and redundancy in mind? Is it feasible to deactivate the problematic feature through the use of a feature flag? What rollback strategies do we consider for the project?
Failure Modes: In which way can the functionality fail? Can failures leave the system in an inconsistent state? Are those failures recoverable? Is a manual intervention required to restore the service after a certain failure?
Testability: What types of tests will be required in the project? manual or automated? unit, integration, component, end to end, performance, security, smoke, pixel tests?
Legal Compliance: Regulations regarding how the application is developed or deployed? Do data protection standards apply? What legislative constraints does the system operate in? Engage the legal team during the discovery phase to ensure compliance and mitigate risks
Reliability: What is the cost to the business of inaccuracy? Can be solved manually for the MVP? Is this system designed for real-time 24/7 operations, or is it based on scheduled batch processing? Is it feasible to schedule downtimes and maintenance periods during batch idle times without adversely affecting operations?
Usability and User Experience: Are there company UX guidelines to follow? How quickly do we need to iterate on UX changes? How important is usability and UX in this project?
Monitoring: What new monitors we need to create? which is the most important KPI to monitor and set alerts on?What types of alert and notifications should be introduced to support this project? Monitoring infrastructure is in place?
Supportability: What levels of support customer/tech are in place? other engineering and operations teams will need to maintain the system post launch? Do we need training and documentation for tech support? Will any tasks omitted from the MVP lead to an increase in the Business As Usual workload for the Operations team? How will the new functionality affect the team's level of firefighting? Any on-call impact?
Configurability: Is a special a Administrator role needed to configure? Can users or administrator configure feature behaviours? Do we need a central configuration? local config files? Is the configuration read-only upon boot-up, or does it need to be dynamically adjusted during runtime?
Extensibility: How easy it is to extent the system with new functionalities or iterate on the existing ones?
Data integrity and Consistency: Tolerance for loss and corruption of data? checksum, journaling, consistency across nodes, data recovery mechanisms?
Scalability: Horizontal? Vertical? Which resources are likely to become bottlenecks?
Authentication: Do you needs users to be identified? How are users identified? Are there Single Sign On standards? Which are the existing authentication systems?
Authorization: To which level of depth and granularity do we limit access? Who maintains this access information? Do we need to maintain groups and roles?

Enjoyed the checklist?

Compatibility in Event driven architectures

TEB — Tue, 03 Oct 2023 09:52:45 GMT

Refer a friend

Welcome to our second article about event driven architectures. Software and systems are in a constant state of evolution, driven by factors such as emerging requirements, regulatory shifts, service deprecation, and data model updates. Consequently, our system design must be robust enough to accommodate both internal and external changes seamlessly. While short-term goals focus on timely releases and immediate value delivery to our customer base, it's imperative to also strategize for long-term system evolution. This dual focus ensures that our architecture can transition smoothly from its current state to future versions, thereby sustaining its operational efficiency and adaptability. The insights offered in this article aim to guide us in navigating the journey of system evolution with a focus on maintaining compatibility.

In an ideal scenario, we would prefer to have every component within a system operating on the same version, ensuring consistent functionality and seamless interaction. However, given the inherently distributed nature of payment systems, microservices architectures and the multitude of actors involved - ranging from banks, payment gateways, to technology vendors and others - achieving simultaneous updates across all parties becomes a logistical challenge. Each actor may have their unique operational constraints, technological resources, and update schedules, which means that version upgrades across the system will inevitably occur in a staggered, non-synchronized manner. Therefore, it becomes crucial to maintain robust backward compatibility measures and have in place mechanisms to handle version mismatches, ensuring smooth operation of the system despite the diversity in component versions.

Compatibility is evaluated from the perspective of the consumer, as it is at the point of message consumption where parsing and interpretation occur. This stage is particularly sensitive to failure, making it crucial to ensure that the received messages are compatible with the consumer's capabilities for accurate interpretation and processing. The previous diagram fully explains the difference between backward and forward compatibility.

Backward compatibility means that consumers with a newer schema version can correctly parse and interpret data from producers with an older schema version.
Forwards compatibility means that consumers with an older schema version can correctly parse data from producers with a newer schema version.

When a modification is designed to be both backward and forward compatible, it is termed as fully compatible. This scenario commonly arises when an optional field is introduced to an existing system. If a change in the schema or specification is neither forward nor backward compatible, then it is an incompatible change or a breaking change. In order to cope with this scenarios we can make use of the parallel change pattern. Certain changes can introduce incompatibilities between the producer and consumer in an event-driven system. Examples of incompatible changes include:

Removing Fields: Eliminating a field that the consumer expects will lead to errors during deserialization or may result in incorrect behavior.
Changing Data Types: Modifying the data type of an existing field can cause type mismatch errors at the consumer end.
Reordering Fields: Changing the order of fields in a fixed-order format like CSV could lead to incorrect data mapping during consumption.
Adding Mandatory Fields: Introducing a new field that is mandatory can break consumers who are not programmed to provide this field.
Enum Restriction: Removing an enum value that the consumer might be using could lead to issues.
Changing Field Semantics: Modifying the meaning or usage of a field without changing its name or type can cause logical errors.
Modifying Default Values: Changing the default value of a field may lead to unexpected behavior for consumers relying on the previous default.
Renaming Fields: Changing the name of an existing field will lead to issues if the consumer expects the old name.
Switching from Single to Repeated Fields: Changing a field from being a single value to a list of values can cause type errors on the consumer side.
Constraints and Validations: Adding or tightening validation rules or constraints on the data can result in previously valid messages being rejected.

Some of these incompatible changes pertain directly to the message's data structure, affecting how the consumer parses and deserializes the incoming data. Others are more subtle and relate to alterations in the business logic that governs how the consumer interprets and processes these messages. Both types of changes can have significant implications for the reliability and accuracy of data consumption in an event-driven system.

Thank you for reading The Engineer Banker. This post is public so feel free to share it.

Introducing incompatible changes in our architecture

The parallel change pattern is a software engineering technique used for evolving a system in a manner that maintains its integrity and continuity. This pattern involves creating a new path or functionality while keeping the old one operational. In essence, it is running the old and new systems in parallel. The new functionality is added and used alongside the old one, with the system being capable of supporting both simultaneously. This approach allows for testing and gradual migration to the new system while maintaining operational stability.

The Parallel Change pattern unfolds in two stages: 'Expansion' and 'Contraction'.

Initially, during the 'Expansion' phase, we seamlessly incorporate the new interface into the existing system. This allows the legacy functionality and the new behavior to coexist harmoniously. Once all system users have transitioned smoothly onto the new schema, we transition to the 'Contraction' phase. During this phase, we systematically eliminate the old behaviour from the system, ensuring a clean and efficient upgrade to the new interface. Let’s understand the pattern with some diagrams.

In essence, this technique or strategy can be used in a number of other situations, namely:

Database Refactoring: A crucial element of evolutionary database design, most refactorings follow the parallel change pattern. This approach allows for a transition period between the old and new schemas until all code is updated.

Deployments: Techniques like canary releases and Blue-Green Deployment use parallel change to incrementally shift users between old and new code versions, reducing risk and simplifying service orchestration in microservices architectures.

API Evolution: For non-backwards-compatible changes to remote APIs like REST services, it can be applied to modify payload requirements or introduce new endpoints to distinguish between versions.

You may want to continue reading the rest of the event-driven architecture series:

Event driven architectures in payments

TEB — Sun, 01 Oct 2023 16:59:23 GMT

Refer a friend

Today, we're excited to present a comprehensive article that delves into the role of event-driven architectures in financial institutions, particularly in the complex ecosystem of banking. A bank is a multifaceted organization, with numerous functional departments such as compliance, risk management, customer service, and transaction processing, each serving as cogs in a well-oiled machine. Remarkably, the technical infrastructure of a bank often mirrors its organizational structure, an observation that aligns with Conway’s Law, which posits that system designs are bound to emulate the communication structures of the organizations that create them.

Understanding the intricacies of how these diverse departments and their corresponding technical systems communicate and interact with each other is key to grasping the agility and scalability required for modern banking. If you're new to the foundational elements of banking systems, we highly recommend that you read our introductory article on the anatomy of payments infrastructure. This will offer you a holistic view of the building blocks that form the underpinnings of a bank's operations.

In such a complex landscape, the need for efficient, robust, and scalable communication between these systems becomes apparent. This is where event-driven architectures come into play. This architectural style facilitates the decoupling of systems, allowing each to operate autonomously while still being part of an integrated whole. Through the use of events, services can publish, subscribe to, and consume data asynchronously, thereby enhancing operational efficiency and responsiveness.

For example, consider a payment processing unit in a bank. When a new transaction comes in, an event can be emitted to notify risk assessment, fraud detection, and customer notification services. These services can independently pick up the event, process it in their own specialized way, and then emit new events to signal completion or trigger additional actions. This way, each unit can operate autonomously, yet still contribute to a unified, streamlined operation.

By embracing event-driven architectures, financial institutions can not only improve internal efficiencies but also adapt more readily to changing business requirements and technological advancements. This contributes to an agile, resilient, and scalable operation capable of meeting the multifaceted demands of modern banking.

Event-Driven Architectures (EDA) have emerged as a crucial technological strategy to meet the agility, scalability, and real-time processing demands. Unlike traditional, request-response architectures that can become bottlenecks in the fast-paced world of transactions, EDAs offer a robust and flexible framework for handling a multitude of operations concurrently.

In an Event-Driven Architecture, the core idea is that changes in state, or events are captured in real-time and processed asynchronously. These events could range from a customer initiating a payment, a batch of transactions getting cleared, to fraud detection algorithms flagging suspicious activity. Each event is an atomic chunk of data that can be acted upon independently, thereby offering the opportunity for highly parallel and distributed processing.

Events flow through the system, being consumed, produced, and often transformed in the process. Each event carries significance and encapsulates specific data.

Let's consider a typical payment journey to understand how EDA could be applied. When a customer initiates a payment, an event is generated and placed on a message queue or broker. Various services can subscribe to these events, each responsible for a particular aspect of the transaction, such as authorization, risk assessment, fund reservation, and more. Because these services operate asynchronously, they can process the event at their convenience, ensuring that system resources are optimally utilized. Once processed, these services can generate additional events that other services can consume, creating a chain of operations linked together through events.

This architecture has several advantages in a payments context. First, it allows for highly decoupled systems. The service responsible for fraud detection, for instance, doesn't need to know anything about the service that handles currency conversion. This modularity makes the system easier to manage, test, and scale. Secondly, the asynchronous nature of EDA ensures that the system is highly responsive and can cope with variable loads, a must-have feature especially during peak transaction times like holidays or during specific market events.

What is an event?

An event is a message that is sent as a reaction to something that has happened, typically a change in state of a resource in a particular system. For example, when you create an account in a bank, the account can transition through multiple states based on external actions that affect the state of the account. Below, you'll find two illustrative examples, among many possibilities, that demonstrate how events interconnect various business processes. The first example elucidates how payment functionality can be enabled for a specific customer following successful completion of Know Your Customer (KYC) procedures. When the customer finishes the KYC steps, the account status undergoes a modification. In response to this state change, the accounts service broadcasts a message to notify relevant systems about the updated account status.

In the second example, we explore a distinct scenario that demonstrates the importance of time as an actor in our systems, particularly in the context of compliance with General Data Protection Regulation (GDPR). Unlike traditional actors in financial systems such as customers, agents, and operations personnel, time can also be an actor affecting the state of specific records.

Let's consider the case where a customer decides to close their account with the financial institution. After the account is closed, the institution's GDPR compliance service schedules the removal of all the customer's personally identifiable information (PII) in accordance with GDPR requirements. This entails a predetermined retention period post-closure, during which the customer's data must be stored securely before it can be purged. The retention period varies based on the type of data and the specific requirements outlined in GDPR, but it serves a critical purpose, allowing for audits, dispute resolutions, or any other legal obligations that may require the institution to access historical data.

As the end of the retention period approaches, the GDPR service becomes an active participant in the system. When the retention deadline is reached, the service emits an event signaling to downstream services that the time for data removal has come. These downstream services, which could range from customer management systems to transaction histories, are then responsible for executing the data manipulation steps required to remove the customer's PII.

Now, what makes this event particularly critical is the stringent nature of GDPR penalties. Non-compliance can result in hefty fines, which makes it essential for all downstream services to respond promptly and effectively to the event emitted by the GDPR service. Moreover, failing to remove customer data post-retention could compromise the customer's privacy and leave the institution vulnerable to data breaches, thereby exacerbating both financial and reputational risks.

It's also worth noting that the GDPR service must ensure that the emitted event is secure, traceable, and logged for accountability. It must also be designed to trigger data removal from backups, logs, and any third-party services that the institution might be using. Hence, the GDPR service plays a vital role not only in automating the data removal process but also in ensuring that it is carried out in full compliance with GDPR standards.

Having explored various examples of events within event-driven architectures, it's time to delve into the categorization of these events. The types of events can be distinguished based on the volume of state and data they encapsulate, leading us to define distinct classifications for them.

Thank you for reading The Engineer Banker. This post is public so feel free to share it.

Notifications

An event is said to have notification semantics when the event itself does not carry information about the resource or record that has been mutated. The event purely signals that something has happened and most of the time has a link for the consumer to fetch more information if needed. Those events go well when the only context needed for the receiving system is the acknowledgement that a particular action has happened. Additional scenarios where event notifications are particularly useful occur when the data payload is so large that it becomes impractical for queues and brokers to handle the entire event along with all its associated data. A prime example is the event generated when a file has been uploaded. In such cases, the notification related to this action will not carry the file itself, but rather a URL where the file can be downloaded. This allows downstream business processes to be triggered efficiently without overwhelming the system's data handling capabilities.

In this illustrative example, a dedicated service serves as the intermediary for connecting to the payment scheme. This service is tasked with downloading a settlement file that contains crucial payment-related information. Upon successful and secure downloading of the settlement file, the service uploads it to an Amazon S3 storage bucket for enhanced security and accessibility. It then initiates a notification event to alert the various consumer services that have registered an interest in the file's content. These consumer services, upon receiving the real-time notification, are provided with the necessary credentials or links to securely access and download the file from the S3 storage. This streamlined process not only ensures data integrity but also facilitates seamless and efficient distribution of critical financial information among the interested parties.

Among these interested consumers is a specialized service responsible for reconciliation. This reconciliation service takes the distributed settlement file and begins the process of parsing its entries, examines each entry to align it with the clearing information it has previously received from other data streams or channels.

Depending on the process that is run at the downstream service it may need context of the action that happened on the upstream service, for this reason some events carry state from the upstream to the downstream to minimize the number of calls to the producer, this pattern is called event-carried state transfer.

Depending in the amount of state that the upstream propagates we will talk about think or fat events.

Delta vs Snapshot Events

Depending on the volume of state information that an upstream service propagates downstream, events can be categorized as either thin or fat. In the context of event-driven architectures, a thin event typically carries minimal state information, often just enough to signal that a change has occurred. It may include identifiers or keys that allow the downstream service to fetch more details if needed. For example, a thin event might merely announce that a new transaction has been posted and provide a transaction ID for reference.

On the other hand, a fat event contains a more comprehensive set of data, capturing a richer snapshot of the state at the time of the event. This might include details like account balances, customer information, and transaction metadata, eliminating the need for a downstream service to make additional calls to gather more context. A fat event could be highly useful in scenarios requiring immediate, complex decision-making based on the event, such as fraud detection mechanisms that need to analyze multiple variables rapidly.

Choosing between thin and fat events often depends on various factors like network bandwidth, the complexity of downstream processing, and latency requirements, allowing for more tailored, efficient system-to-system communication.

A snapshot event serves as a particular subtype of fat events. A snapshot event captures the complete current state of a given object or system at a specific point in time. Unlike delta or incremental events, which only relay changes, snapshot events deliver a full data set. This allows downstream systems to either synchronize or initialize their states based on this comprehensive snapshot. While snapshot events grant complete visibility into the state of upstream systems, the onus is on those systems to track any changes between successive snapshots, calculate the differences or deltas, and accordingly update their state or trigger actions.

Serialization

Serialization of events is the process of converting the structured data of an event into a format that can be easily stored or transmitted, and later reconstructed. In the context of event-driven architectures, serialization allows events to be packaged into a format that can be sent across different services, systems, or network components. This format could be JSON, XML, Protobuf, Avro, or other serialization formats. Once the event reaches its destination, it is deserialized back into its original structure for further processing or analysis. Serialization ensures that complex data structures within the event are simplified into a format that can be easily shared and understood across different parts of a system.

In event-driven architectures, one of the pivotal considerations is the choice of serialization for events or messages that traverse the system. This is crucial for both performance and compatibility among the diverse services that may be consuming or producing events. Three popular serialization options are JSON, Protocol Buffers (protobuf), and Avro, each with its own set of advantages and disadvantages.

JSON (JavaScript Object Notation) is a text-based, human-readable format that's easy to understand and widely supported across programming languages. It's the go-to choice for many developers due to its simplicity and ease of use. However, the text-based nature of JSON can be a drawback in terms of size and speed, making it less efficient for transmitting large volumes of data or for real-time processing requirements commonly encountered in payments systems.

Protocol Buffers, or protobuf, is a binary serialization format developed by Google. Unlike JSON, protobuf messages are not self-describing, meaning they require a predefined schema to be interpreted. However, they offer significant advantages in terms of speed and size, making them highly efficient for transmitting large or complex datasets. Additionally, protobuf has built-in support for versioning, a crucial feature when dealing with evolving data structures in a distributed system.

Avro, developed by Apache, is another binary serialization format that is somewhat of a middle-ground between JSON and protobuf. Like protobuf, Avro is efficient in terms of both size and speed. Unlike protobuf, Avro schemas are typically sent along with the data, making Avro messages self-describing. This facilitates greater flexibility when evolving schemas but at the cost of slightly larger message sizes.

The choice of serialization in event-driven architectures is highly contextual. JSON is easy to use and widely supported but may not be the best choice for systems requiring high throughput or low latency. Protobuf offers efficiencies in size and speed but requires a predefined schema, making it less flexible for certain use-cases. Avro strikes a balance, offering both efficiency and flexibility, but may result in slightly larger messages due to the inclusion of schema information.

We have discussed event-driven architectures in financial environments. One final consideration to take into consideration when introducing EDAs into your technological stack is the architecture allows for easy adaptability and evolution. As business needs change, new services can be added or existing ones modified without disrupting the entire system. If a new regulation mandates additional checks before processing a payment, a new service can be introduced to handle that specific requirement, subscribed to the relevant events, and deployed—without a system-wide overhaul.

You may want to continue reading the rest of the event-driven architecture series: