Software Design Patterns for a Distributed Codebase

November 3, 2021

As Abnormal grows, we have to maintain a scalable codebase across all areas of engineering to prevent issues around testing, maintainability, and documentation. When organizations scale, a common problem is that the codebase becomes cluttered, with multiple teams writing different code that accomplishes the same task. While Abnormal is in this phase of rapid growth, we want to avoid this problem, so designing code that is extendable and generalizable has taken on a higher priority.

Ideally, our codebase contains minimal unused or deprecated modules, leverages common and reusable modules as opposed to one-off solutions, and is written entirely in the same voice. This is challenging because there are varying levels of engineers contributing to the codebase at any point in time, and we want to avoid complexity in order to move with velocity.

One way we’ve tackled this challenge is through implementing design patterns, which are standardized systems that present a way of architecting a solution based on the usage and constraints of the problem being solved. In this post, we present examples of a few such design patterns we follow at Abnormal.

Strategy Design Pattern

The strategy design pattern bundles together several different implementations of the same task. This is particularly useful when there are many different strategies for the same task given different preconditions, where only one strategy should be executed at runtime.

Our attachment signals builder uses this design pattern pretty extensively. At the core, we have several different builders for different types of attachments, such as HTMLAttachmentSignalsBuilder for HTML attachments and DocxAttachmentSignalBuilder for Microsoft Word documents. We must have multiple because we don’t know which SignalsBuilder to use until we know what type of attachment a message contains.

Codebase client 1

In this diagram, the attachment signals builder acts as the context class, storing the different SignalsBuilders that will be used to extract signals from an attachment.

The AbstractAttachmentSignalsBuilder specifies the base functionality of any signals builder but does not provide any implementations. Each subclass provides the implementation of the base class. This design pattern allows ML engineers to extend our attachment signals build capability to existing and new attachment types without having to change the AttachmentSignalsBuilder.

Template Design Pattern

This design pattern is a pretty common one in our codebase. It defines a template class with some boilerplate functionality, while also specifying other functions that have to be extended by subclasses. It doesn’t rely on a context class or an interface to specify which subclass to use, unlike the strategy design pattern from before.

There are several components of our detection system which rely on deterministic rules for making decisions on a message. Though each component implements different business logic, they all require some shared functions.

Codebase template 2

The framework class RulesBasedModel defines the reusable components, as well as hook functions that are not implemented in the framework class but are implemented in the subclasses. The developer builds new subclasses using the shared boilerplate code from the parent class.

The template design pattern is useful in cases like these, where two or more classes demonstrate similar functionality, but not enough to use a single interface for both.

Adapter Design Pattern

The adapter design pattern communicates between two interfaces that would otherwise not be able to communicate with one another. It has a pretty straightforward use case: you use this design pattern to build adapters to translate between two otherwise incompatible interfaces.

Our detection system uses several types of models developed on third-party libraries including SKLearn, Keras, and others. Each detector uses its own library-specific model during prediction, which means the way that data feeds into the model varies from detector to detector. Therefore, we created adapters for each library’s model. Below is an example of the adapter for SKLearn models.

Codebase modelwrapper 3

In this example, SKLearnModelWrapper is an adapter between the model wrapper and the third-party SKLearn model object. The ModelWrapper can now be used for any SKLearn model through this adapter. Developers can also easily extend the system to any ML library by creating an adapter for that third-party model object.

Composite Design Pattern

Composite objects are objects that are composed of other objects. This pattern is useful in cases where we have hierarchical structures within composite objects, such as tree-like structures.

For one of our detectors, user attributes are extracted through nested logical expressions. These expressions form a hierarchy given the nature of the nesting, and a composite design pattern can be used to specify a single structure regardless of the size of the expression.

The example below uses the following classes to illustrate this:

  • ProcessedEventPredicate: The base class of an operator. When this class is called, it runs check(), which executes the operation.

  • ProcessedEventPredicate subclasses: Each operation extends the ProcessedEventPredicate class by implementing the logic in check(). Each subclass also contains a predicates list for the predicates it performs the operation on. This list can also contain either boolean values or other ProcessedEventPredicate subclasses.

Codebase 4

When the nested logical expression is run in an extractor, the composite design pattern ensures that all of the operators and operands are compatible with the extraction method. This makes it easier to use operators and operands as abstractions, especially in a nested structure.

Key Takeaways

As engineers, we are constantly solving complex problems. To maintain a sense of consistency across teams working on the same codebase, we need some high-order values to abide by. Here are some of the general principles we like to follow:

  • Single Responsibility: Every module, class, or function in a computer program should have responsibility over a single part of that program's functionality, which it should encapsulate. All of these services should be narrowly aligned with that responsibility.
  • Open/Close: Software entities should be open for extension, but closed for modification. We want to build our codebase such that we can extend its functionality without having to change its source code.
  • Liskov Substitution: Objects of a superclass should be replaceable with objects of their subclasses without breaking the application. This requires the objects of the subclasses to behave in the same way as the objects of the superclass. Any subclass must be able to replace the usage of a superclass in the codebase.
  • Interface Segregation Principle: Clients should not be forced to depend on interfaces that they don’t use. Interfaces should be developed to suit a very specific client need—we never want a client to rely on an interface with functions that they will not use.
  • Dependency Inversion Principle: High-level modules should not depend on low-level modules and abstractions should not depend on details. Details should depend on abstractions.
  • Composition over Inheritance: Too many levels of inheritance in a codebase can cause difficulties in testing, maintenance, and code extensibility. To reduce these sorts of problems, composition is generally recommended.
  • Pattern as a Last Resort: While patterns are great for reducing redundancy in extending existing functionality, code should not be instinctively written in the style of an existing pattern. Rather, after a new design is scoped out, it should only be fitted into a pattern if the use case and interface are identical. This allows us to prevent issues with tight coupling, complexity, and confusion that may arise from retrofitted code.

These principles, along with their corresponding design patterns, are just a few of the more commonly used ones at Abnormal. By following these principles, we ensure that our codebase maintains a sense of cohesion, extensibility, and clarity across teams at the company.

We hope this will give a good framework for anyone interested in using design strategies in their own work! Design patterns used in this blog are based on the principles described in Source Making.

If you’re interested in joining the Abnormal engineering team, we’re hiring! Check out our open roles and apply on our Careers page.

Related Posts

B 12 03 22 SIEM
Learn about Abnormal’s enhanced SIEM export schema, which provides centralized visibility into email threats
Read More
Blog phishing cover
The phishing email is one of the oldest and most successful types of cyberattacks. Attackers have long used phishing as a common attack vector to steal sensitive information or credentials from their victims. While most phishing emails are relatively simple to spot, the number of successful attacks has grown in recent years.
Read More
Blog brand cover
For those of you who have visited the Abnormal website over the last month, you’ve seen something different—a redesigned brand focused on precision. It’s new and innovative, and different from any other cybersecurity company, because it was created with one thing in mind: our customers.
Read More
B 11 22 21 AAA
At Abnormal, our customers have always been our biggest priority. Customer obsession is one of our five company values, and we live this every single day as we provide the best email security protection available for the hundreds of companies who entrust us to protect their mailboxes.
Read More
Blog microsoft abnormal cover
Before we jump into modern threats, I think it’s important to set the stage ​​since email has been around. Since email existed, threat actors targeted email users with malicious messages, general spam, and different ways to take advantage of the platform. Then of course, more dangerous attacks started to come up… things like malware and other viruses.
Read More
Blog black friday scam cover
While cybersecurity awareness is a year-round venture, it is especially important to be mindful during certain times of the year. With Thanksgiving here in the United States on Thursday, our thoughts will likely be on our family and friends and everything we have to be thankful for this holiday season.
Read More
Blog automation workflows cover
Our newest platform capabilities help customers streamline critical security workflows, like triaging phishing mailbox submissions or triggering tickets to investigate account takeovers, through automated playbooks. Doing so can decrease mean time to respond (MTTR) to incidents, further reducing any potential risk to the organization and eliminating manual workflows to save time and increase the efficiency of IT and security teams.
Read More
Blog tsa scam cover
On November 9, 2021, we identified an unusual phishing email that claimed to be from “Immigration Visa and Travel,” inviting the recipient to renew their membership in the TSA PreCheck program. The email wasn’t sent from a .gov domain, but the average consumer might not immediately reject it as a scam, particularly because it had the term “immigrationvisaforms” in the domain. The email instructed the user to renew their membership at another quasi-legitimate-looking website.
Read More
Blog pyspark cover
At Abnormal Security, we use a data science-based approach to keep our customers safe from the most advanced email attacks. This requires processing huge amounts of data to train machine learning models, build datasets, and otherwise model the typical behavior of the organizations we’re protecting.
Read More
Blog tiktok attack cover
As major social media platforms have expanded the ability of creators to monetize their content in the last few years, they and their users have increasingly found themselves the targets of malicious activity. TikTok is now no exception.
Read More
Blog ransomware guide cover
While various state agencies and the private sector keep track of ransomware attacks and related tactics worldwide, malicious actors change and evolve their ransomware strategies all the time. We’ve put together a comprehensive guide that will define ransomware, how to detect it, and what steps to take if you’ve fallen victim to a ransomware virus attack.
Read More
Blog detection efficacy cover
One of the key objectives of the Abnormal platform is to provide the highest precision detection to block all never-before-seen attacks. This ranges from socially-engineered attacks to account takeovers to everyday spam, and the platform does it without customers needing to create countless rules like with traditional secure email gateways.
Read More