Abstract White Corner

8 Key Differences Between Ineffective and Effective Machine Learning Engineers

At Abnormal, we pride ourselves on our excellent machine learning engineering team. Here are some patterns we use to distinguish between effective and ineffective ML engineers.

At Abnormal Security we pride ourselves on our excellent machine learning (ML) engineering team. Every day, our ML engineers solve problems that make our groundbreaking technology even more effective at stopping the most advanced socially-engineered attacks.

As the company grows and our team expands, we recognize our success depends on inviting the right people to join us on our mission to build the next generation of email security. This is why we have taken the time to define what effectiveness means at Abnormal.

Here, we share some patterns that we use to distinguish between effective and ineffective ML engineers.

1. Opting for Simplicity

An ineffective ML engineer uses complex algorithms without justification. They are quick to throw away existing solutions they don’t fully understand, dismissing heuristics or manual transformations in favor of fancy models.

An effective ML engineer starts by thoroughly studying the problem and the data first. They like to use the simplest approach that can completely solve the problem. They fully understand the current stack before proposing new solutions and don’t increase complexity without a sound justification.

2. Focusing on Impact

An ineffective ML engineer spends weeks on projects that have no clear path to production. They don’t think about the environments in which their models will run online, or how their work will impact the product.

An effective ML engineer thinks about the online launch from the beginning of each project. They chart the path from offline experimentation to product impact and design their offline test plan to derisk the biggest uncertainties.

3. Embracing Versatility

An ineffective ML engineer does not understand the software systems they rely upon. They become blocked easily when the exact datasets or tools they need are not available and rely heavily on software teams to support them.

An effective ML engineer is a strong programmer. They are comfortable outside of notebooks and write well-tested and readable data pipelines. They can read and write serving code as well as deploy their own models and feature serving systems when necessary.

4. Avoiding Assumptions

An ineffective ML engineer makes blind assumptions. They might assume that a feature is always accurate, that the labels they use for training represent ground truth, or that a mean is a good default for all of their continuous features. An ineffective ML engineer doesn’t question where their dataset came from, or how their model will interact with the other parts of the system.

An effective ML engineer always questions their assumptions. They have good instincts, but they aren’t afraid to challenge them. They ask questions about anything they are unsure about and write down the answers so they can reference them later. An effective ML engineer loves data wrangling—starting with the client-side signals that feed the model features and ending with the product metrics that respond to model performance.

5. Prioritizing Reproducibility

An ineffective ML engineer views the trained model as the output of their work. They don’t prioritize cleaning, testing, or refactoring their data processing or model training code.

An effective ML engineer understands the importance of reproducibility and views reproducible training pipelines as the output of their work. They refactor and clean their training code as part of the modeling process, and they write runbooks that are detailed enough for a new hire to understand.

6. Managing Uncertainty

An ineffective ML engineer is unwilling to make commitments for how long something will take, or how much of an improvement they will be able to make.

An effective ML engineer knows that ML projects have high uncertainty but also understands that a software organization cannot function without reliable estimates. They can decompose a problem and propose a path that accounts for multiple possible directions. An effective ML engineer is detail-oriented and mitigates risks early with sanity checks and simple tests.

7. Overcoming Roadblocks

An ineffective ML engineer accepts constraints at face value. They struggle to understand why certain things are hard or to communicate the problems they face to their team and leadership.

An effective ML engineer questions every constraint until they fully understand it. They propose new solutions based on first principles, external resources, or experience.

8. Designing for Resilience

An ineffective ML engineer doesn’t prioritize change resilience. Their models and features adapt poorly to infrastructure changes or data distribution shifts. They optimize for today’s performance at the expense of tomorrow’s generalizability.

An effective ML engineer produces generalizable systems. Their models and features don’t fall over in the face of distribution shifts like new customers, products, or platforms. They prefer simpler and more adaptable solutions.

Embodying Effectiveness as an ML Engineer

Effectiveness means different things in different organizations. An ML engineer in a small startup needs to prioritize versatility and velocity, whereas the most effective engineers at large companies need to build on the work of others to drive impact. Growing as an ML engineer requires understanding what effectiveness means in your organization.

For us, keeping our customers safe from the most dangerous cyber threats requires a flexible team that knows what to prioritize and how to adapt their approach based on new data. An effective ML engineer at Abnormal enjoys solving complex problems and identifying new ways to stay ahead of ever-changing threats. If this sounds like you, we’d love to hear from you! Check out our open roles to learn more.


Prevent the Attacks That Matter Most

Get the Latest Email Security Insights

Subscribe to our newsletter to receive updates on the latest attacks and new trends in the email threat landscape.

Demo 2x 1

See the Abnormal Solution to the Email Security Problem

Protect your organization from the attacks that matter most with Abnormal Integrated Cloud Email Security.

Related Posts

B 05 11 22 Scaling Out Redis
As we’ve scaled our customer base, the size of our datasets has also grown. With our rapid expansion, we were on track to hit the data storage limit of our Redis server in two months, so we needed to figure out a way to scale beyond this—and fast!
Read More
B 05 17 22 Impersonation Attack
See how threat actors used a single mailbox compromise and spoofed domains to subtly impersonate individuals and businesses to coerce victims to pay fraudulent vendor invoices.
Read More
B 05 14 22 Best Workplace
We are over the moon to announce Abnormal has been named one of Inc. Magazine's Best Workplaces of 2022! Learn more about our commitment to our workforce.
Read More
B 05 13 22 Spring Product Release
This quarter, the team at Abnormal launched new features to improve lateral attack detection, role-based access control (RBAC), and explainable AI. Take a deep dive into all of the latest product enhancements.
Read More
B 05 11 22 Champion Finalist
Abnormal has been selected as a Security Customer Champion finalist in the Microsoft Security Excellence Awards! Here’s a look at why.
Read More
Blog series c cover
When we raised our Series B funding 18 months ago, I promised our customers greater value, more capabilities, and better customer support. We’ve delivered on each of those promises and as we receive an even larger investment, I’m excited about how we can continue to further deliver on each of them.
Read More
B 05 09 22 Partner Community
It’s an honor to be named one of CRN’s 2022 Women of the Channel. Here’s why I appreciate the award and what I love about being a Channel Account Manager at Abnormal.
Read More
B 05 05 22 Fast Facts
Watch this short video to learn current trends and key issues in cloud email security, including how to protect your organization against modern threats.
Read More
B 05 03 22
Like all threats in the cyber threat landscape, ransomware will continue to evolve over time. This post builds on our prior research and looks at the changes we observed in the ransomware threat landscape in the first quarter of 2022.
Read More
B 04 28 22 8 Key Differences
At Abnormal, we pride ourselves on our excellent machine learning engineering team. Here are some patterns we use to distinguish between effective and ineffective ML engineers.
Read More
B 04 26 22 Webinar Re Replacing Your SEG
Learn how Microsoft 365 and Abnormal work together to provide comprehensive defense-in-depth protection in part two of our webinar recap.
Read More
Blog mitigate threats cover
Learn about the most common socially-engineered attacks and why these tactics are still so successful—despite a growing awareness from employees.
Read More