Million Ads Machine: How Nexxen Runs an Accurate, High-Performance Ad Scoring Platform
In the fast-paced world of digital advertising, the ability to efficiently rank and score millions of ads in real-time is paramount to success. As advertisers increasingly demand measurable results tailored to their unique campaign objectives, the challenge for Demand-Side Platforms (DSPs) intensifies. How do we strike the perfect balance between speed, accuracy, and scalability in a landscape marked by fluctuating demands and diverse goals? Feature Engineering Effective feature engineering is crucial for extracting actionable insights from complex, sparse datasets. At Nexxen, we distill thousands of candidates’ features into a few hundred high-impact signal-based features, which is key to optimizing model performance and enhancing prediction accuracy. Our data scientists utilize a proprietary query engine to execute a variety of Spark and MapReduce jobs, deconstructing data across multiple dimensions. We’ve developed a framework that detects feature discrepancies, improving training data quality. This system combines automated pipelines with on-demand SparkSQL queries, allowing for comparisons between training data and incoming ad requests, ensuring that the models remain aligned with the latest data distributions. Ad Scoring and Ranking Using Machine-Learning Models Before scoring and ranking, a series of services handle pre-processing steps like filtering and discarding based on specific rules. These services leverage in-memory caches and distributed key-value stores for fast retrieval of metadata from relational databases and object stores. These lookups occur in milliseconds, crucial for ensuring real-time performance. When an impression request arrives, the scoring system uses the deserialized trained models loaded into memory for immediate scoring and ranking. Requests are transformed into feature vectors, which are then scored using a Directed Acyclic Graph (DAG), where each machine learning model acts as a node in the DAG. The DAG structure allows for dependency-based execution, optimizing for various KPIs like CPA (Cost Per Action), CPC (Cost Per Click), or Viewability. The complete bidding workflow—including selection, filtering, and scoring—occurs within a few milliseconds, enabling Nexxen to handle millions of requests per second while maintaining high throughput and minimal latency. Below is a high-level design of such a system: A/B Testing and Custom Bidding Strategies The scoring platform provides A/B testing through user-split methodologies, to quantify campaign lift while minimizing expenses commonly associated with control group impressions. Distinct machine learning model versions can be assigned unique budget caps, facilitating performance benchmarking and adaptive budget allocation. Advertisers can further optimize their bidding algorithms by applying bid multipliers across various targeting vectors, providing increased flexibility to maximize campaign effectiveness. Observability Our observability infrastructure is divided into two core components: 1. Model Generation and Training: Tracks the success rates of data collection, training, and model distribution. 2. Model Performance: Monitor real-time performance metrics such as latency, throughput, and accuracy for each deployed model. We leverage a time-series database to collect high-resolution metrics and generate dynamic dashboards. These dashboards allow us to separate signal from noise, providing insights into true performance anomalies. Model Release and Versioning Our CI/CD pipeline integrates GitLab and Jenkins for version control, build automation, and deployment. This setup enables seamless rollout of new machine learning models or rollback to previous versions based on real-time performance metrics, ensuring both agility and reliability in model deployment. Looking Ahead As Nexxen looks to the future, a key focus will be leveraging larger and more sophisticated AI models to tackle challenges of AI-driven ad fraud and navigating the potential for AI-powered algorithms to perpetuate biases present in training data, leading to discriminatory ad targeting and reinforcing existing inequalities. By continuously refining its technology and methodologies, Nexxen is committed to developing strategies and further expand customization on our scoring platform that address these issues while ensuring efficient predictions and maintaining high performance. Read Next
From Insight to Activation: Nexxen’s Formula for Personalization in a Privacy-Conscious World
World Conway’s Law, a principle coined by computer programmer Melvin Conway in 1967, states that “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” In simple terms, the way teams are structured within a company directly influences the architecture of the software they build. If teams are siloed or fragmented, the software systems they create will reflect that disjointedness, and vice versa – collaborative teams tend to build more cohesive systems. This is best explained via the following simplistic illustration: At Nexxen, we’ve witnessed this law in action many times, both in positive and challenging ways. Below are a few real-world examples where Conway’s Law was applied, intentionally or otherwise, and how it shaped our technical architecture and team structure. The Monolith Several years ago, Nexxen acquired a successful ad-tech firm that had built an impressive and feature-rich platform. However, it came with one significant architectural challenge: it was built as a monolith. A single, unified codebase where all functionality is tightly coupled. While such systems may be easier to develop in the early stages, they come with several long-term downsides. Such applications usually demonstrate slower development and scaling limitations given the butterfly effect, where for example, changing one line of code in the UI may impact the data-science codebase. It was clear we needed to break down this monolith and transition to a distributed architecture with microservices. Distributed services allow different components of the system (e.g., runtime, front-end, data) to operate independently, making the system easier to scale, maintain, and deploy. To make this transition, we reorganized the original monolithic development teams into several smaller groups, each focused on a specific domain – such as runtime, front-end, and data. These new teams, now more autonomous, were incentivized to take ownership of their respective areas, leading to the natural evolution of independent services. Each group began decoupling their domain from the monolith and building it as a standalone service. As Conway’s Law dictates, as our communication structure shifted, the software followed, evolving from a monolithic architecture into distributed microservices Distributed teams lead to misalignment Not all outcomes of the newly distributed team structure were purely positive. As teams became more independent, we noticed an increase in the rate of production issues. The issue wasn’t with the teams’ capabilities, but rather with a lack of alignment between them. As each team focused on its own service, communication across teams diminished, and this misalignment led to inconsistencies. Because the teams were operating in silos, the systems they built reflected that disconnection. The architecture had become fragmented, with services not always communicating well or adhering to shared practices. Fearing the return of the monolith, we introduced a more structured communication framework, leaving the teams independent. For example, we implemented a rigid Slack structure, ensuring all teams had dedicated channels for cross-team collaboration, changes, and feedback. We created clear guidelines for communicating changes early and soliciting feedback across teams. Another example would be the enforcement of regular cross-team meetings and a shared response system that improved communication, leading to faster resolution times and better alignment across services. By improving the communication structure, we were able to reduce production issues and bring the architecture into better alignment with the business’s needs. Freedom vs. Alignment Another example of Conway’s Law at play can be seen in our front-end development teams. Nexxen operates several independent product lines, each with its own front-end application. Each product line is supported by its own development team, leading to a natural separation in how teams approach their work. Over time, this autonomy led to diverse tool choices – teams were using different logging frameworks, testing tools, monitoring solutions, and even CI/CD pipelines. There are clear advantages to allowing teams to choose their own tools, such as increased ownership and innovation, but it raised the challenge of inconsistent practices. Each team’s differing choices in tools made it harder to maintain, monitor, and support these applications at a company-wide level. To address this, we reorganized the teams under a single department, while maintaining the independence of each team’s product line. This shared management layer ensured alignment on core tools and protocols (logging, testing, monitoring, etc.) while preserving the benefits of autonomy where it made sense. This approach allowed us to standardize critical infrastructure while giving teams the freedom to innovate within their domains. Conclusion Conway’s Law has been a guiding principle at Nexxen, sometimes intentionally and other times revealed through experience. Whether breaking down a monolith, dealing with the challenges of distributed teams, or balancing autonomy with alignment, we’ve learned that organizational structure and communication are just as critical to system design as the technologies we choose. By being mindful of how teams interact, we’ve been able to shape our architecture to better serve the business – and ultimately, our customers. Read Next
Life At Nexxen with Dominik Weber
For the latest installment of Life at Nexxen, we spoke with Marcie Kaufman, VP of Legal. Marcie shared how being a fashion buyer set the foundation for her legal career, the latest books she’s read, and what brought her to ad tech.
How the Nexxen SSP Scales: Our Technical Approach to High-Performance Systems
Conway’s Law, a principle coined by computer programmer Melvin Conway in 1967, states that “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” In simple terms, the way teams are structured within a company directly influences the architecture of the software they build. If teams are siloed or fragmented, the software systems they create will reflect that disjointedness, and vice versa – collaborative teams tend to build more cohesive systems. This is best explained via the following simplistic illustration: At Nexxen, we’ve witnessed this law in action many times, both in positive and challenging ways. Below are a few real-world examples where Conway’s Law was applied, intentionally or otherwise, and how it shaped our technical architecture and team structure. The Monolith Several years ago, Nexxen acquired a successful ad-tech firm that had built an impressive and feature-rich platform. However, it came with one significant architectural challenge: it was built as a monolith. A single, unified codebase where all functionality is tightly coupled. While such systems may be easier to develop in the early stages, they come with several long-term downsides. Such applications usually demonstrate slower development and scaling limitations given the butterfly effect, where for example, changing one line of code in the UI may impact the data-science codebase. It was clear we needed to break down this monolith and transition to a distributed architecture with microservices. Distributed services allow different components of the system (e.g., runtime, front-end, data) to operate independently, making the system easier to scale, maintain, and deploy. To make this transition, we reorganized the original monolithic development teams into several smaller groups, each focused on a specific domain – such as runtime, front-end, and data. These new teams, now more autonomous, were incentivized to take ownership of their respective areas, leading to the natural evolution of independent services. Each group began decoupling their domain from the monolith and building it as a standalone service. As Conway’s Law dictates, as our communication structure shifted, the software followed, evolving from a monolithic architecture into distributed microservices Distributed teams lead to misalignment Not all outcomes of the newly distributed team structure were purely positive. As teams became more independent, we noticed an increase in the rate of production issues. The issue wasn’t with the teams’ capabilities, but rather with a lack of alignment between them. As each team focused on its own service, communication across teams diminished, and this misalignment led to inconsistencies. Because the teams were operating in silos, the systems they built reflected that disconnection. The architecture had become fragmented, with services not always communicating well or adhering to shared practices. Fearing the return of the monolith, we introduced a more structured communication framework, leaving the teams independent. For example, we implemented a rigid Slack structure, ensuring all teams had dedicated channels for cross-team collaboration, changes, and feedback. We created clear guidelines for communicating changes early and soliciting feedback across teams. Another example would be the enforcement of regular cross-team meetings and a shared response system that improved communication, leading to faster resolution times and better alignment across services. By improving the communication structure, we were able to reduce production issues and bring the architecture into better alignment with the business’s needs. Freedom vs. Alignment Another example of Conway’s Law at play can be seen in our front-end development teams. Nexxen operates several independent product lines, each with its own front-end application. Each product line is supported by its own development team, leading to a natural separation in how teams approach their work. Over time, this autonomy led to diverse tool choices – teams were using different logging frameworks, testing tools, monitoring solutions, and even CI/CD pipelines. There are clear advantages to allowing teams to choose their own tools, such as increased ownership and innovation, but it raised the challenge of inconsistent practices. Each team’s differing choices in tools made it harder to maintain, monitor, and support these applications at a company-wide level. To address this, we reorganized the teams under a single department, while maintaining the independence of each team’s product line. This shared management layer ensured alignment on core tools and protocols (logging, testing, monitoring, etc.) while preserving the benefits of autonomy where it made sense. This approach allowed us to standardize critical infrastructure while giving teams the freedom to innovate within their domains. Conclusion Conway’s Law has been a guiding principle at Nexxen, sometimes intentionally and other times revealed through experience. Whether breaking down a monolith, dealing with the challenges of distributed teams, or balancing autonomy with alignment, we’ve learned that organizational structure and communication are just as critical to system design as the technologies we choose. By being mindful of how teams interact, we’ve been able to shape our architecture to better serve the business – and ultimately, our customers. Read Next
Conway’s Law of Engineering Management
Conway’s Law, a principle coined by computer programmer Melvin Conway in 1967, states that “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” In simple terms, the way teams are structured within a company directly influences the architecture of the software they build. If teams are siloed or fragmented, the software systems they create will reflect that disjointedness, and vice versa – collaborative teams tend to build more cohesive systems. This is best explained via the following simplistic illustration: At Nexxen, we’ve witnessed this law in action many times, both in positive and challenging ways. Below are a few real-world examples where Conway’s Law was applied, intentionally or otherwise, and how it shaped our technical architecture and team structure. The Monolith Several years ago, Nexxen acquired a successful ad-tech firm that had built an impressive and feature-rich platform. However, it came with one significant architectural challenge: it was built as a monolith. A single, unified codebase where all functionality is tightly coupled. While such systems may be easier to develop in the early stages, they come with several long-term downsides. Such applications usually demonstrate slower development and scaling limitations given the butterfly effect, where for example, changing one line of code in the UI may impact the data-science codebase. It was clear we needed to break down this monolith and transition to a distributed architecture with microservices. Distributed services allow different components of the system (e.g., runtime, front-end, data) to operate independently, making the system easier to scale, maintain, and deploy. To make this transition, we reorganized the original monolithic development teams into several smaller groups, each focused on a specific domain – such as runtime, front-end, and data. These new teams, now more autonomous, were incentivized to take ownership of their respective areas, leading to the natural evolution of independent services. Each group began decoupling their domain from the monolith and building it as a standalone service. As Conway’s Law dictates, as our communication structure shifted, the software followed, evolving from a monolithic architecture into distributed microservices Distributed teams lead to misalignment Not all outcomes of the newly distributed team structure were purely positive. As teams became more independent, we noticed an increase in the rate of production issues. The issue wasn’t with the teams’ capabilities, but rather with a lack of alignment between them. As each team focused on its own service, communication across teams diminished, and this misalignment led to inconsistencies. Because the teams were operating in silos, the systems they built reflected that disconnection. The architecture had become fragmented, with services not always communicating well or adhering to shared practices. Fearing the return of the monolith, we introduced a more structured communication framework, leaving the teams independent. For example, we implemented a rigid Slack structure, ensuring all teams had dedicated channels for cross-team collaboration, changes, and feedback. We created clear guidelines for communicating changes early and soliciting feedback across teams. Another example would be the enforcement of regular cross-team meetings and a shared response system that improved communication, leading to faster resolution times and better alignment across services. By improving the communication structure, we were able to reduce production issues and bring the architecture into better alignment with the business’s needs. Freedom vs. Alignment Another example of Conway’s Law at play can be seen in our front-end development teams. Nexxen operates several independent product lines, each with its own front-end application. Each product line is supported by its own development team, leading to a natural separation in how teams approach their work. Over time, this autonomy led to diverse tool choices – teams were using different logging frameworks, testing tools, monitoring solutions, and even CI/CD pipelines. There are clear advantages to allowing teams to choose their own tools, such as increased ownership and innovation, but it raised the challenge of inconsistent practices. Each team’s differing choices in tools made it harder to maintain, monitor, and support these applications at a company-wide level. To address this, we reorganized the teams under a single department, while maintaining the independence of each team’s product line. This shared management layer ensured alignment on core tools and protocols (logging, testing, monitoring, etc.) while preserving the benefits of autonomy where it made sense. This approach allowed us to standardize critical infrastructure while giving teams the freedom to innovate within their domains. Conclusion Conway’s Law has been a guiding principle at Nexxen, sometimes intentionally and other times revealed through experience. Whether breaking down a monolith, dealing with the challenges of distributed teams, or balancing autonomy with alignment, we’ve learned that organizational structure and communication are just as critical to system design as the technologies we choose. By being mindful of how teams interact, we’ve been able to shape our architecture to better serve the business – and ultimately, our customers. Read Next
Life At Nexxen with Marcie Kaufman
For the latest installment of Life at Nexxen, we spoke with Marcie Kaufman, VP of Legal. Marcie shared how being a fashion buyer set the foundation for her legal career, the latest books she’s read, and what brought her to ad tech.
Life At Nexxen with Jessie Brickner
For the latest installment of Life at Nexxen, we spoke with Jessie Brickner, Engineering Manager on the SSP side. We talked about lake fishing, nearly a decade of systems work, and more.
Partner Spotlight with Raphael Rivilla
We recently sat down with Raphael Rivilla, Chief Media Officer, from Marcus Thomas to discuss their award-winning campaign with Troy-Bilt. Read more of our conversation below: Congratulations on your big Digiday Content Award win for Best Interactive Content Piece for Troy-Bilt’s “Low, Slow and Mow” campaign! Voice-to-Action (VTA) is a fresh innovation in CTV which is just starting to get traction with brands; what excited you most about using VTA for this campaign? Marcus Thomas and Troy-Bilt have always leveraged new technology as part of our test and learn budgets. We were the first Outdoor Power Equipment company to leverage Watson’s AI when it first came out, so when we heard about this new extension of CTV as a way to distribute our Low, Slow, and Mow recipe content, then it was a no-brainer. We could literally send our recipes to interested consumers from the CTV prompt. Can you speak about how you collaborated with Nexxen Studio on developing your creative assets? Marcus Thomas media and creative leads worked together with the Nexxen Studio team to ensure the holistic look and feel of the Low, Slow & Mow campaign was captured in the Voice-to-Action CTV unit. The branded frame was refined to ensure the animation and introduction of the voice invocation was cohesive with the Troy-Bilt brand asset, resulting in a comprehensive viewing experience for consumers with this first-to-market approach. In partnership with both Nexxen Studio & Say It Now, Marcus Thomas also provided creative direction on the script for the Alexa skill which ultimately redirected viewers to the landing page to view additional recipes. When we heard about this new extension of CTV as a way to distribute our Low, Slow, and Mow recipe content, then it was a no-brainer. When we heard about this new extension of CTV as a way to distribute our Low, Slow, and Mow recipe content, then it was a no-brainer. Your campaign obviously resonated with audiences, so what was your approach to ensuring that this creative was activated programmatically in a way that would achieve your desired results? We have specific target audiences for Troy-bilt so we ensured that our CTV spots targeted these audiences while layering in audiences that engaged with voice-assistants. What is one piece of advice you’d give to other agencies when they plan cross-platform campaigns? Think about the value exchange and the consumer journey and how you can deliver content of value, first. Second, make sure the content can be modular so it can be distributed and shared across various formats. The addition of AI in platforms to customize your existing content to dynamically fit platforms and audiences makes it even easier now. Read Next Connect With Us Learn how you can effectively and meaningfully leverage today’s video and CTV opportunities with our end-to-end platform, data and insights. Contact Us
Partner Spotlight with Kevin VanValkenburgh
We sat down with Kevin VanValkenburgh, Media Director at Guru, to learn more about the agency’s approach to successfully target consumers and his thoughts on Google’s recent announcement. Guru describes itself as a purpose-driven agency with a mission to increase joy and reduce suffering. What does that mean for brands you work with? We work on purpose driven brands only and that means unique products and services with very diverse audiences and sometimes smaller budgets. The joy comes from creating the red thread that ties relevant messages to these multiple unique audience segments. Suffering is reduced by encouraging consumers to try products and services that are not only good for them, but for the planet. What is the value that Nexxen delivers that led you to select us not just as your sole ad tech partner for the Throat Coat campaign, but also Traditional Medicinals general branding campaigns? Having in-depth audience tools to help us find not just Conscious Consumers who are the broadest targets for our purpose driven clients but also being able to refine and enrich these audiences showing interest and giving off in-market signals is a big reason we partner with Nexxen. As a small, purpose driven agency, we needed help to plan a path for our team to take control of our media investments with a DSP that is intuitive and easy to learn. Bonus points for being connected to a real time updating audience platform and having a great set of account leads and trainers to guide the way. Having in depth audience tools to help us find consumers and being able to refine and enrich these audiences is a big reason we partner with Nexxen. Having in depth audience tools to help us find consumers and being able to refine and enrich these audiences is a big reason we partner with Nexxen. Switching topics a bit here–what are your thoughts on Google’s recent rollback on their plans to eliminate cookies? Were you surprised? Truthfully, I don’t think it matters what Google does or doesn’t do around 3rd party tracking pixels anymore. It is estimated that 70% of consumers no longer have a cookie, and within Chrome, 40% of consumers have manually disabled cookies themselves. I truly believe the best way to move forward in digital to get the best targeting and measurement is using unique ids that are 1st party data from brands and other sources that can be enriched and pushed across multiple platforms instead of just one. I don’t think it’s back to the 3rd party cookie at all and between tech builders and government the 3rd party cookie will still be dead. It probably is now with things that consumers and tech have allowed to enhance privacy. If you had one piece of advice to give to other media buyers/planners in the industry who want to improve programmatic performance for their clients, what would it be? Focus on audience and inventory. These are the two most important places you make mistakes in programmatic. Create the relevancy of the message to the targets and serve them wherever they are, especially outside the walled gardens. Our modern view of digital is that content, audience, platforms and inventory are the pillars that are one fail all fail. They sit on a strong base of measurement with a roof of AI/machine learning. Our campaigns and creative/connections internal alignment from step one with a client allows us to just do it better at connecting messages and content to drive results. This is especially true for our portfolio of purpose driven brands who can’t afford an old school batch and blast approach to targeting by traditional demography or with old 3rd party sources as they need their budgets to deliver results to the bottom line, not just a meaningless media report. Read Next Connect With Us Learn how you can effectively and meaningfully leverage today’s video and CTV opportunities with our end-to-end platform, data and insights. Contact Us
Driving emotion across the funnel: TV advertising at the Euros
The UEFA European Championship 2024 [Euros] kicked off a Summer of sport that was filled with moments of high drama, emotion and surprises. But those moments weren’t just confined to on the pitch, or even the participating teams. The brands who looked to capitalize on the event created advertising that ticked all these boxes too. At Nexxen, our Studio Insights team is constantly testing ads to see how they perform on sets of key metrics. And over the course of the first week of the Euros, we tested eleven different ads, for ten different brands. The insights gained from this thorough process gives a one-of-its-kind analysis of this summer’s football advertising in the UK. Consumers are Proud and Happy Our analysis shows us that overall, brand advertising that aimed to cash in on the Euros was a hit. In fact, in the key 18 to 34 bracket, every ad we tested performed above the norm. And for the General Population, 82% of the ads performed better than average. Furthermore, 91% of the ads outperformed on Brand Recall, with an average performance of 36% better than the UK Norm. Ironically for a country more used to disappointment, optimism still won out – 11% of respondents felt intense levels of happiness, which is 40% above the UK Norm. And appropriately for ads cashing in on the national team, pride was also a top emotion, being 112% higher than the UK Norm. That said, it’s often the ads that take a contrary angle that triumph. So BHF’s moving ad ‘Till I Die’ evoked intense Sadness, and made it stand out for the crowd, as evidenced by its high scores across the board, particularly in Brand Recall. Having conducted a thorough analysis, we’ve identified the groups most likely to respond to ads around the Euros – which we call our Opportunity Audience. This group is Males, 25 to 34 year olds, and Small to Medium families (married 1-2 children). Overall, audiences were suitably impressed by the ads we analysed, and the brands who looked to capitalize on the UEFA Euro 2024 tournament should be as proud – and happy as the viewers of their ads. Download Report The UEFA European Championship 2024 [Euros] kicked off a Summer of sport that was filled with moments of high drama, emotion and surprises. But those moments weren’t just confined to on the pitch, or even the participating teams. The brands who looked to capitalize on the event created advertising that ticked all these boxes too. At Nexxen, our Studio Insights team is constantly testing ads to see how they perform on sets of key metrics. And over the course of the first week of the Euros, we tested eleven different ads, for ten different brands. The insights gained from this thorough process gives a one-of-its-kind analysis of this summer’s football advertising in the UK. Consumers are Proud and Happy Our analysis shows us that overall, brand advertising that aimed to cash in on the Euros was a hit. In fact, in the key 18 to 34 bracket, every ad we tested performed above the norm. And for the General Population, 82% of the ads performed better than average. Furthermore, 91% of the ads outperformed on Brand Recall, with an average performance of 36% better than the UK Norm. Ironically for a country more used to disappointment, optimism still won out – 11% of respondents felt intense levels of happiness, which is 40% above the UK Norm. And appropriately for ads cashing in on the national team, pride was also a top emotion, being 112% higher than the UK Norm. That said, it’s often the ads that take a contrary angle that triumph. So BHF’s moving ad ‘Till I Die’ evoked intense Sadness, and made it stand out for the crowd, as evidenced by its high scores across the board, particularly in Brand Recall. Having conducted a thorough analysis, we’ve identified the groups most likely to respond to ads around the Euros – which we call our Opportunity Audience. This group is Males, 25 to 34 year olds, and Small to Medium families (married 1-2 children). Overall, audiences were suitably impressed by the ads we analysed, and the brands who looked to capitalize on the UEFA Euro 2024 tournament should be as proud – and happy as the viewers of their ads. Download Report Read Next