Lessons in Testing, Performance, and Legacy Systems from /dev/mtl 2025

The /dev/mtl 2025 conference, held in Montreal—the birthplace of the search engine—brought together approximately 150 developers to discuss common challenges and innovations. Montreal's history as the home of Archie, an early tool for finding files across FTP servers, mirrors the conference's spirit: focused solutions to specific issues that ultimately resonate across the broader tech landscape. This cross-community event, organized by a coalition of 14 local tech groups including Java and Python user groups, CNCF, AWS meetups, and Women in AI, featured 21 speakers presenting in both French and English across three tracks at École de technologie supérieure (ÉTS). The conference highlighted critical lessons in managing testing complexity, optimizing performance, and navigating legacy systems.

Unchecked Complexity Makes Testing Unpredictable

One key theme addressed was the challenge of unchecked complexity in testing. Gleb Bahmutov, Senior Director of Engineering at Mercari, presented on "Feature Flags And End-to-End Testing." He highlighted how feature flags, while beneficial for incremental releases and experiments, can exponentially complicate end-to-end testing. For instance, three flags with two states each result in eight testable states, a number that doubles with each additional flag. This leads to unpredictable test failures due to percentage rollouts and environment misalignment. Bahmutov advocated for per-user control of feature flags, keeping them as production-like as possible and targeting individual user IDs in tools like LaunchDarkly. The overarching lesson emphasized lifecycle discipline: treating flags as temporary, migrating tests as defaults change, archiving completed flags, and aggressively retiring old ones. He also advised against developing custom feature flag systems.

Invisible Complexity Impacts Performance

The issue of invisible complexity impacting performance was tackled by Reza Madabadi, Software Developer at 360.Agency, in his talk "Why Your Database Hates You: The N+1 Query Problem." Madabadi pointed out that while tools like MySQL's EXPLAIN can show long query times, they often don't reveal the root cause. He stressed the importance of tracing to uncover how a single request can cascade into thousands of database calls—the N+1 problem. This issue often stems from Object Relational Mappers (ORMs) and lazy loading in frameworks like Java and Hibernate, where ORMs perform one query for a list and N subsequent queries to hydrate associations. Madabadi discussed partial fixes like join fetches and Hibernate batch size tweaks but emphasized that a more sustainable solution lies in designing for Data Transfer Objects (DTOs) directly. This involves using entity-based CRUD where appropriate and crafting targeted SELECT DTO queries for specific data needs. Pairing this with local tracing tools such as Digma can help identify N+1 patterns early.

Improvements Take A Dedicated Approach

Lucian Condrea, a freelance full-stack developer and contributor at Tribe Social, shared his journey with software testing in his talk "My Journey With Software Testing." Condrea recounted his struggle with manual testing, slow QA, and a lack of strategy, where leadership perceived "QA as too slow" without understanding the underlying cognitive load. His breakthrough came from intentional learning, building a "testing wishlist," and dedicating 30-minute daily practice sessions. He cited inspirations like Kent C. Dodds for clarifying the importance of tests and Nicolas Carlo’s “Legacy Code: First Aid Kit” for practical approaches to creating testable code boundaries. Condrea adopted a pragmatic philosophy: "Tests should serve you, not the other way around." He advised focusing on readable, co-located integration tests for optimal return on investment, advocating for minimal mocking and only essential end-to-end coverage. His key takeaway was to reflect on strategy, not dogma, and to build deliberate habits that instill confidence rather than resentment in testing.

Lessons From Developers For Everyone

The conference crystallized several universal lessons for developers:

Legacy Is Our Shared Starting Point

The reality is that most meaningful work happens within existing legacy systems. Rather than aspiring for greenfield projects, developers must learn to navigate and evolve within existing constraints. Accepting this transforms legacy management from a "shameful side quest" into a core design challenge focused on change without breaking past promises.

Feedback Loops Beat Raw Velocity

While speed is important, understanding and measuring system behavior is equally crucial. Teams with trusted feedback loops—whether through observability, informative test failures, or metrics on user experience—are calmer and more effective. Without the ability to observe, reasoning becomes impossible, and perceived speed is merely working in the dark.

Guardrails Over Heroics

Strong teams design systems with guardrails that make safe behavior the default. This includes defensible defaults, explicit lifecycles for features and configurations, and constraints to prevent complexity from escalating. Treating experiments, flags, and tests as living systems with an end-of-life plan reduces the reliance on individual "rockstars" and enables normal teams to maintain stability and security.

Tools Change, Habits Compound

Underlying all specific technologies, the most significant leverage comes from consistent habits, not fleeting tools. Speakers frequently highlighted the value of small, repeatable practices: dedicating time to improve tests, routinely inspecting system behavior, and proactively retiring complexity. The future of systems depends more on disciplined habits than on the next big technological breakthrough.

A session on secrets security was also presented at this developer-focused event. It was encouraging to see attendees, who may not regularly interact with security teams, immediately recognize the dangers of plaintext credentials and show eagerness to embrace available solutions for safer and more efficient work.

Ultimately, innovations arise from persistence in addressing real issues. The conference reinforced that regardless of the enterprise technology domain, the principles remain consistent: embrace legacy, integrate observability, establish guardrails, and cultivate habits that make the secure and efficient path the easiest. By doing so across testing, performance, search, and security, the next major breakthrough will likely emerge not from a single invention, but from thousands of small, deliberate improvements made by dedicated teams, much like those who gathered at /dev/mtl.