TL,DR: Thank You, thoughtbot
This post is a big ol’ Thank You post to the team at thoughtbot (a software consultancy company) for their generosity in creating and managing open source software projects over a lifetime of a project I work on a fair amount these days – PyPI.org, the largest free repository of Python packages.
Open Source Software (OSS) can be a force-multiplier for any software development project, as it reduces the need for the developer to write all of the code necessary to finish a task/project/website/etc. The motivation to share software as free and open for others to use can differ wildly from one person to the next, but the fact that anyone does is much appreciated, thanks again!
History Class
Back in 2013, when Macklemore & Ryan Lewis’ Thrift Shop was all the rage, is when I probably became aware of thoughtbot, as I was building up Datadog’s operations team and practices. We used Opscode Chef to manage the ever-growing complexity of our fleet of cloud instances. The earliest reference I can still dig up is my own blog, The Importance of Dependency Testing (generally still good advice!) and the super helpful tools and blogs they wrote to help folks understand not only the code they shared, but how they used it, and why.
I incorporated one of their tools into one of my own tools’ testing workflows, to help users of my tool have a better experience. It worked as advertised, and I used their tool again in early 2014 to help do the same for more “official” open source software.
While many of us have moved on from Chef software, I’ll never forget the #ChefFriends we made along the way. 🥰
Narrator: Mike wasn’t part of this next bit, the details might be a bit wrong, sorry!
In 2015, the Python Software Foundation (PSF) was engaged in rebuilding PyPI to meet new demands, launching in 2018. Beyond the new underlying functionality, Nicole Harris led the user interface redesign. You can still see her mock prototype and its inner workings – most of which remains active in PyPI today.
A couple of OSS libraries were copied into the codebase back then – bourbon and neat – two projects that made writing Sass (SCSS) style-related code a little easier – both of which are now deprecated. PyPI briefly included the third project, bitters, but removed it prior to production launch as it wasn’t needed.
These tools were especially important during the times of the second browser war, providing developers simpler methods to create layouts, mix in functionality that would create better browser support for end users, despite lack of feature parity across all browsers, leading to a better end-user experience.
PyPI’s new design needed a grid layout, which wasn’t a standard yet, and neat provided clean way to make a grid layout in CSS work properly for browsers of that era (Chrome 10+, Firefox 29+, Internet Explorer 9+, Safari 5.1+, et al). If I’m reading the source correctly, neat had functions to create visual tables and place items correctly. bourbon provided behaviors (mixins and selectors) like @clearfix, and @ellipsis, utilities to prevent the need to repeat a given set of statements. Both libraries without having to write a lot of custom code to work well across browsers.
Back in 2015 this amounted to ~7,000 lines of code, and thanks to thoughtbot’s sharing of libraries they developed internally to help their clients, PyPI has a good user interface, launching to the public in 2018.
Getting into the game
I started contributing to PyPI’s codebase in 2021, with typo fixes, documentation updates, and development environment improvements – common ways I’ve found are usually welcome to OSS projects, as they are easily overlooked by the current maintainers, who are often busy with other things and overlook when an instruction is wrong. I am still occasionally tickled by the fact that my path to working at the Python Software Foundation started with writing JavaScript and CSS.
Some of my early work was dropping support for Internet Explorer 11, and as I learned more about the codebase, I spotted the bourbon/neat inclusions. In early 2023 I mentioned that these were no longer maintained as of late 2019. PyPI was using bourbon 4.2.5 and neat 1.7.2 at that time, both released in early 2015.
Over time, neat 2.0 changed its external-facing API, and bourbon 5.0 did the same for some functions. Due to how bourbon and neat were included in the repository – copied the source, not via an installation package, commonly known as “vendoring” – made updates a little harder to reason about, especially since minor changes had been added to the vendored code over time. (No criticism to the vendoring method, it worked with the tools and approaches for that era!)
PyPI’s versions were effectively “stuck” in 2015, and what was there worked well, but didn’t take advantage of the evolving browser standards that changed the need for these libraries over time.
Do the thing
I recently decided to spend a weekend day to see how much I could remove/replace to continue to keep PyPI style code modern, working, and maintainable.
I learned that the only functionality PyPI still used from neat was @span-columns(), and that could be replaced with a relatively small custom mixin using native CSS Grid (specifically grid-template-columns, grid-column, and column-gap), all widely available since 2018. Replacing ~1,100 lines of neat with this SCSS was pretty straightforward:
$grid-columns: 12;
@mixin grid-container {
display: grid;
grid-template-columns: repeat($grid-columns, 1fr);
column-gap: $half-spacing-unit;
}
The resulting compiled CSS was effectively the same visual behavior, but now used native CSS styles and is identified as browsers as using grid syntax, which opens up the ability to further transform to more responsive layouts in the future if needed.
On the bourbon side, thoughtbot had produced a Migrating from v4 to v5 guide in 2018, which included helpful suggestions on replacements or adaptations. In the intervening years, bourbon was updated to version 7.3.0, and then finally deprecated, and thoughbot again shared a guide for replacing it with alternative approaches earlier this year.
Using this guide, and now more familiar with the SCSS code used, I was able to update bourbon to the final version, and then fully replace any used functionality remaining with adequate replacements, removing another ~4,400 lines of code from the codebase.
Finishing the job
Testing website visual layouts is a whole discipline on its own, and PyPI doesn’t have a test suite that covers visual layouts, but we have a pretty robust local development environment. I used that to validate that each change I made had the desired affect as I was making them. Thankfully, most of the usages were hyper-specific to a given page, so there were only a handful of places to confirm manually.
Once I was done with all my changes, I captured the compiled distribution CSS file and ran a semantic difference checker against the file from before the changes and after. The open source tool I used was difftastic , which was able to point out the differences between the two compiled, minimized CSS files, and help me confirm that the only changes were the ones I expected. Thanks Wilfred!
Once done, I opened a pull request for review, and it made it out to production with no issues. 🥳
What have we learned, Mike?
Well, first and foremost, open source is continuing to grow, so big thanks to any and all who share their code with the world and take the time to maintain. Truly, we are stronger together.
Let’s give a round of applause to the team at thoughtbot. While I generally write less Ruby/JavaScript/CSS these days, and their code is no longer inside PyPI, I know that when I come across these folks’ tools or libraries, it’s likely to be a cut above others on the quality spectrum. Their approach is what I’d consider exemplary on how to help user navigate changes to open source from “cradle to grave” – providing essential, useful software for others, showing them how to use it, update it, and finally retire it from use. PyPI benefited from these projects for ~9 years, until no longer needed, so we salute them! 🫡
And finally, consider this a call to action to anyone out there to get involved in open source. For some ideas and approaches, check out the chat I had with my fellow AWS Hero Chris Williams last year on vBrownBag:
Thanks for reading, now go out and get involved!