Were URLs a Bad Idea? Reflecting on Java's Evolution in URL Handling

About Neil Madden

When reflecting on 26 years of Java changes, I began to consider the new HttpClient library introduced in Java 11. Historically, fetching a URL involved using URL.openConnection(). This mechanism was initially conceived as a generic way to retrieve content from any URL, encompassing files, web resources, FTP servers, and more. It was a pluggable system designed to theoretically support all types of URLs – a concept highly regarded in the 90s and early 2000s.

However, this universal approach came with a number of significant drawbacks:

• Varying Implications: Fetching different URL types can lead to drastically different security, performance, and failure scenarios. For instance, do we truly want to accept mailto: URLs or javascript: "URLs"? The answer is unequivocally no.
• Lowest-Common-Denominator API: The API's generic nature forced it to operate at the lowest common denominator. This meant that if developers needed to configure options specific to a particular protocol, they were required to cast the returned URLConnection to a more specific subclass, thereby sacrificing the very generality the API aimed to provide.

In contrast, the new HttpClient in Java 11 is significantly more effective at handling HTTP requests, primarily because it is designed specifically for HTTP/HTTPS. This specialization, it seems, is a considerable improvement.

Indeed, in the vast majority of modern applications, the uniformity once touted for URLs is no longer a desirable attribute. Most applications and libraries are optimized to handle essentially a single type of URL, and they benefit immensely from this focused approach. This raises a pertinent question: are there still genuine use cases where accepting a URL of any (or nearly any) scheme remains truly useful?