ASN1_STRING type is now opaque

Previous posts about the upcoming OpenSSL 4.0 release:

  1. removing ENGINE code
  2. removing deprecated functions for creating or modifying custom METHODS
  3. no longer registering a function via atexit function
  4. adding ECH support
  5. removing SSLv3 and SSLv2 Client Hello

Summary

The ASN1_STRING structure can no longer be accessed directly. Instead, accessor functions must be used.

While these accessor functions have been available since OpenSSL 1.0.1, this change is being made now to enable future work improving X509 memory efficiency. Requiring accessor functions will allow ASN1 strings to be stored as pointers to data in read only memory instead of making duplicate copies.

Given a structure pointer of the form:

ASN1_STRING *foo; 

The most common case by far requires a straightforward change:

For extensive discussion and many examples of upstream patches, please see Make ASN1_STRING opaque and X509 more memory-efficient in 4.x. The actual change was PR #29862.

Additionally, the ASN1_OBJECT_new() function has been deprecated.

In a related development, many X509 functions have had their arguments and return values made const. See Constification of X509 functions.

Details

Paul Dale, one of the OpenSSL Library’s committers, wrote about OpenSSL 3.0:

The groundwork for the OpenSSL 3.0 changes was laid out in OpenSSL 1.1. [Most] internal data structures were made opaque. This critical change permits the project to extend and expand functionality and rework internal infrastructure without having to do it while wearing a straitjacket.

Before OpenSSL 1.1, everything was open and available to users and many APIs that should have been for internal use only were globally visible and used by applications. Needless to say, the transition for our users to OpenSSL 1.1 was not without pain. Any future transitions need to be as smooth and painless as possible.

One data structure that was not made opaque was ASN1_STRING, which is part of the Abstract Syntax Notation One (ASN.1) implementation. It’s the foundation of other formats including X.509, which is used for public key certificates.

As a result, application code might look at or even modify ASN1_STRING buffers at any time. As long as that’s true, it’ll be impossible for OpenSSL developers to improve the memory efficiency of the X509 object. As Paul put it, the OpenSSL Library is “wearing a straitjacket”.

Bob Beck, a software engineer at the OpenSSL Corporation, and David Benjamin, a software engineer at Google working on BoringSSL, documented a transition plan. In addition to extensive discussion, that GitHub issue documents a large number of commits to upstream projects to prepare for OpenSSL 4.0. Many of those patches were submitted by Theo Buehler, who is a contributor to LibreSSL.