Is stdlib versioned?

[awvwgk]

While writing package files for stdlib (see #291) I encountered a quite simple issue what puzzled me: What is the version number of stdlib?

It turns out that some package files simply don't work if there is no version number specified, i.e. pkg-config will not use any pc-file without a Version entry. Not specifying any version is therefore not possible if I want to use stdlib in a non-CMake project. I could just define a dummy version number for my purpose, but this calls for trouble.

Maybe this issue has been discussed already somewhere, but I didn't found any hint on this here.

[milancurcic]

Currently, no. It should be. You can think of stdlib being in perpetual 0.1.0, but that version number is not specified anywhere.

There have been discussions scattered across this repo, monthly call, and Discourse:

• Suggested change in directory structure #216
• July 2020 discussion on stdlib structure
• Stabilization of Rust language and stdlib features

TL'DR:

• We started with the idea of having 2 separate namespaces/APIs, experimental and stable. Everything goes into experimental (equivalent to 0.x.x), and when parts of stdlib make it into the Standard, they become "stable" (equivalent to 1.x.x). Separate namespaces/APIs means that experimental functions are in separate modules from stable ones. So there could be a stdlib_experimental_io module, and a stdlib_stable_io module. Functions can exist only in one or the other. But of course, this wasn't too meaningful because we don't have anything stable. So every module has the word "experimental" in it.
• This structure was challenged by @wclodius2 in Suggested change in directory structure #216. I mostly agreed and we discussed it on a call and had a follow-up discussion on Discourse.
• We tentatively agreed to unify "experimental" and (future) "stable" into the same namespace, and start semantic versioning. We didn't start it in practice.
• We also agreed to keep track of what's experimental or stable on a per procedure basis, by noting in the source code docstrings and spec docs, similar to Rust.

So, let's start versioning and making releases, like we do for fpm.

[awvwgk]

Thanks for the comprehensive summary. Really looking forward to see the first release of stdlib.

[awvwgk]

With #291 merged there is now a dummy version in place.

[jvdp1]

So, let's start versioning and making releases, like we do for fpm.

I agree with that. A couple of questions:

• Should we start with, let say, the version 0.1.0?
• Would all procedures still remain in the "experimental" namespace? Or should some produres be associated with the new version?

[awvwgk]

Version 0.1.0 is a reasonable choice.

[ivan-pi]

* Would all procedures still remain in the "experimental" namespace? Or should some produres be associated with the new version?

According to the workflow established in #94:

In order to move from experimental to release, the specification document must be approved by the wide community and the standards committee (informally). If that happens, it has now been blessed for broad use and we can move the code into the main section of stdlib, and the particular specification document becomes part of the Fortran Standard Library.

Personally, I would preserve everything in experimental for now.

@certik also suggested to introduce release notes, for example: https://github.com/sympy/sympy/wiki/Writing-Release-Notes

[awvwgk]

Any progress on this issue? I could go ahead and submit a patch to bump the version to 0.1.0 and create a release tag once merged, if we are looking for volunteers.

[milancurcic]

Yes, I'm pretty sure this is just waiting for a volunteer. If you have time to do it, please go ahead. Do you think we need to adjust the workflow in any way?

And more specific questions that come to mind:

• When do we bump version? On any change to the code? (I think so)
• When do we tag/release? On any version bump? (I think so)
• Source or binary releases or both?
• For source release, pre-processed or not? If pre-processed, how many ranks to include in the release?

Obviously, the simplest kind of release we could do initially is just source release, not pre-processed, and let the user do all that, as they do now from master.

Looking at the scope of my questions and potential answers, I think this would make a good GSoC project.

[milancurcic]

@jvdp1 @ivan-pi I think the experimental vs. release thing is less meaningful now as we agreed to move forward with semantic versioning. There are no separate namespaces for experimental and release. We simply track their status in the docstrings, but otherwise they live in the same namespace.

[awvwgk]

When do we bump version? On any change to the code? (I think so)

I haven't been watching stdlib that closely, so from an outside perspective the lifetime of a feature patch seems to measure in weeks or months. While there seems to be a preference for a short release cycle, the project currently feels rather slow-paced. Maybe setting a regular release schedule, like four times a year, is more suitable for the current project dynamic.

My general recommendation for versioning a library would be (in the zero major version stage), which is almost inline with the semantic versioning recommendation:

• bump major version for first modules, procedures, ... entering stable
• bump minor version for adding new modules, procedures, ...
• bump minor version for API (and ABI) breaking changes to existing modules, procedures
• bump patch version for API (and ABI) compatible fixes
• don't bump the version for changes on the documentation, build files, CI workflows

Personally I think ABI changes in Fortran are as important as API changes, even if they are more subtle and difficult to catch, but this is up to your judgement.

When do we tag/release? On any version bump? (I think so)

Releasing on a version bump is always a good idea.

Source or binary releases or both?

Submitting to conda-forge might be an idea, in this case a shared library would be preferred and ABI compatibility becomes important.

For source release, pre-processed or not? If pre-processed, how many ranks to include in the release?

This requires to generate build files (CMake and make) as well. It is a common practice for autoconf projects to upload releases after the autoconf step with all the configure scripts in place, it doesn't really make sense for CMake based projects in my opinion.

[ivan-pi]

@jvdp1 @ivan-pi I think the experimental vs. release thing is less meaningful now as we agreed to move forward with semantic versioning. There are no separate namespaces for experimental and release. We simply track their status in the docstrings, but otherwise they live in the same namespace.

Yes. If I recall correctly, one of my issues was that stdlib_experimental_... was too verbose. Once a routine would be stabilized, I would need to edit any programs that already depend on it. With the current docstring practice, you simply need to avoid experimental routines (currently still all of them?) if you are afraid of breaking changes. But for those of us who are willing to test and adopt stdlib it makes the process easier.

[milancurcic]

@awvwgk almost all of it sounds good to me.

Maybe setting a regular release schedule, like four times a year, is more suitable for the current project dynamic.

This is at odds with the version bump + release model you suggested in the rest of the message. And I don't think it makes sense without setting goals. The pace is slow because of lack of manpower (we struggle even with finding people to review PRs), and not because we think a slow pace is good for the project. A calendar-based schedule would then even further cement the project dynamic.

[awvwgk]

The pace is slow because of lack of manpower (we struggle even with finding people to review PRs), and not because we think a slow pace is good for the project.

Didn't meant to judge anyone. I was scrolling through the closed PRs and the commit history to get a quick overview over the recent activity to make a half-qualified comment, since I'm not actively watching the activity on stdlib (yet).

This is at odds with the version bump + release model you suggested in the rest of the message.

Never thought about it that way. Usually I make a version bump together with a release and decide on the kind of version bump based on the changes since the last version. Therefore, the version on the git default branch in between releases is usually ill-defined, it is something more than the last version but can't have a bumped version, because it is not yet complete. I don't really know if there is a way to clearly define a version number on a git branch in between releases (e.g. git --describe --tags uses the last version and adds the number of commits as revisions).

[milancurcic]

Ah, I think I understand now. In the scheduled released model, you'd simply accumulate all contributions in the default branch and do one version bump when the schedule is up.

[milancurcic]

I was thinking that every contribution to the code is a version bump + release. Considering the slow pace, I think it would work well.

[jvdp1]

I am currently quite busy with other stuffs. So if you would like to do it, please do so. Regarding the versioning, my understanding was as follows: * stdlib as a whole is versioned (let say, 0.1.0) * All new procedures go first in an experimental phase; * when there is an agreement about an experimental procedure, it is "moved" to the release version and it gets the updated version of stdlib (e..g, 0.1.1). Actually, only the specs and docs need to be changed. The doc would contain something like: `stable since 0.1.1`(instead of the current 'experimental'). It would remain so until the same procedure is modified. it seems to work like this with the Rust standard library ( https://doc.rust-lang.org/std/index.html). Le jeu. 4 févr. 2021 à 19:00, Milan Curcic <[email protected]> a écrit :

…

I was thinking that every contribution to the code is a version bump + release. Considering the slow pace, I think it would work well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5RO7HZUFBQTGMGREICBJTS5LOEBANCNFSM4VZNONWA> .

[wclodius2]

What I suggest:

1. That the codebase be reviewed on a regular schedule to determine whether some "experimental" code has matured enough to warrant a new official release. I suggest a review once a quarter.
2. That a code only be considered mature if at least six months has passed without a change in its API, and at least three months have passed since a change in its algorithms has occurred, including changes in it's called routines. This is a minimum requirement. If the code is particularly tricky or little used a longer delay may be appropriate.
3. That the releases be numbered n.m.l, where n is the zero based year since the first release, m represents the zero based m'th release of that year, and l be a fix release if the m'th release proves to be a regression, i.e. it should always be 0. This means that the first release should be 0.0.0.
4. That the individual routines be marked with the date of their last semantic (non-comment) change so that criteria 3 can be easily evaluated, and either a flag indicating whether or not they have been officially released, or the version of the first release that the code was considered stable.
5. That the flag to indicate that they have not been officially released be one of:

• Experimental
• Trial - a shorter synonym for experimental
• Alpha - a code whose API may change
• Beta - a code whose algorithm may change

6. Tracking the effects of dependencies may be difficult. What happens if routine A is not explicitly changed, but a routine A calls in a different module, say B, is changed.
7. What is the version of a module file that contains officially released procedures and experimental procedures.

[certik]

We can simply periodically review the code base and propose things to mature from experimental to stable. We can vote or even having a unanimous decision would be ideal. We should all feel good and on board before we put things into stable.

Regarding the other mechanics, I am fine I think with any of the proposed systems.

[awvwgk]

Did we reach an agreement here on the criteria to version stdlib?

[jvdp1]

Did we reach an agreement here on the criteria to version stdlib?

I don't think so. Should we discuss it tomorrow during the call?

[awvwgk]

stdlib has been in version 0.0.0 for quite some time now, maybe now with the incoming FortranCon would be a good point to roll out the first actual release. What do you think?

[jvdp1]

I think it is indeed a good time, especially now that it is supported by fpm. Le jeu. 16 sept. 2021 à 20:32, Sebastian Ehlert ***@***.***> a écrit :

…

stdlib has been in version 0.0.0 for quite some time now, maybe now with the incoming FortranCon would be a good point to roll out the first actual release. What do you think? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5RO7BFHAHLJNQCUB2YW63UCIZ4VANCNFSM4VZNONWA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[awvwgk]

I created a new milestone to collect all features for the first release: v0.1.0. Please feel free to associate any issue or patch with this milestone if it should be included in the first release, the release will be on hold until all issues are resolved and all patches merged.

The patch for the version bump to 0.1.0 can be found in #538.