This document describes procedures for people assisting with FOLIO DevOps to add a new back-end module to the platform-complete and folio-ansible configuration, and make it available in the folio-snapshot reference environments, and make it available in the Vagrant box VMs for snapshot.
The developers responsible for that module would have already ensured that the module is ready, its repository is established, and is operating in FOLIO CI. They would have added a Jira ticket, assigned to our team so that we can schedule and track the work.
Verification and preparation
This is the relevant guide that they would have followed. Still, we need to follow it again. There are tips not repeated here, and it defines verification steps that would be wise to re-follow before committing to infrastructure configuration.
If they have not indicated on their Jira ticket that they have verified that the module will install on a local VM, then remind them to follow the Ensure recent local VM instructions. Otherwise it could be a huge waste of your time.
Investigate the module README and LaunchDescriptor in case there are other special requirements.
Add to platform-complete snapshot branch.
Make branch in folio-ansible to add to a
Make branch in folio-infrastructure to configure Jenkins for a test build.
Configure and test via a special Jenkins job.
Remove testing configuration.
Merge PR to folio-ansible, then to folio-infrastructure.
Follow the daily backend builds to verify.
Add to snapshot platform
This is the procedure for adding a backend module to the “snapshot” branch of the Stripes Platform platform-complete.
The normal process is that a UI module will require interfaces that are provided by back-end modules (declared in the “okapiInterfaces” section of their package.json file). With this, the relevant back-end modules are automatically included. (See separate procedure to install a new front-end module.)
However often the front-end and back-end are not yet ready for each other.
If so, then temporarily add the new back-end module to the
install-extras.json file of the “snapshot” branch of platform-complete.
After the back-end development is eventually ready, then a front-end module will require this module.
When that happens, then the backend module will be automatically included via the normal process, so it can then be removed from this
However some back-end modules are never required by front-end modules. In these cases they are always configured in that file.
If approaching the daily reference environment build times, then the timing of this task can be difficult.
The build-platform-complete-snapshot Jenkins build happens every hour, starting at about 19 minutes past and taking about 32 minutes. This will regenerate the yarn.lock and install files of the Platform, and automatically merge them.
So there will be a short window to prepare the branch and pull-request builds, then merge to “snapshot” branch, then its branch build.
After the next hourly “build-platform-complete-snapshot” run, then verify that the new backend module is now in the okapi-install.json file.
If there is trouble, then revert the addition to the Platform, and investigate and report the issues.
Make a branch in folio-ansible,
Add the module to the
This file assists with building the VMs.
It also currently provides special configuration for some modules
to over-ride LaunchDescriptor settings, or to provide other additional settings.
Follow the format for the entries of other modules.
The “snapshot” file is alphabetically sorted, as Okapi handles the dependency resolution.
The module is declared in the main
If the module is not to be pulled in by a front-end UI module via a platform, then it needs to be also declared in the “
Often that is the case because the development of a UI module is not yet ready to require it.
add_modules section could later be tidied, when that is finally happening.)
This section explains the preparation for conducting a run of
Make a branch in folio-infrastructure,
Add the configuration for the Jenkins job …
Add to the Jenkinsfile to refer to the folio-infrastructure branch:
branches variable to be
Next refer the git submodule to the folio-ansible branch. Take care, as this is a tab-delimited file.
[submodule "CI/ansible/folio-ansible"] path = CI/ansible/folio-ansible url = https://github.com/folio-org/folio-ansible branch = FOLIO-2467-refenvs-ncip # <<< add this line <<<
Push the folio-ansible branch and the folio-infrastructure branch.
Jenkins testing configuration
So, select the
Configure link in the top-left panel.
Then in the section “Pipeline : Pipeline script from SCM : Branches to build” replace “Branch specifier” from
refs/heads/FOLIO-2467-refenvs-ncip and then “Save”.
Run folio-snapshot-test build
If there have been master changes for folio-ansible and folio-infrastructure since branching, then do merge master to branch, and deal with conflicts.
Before proceeding, visit AWS EC2 - the instance is stopped to save a buck, so temporarily start.
Now run the test build. Select the
Build with Parameters link in the top-left panel.
Follow the results of this build run via its
Console Output and full log.
The AWS instance will be removed and rebuilt. It takes approximately 35-40 minutes.
If not success, then try to interpret the Jenkins output logfile. In some cases, may need to ssh to the instance and inspect the Okapi logfile.
Verify the test build
After success, then ensure that the new back-end module is in place. (Example ‘curl’ of course needs token and tenant headers.)
curl -s -S \ https://folio-snapshot-test-okapi.dev.folio.org/_/proxy/tenants/diku/modules \ | jq -r '. | select(.id | match("mod-"))' | sort
Do ssh login to the AWS instance (search the Jenkins build output for “ec2-“ to get the DNS name) and confirm that the module’s docker logs are not spewing errors, and that its docker has not terminated.
Could also verify via the front-end “Settings : Software versions” page (e.g. folio-snapshot-test.dev.folio.org).
Now that there is happiness, prepare and merge the pull-request to folio-ansible.
Remove the test configuration …
- At the Jenkins interface for “folio-snapshot-test”, revisit its
Configurepage and revert the “Branch specifier” to be
*/master(which also signals to other DevOps that you have finished with the test builds).
- Visit AWS EC2 - the “folio-snapshot-test” instance needs to be in “stopped” state to save a buck.
Now make the PR.
Be aware of the timing of the automated back-end build jobs, as explained at the reference environments.
(If you need to manually run these jobs outside the normal automated schedule, then may need to ask #hosted-reference-envs channel, as people depend on these environments.)
Inspect build results
Await the scheduled automated build.
Verify the results of the backend build, as done above.
curl -s -S \ https://folio-snapshot-okapi.dev.folio.org/_/proxy/tenants/diku/modules \ | jq -r '. | select(.id | match("mod-"))' | sort
Or inspect via the front-end.
Delete old branches.
Visit AWS EC2 - the “folio-snapshot-test” instance needs to be in “stopped” state to save a buck.
Comment and close the Jira ticket.