Skip FOLIO Project Navigation

Introduction

This guide provides an overview of the ModuleDescriptor.

The ModuleDescriptor (MD) is introduced in the Okapi Guide (e.g. here and here).

The MD adheres to the ModuleDescriptor.json schema.

Maintenance

Back-end modules

For back-end modules, a template is maintained. This is usually at ./descriptors/ModuleDescriptor-template.json and is transformed by the module’s build process into the target/ModuleDescriptor.json file. See example at mod-notes.

To validate changes, utilise a JSON Schema validator such as z-schema:

cd mod-notes
jq '.' descriptors/ModuleDescriptor-template.json
z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  ./descriptors/ModuleDescriptor-template.json

Front-end modules

For front-end UI modules, the ModuleDescriptor.json is generated from various information that is maintained in the package.json file. See example at ui-users.

To validate changes, first use stripes-cli to generate the MD. Then utilise a JSON Schema validator such as z-schema:

cd ui-users
jq '.' package.json
stripes mod descriptor --full --output /tmp
z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  /tmp/users.json

Registry

As part of the continuous integration process, each ModuleDescriptor.json is published to the FOLIO Registry at https://folio-registry.aws.indexdata.com/

ModuleDescriptor properties

Introduction

Refer to the general introduction above.

General MD properties

Each main property is briefly described in the ModuleDescriptor.json schema.

The following sub-sections explain some properties in more detail …

TODO: Provide brief explanations of some sections.

metadata

The “metadata” section enables some additional descriptive items. The MD schema enables any JSON object.

LaunchDescriptor properties

Introduction

The LaunchDescriptor (LD) is introduced in the Okapi Guide (e.g. at sections Deployment and Discovery and Deployment and Auto-deployment).

The LD adheres to the LaunchDescriptor.json schema.

As explained in the Okapi Guide, the LaunchDescriptor can be a separate descriptor, or be part of the ModuleDescriptor. The LaunchDescriptor can utilise various methods for deployment.

Default LD Docker properties

For the suite of back-end modules that are hosted in the folio-org GitHub organization, each one has a LaunchDescriptor for Docker, and the LD is included in the module’s ModuleDescriptor file.

For a back-end module to be included in the reference environments, it must have such a LaunchDescriptor.

This enables ready default deployment. Each module’s LD settings are used directly in the FOLIO reference environments. Note that those installations have a small amount of data and low activity load. Other installations would probably adjust or replace these LDs.

The "env" section of the "launchDescriptor" properties contains an enumeration of variables (such as DB_USERNAME and JAVA_OPTIONS) and sample values. It is considered best practice to override these values in the Launch Descriptors created for production environments.

General LD properties

Each main property is briefly described in the LaunchDescriptor.json schema.

The following sub-sections explain some properties in more detail.

Refer to the example below.

dockerCMD

Optional. Over-ride the CMD of Dockerfile. An array of string items.

Example LDs that use this: mod-login

Memory

Limit each container’s memory usage.

All LDs must have the memory setting.

The setting must be expressed as bytes.

Take care to have the correct number of digits, typically nine.

As noted above, this memory level is appropriate for running a basic system such as the “folio-snapshot-load” reference environment and the Vagrant VMs. So this default container memory setting should be as low as possible.

NOTE: 20191118: Module developers: please determine the lowest possible container memory allocation, and adjust your setting (see FOLIO-2315).

With the new directions for LaunchDescriptors and Dockerfiles (see FOLIO-2358), and as shown in the example below, two-thirds of the memory allocation will be reserved for heap space.

HostPort binding

Okapi will map the “%p” value to the relevant port for this container.

env

The default environment for deployment.

This is an array of items, each with properties: name (required), value, description.

All LDs for Java-based modules must have the base JAVA_OPTIONS setting.

If the module uses a database, then it must provide the standard set of DB_ database settings shown in the example.

Other environment variables can also be documented here (see examples). Their default values would need to make sense in the FOLIO reference environments where these LaunchDescriptors will be used as-is. The defaults also need to make sense in a cluster.

env JAVA_OPTIONS

This environment variable must at least have the setting as shown in the example, which enables Java to configure the specified memory for the container.

The module’s Dockerfile needs to use a base image that has the feature “UseContainerSupport” – which was backported to Java 8 (8u191+). Use that in conjunction with “MaxRAMPercentage”.

Use the folioci/alpine-jre-openjdk8:latest

NOTE: 20191118: This is now being applied to all modules. See FOLIO-2358.

Other necessary options can be appended (e.g. -XX:+PrintFlagsFinal).

See one explanation of Java memory footprint, monitoring, and profiling.

Also note that “setting -Xmx and -Xms disables the automatic heap sizing”.

env DB environment

If the module uses a database, then it must provide the standard set of DB_ settings shown in the example.

Some need explanation:

  • DB_HOST keyword “postgres” is automatically mapped by the relevant system.
  • DB_DATABASE is “okapi_modules” used by all modules.

Example LaunchDescriptors

The following example is for mod-notes which does use a database.

As explained above, this form needs the new folio docker image.

  "launchDescriptor": {
    "dockerImage": "${artifactId}:${version}",
    "dockerPull": false,
    "dockerArgs": {
      "HostConfig": {
        "Memory": 357913941,
        "PortBindings": { "8081/tcp": [ { "HostPort": "%p" } ] }
      }
    },
    "env": [
      { "name": "JAVA_OPTIONS",
        "value": "-XX:MaxRAMPercentage=66.0"
      },
      { "name": "DB_HOST", "value": "postgres" },
      { "name": "DB_PORT", "value": "5432" },
      { "name": "DB_USERNAME", "value": "folio_admin" },
      { "name": "DB_PASSWORD", "value": "folio_admin" },
      { "name": "DB_DATABASE", "value": "okapi_modules" },
      { "name": "DB_QUERYTIMEOUT", "value": "60000" },
      { "name": "DB_CHARSET", "value": "UTF-8" },
      { "name": "DB_MAXPOOLSIZE", "value": "5" }
    ]
  }

Other examples:

Testing the modifications

Use jq:

jq '.' descriptors/ModuleDescriptor-template.json

Use z-schema:

z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  descriptors/ModuleDescriptor-template.json

Deploy the module as a Docker container with a local FOLIO Vagrant VM (e.g. folio/testing). See instructions.

Inspect the container, and gather some crude statistics:

vagrant ssh
sudo docker ps  # and find your container ID
sudo docker stats [CONTAINER...]
sudo docker exec -it CONTAINER bash
pmap -x 1

See other monitoring notes above.