Running LTD Mason on Travis¶
If your project already uses Travis CI for continuous integration, you may want to run LTD Mason there for documentation deployment as well. Projects well suited for Travis-based Mason deployments are:
Documentation for EUPS-based packages need to be run from a Jenkins-based CI environment. See Running LTD Mason on Jenkins.
Composing .travis.yml for LSST the Docs¶
The exact construction of this file depends on how you need to use Travis to run tests for your project, or if you are only building documentation. This guide will outline how to deploy documentation in both cases.
.travis.yml, we recommend using Travis’ validator, available online or through the
travis command line app.
Environment variables and secrets¶
Regardless of context, all projects using Travis to deploy to LSST the Docs need to configure the Travis CI build environment through environment variables.
This is a sample
env: global: - LTD_MASON_BUILD=false # disable builds in regular text matrix - LTD_MASON_PRODUCT="sims-operations" - LTD_KEEPER_URL="https://keeper.lsst.codes" # travis encrypt "LTD_MASON_AWS_ID=... LTD_MASON_AWS_SECRET=..." - secure: "K332NczWP7D7qczaqXmcFq1nNT6/PeZ14rtRK0sFDo7/7u6KwLvhe5biXOgmKJmoOFlXoBg93uRn3tewRKvVpffPzrWwOlOHMNPR90X6uOubFxnd/IXFSWXOtEoKguKv71kTJANJvjhJkEtKm0cjrfAIBo8hMiiOqR9782oB/FQ=" # travis encrypt "LTD_KEEPER_USER=... LTD_KEEPER_PASSWORD=..." - secure: "B3bdOBOnrhb+YFaFKG0LDZ5ETGb8MnH3VdmkmrM9BNmcO9X8q39gqI9cXVaOFrqLHCBkAtLZus5pGeEetxG/87gAE04JDPWjSMCkMyszUMn8fpv3WgRGxmhKxNAlHHVFOiRJLaE2j58YnQtcXYLyIeGWe3QH6Zk+LHHaOJqtqeU="
By including these environment variables under
env.global we ensure that the entire set is applied to all builds; including environment variables directly in
env creates a build matrix.
LTD_MASON_BUILD=false should always be present to ensure that LTD Mason is only run on the build you specify, rather than all entries of a Travis build matrix.
LTD_MASON_PRODUCT should be set to the project’s slug (specified when the project was created on LTD Keeper).
LTD_KEEPER_URL is the URL of the Keeper API server. For LSST the Docs this is
The next two lines contain encrypted environment variable settings for
LTD_KEEPER_PASSWORD (credentials for Travis to work with LTD Keeper) and
LTD_MASON_AWS_SECRET (credentials for Travis to upload to AWS S3).
Encrypting variables requires the Travis gem, which can be installed with:
gem install travis
From the root of your Git repository, encrypt the environment variables using
travis encrypt "LTD_MASON_AWS_ID=... LTD_MASON_AWS_SECRET=..." travis encrypt "LTD_KEEPER_USER=... LTD_KEEPER_PASSWORD=..."
Paste the results as list items into the
env.global section of
Alternatively you could pass the
--add env.global flag to
travis encrypt, but note that your entire
.travis.yml file will be reformatted.
These settings can be securely checked into GitHub since only Travis possesses the private key needed to decrypt those
You may need to ask an LSST the Docs administrator to add these encrypted lines since the Keeper and AWS credentials are not widely distributed.
Travis setup for a pure Sphinx repository¶
travis.yml file below shows how Travis can be configured to build a pure-documentation repository (such as a Technote).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
sudo: false language: python matrix: include: # This is the ltd-mason documentation deployment build - python: "3.5" env: LTD_MASON_BUILD=true install: - pip install -r requirements.txt - pip install ltd-mason script: - sphinx-build -b html -a -n -W -d _build/doctree . _build/html after_success: - ltd-mason-travis --html-dir _build/html env: global: - LTD_MASON_BUILD=false # disable builds in regular text matrix - LTD_MASON_PRODUCT="dmtn-013" # travis encrypt "LTD_MASON_AWS_ID=... LTD_MASON_AWS_SECRET=... LTD_KEEPER_URL=... LTD_KEEPER_USER=... LTD_KEEPER_PASSWORD=..." --add env.global - secure: "edPD6RMPjPCvuuqtXc3nmQ2T5tvVWWFnhC957tmqyzvFS++cEZLXRCRRenlxd+9ygm7qONWXtcvJeyWiaVi9pooeFpIKlcVWeJQAZD+oLsONrFBzdPQrP8ObaeD845T9meFR+k48uFpvB2yHc3e0ZUczhxJbKUxSQzlX1xRIVh9YwwWCLbBitNHcTNsnNDfdvxFw5w3CTkWdd8j6962eOzOQHgO/Ta5B71Ab/0XwVWEh7C5mE3jNq4GRLlEXp9vyDr0DzHihVN/XkMMeSTrUj20pHePvxJh0lJi6zmseX5RCKu1hVe7dvv2FUySVB/BzW6O4kFgS7L3ynaF5jGhiIjkGMwIgd+o1ucDbuPtFqk6e1SBGwc2y2ZVijN/D2ZeQOWGDx2lUrVbdmP3MW5YQDuHfep/8H0npmvd/pUlnIjT/cxSMVW0rQlH0O+ZyTbR30lX1JRCzHgjDqX78m1JHaaOdAjcJ+3GfLRr7vWaCr4mb9NbKMCgtkt6efN6E/cfTB8xbFC0x/TF2QcKIWGQKWxCnGuXgqXlOsoR367kgqWps5l5jGmndth24/sK0UWBlqE9Vkhi3ea58Uprh6RjlMvG/6syAhhsDR1u+T48T3ABpKJHhkrbRncxpiWyki3s9t8z85MLD2dBqPF8RanaeI/86ecoNln1imcWRViCwyFQ="
In this Travis configuration, the Sphinx documentation repository’s dependencies specified in a
requirements.txt are installed, along with
script stage we build the Sphinx project with
The flags and options are as follows:
-b htmlto build HTML.
-ato force a complete build. This should be an issue on Travis, but including the flag does protect against cases where the HTML build output was accidentally checked into the Git repository.
-n -Wactivates ‘nitpicky mode’ where broken references become warnings, and warnings are elevated to errors that cause
1and fail the build.
.is the root directory of the documentation relative to the root of the Git repository; here the Sphinx project resides at the root of the Git repository.
_build/htmlspecifies an output directory for the built HTML.
Note: Unlike the
ltd-mason build tool,
ltd-mason-travis defers the Sphinx build step to the Travis environment (Travis excels at executing and logging scripts already).
If the Sphinx build was successful, the
after_success stage is executed.
Here we run
ltd-mason-travis to deploy the built documentation.
The HTML output directory from
sphinx-build must match the
--html-dir argument for
Note that we use
matrix.include to include a single Travis build.
This technique is for forwards compatibility if continuous integration testing against multiple environments is later added to a project—such as the next example.
Travis setup for LSST the Docs deployment alongside Python CI¶
In many cases you will want to use Travis for continuous integration of a software package, in addition to documentation deployment.
travis.yml sample shows how this can be achieved.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
sudo: false language: python python: - '2.7' - '3.4' - '3.5' - '3.5-dev' matrix: allow_failures: - python: "3.5-dev" include: # This is the ltd-mason documentation deployment build - python: "3.5" env: LTD_MASON_BUILD=true install: - pip install -r requirements.txt - pip install ltd-mason - pip install -e . script: - py.test --flake8 --cov=ltdmason - sphinx-build -b html -a -n -W -d docs/_build/doctree docs docs/_build/html after_success: - ltd-mason-travis --html-dir docs/_build/html env: global: - LTD_MASON_BUILD=false # disable builds in regular text matrix - LTD_MASON_PRODUCT="ltd-mason" # travis encrypt "LTD_MASON_AWS_ID=... LTD_MASON_AWS_SECRET=... LTD_KEEPER_URL=... LTD_KEEPER_USER=... LTD_KEEPER_PASSWORD=..." --add env.global - secure: "CIpaoNzWwEQngjmj0/OQBRUOnkT9Rq8273N5ZgXmZTtVSliukfJMROQnp9m42x3a2XFamaYV60mmuAvMRNU8VHi4nePxF2vp7utVnp8cF4zFQQzL6KnN2rqWv0H3Snqc1sfMT2n4H9qgBlYG7w5Cv52VIXdwh8MqGSxl8HAiYgqcVNJ+q1Rxeb1Yk+Bv3VW6O0/K4AlrhGY2Gl/zbwgM4ph0K0UvT1IZg8ZjCdddOpgwxPq66kvzHNcpCR6JUnvy5vRVH+IgC83Ar+oJqOA/4pizcFccriLF7nANkVJMrRSL8B1h2IHuuGYpC2VzDPMlAuEPmU6t8QAhVCOq9BSy98902TgKkvt4enPcxS2iNqMoOJSNUW7q9yqvVacz4JApJfHWlq5K7uTy00p4XHV4TUs+9NEgBUCwEFE5CXcRQvg+Y2y1wqUUkH+12nb1Nv4CdGxG6k7yG+eM+qmANJ87jZK9vX0RmDLKXuA3gpJyVomrAKX1+MqqwD0Qu885AUsHCQevO+oDmXv6nKLK/x2ZeyHQrgWISj3LXU6B7LarLrqsrE7JWTwgo/iX6xiVHS422tj94/+rab3JarBWe+ntdG9rZBdILU92kLqzgMA570ryVxtsnu8GnzOB0/3yvdtW+duAgrrBUusBcg9E/Kz/68Cm5RbMLyjaeA6HxP6mfM4="
python section defines a matrix of builds against several versions of Python.
During each build, the lines specified in the
install section are executed, followed by the
install stage first installs the Python project’s development dependencies specified in
requirements.txt, followed by the
ltd-mason package and the Python package itself.
script stage we run tests (
py.test, in this example) and then
Again, we have configured Sphinx to fail the build on any warnings so you effectively get CI of the documentation in addition to code.
Finally, in the
after_success stage we trigger
ltd-mason-travis to deploy the previously-built HTML resources.
For Travis projects that CI code we want
ltd-mason-travis to run in the
after_success stage to ensure that documentation is only deployed for commits that don’t have issues.
Notice that each environment in the build matrix (Python 2.7, 3.4, and so on) executes
However, we only want a single documentation deployment from each code push.
We solve this by setting the default value of
false, which causes
ltd-mason-build to skip deployment.
It’s only in the extra build environment specified in
matrix.include that we change
This technique allows you to create a build matrix for CI without worrying about multiple accidental documentation deployments.