Serge NOEL
300 lines
13 KiB
Plaintext
300 lines
13 KiB
Plaintext
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta name="generator" content="HTML Tidy for HTML5 for Apple macOS version 5.8.0">
|
|
<title>JMRI: Continuous Integration</title>
|
|
<meta name="author" content="Bob Jacobsen">
|
|
<meta name="keywords" content="JMRI technical code Continuous Integration jenkins test testing ci workflow">
|
|
<!--#include virtual="/help/en/parts/Style.shtml" -->
|
|
</head>
|
|
<body>
|
|
<!--#include virtual="/help/en/parts/Header.shtml" -->
|
|
|
|
<div id="mBody">
|
|
<!--#include virtual="Sidebar.shtml" -->
|
|
|
|
<div id="mainContent">
|
|
<h1>JMRI Code: Continuous Integration</h1>
|
|
|
|
<p>"Continuous Integration" is the process of rebuilding a system every time it changes, so
|
|
that you rapidly learn of new problems and are motivated to remove existing ones.</p>
|
|
|
|
<p>The core of our system is a series of
|
|
<a href="#mandatory">GitHub Actions</a> and
|
|
<a href="#jenkins">Jenkins</a> jobs that routinely
|
|
build the code, run checks, and creates downloadable installers. That means that we can and
|
|
do put changes into our users hands very quickly. There's a <a href="CI-status.shtml">CI
|
|
status page</a> that shows the combined status of that.</p>
|
|
|
|
<p>We divide these into three groups:</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<a href="#mandatory">Mandatory checks</a> that must be OK before the change will be
|
|
merged.
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#optional">Optional checks</a> that provide additional information on the code.
|
|
These don't have to be completely cleared before a change is merged, but their
|
|
observations should be considered and their metrics should be improving.
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#independent">Independent checks</a> that are run by our <a href=
|
|
"#jenkins">Jenkins instance</a> on a periodic basis to look for deeper problems.
|
|
</li>
|
|
</ul>
|
|
|
|
<p>These checks can be run locally by developers before uploading changes to Github,
|
|
see <a href="JUnit.shtml#run">Running JUnit Tests</a>.</p>
|
|
|
|
<h2 id="mandatory">Mandatory Checks</h2>
|
|
<p>We use multiple
|
|
<a href="https://docs.github.com/en/actions">GitHub actions</a>
|
|
to test every proposed change entered into our GitHub code repository
|
|
before it gets merged.</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<a href="#winCI">Windows CI tests</a>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#headless">Headless Tests</a>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#separate">Separate Tests</a>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#static">Static Analysis</a>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="#process">Process steps</a>
|
|
</li>
|
|
</ul>
|
|
<p>These checks are defined by files in the <code>.github/</code> directory.</p>
|
|
|
|
<p>They normally run both on PRs to the main <code>JMRI/JMRI</code> repository,
|
|
but also on pushes and PRs to your own repository.</p>
|
|
|
|
<p>If you want to turn that off,</p>
|
|
<a href="images/ActionSettings.png"><img src="images/ActionSettings.png"
|
|
class="floatRight" alt="github actions"></a>
|
|
|
|
<ol>
|
|
<li>On the page for your own repository, select "Settings"</li>
|
|
|
|
<li>From the left sidebar, select "Actions"</li>
|
|
|
|
<li>Pick on option for what can run. The bottom option turns everything off for you
|
|
locally.</li>
|
|
</ol>
|
|
|
|
<p>Note that if you turn this off, the "Actions" tab won't show in the GitHub web interface
|
|
until you turn it back on.</p>
|
|
|
|
<h3 id="winCI">Windows CI tests</h3>
|
|
<p>This runs "AllTest", our test suite of over 30,000
|
|
<a href="JUnit.shtml">JUnit tests</a> on a Windows server. A screen buffer (not a real
|
|
screen) is used for all the GUI tests. All tests must pass.</p>
|
|
|
|
<h3 id="headless">Headless tests</h3>
|
|
<p>Reruns the JUnit test suite in "java.awt.Headless=true" mode to ensure that
|
|
the parts of JMRI that are meant to run without a GUI really can. This is about 80%
|
|
of the test suite. These are run on Linux. All tests must pass.</p>
|
|
|
|
<h3 id="separate">Separate tests</h3>
|
|
<p>About a hundred JUnit graphical tests are run separately from AllTest
|
|
to ensure they have a clean environment. These are run on Linux. All tests must pass.</p>
|
|
|
|
<h3 id="static">Static analysis</h3>
|
|
<p>This runs a series of static analysis checks:</p>
|
|
<ul>
|
|
<li>A pass of the ECJ compiler with warnings on
|
|
<li><a href="SpotBugs.shtml">Spotbugs</a>
|
|
<li>CheckStyle
|
|
<li>A Javadoc generation and check
|
|
<li>A scan of the help files with htmllint
|
|
<li>And a set of architecture checks of interpackage references, etc.
|
|
</ul>
|
|
<p>Any errors or warnings fail this step.</p>
|
|
|
|
<h3 id="process">Process steps</h3>
|
|
<p>There are a few additional actions that are used as process controls:</p>
|
|
<ul>
|
|
<li>Checks consistency of TypeScript and Javascript files.
|
|
<li>Set flags that represent whether a PR might need localization,
|
|
and whether it contains updated help files.
|
|
<li>If needed, reminds the author to add to the release note
|
|
<li>Prevent PRs marked "WIP" (Work In Progress) from <a href="https://github.com/apps/wip">being merged</a>
|
|
<li>Require that PRs be approved by a 2nd party before being merged
|
|
<li>And enforce a 24-hour waiting period so that the world-wide team
|
|
can get a look at the PR.
|
|
</ul>
|
|
|
|
<h2 id="optional">Optional Checks</h2>
|
|
|
|
<p>We also run advisory checks on every pull request (PR). Although we don't require that they
|
|
have zero warnings, we strongly
|
|
recommend that people look at them and try to improve their metrics of test coverage,
|
|
simplicity and understandability, etc.</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<a href="#CodeClimate">CodeClimate</a>
|
|
</li>
|
|
</ul>
|
|
|
|
<h3 id="CodeClimate">Code Climate</h3>
|
|
<p>The <a href="https://codeclimate.com/github/JMRI/JMRI/">Code Climate</a> points out places
|
|
where the code appears to be complex in various ways. (It also provides coverage information,
|
|
though <a href="#jacoco">JaCoCo</a> seems to do a better job of that.) We recommend you
|
|
look at those results and make updates where they make sense, so that things are continuously
|
|
improving, but not every suggestion it makes is worth it, or even appropriate. We don't
|
|
require this to be clean before merging.</p>
|
|
|
|
<p>Code Climate is controlled by the .codeclimate.yml file.</p>
|
|
|
|
<h2 id="independent">Independent Checks with Jenkins</h2>
|
|
<p>JMRI uses the <a href="https://www.jenkins.io/">Jenkins</a> continuous integration engine for
|
|
integration and test builds. This includes keeping our <a href="WebSite.shtml">website</a>
|
|
up to date as changes are committed, building installer packages from development builds, and
|
|
building final releases.</p>
|
|
|
|
<p>Our primary Jenkins engine is at <a href=
|
|
"https://builds.jmri.org/jenkins/">https://builds.jmri.org/jenkins/</a>. Because it's hosted
|
|
outside the US, we force its Java locale to US English by setting the <code>LC_ALL</code>
|
|
environment variable to <code>en_US</code> in the master Jenkins configuration settings.</p>
|
|
|
|
<p id="jenkins">Specific Jenkins results of interest:</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<a href="https://builds.jmri.org/jenkins/job/development/job/builds/">Builds</a> page,
|
|
showing status of the builds done after every series of commits. Successful builds are
|
|
followed by building installers via the <a href=
|
|
"https://builds.jmri.org/jenkins/job/development/job/packages">Packages</a> job.
|
|
These are sometimes referred to as "development builds".
|
|
</li>
|
|
|
|
<li id="jacoco">
|
|
<a href="https://builds.jmri.org/jenkins/job/development/job/jacoco/">JaCoCo</a> page,
|
|
showing test coverage from our JUnit tests. There's both a summary of overall
|
|
coverage, and detailed information on each
|
|
<a href="https://builds.jmri.org/jenkins/job/development/job/jacoco/lastBuild/jacoco/jmri.implementation/">class</a> and
|
|
<a href="https://builds.jmri.org/jenkins/job/development/job/jacoco/lastBuild/jacoco/jmri.implementation/AbstractAudio/">method</a>.
|
|
</li>
|
|
<li>
|
|
<a href="https://builds.jmri.org/jenkins/job/development/job/spotbugs/">SpotBugs</a>
|
|
page, showing status of the more-extensive <a href="SpotBugs.shtml">SpotBugs</a> tests
|
|
done periodically. There are also pages of the <a href=
|
|
"https://builds.jmri.org/jenkins/job/development/job/spotbugs/changes">most recent
|
|
changes</a> and the <a href=
|
|
"https://builds.jmri.org/jenkins/job/development/job/spotbugs/lastBuild/">most recent
|
|
results</a>.
|
|
<p>For more information on JMRI's use of SpotBugs, see the <a href=
|
|
"SpotBugs.shtml">separate SpotBugs page</a>.</p>
|
|
</li>
|
|
</ul>
|
|
|
|
<h3>Jenkins Integration with NetBeans</h3>
|
|
|
|
<p>If you are developing in the <a href="NetBeans.shtml">NetBeans 7.0</a> environment, it is
|
|
possible to integrate feedback from the Jenkins Continuous Integration (CI) engine.</p>
|
|
|
|
<p>Within the NetBeans environment, reference is made to the Hudson CI engine - Jenkins is a
|
|
<a href="https://en.wikipedia.org/wiki/Fork_(software_development)">"fork"</a> of the Hudson
|
|
code and supports the same <a href="https://en.wikipedia.org/wiki/API">API</a>.</p>
|
|
|
|
<p>Integration into NetBeans is achieved by executing the following steps:</p>
|
|
|
|
<ol>
|
|
<li>Open NetBeans</li>
|
|
|
|
<li>Switch to the 'Services' window (shortcut key 'Ctrl+5')</li>
|
|
|
|
<li>Right-click (Ctrl-click on Mac) the entry 'Hudson Builders' and choose 'Add Hudson
|
|
Instance...'
|
|
<p><img src="images/NetBeansJenkinsAdd.png" alt="Add Hudson Instance...">
|
|
</p>
|
|
</li>
|
|
|
|
<li>In the resulting pop-up, complete the following:
|
|
<dl>
|
|
<dt>Name</dt>
|
|
|
|
<dd>JMRI</dd>
|
|
|
|
<dt>URL</dt>
|
|
|
|
<dd>https://builds.jmri.org/jenkins/</dd>
|
|
|
|
<dt>Auto refresh every X minutes</dt>
|
|
|
|
<dd>60 (change from the default of 5 to avoid overloading the CI server)</dd>
|
|
</dl>
|
|
<img src="images/NetBeansJenkinsAddDialog.png" alt="Add Hudson Instance dialog">
|
|
</li>
|
|
</ol>
|
|
|
|
<p>From now on, the current status of the Jenkins builds can be reviewed in the 'Services'
|
|
window by expanding the 'JMRI' entry under 'Hudson Builders'.</p>
|
|
|
|
<p><img src="images/NetBeansJenkinsOverview.png" alt="Jenkins Overview">
|
|
</p>
|
|
|
|
<h3 id="architectureTests">Architecture Tests</h3>
|
|
<p><a href="https://github.com/JMRI/JMRI/blob/master/java/test/jmri/ArchitectureTest.java">ArchitectureTest</a>
|
|
contains rules which seek to maintain and improve the overall system architecture of JMRI.
|
|
<br/>For example, the server classes should not depend on code within the system connection hardware types.</p>
|
|
|
|
<pre style="font-family: monospace;"> noClasses()
|
|
.that().resideInAPackage("jmri.jmris")
|
|
.should().dependOnClassesThat().resideInAPackage("jmri.jmrix..");</pre>
|
|
|
|
<p><a href="https://github.com/JMRI/JMRI/blob/master/java/test/jmri/ArchitectureCheck.java"
|
|
>ArchitectureCheck</a> uses FreezingArchRule to collect issues in the archunit_store directory.
|
|
<br/>Should new issues be encountered, they'll be reported as errors.
|
|
<br/>Please fix them rather than committing a larger store.
|
|
</p>
|
|
|
|
<p>The Test classes also undergo an Architecture Test,
|
|
<a href="https://github.com/JMRI/JMRI/blob/master/java/test/jmri/TestArchitectureTest.java"
|
|
>TestArchitectureTest</a>.</p>
|
|
|
|
|
|
<p>Running <pre style="font-family: monospace;"> ant test-single -Dtest.includes=jmri.ArchitectureTest</pre>
|
|
will fail as no tests are executed.
|
|
The build file specifies which JUnit Runners are Launched, i.e.
|
|
<pre style="font-family: monospace;"> --include-engine=junit-jupiter --include-engine=junit-vintage</pre>
|
|
<br/>Within maven no launchers are specified ( hence all launchers are included ),
|
|
though the Architecture tests are excluded from standard CI test executions.</p>
|
|
|
|
<p>These tests can be run with
|
|
<pre style="font-family: monospace;"> mvn -Dtest=jmri.ArchitectureTest,jmri.TestArchitectureTest,jmri.util.FileLineEndingsCheck test</pre>
|
|
|
|
<h3 id="cucumberTests">Cucumber Tests</h3>
|
|
<p><a href="https://en.wikipedia.org/wiki/Cucumber_(software)">Cucumber Tests</a> allow
|
|
expected software behaviours to be specified in a logical language.
|
|
<br/>These are used to test the Web Server output ( in Firefox and Google Chrome ),
|
|
along with Application launches using simulation profiles for various hardware systems.
|
|
<br/>The Features and Step Definitions files are in the <a href="https://github.com/JMRI/JMRI/tree/master/java/acceptancetest"
|
|
>/java/acceptancetest</a> folder.
|
|
<br/>The ant command runs all tests with the class name RunCucumberIT.class ,
|
|
so both the apps and jmri tests are performed.</p>
|
|
<pre style="font-family: monospace;"> ant cucumber</pre>
|
|
|
|
|
|
<!--#include virtual="/help/en/parts/Footer.shtml" -->
|
|
</div>
|
|
<!-- closes #mainContent-->
|
|
</div>
|
|
<!-- closes #mBody-->
|
|
<script src="/js/help.js"></script>
|
|
</body>
|
|
</html>
|