Jekyll2021-08-05T12:44:04+00:00https://blog.networktocode.com/feed.xmlThe NTC MagNetwork to Codeinfo@networktocode.comNautobot Plugin: Single Source of Truth (SSoT)2021-08-05T00:00:00+00:002021-08-05T00:00:00+00:00https://blog.networktocode.com/post/nautobot-ssot-plugin<p>In managing networks, there are <em>systems of record</em> (SoR) that own distinct sets of information about the network, and <em>sources of truth</em> (SoT) from which users and network automation tools pull their required data. All too commonly, these are one and the same, with each SoR serving as an SoT, which has the undesirable consequence that the user or automation needs to consult multiple SoT in order to do their job. Worse, in many cases there is overlap or intersection between various SoR – for example, device hardware inventory might be managed in one SoR, while device configuration might belong to a different SoR – which requires the consumer to cross-correlate information between SoTs in order to gain the full picture. Fortunately, there’s a solution – the introduction of a unified <em>single source of truth</em> (SSoT) that aggregates data from multiple SoR and provides a single unified point of access to this data. Towards this end, we are now introducing the <a href="https://pypi.org/project/nautobot-ssot/">Single Source of Truth (SSoT) plugin</a> for <a href="https://www.networktocode.com/nautobot/">Nautobot</a>.</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/ssot-concept.png" alt="Systems of Record and Single Source of Truth" /></p> <h2 id="overview">Overview</h2> <p>This open-source plugin is designed to enable you to use Nautobot as your network’s SSoT, unifying data from any number of SoR behind Nautobot’s UI and APIs. Additionally, this plugin makes it quick and easy to develop and integrate specialized Nautobot <a href="https://nautobot.readthedocs.io/en/stable/additional-features/jobs/">Jobs</a> that use <a href="https://github.com/networktocode/diffsync">DiffSync</a> to synchronize data from other systems (“Data Sources”) into Nautobot and/or from Nautobot to other systems (“Data Targets”) as desired. In short, whatever system(s) you need to synchronize with Nautobot, you can support via Jobs (either open-source or custom-built) that use this plugin.</p> <blockquote> <p>Please note that there is a 1:1 correlation between Nautobot plugins and Nautobot Apps. The terms can be used interchangeably.</p> </blockquote> <p>Key features of the Nautobot SSoT plugin include:</p> <h3 id="ssot-dashboard">SSoT Dashboard</h3> <p>The Single Source of Truth Dashboard UI lists available Data Sources and Data Targets and summarizes recent synchronization history.</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/dashboard-populated.png" alt="Screenshot of dashboard UI including several custom Jobs" /></p> <h3 id="data-source--data-target-views">Data Source / Data Target Views</h3> <p>Each installed Data Source or Data Target automatically creates a corresponding detailed landing page that displays its configuration, data mappings between the source/target and Nautobot, and sync history.</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/detail-view-servicenow.png" alt="Screenshot of detailed Data Target view showing ServiceNow integration" /></p> <h3 id="ssot-job-result-database-and-views">SSoT Job Result Database and Views</h3> <p>After executing a data synchronization Job, its impact (diffs, number of changes overall), outcome (successes/failures/errors), and detailed logs are automatically saved in Nautobot’s database, and can be reviewed in detail according to your needs.</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/result-view-servicenow.png" alt="Screenshot of detailed synchronization result view showing ServiceNow integration" /></p> <h2 id="installing-the-plugin">Installing the Plugin</h2> <p>The plugin is available as a Python package in PyPI and can be installed atop an existing Nautobot installation using <code class="language-plaintext highlighter-rouge">pip</code>:</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>nautobot-ssot </code></pre></div></div> <blockquote> <p>This plugin is compatible with Nautobot 1.0.3 and higher.</p> </blockquote> <p>Once installed, the plugin needs to be enabled in your <code class="language-plaintext highlighter-rouge">nautobot_config.py</code>:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># nautobot_config.py </span><span class="n">PLUGINS</span> <span class="o">=</span> <span class="p">[</span> <span class="c1"># ..., </span> <span class="s">"nautobot_ssot"</span><span class="p">,</span> <span class="p">]</span> </code></pre></div></div> <h2 id="using-the-plugin">Using the Plugin</h2> <p>As initially installed, the plugin provides example Data Source and Data Target jobs for syncing some basic data (Regions and Sites) between your Nautobot installation and a remote Nautobot instance (such as <a href="https://demo.nautobot.com">demo.nautobot.com</a>). We’ll use those example Jobs for this introduction.</p> <h3 id="accessing-the-ssot-dashboard">Accessing the SSoT Dashboard</h3> <p>You can begin by selecting <strong>Plugins &gt; Single Source of Truth &gt; Dashboard</strong> from the navigation bar (or navigating to <code class="language-plaintext highlighter-rouge">/plugins/ssot/</code>) on your Nautobot instance to view the SSoT dashboard:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/dashboard-initial.png" alt="Screenshot of dashboard as seen on initial installation" /></p> <h3 id="running-a-data-synchronization-job">Running a Data Synchronization Job</h3> <p>From here you can click the <strong>Sync</strong> button next to any of the available Data Sources or Data Targets - for this example, let’s click the <strong>Sync</strong> button under <strong>Example Data Source</strong> to begin syncing data <em>from</em> a remote Nautobot instance <em>to</em> your local Nautobot instance. This will open a web form (which should look familiar if you have any prior experience with Nautobot Jobs) prompting you for any necessary inputs and options for the data sync:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/job-submission.png" alt="Screenshot of Job form" /></p> <blockquote> <p>Note that by default the <strong>Dry Run</strong> option is selected; if this is checked, the Job will report on what <em>would</em> be changed but will not actually make any database updates. For sake of a more interesting example, make sure to <em>uncheck</em> this option before clicking the <strong>Run Job</strong> button.</p> </blockquote> <p>After submitting the Job, you will be redirected to a standard Nautobot <strong>Job Result</strong> view, which will refresh automatically until the Job has run to completion.</p> <h3 id="viewing-synchronization-results">Viewing Synchronization Results</h3> <p>Once the Job has finished, a button labeled <strong>SSoT Sync Details</strong> will appear at the top right of the page:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/job-result-view.png" alt="Screenshot of Job Result view" /></p> <p>Clicking this button will redirect you to the SSoT plugin’s detailed view of this sync, including the diffs between the two systems that DiffSync detected and the detailed logs of exactly what got changed:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/sync-detail-view.png" alt="Screenshot of Sync detail view" /></p> <p>This view describes in detail everything that occurred during the data synchronization attempt. The primary <strong>Data Sync</strong> tab summarizes the overall outcome of the sync attempt, including a view of the diffs (if any) identified by DiffSync and a summary of the actions taken (create, update, delete) and their outcomes (success, failure, error).</p> <p>The <strong>Job Logs</strong> tab shows any general status messages generated by the data synchronization Job as it executed; this is roughly equivalent to the Nautobot <strong>Job Result</strong> view shown earlier.</p> <p>The <strong>Sync Logs</strong> tab shows the logs captured from DiffSync regarding the individual data records being synchronized, details of any contents or changes of these records, and other detailed information. (Sync logs can also be accessed directly via the <strong>Plugins &gt; Single Source of Truth &gt; Logs</strong> menu item if desired.)</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/sync-logs-view.png" alt="Screenshot of Sync Logs view" /></p> <h3 id="reviewing-history-and-job-details">Reviewing History and Job Details</h3> <p>You can return to the dashboard view and see that it now summarizes the recent sync history, including this successful sync:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/dashboard-one-result.png" alt="Screenshot of dashboard with a history of one sync" /></p> <p>From here, you can click on the <strong>Example Data Source</strong> link to inspect the detailed view of this Job, including its execution history and the data mappings that it includes in the sync:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/data-source-detail-view.png" alt="Screenshot of Data Source detail view" /></p> <h3 id="running-again">Running Again</h3> <p>If you run this Job a second time, you will see that, thanks to DiffSync, the Job detects that nothing has changed that needs to be resynced, and completes quickly:</p> <p><img src="../../../static/images/blog_posts/nautobot-ssot/sync-detail-view-no-changes.png" alt="Screenshot of Sync detail view showing no new changes" /></p> <h2 id="finding-creating-and-enabling-more-jobs">Finding, Creating, and Enabling More Jobs</h2> <p>Now that you have some idea of how this plugin works and what it’s capable of, you probably want to investigate how to integrate your own specific SoR systems with Nautobot via this plugin.</p> <blockquote> <p>Keep an eye on this space, as we’ll be open-sourcing several example Data Source and Data Target Jobs in the near future for use with this plugin! You may also browse the <a href="https://www.networktocode.com/nautobot/apps/">Nautobot App Ecosystem</a> under the <strong>Single Source of Truth</strong> category to see what’s available already and what’s coming up on the horizon.</p> </blockquote> <p>We also have <a href="https://github.com/nautobot/nautobot-plugin-ssot/blob/develop/docs/developing_jobs.md">documentation</a> on how to develop your own custom Jobs from scratch to support whatever SoR you want to integrate.</p> <p>In any case, once the desired Job(s) have been developed, you can install them into your Nautobot system like any other Nautobot Job:</p> <ul> <li>by <a href="https://nautobot.readthedocs.io/en/stable/plugins/development/#including-jobs">packaging Jobs into a Nautobot plugin</a>, which can then be installed into Nautobot’s virtual environment</li> <li>by <a href="https://nautobot.readthedocs.io/en/stable/models/extras/gitrepository/#jobs">committing Jobs into a Git repository</a>, which can be configured in Nautobot and refreshed on demand</li> <li>by <a href="https://nautobot.readthedocs.io/en/stable/additional-features/jobs/#writing-jobs">manual installation of individual Job source files</a> to Nautobot’s <code class="language-plaintext highlighter-rouge">JOBS_ROOT</code> directory</li> </ul> <h2 id="next-steps">Next Steps</h2> <p>Hopefully you can now see the value that this plugin provides as a framework for integrating any number of other SoT and SoR with Nautobot as the unified SSoT. Keeping data synchronized between these systems, especially via automation such as this, makes our lives easier and reduces the likelihood of trouble resulting from out-of-sync information. If you do develop your own Data Source and/or Data Target Job with this plugin, we’d love to hear all about it!</p> <p>-Glenn</p>Glenn MatthewsIn managing networks, there are systems of record (SoR) that own distinct sets of information about the network, and sources of truth (SoT) from which users and network automation tools pull their required data. All too commonly, these are one and the same, with each SoR serving as an SoT, which has the undesirable consequence that the user or automation needs to consult multiple SoT in order to do their job. Worse, in many cases there is overlap or intersection between various SoR – for example, device hardware inventory might be managed in one SoR, while device configuration might belong to a different SoR – which requires the consumer to cross-correlate information between SoTs in order to gain the full picture. Fortunately, there’s a solution – the introduction of a unified single source of truth (SSoT) that aggregates data from multiple SoR and provides a single unified point of access to this data. Towards this end, we are now introducing the Single Source of Truth (SSoT) plugin for Nautobot.Manipulating Data with Jinja Map and Selectattr2021-08-03T00:00:00+00:002021-08-03T00:00:00+00:00https://blog.networktocode.com/post/jinja-map-review<p>Manipulating data with the Jinja filters, Map, Selectattr, and Select provides quick and efficient ways to create new data structures. This can simplify creating lookup dictionaries and avoids complex loops or Jinja templating options.</p> <p>In many instances in ansible, I’ll use an ordered list of dictionaries as my data model; this specifically keeps the preferred ordering for the device and type of configuration as well as makes it easy to do operations on all the members of that list. For some operations that rely on specific dictionaries, it can be convoluted to dig that information out of the list, or work on a subset of the list, without going through loops and conditionals, without these tools.</p> <p>Like all good overviews, it’s best explained with an example. We’ll start with this example data structure for some selected interfaces, a commonly used pattern.</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="na">interfaces</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">mgmt"</span> <span class="na">enable</span><span class="pi">:</span> <span class="no">true</span> <span class="na">dhcp</span><span class="pi">:</span> <span class="no">true</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">no</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">management"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">1/1/1"</span> <span class="na">enable</span><span class="pi">:</span> <span class="no">true</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">10.1.1.2/24</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">uplink</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">core-1"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">1/1/2"</span> <span class="na">enable</span><span class="pi">:</span> <span class="no">true</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">10.1.2.2/24</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">uplink</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">core-2"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">1/1/3"</span> <span class="na">enable</span><span class="pi">:</span> <span class="no">true</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">10.1.3.1/24</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">peerlink</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">agg-2"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vlan42"</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">10.42.0.2/24</span> <span class="na">ip_helper</span><span class="pi">:</span> <span class="no">false</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">inband</span><span class="nv"> </span><span class="s">management"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vlan44"</span> <span class="na">ip_address</span><span class="pi">:</span> <span class="s">10.44.0.3/24</span> <span class="na">ip_helper</span><span class="pi">:</span> <span class="no">true</span> <span class="na">fhrp</span><span class="pi">:</span> <span class="no">true</span> <span class="na">description</span><span class="pi">:</span> <span class="s2">"</span><span class="s">wifi</span><span class="nv"> </span><span class="s">aps"</span> </code></pre></div></div> <h2 id="selecattr">Selecattr</h2> <p>Selectattr takes in a list of dictionaries and applies a test to each dictionary.</p> <p>One of the first use cases might be to get all of the interfaces that have an IP address defined:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{{</span> <span class="nv">interfaces</span> <span class="o">| </span><span class="nf">selectattr</span><span class="p">(</span><span class="s1">'ip_address'</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>With the selectattr filter, a Jinja test can be used as the second argument; this gives us a lot of flexibility when selecting our data. Additional arguments can be specified as inputs to the test specified in the second arg.</p> <p>One thing to be careful of with the selectattr filter with this example: The attribute you use must be defined, but not all of our interfaces have ip_address defined. When the default test is applied to the value of the attribute (bool), the var doesn’t exist, which will result in an error. We’ll hit that error wherever ip_address is not defined in our interfaces data structure.</p> <p>To get around this in our data, we specify the ‘defined’ parameter to test whether the attribute is there or not:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{{</span> <span class="nv">interfaces</span> <span class="o">| </span><span class="nf">selectattr</span><span class="p">(</span><span class="s1">'ip_address'</span><span class="p">,</span> <span class="s1">'defined'</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>This returns a list of dictionaries where the ip_address attribute is present. Easy enough.</p> <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"enable"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nl">"ip_address"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.1.1.2/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1/1/1"</span><span class="w"> </span><span class="p">},</span><span class="w"> </span><span class="p">{</span><span class="err">...</span><span class="p">},</span><span class="w"> </span><span class="p">]</span><span class="w"> </span></code></pre></div></div> <p>To be even more granular, we can chain the output to another selectattr filter and further test the ip_address or another attribute. For example, we can pass the list of interfaces with IP addresses and test whether the IP address on the interface is in another list of IP addresses:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{{</span> <span class="nv">interfaces</span> <span class="o">| </span><span class="nf">selectattr</span><span class="p">(</span><span class="s1">'ip_address'</span><span class="p">,</span> <span class="s1">'defined'</span><span class="p">)</span> <span class="o">| </span><span class="nf">selectattr</span><span class="p">(</span><span class="s1">'ip_address'</span><span class="p">,</span> <span class="s1">'in'</span><span class="p">,</span> <span class="s1">'[10.1.3.1/24]'</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>Which gives us this output:</p> <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="p">[</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"peerlink to agg-2"</span><span class="p">,</span><span class="w"> </span><span class="nl">"enable"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nl">"ip_address"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.1.3.1/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1/1/3"</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">]</span><span class="w"> </span></code></pre></div></div> <h2 id="map">Map</h2> <p>Map allows us to turn a list of dictionaries into a simpler list or flatten the list of dictionaries. We specify the values of the list by giving Map the attribute we want to take from the dictionaries.</p> <p>Now, let’s get a list of the IP addresses. We could easily use this in a route-map, prefix-list, or ACL.</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">{{</span> <span class="nv">interfaces | map(attribute='ip_address'</span><span class="pi">,</span> <span class="nv">default='n/a')</span> <span class="pi">}}</span> </code></pre></div></div> <p>Gives the output:</p> <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="p">[</span><span class="w"> </span><span class="s2">"n/a"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.1.1.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.1.2.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.1.3.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.42.0.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.44.0.1/24"</span><span class="w"> </span><span class="p">]</span><span class="w"> </span></code></pre></div></div> <p>Similar to the selectattr, Map expects the attribute to be defined. In this case we have to specify a default value in case the dictionary doesn’t contain the specified attribute, ip_address in this case.</p> <p>It might be useful to ONLY have IP addresses in our list of IPs. Here we can combine the selectattr by filtering down the dictionaries where the ip_address attribute is defined and then using Map to filter the dictionaries down to the values of the ip_address attribute.</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">{{</span> <span class="nv">interfaces | selectattr('ip_address'</span><span class="pi">,</span> <span class="s1">'</span><span class="s">defined'</span><span class="nv">) | map(attribute='ip_address')</span> <span class="pi">}}</span> </code></pre></div></div> <p>Output:</p> <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="w"> </span><span class="s2">"10.1.1.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.1.2.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.1.3.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.42.0.1/24"</span><span class="p">,</span><span class="w"> </span><span class="s2">"10.44.0.1/24"</span><span class="w"> </span><span class="p">]</span><span class="w"> </span></code></pre></div></div> <p>The Map filter also gives us the ability to input another filter to use on the data in a list; this allows us to use a filter that doesn’t support working on lists to work on each element of the list without using a loop. Most of the Jinja built-in filters will work without the Map filter, however.</p> <h2 id="other-useful-combinations">Other Useful Combinations</h2> <p>If we need to look up IP addresses by interface name, we can instead use the items2dict filter. This gives us the ability to map attributes of the dictionary to new keys and values. This is useful when specific addresses are used in specific ways, such as network-specific ACLs, prefix-lists, bgp neighbors, and others that need to reference interfaces in specific ways or different orders.</p> <p>We first use selectattr to filter down to which dictionaries have an ip_address, then use items2dict to output a new dictionary where the interface name is the key and the ip_address is the value.</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{{</span> <span class="nv">interfaces</span> <span class="o">| </span><span class="nf">selectattr</span><span class="p">(</span><span class="s1">'ip_address'</span><span class="p">,</span> <span class="s1">'defined'</span><span class="p">)</span> <span class="o">| </span><span class="nf">items2dict</span><span class="p">(</span><span class="nv">key_name</span><span class="o">=</span><span class="s1">'name'</span><span class="p">,</span> <span class="nv">value_name</span><span class="o">=</span><span class="s1">'ip_address'</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>This gives us a nice tidy lookup dictionary without any loops and a relatively readable call:</p> <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w"> </span><span class="nl">"1/1/1"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.1.1.1/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"1/1/2"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.1.2.1/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"1/1/3"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.1.3.1/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"vlan42"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.42.0.1/24"</span><span class="p">,</span><span class="w"> </span><span class="nl">"vlan44"</span><span class="p">:</span><span class="w"> </span><span class="s2">"10.44.0.1/24"</span><span class="w"> </span><span class="p">}</span><span class="w"> </span></code></pre></div></div> <h2 id="references">References:</h2> <ul> <li><a href="https://jinja.palletsprojects.com/en/2.11.x/templates/#list-of-builtin-filters">Jinja Builtin Filters</a></li> </ul> <p>All in all, these are very handy tools to use to get new views of the same data without having to resort to too much looping or other odd manipulations. I hope you find them useful!</p> <p>-Stephen Corry</p>Stephen CorryManipulating data with the Jinja filters, Map, Selectattr, and Select provides quick and efficient ways to create new data structures. This can simplify creating lookup dictionaries and avoids complex loops or Jinja templating options.Nautobot, Beyond Network Source of Truth2021-07-29T00:00:00+00:002021-07-29T00:00:00+00:00https://blog.networktocode.com/post/nautobot-beyond-network-source-of-truth<p>As the title suggests, this post will use Nautobot for something other than a Network Source of Truth. This post will use Nautobot to solve one of life’s most difficult questions, perhaps one of the most frequently asked questions. This question is known to ruin the evening, but with Nautobot’s help, this no longer has to be the case: The question Nautobot will answer for us is, “What’s for dinner?”</p> <p>This post will walk through using one new feature introduced in Nautobot version 1.1 and one existing feature, in order to have each Site in Nautobot provide a recipe recommendation. The two features are independent, but often work hand in hand to provide dynamic data to a specific entry in Nautobot. The primary feature that will provide the Site with a recipe is <a href="https://nautobot.readthedocs.io/en/v1.1.0/models/extras/customlink/">Custom Links</a>. The Custom Link will make use of a <a href="https://nautobot.readthedocs.io/en/v1.1.0/plugins/development/#including-jinja2-filters">Custom Jinja2 Filter</a> in order to generate a recommendation from a third-party API.</p> <h2 id="high-level-architecture">High-Level Architecture</h2> <p>The API used by the Custom Jinja2 Filter will be <a href="https://spoonacular.com/food-api/docs">Spoonacular</a>. <em>Spoonacular</em> requires an account and token to use the API, but it allows for several API requests per day for free. The API also has several filtering capabilities, the two that will be used here are: <strong>cuisine</strong> and <strong>ingredients</strong>. A <a href="https://nautobot.readthedocs.io/en/v1.1.0/plugins/development/">Nautobot Plugin</a> needs to be created in order to give Nautobot access to the Custom Jinja2 Filter, which will also provide access to the <em>Spoonacular</em> API token using <a href="https://nautobot.readthedocs.io/en/v1.1.0/plugins/#configure-the-plugin">PLUGINS_CONFIG</a> in nautbot_config.py.</p> <p>The Custom Link will be added to the <code class="language-plaintext highlighter-rouge">dcim | site</code> Content Type, and will use the Site object to pass the Jinja2 Filter a cuisine and an ingredient. The cuisine will be obtained from the Site’s parent Region’s <code class="language-plaintext highlighter-rouge">slug</code> field, and the Site’s <code class="language-plaintext highlighter-rouge">slug</code> field will be used to provide an ingredient. This means that:</p> <ul> <li>Each Region that is associated with a Site must have a valid <em>Spoonacular</em> cuisine for its slug value</li> <li>Each Site must be associated to a Region</li> <li>Each Site must have an ingredient for its slug value</li> </ul> <blockquote> <p>A list of acceptable cuisines for the API can be found at <a href="https://spoonacular.com/food-api/docs#Cuisines">Spoonacular Cuisines</a>.</p> </blockquote> <h2 id="project-structure">Project Structure</h2> <p>This Plugin will only provide a Jinja2 Filter, so the file structure is very simple:</p> <ul> <li><code class="language-plaintext highlighter-rouge">nautobot_plugin_recipe_filter</code> directory for the Plugin App</li> <li><code class="language-plaintext highlighter-rouge">nautobot_plugin_recipe_filter/__init__.py</code> file with the PluginConfig definition</li> <li><code class="language-plaintext highlighter-rouge">nautobot_plugin_recipe_filter/recipe_filters.py</code> for providing the filter to get recipe recommendations</li> <li><code class="language-plaintext highlighter-rouge">pyproject.toml</code>, <code class="language-plaintext highlighter-rouge">README.md</code>, and <code class="language-plaintext highlighter-rouge">LICENSE</code> for installing the Plugin in Nautobot.</li> </ul> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>tree nautobot-plugin-recipe-filter <span class="nb">.</span> ├── nautobot_plugin_recipe_filter │ ├── __init__.py │ └── recipe_filters.py ├── LICENSE ├── pyproject.toml └── README.md </code></pre></div></div> <h3 id="nautobot_plugin_recipe_filterinitpy">nautobot_plugin_recipe_filter/<strong>init</strong>.py</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">"""Plugin declaration for nautobot_plugin_recipe_filter."""</span> <span class="n">__version__</span> <span class="o">=</span> <span class="s">"0.1.0"</span> <span class="kn">from</span> <span class="nn">nautobot.extras.plugins</span> <span class="kn">import</span> <span class="n">PluginConfig</span> <span class="k">class</span> <span class="nc">NautobotPluginRecipeFilterConfig</span><span class="p">(</span><span class="n">PluginConfig</span><span class="p">):</span> <span class="s">"""Plugin configuration for the nautobot_plugin_recipe_filter plugin."""</span> <span class="n">name</span> <span class="o">=</span> <span class="s">"nautobot_plugin_recipe_filter"</span> <span class="n">verbose_name</span> <span class="o">=</span> <span class="s">"Nautobot Plugin Recipe Filter"</span> <span class="n">version</span> <span class="o">=</span> <span class="n">__version__</span> <span class="n">author</span> <span class="o">=</span> <span class="s">"Network to Code, LLC"</span> <span class="n">description</span> <span class="o">=</span> <span class="s">"Nautobot Plugin Jinja2 Filter for Recipe Recommendations"</span> <span class="n">required_settings</span> <span class="o">=</span> <span class="p">[</span><span class="s">"spoonacular_token"</span><span class="p">]</span> <span class="n">min_version</span> <span class="o">=</span> <span class="s">"1.1.0"</span> <span class="n">max_version</span> <span class="o">=</span> <span class="s">"1.9999"</span> <span class="n">default_settings</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">caching_config</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">jinja_filters</span> <span class="o">=</span> <span class="s">"recipe_filters.py"</span> <span class="n">config</span> <span class="o">=</span> <span class="n">NautobotPluginRecipeFilterConfig</span> </code></pre></div></div> <h3 id="pyprojecttoml">pyproject.toml</h3> <div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">[tool.poetry]</span> <span class="py">name</span> <span class="p">=</span> <span class="s">"nautobot-plugin-recipe-filter"</span> <span class="py">version</span> <span class="p">=</span> <span class="s">"0.1.0"</span> <span class="py">description</span> <span class="p">=</span> <span class="s">"Nautobot Plugin Jinja2 Filter for Recipe Recommendations"</span> <span class="py">authors</span> <span class="p">=</span> <span class="p">[</span><span class="s">"Network to Code, LLC &lt;info@networktocode.com&gt;"</span><span class="p">]</span> <span class="py">license</span> <span class="p">=</span> <span class="s">"Apache-2.0"</span> <span class="py">readme</span> <span class="p">=</span> <span class="s">"README.md"</span> <span class="py">homepage</span> <span class="p">=</span> <span class="s">"https://github.com/networktocode/nautobot-plugin-recipe-filter"</span> <span class="py">repository</span> <span class="p">=</span> <span class="s">"https://github.com/networktocode/nautobot-plugin-recipe-filter"</span> <span class="py">keywords</span> <span class="p">=</span> <span class="p">[</span><span class="s">"nautobot"</span><span class="p">,</span> <span class="s">"nautobot-plugin"</span><span class="p">]</span> <span class="py">include</span> <span class="p">=</span> <span class="p">[</span><span class="s">"LICENSE"</span><span class="p">,</span> <span class="s">"README.md"</span><span class="p">]</span> <span class="py">packages</span> <span class="p">=</span> <span class="p">[</span> <span class="err">{</span> <span class="err">include</span> <span class="err">=</span> <span class="s">"nautobot_plugin_recipe_filter"</span> <span class="err">}</span><span class="p">,</span> <span class="p">]</span> <span class="nn">[tool.poetry.dependencies]</span> <span class="py">python</span> <span class="p">=</span> <span class="s">"^3.6"</span> <span class="py">nautobot</span> <span class="p">=</span> <span class="s">"^v1.1.0"</span> <span class="py">requests</span> <span class="p">=</span> <span class="s">"*"</span> </code></pre></div></div> <h2 id="creating-the-custom-jinja2-filter">Creating the Custom Jinja2 Filter</h2> <p>Creating a Custom Jinja2 Filter that can be consumed within Nautobot follows the standard <a href="https://jinja.palletsprojects.com/en/latest/api/#custom-filters">Jinja2 process</a>, and is auto-registered to the Nautobot Environment when a Plugin is installed and enabled on the Nautobot instance. Each Plugin can define the name of the file where Jinja2 Filters should be loaded from using the PluginConfig; the default location is <code class="language-plaintext highlighter-rouge">jinja_filters.py</code>. The above PluginConfig specified <code class="language-plaintext highlighter-rouge">recipe_filters.py</code>, so the filters made available by this plugin will go in that file.</p> <p>In order to register the Jinja2 Filter with Nautobot, the Filter function is decorated with <code class="language-plaintext highlighter-rouge">@library.filter</code>, where <code class="language-plaintext highlighter-rouge">library</code> is imported from the <code class="language-plaintext highlighter-rouge">django_jinja</code> library. The <a href="https://niwinz.github.io/django-jinja/latest/">django_jinja</a> library is included with Nautobot in order to support the Custom Jinja2 Filters feature. This Filter function will use the <a href="https://docs.python-requests.org/en/master/index.html">requests</a> library to make API calls to the <em>Spoonacular</em> API. The Jinja2 Filter will return the URL to the recipe recommended by <em>Spoonacular</em>. In order to interact with the <em>Spoonacular</em> API, the token will be retrieved using the <code class="language-plaintext highlighter-rouge">spoonacular_token</code> setting in the <code class="language-plaintext highlighter-rouge">PLUGINS_CONFIG</code>; this setting is marked as required in the above PluginConfig.</p> <h3 id="recipe_filterspy">recipe_filters.py</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">"""Custom filters for nautobot_plugin_recipe_filter."""</span> <span class="kn">import</span> <span class="nn">requests</span> <span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span> <span class="kn">from</span> <span class="nn">django_jinja</span> <span class="kn">import</span> <span class="n">library</span> <span class="n">SPOONACULAR_TOKEN</span> <span class="o">=</span> <span class="n">settings</span><span class="p">.</span><span class="n">PLUGINS_CONFIG</span><span class="p">[</span><span class="s">"nautobot_plugin_recipe_filter"</span><span class="p">][</span><span class="s">"spoonacular_token"</span><span class="p">]</span> <span class="n">SPOONACULAR_URL</span> <span class="o">=</span> <span class="s">"https://api.spoonacular.com/recipes"</span> <span class="n">SPOONACULAR_RECOMMENDATION_URL</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">SPOONACULAR_URL</span><span class="si">}</span><span class="s">/complexSearch"</span> <span class="c1"># URL to get recommendations </span><span class="n">SPOONACULAR_RECIPE_URL</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">SPOONACULAR_URL</span><span class="si">}</span><span class="s">//information"</span> <span class="c1"># URL to get details of recommendation </span> <span class="o">@</span><span class="n">library</span><span class="p">.</span><span class="nb">filter</span> <span class="k">def</span> <span class="nf">get_recipe_url</span><span class="p">(</span><span class="n">cuisine</span><span class="p">:</span><span class="nb">str</span><span class="p">,</span> <span class="n">ingredient</span><span class="p">:</span><span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span> <span class="s">""" Get Recipe recommendation based on cuisine and single ingredient. Args: cuisine (str): The cuisine derived from Nautobot's Region.slug value. ingredient (str): The ingredient to include in the recipe based on Nautobot's Site.slug value. Returns: str: The URL of the recommended recipe. """</span> <span class="n">recommendation_params</span> <span class="o">=</span> <span class="p">{</span> <span class="s">"apiKey"</span><span class="p">:</span> <span class="n">SPOONACULAR_TOKEN</span><span class="p">,</span> <span class="s">"number"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="c1"># Limit number of responses to a single recipe </span> <span class="s">"cuisine"</span><span class="p">:</span> <span class="n">cuisine</span><span class="p">,</span> <span class="s">"includeIngredients"</span><span class="p">:</span> <span class="n">ingredient</span><span class="p">,</span> <span class="p">}</span> <span class="c1"># Get recipe recommendation </span> <span class="n">recommendation_response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">=</span><span class="n">SPOONACULAR_RECOMMENDATION_URL</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">recommendation_params</span><span class="p">)</span> <span class="n">recommendation_response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span> <span class="n">recipe_id</span> <span class="o">=</span> <span class="n">recommendation_response</span><span class="p">.</span><span class="n">json</span><span class="p">()[</span><span class="s">"results"</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">"id"</span><span class="p">]</span> <span class="n">recipe_params</span> <span class="o">=</span> <span class="p">{</span><span class="s">"apiKey"</span><span class="p">:</span> <span class="n">SPOONACULAR_TOKEN</span><span class="p">}</span> <span class="c1"># Get recipe detailed information </span> <span class="n">recipe_response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">=</span><span class="n">SPOONACULAR_RECIPE_URL</span><span class="p">.</span><span class="nb">format</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="n">recipe_id</span><span class="p">),</span> <span class="n">params</span><span class="o">=</span><span class="n">recipe_params</span><span class="p">)</span> <span class="n">recipe_response</span><span class="p">.</span><span class="n">raise_for_status</span><span class="p">()</span> <span class="n">recipe_json</span> <span class="o">=</span> <span class="n">recipe_response</span><span class="p">.</span><span class="n">json</span><span class="p">()</span> <span class="c1"># Get URL to recipe on Spoonacular's website </span> <span class="n">recipe_url</span> <span class="o">=</span> <span class="n">recipe_json</span><span class="p">[</span><span class="s">"spoonacularSourceUrl"</span><span class="p">]</span> <span class="k">return</span> <span class="n">recipe_url</span> </code></pre></div></div> <p>The Plugin is ready to be installed into the Nautobot environment. See the <a href="https://nautobot.readthedocs.io/en/latest/plugins/#installing-plugins">Plugin Installation Guide</a> for details. The <em>Database Migration</em> and <em>Collect Static Files</em> steps can be skipped, since the Plugin does not use a database table and does not provide static files.</p> <h3 id="example-config-settings">Example Config Settings</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">PLUGINS</span> <span class="o">=</span> <span class="p">[</span><span class="s">"nautobot_plugin_recipe_filter"</span><span class="p">]</span> <span class="n">PLUGINS_CONFIG</span> <span class="o">=</span> <span class="p">{</span> <span class="s">"nautobot_plugin_recipe_filter"</span><span class="p">:</span> <span class="p">{</span> <span class="s">"spoonacular_token"</span><span class="p">:</span> <span class="s">"abc123"</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The Jinja2 Filters provided by Plugins are available in Nautobot wherever Jinja2 text fields are used. This post will showcase using the above Jinja2 Filter to create a Custom Link; however, the Filter could also be used by the new <a href="https://nautobot.readthedocs.io/en/v1.1.0/additional-features/computed-fields/">Computed Fields</a> feature added in v1.1.</p> <blockquote> <p>The reason for using Custom Links in this instance is that the goal is to provide a link to another site with the recipe and cooking instructions. Computed Fields are designed to be text fields, and have HTML escaped; this means that an HTML hyperlink would be rendered as plain-text and not be clickable.</p> </blockquote> <h2 id="creating-a-custom-link">Creating a Custom Link</h2> <p>Nautobot’s Custom Link feature allows hyperlinks and groups of hyperlinks to be added to the upper-right side of pages displaying a single entry (DetailViews). Creating a Custom Link has several configuration options:</p> <ul> <li><em>Content Type</em>: Sets the Model that will have its entries display the hyperlink</li> <li><em>Name</em>: The name used to identify the Custom Link within Nautobot</li> <li><em>Text</em>: A Jinja2 expression that will be rendered and used as the text displayed for the hyperlink</li> <li><em>URL</em>: The URL that will be used for the hyperlink</li> <li><em>Weight</em>: Determines the ordering of the hyperlinks displayed</li> <li><em>Group Name</em>: Allows multiple hyperlinks to be displayed under a single drop-down button</li> <li><em>Button Class</em>: Determines the color of the button used for the hyperlink</li> <li><em>New Window</em>: Determines whether clicking the hyperlink should open the URL in a new tab or the current tab</li> </ul> <h3 id="guide-to-create-a-custom-link">Guide to Create a Custom Link</h3> <p>There are two steps to creating most objects in Nautobot:</p> <ul> <li>Browsing to the form to create the object from the main Navigation Bar</li> <li>Filling out and submitting the form</li> </ul> <h4 id="browsing-to-add-custom-link-form">Browsing to Add Custom Link Form</h4> <p>Creating a Custom Link is done in the Web UI by browsing to <strong>Extensibility</strong> &gt; <strong>Custom Links</strong>, and clicking the <strong>+</strong> icon to bring up the form to create a new Custom Link.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/browse-to-add-custom-link.png" alt="browse_create_custom_link" /></p> <h4 id="filling-out-the-add-custom-link-form">Filling Out the Add Custom Link Form</h4> <p>The Custom Link used to provide a hyperlink to <em>Spoonacular’s</em> recipe recommendation specifies a <em>Content Type</em> of <code class="language-plaintext highlighter-rouge">dcim | site</code>, and a <em>Name</em> and <em>Text</em> of <code class="language-plaintext highlighter-rouge">Recipe Recommendation</code>. The <em>URL</em> uses the Jinja2 Filter defined above to obtain the URL to the recipe, and clicks will open the hyperlink in a new tab.</p> <p>The Jinja2 expression for the URL makes use of the variable <code class="language-plaintext highlighter-rouge">obj</code>, which is the Django ORM object for the database record. Since the Content Type for this Custom Link uses the Site model, it will be a record from the Site database table. Nautobot Site entries have foreign key fields to the Region, so both the Region’s slug (cuisine) and Site’s slug (ingredient) can be passed to the Custom Jinja2 Filter.</p> <p>Once all required fields are filled out on the form, clicking <strong>Create</strong> will create the Custom Link.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/create-custom-link.png" alt="create_custom_link" /></p> <h2 id="using-the-custom-link">Using the Custom Link</h2> <p>The Custom Link created above is displayed on every Site page. If the Jinja2 expression fails to render an HTML link, then the Custom Link button will not be clickable. In order to showcase the Custom Link, a guide is provided below to create a Region and Site, which results in a valid URL being created for the Site.</p> <h3 id="guide-to-create-a-region">Guide to Create a Region</h3> <p>In order for a valid link to be shown, each Site must belong to a Region that has a <em>slug</em> value with a valid cuisine.</p> <h4 id="browsing-to-add-region-form">Browsing to Add Region Form</h4> <p>Creating a Region is done by browsing to <strong>Organization</strong> &gt; <strong>Regions</strong>, and clicking the <strong>+</strong> icon to bring up the form to create a new Region.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/browse-to-add-region.png" alt="browse_to_add_region" /></p> <h4 id="filling-out-the-add-region-form">Filling Out the Add Region Form</h4> <p>The Region used in this example is <code class="language-plaintext highlighter-rouge">Spain</code>, which correlates to the <code class="language-plaintext highlighter-rouge">spanish</code> cuisine used by the <em>Spoonacular</em> API.</p> <p>Once all required fields have been filled out on the form, clicking <strong>Create</strong> will create the Region.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/create-region.png" alt="create_region" /></p> <h3 id="guide-to-create-a-site">Guide to Create a Site</h3> <p>In addition to each Site being linked to a Region, the Site must also use an ingredient for the value of its <em>slug</em> field.</p> <h4 id="browsing-to-add-site-form">Browsing to Add Site Form</h4> <p>Creating a Site is done by browsing to <strong>Organization</strong> &gt; <strong>Sites</strong>, and clicking the <strong>+</strong> icon to bring up the form to create a new Site.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/browse-to-add-site.png" alt="browse_to_add_site" /></p> <h4 id="filling-out-the-add-site-form">Filling Out the Add Site Form</h4> <p>The Site in this example is <code class="language-plaintext highlighter-rouge">Madrid</code>, and is assigned to the <code class="language-plaintext highlighter-rouge">Spain</code> Region created above. The slug (ingredient) value uses <code class="language-plaintext highlighter-rouge">rice</code>. In Nautobot, Sites require the <em>status</em> to be set; this Site uses the built-in <code class="language-plaintext highlighter-rouge">Active</code> Status.</p> <p>The Site form is longer than the previous forms, but scrolling to the bottom of the form displays the familiar <strong>Create</strong> button to create the Site.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/create-site.png" alt="create_site" /></p> <h3 id="viewing-the-recipe-recommendation">Viewing the Recipe Recommendation</h3> <p>Clicking on the Create button above redirects to the newly created Site Page with the Custom Link added to it.</p> <h4 id="viewing-the-custom-link-on-the-site-page">Viewing the Custom Link on the Site Page</h4> <p>The Custom Link for the recipe recommendation is displayed in the top-right corner of the page.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/site-created.png" alt="site_created" /></p> <h4 id="clicking-the-link-to-view-the-recipe">Clicking the Link to View the Recipe</h4> <p>Since the check box to open the hyperlink in a new tab was selected on the Create Custom Link form, clicking the Custom Link on the Site page opens the <em>Spoonacular</em> recipe page in a new tab. The recommendation returned in the example is Paella.</p> <p><img src="../../../static/images/blog_posts/nautobot-beyond-network-source-of-truth/spoonacular-recommendation.png" alt="spoonacular_recommendation" /></p> <h2 id="conclusion">Conclusion</h2> <p>Nautobot is more than a Source of Truth application. The newly added Custom Jinja2 Filters provide an easy way to perform more complex operations for fields that are rendered through Jinja2. An example of a field that supports Jinja2 expressions is the URL field in Custom Links. Custom Links can be added to Pages dynamically, and provide a convenient way of giving Users access to related pages.</p> <p>The recipe recommendation example used here is perhaps not practical, but something similar could be built for linking to other Source of Truth systems. Maybe Nautobot is not the System of Record for certain data points, but is instead used to aggregate multiple data points into a single place for read-only operations. The Jinja2 Filter could look up the URL to the data point in the Application used as the System of Record. It might be handy to have a link in Nautobot that is able to take users to that page in order to manage the data there.</p>Jacob McGillAs the title suggests, this post will use Nautobot for something other than a Network Source of Truth. This post will use Nautobot to solve one of life’s most difficult questions, perhaps one of the most frequently asked questions. This question is known to ruin the evening, but with Nautobot’s help, this no longer has to be the case: The question Nautobot will answer for us is, “What’s for dinner?”Ansible Execution Strategies2021-07-27T00:00:00+00:002021-07-27T00:00:00+00:00https://blog.networktocode.com/post/ansible-execution-strategies<p>Adjusting Ansible playbook execution strategies is a use case that has come up several times. The default strategy Ansible uses, is called <code class="language-plaintext highlighter-rouge">linear</code>, and it works great a majority of the time, but what if you’re automating a task that need to be done in a specific order or way. Strategies can be used to define how a playbook should be executed. This includes the ability to run in serial, run in batches, and even, in extreme cases, allows the ability to write a custom strategy plugin.</p> <p>In this blog post I will provide multiple examples demonstrating the different strategies and settings. The playbooks that will be used will be dramatically simplified to allow the focus to be on the strategies in use rather than the content of the playbooks.</p> <h2 id="strategy-basics">Strategy Basics</h2> <ul> <li><strong>Linear(Default)</strong>: Playbook runs in a linear execution, each task is run in order and the next task is not started until all hosts are finished with the current task.</li> <li><strong>Debug</strong>: The execution is done in a debug mode that allows the user to interactively run the playbook for troubleshooting purposes.</li> <li><strong>Free</strong>: Playbook is executed linearly, but each host runs the tasks at its own pace and has no requirement for all hosts to finish a task before continuing to the next task.</li> <li><strong>Host Pinned</strong>: Operating like the <code class="language-plaintext highlighter-rouge">free</code> strategy, <code class="language-plaintext highlighter-rouge">host_pinned</code> will run tasks as fast as it can on the number of hosts specified in the <code class="language-plaintext highlighter-rouge">serial</code> keyword, immediately starting a host as soon as one ends.</li> </ul> <blockquote> <p>Note: The strategy <code class="language-plaintext highlighter-rouge">host_pinned</code> may operate as one would expect <code class="language-plaintext highlighter-rouge">free</code> to run, so give it try to make sure you understand the subtle difference.</p> </blockquote> <p>To see available strategies, use <code class="language-plaintext highlighter-rouge">ansible-doc -t strategy -l</code> command.</p> <p>Strategies can be set per <code class="language-plaintext highlighter-rouge">PLAY</code> or under the <code class="language-plaintext highlighter-rouge">[defaults]</code> group in the <code class="language-plaintext highlighter-rouge">ansible.cfg</code> file.</p> <blockquote> <p>Note: Remember, a playbook often has a single play in it, but can have multiple plays.</p> </blockquote> <h2 id="other-execution-tune-ables">Other Execution Tune-ables</h2> <p>Ansible also provides some keywords that can be used to further tune how a playbook runs.</p> <ul> <li><strong>Forks</strong>: The number of simultaneous connections a task can use to perform work. <ul> <li>Forks can be tuned based on the processing power available. The default is set to 5. If processing power on the Ansible control machine is not a concern this can be modified. This allows for playbook execution to be faster. <blockquote> <p>Note: Setting the <code class="language-plaintext highlighter-rouge">fork</code> value to <code class="language-plaintext highlighter-rouge">1</code> effectively makes each task run on a single host before the next host starts the same task.</p> </blockquote> </li> </ul> </li> <li><strong>Run Once</strong>: This option is exactly as it sounds. It’s a task that should be run only once. <ul> <li>Great for operations that need to be done only once for the group. <strong>Not once per host</strong>.</li> <li>Good example would be creating a directory on the <code class="language-plaintext highlighter-rouge">control machine</code> for storing configuration files. You do NOT need every host to create the directory, this needs to be completed only one time.</li> </ul> </li> <li><strong>Serial</strong>: The number of hosts that Ansible should be running side by side. <ul> <li>Great for playbooks that include redundant devices that should not be worked on at the same time.</li> <li>Great for when there is <a href="./post/Serialize-Ansible-Tasks/">possible race conditions</a>.</li> </ul> </li> <li><strong>Throttle</strong>: Can be used in the task definition to limit the number of hosts the task is executed on. Similar to fork, but at a task level.</li> <li><strong>Order</strong>: Order specifies how the hosts for a given play are going to be executed. By default the order comes from the inventory file, but other options exist that can provide flexibility if inventory hostnames provide insights into a given environment. <ul> <li><code class="language-plaintext highlighter-rouge">reverse_inventory</code>: Is the opposite of the default mode explained previously. It reverses the order from the inventory file.</li> <li><code class="language-plaintext highlighter-rouge">shuffle</code>: Random order.</li> <li><code class="language-plaintext highlighter-rouge">sorted</code>: Alphabetical.</li> <li><code class="language-plaintext highlighter-rouge">reverse_sorted</code>: Reverse alphabetical.</li> </ul> </li> </ul> <h2 id="strategies-and-keyword-interrelationship">Strategies and Keyword Interrelationship</h2> <p>This is where I found myself struggling when learning these concepts. Ansible provides all the flexibility listed above which provides great power, but there is a relationship between the options that can cause an unexpected behavior.</p> <p>The key relationship to be aware of is <code class="language-plaintext highlighter-rouge">throttle</code>. When using it in parallel with <code class="language-plaintext highlighter-rouge">forks</code> or <code class="language-plaintext highlighter-rouge">serial</code>, <code class="language-plaintext highlighter-rouge">throttle</code> must be less than <code class="language-plaintext highlighter-rouge">forks</code> or <code class="language-plaintext highlighter-rouge">serial</code>.</p> <p>Now that I have summarized the options available, I will demo some of the options to solidify what the strategies accomplish.</p> <blockquote> <p>Note: The default strategy <code class="language-plaintext highlighter-rouge">linear</code>, will not be demonstrated.</p> </blockquote> <h2 id="free-strategy">Free Strategy</h2> <p>The playbook has two tasks.</p> <ol> <li>Print the hostname.</li> <li>Print the network OS.</li> </ol> <p>The playbook:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">strategy</span><span class="pi">:</span> <span class="s2">"</span><span class="s">free"</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">inventory_hostname</span><span class="nv"> </span><span class="s">}}"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10015:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">OS"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">}}"</span> <span class="nn">...</span> </code></pre></div></div> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">▶ ansible-playbook -i inventory.cfg pb.yml</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="s">PLAY RECAP ******************************************************************************************************************************</span> <span class="na">csr1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> </code></pre></div></div> <p>Looking into the execution of the playbook, the first thing I notice is each task isn’t completed before the next task is executed. <code class="language-plaintext highlighter-rouge">free</code> allows Ansible to run each host independently and execute the tasks as quick as it can. <code class="language-plaintext highlighter-rouge">free</code> is a great option when the tasks aren’t dependent on one another, or the hosts themselves aren’t dependent on one another. A good example might be collecting operation state data after a change.</p> <h2 id="debug-strategy">Debug Strategy</h2> <p>Very helpful when a playbook is running into issues, and the failed output isn’t helpful enough to identify the problem.</p> <p>The playbook:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">strategy</span><span class="pi">:</span> <span class="s2">"</span><span class="s">debug"</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">inventory_hostnames</span><span class="nv"> </span><span class="s">}}"</span> <span class="na">debugger</span><span class="pi">:</span> <span class="s">on_failed</span> </code></pre></div></div> <blockquote> <p>Notice: For demonstration I misspelled the variable name by adding an <code class="language-plaintext highlighter-rouge">s</code> to the end.</p> </blockquote> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">▶ ansible-playbook -i inventory.cfg pb.yml</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">fatal</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]:</span> <span class="s">FAILED! =&gt; {"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">The</span><span class="nv"> </span><span class="s">task</span><span class="nv"> </span><span class="s">includes</span><span class="nv"> </span><span class="s">an</span><span class="nv"> </span><span class="s">option</span><span class="nv"> </span><span class="s">with</span><span class="nv"> </span><span class="s">an</span><span class="nv"> </span><span class="s">undefined</span><span class="nv"> </span><span class="s">variable.</span><span class="nv"> </span><span class="s">The</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">was:</span><span class="nv"> </span><span class="s">'inventory_hostnames'</span><span class="nv"> </span><span class="s">is</span><span class="nv"> </span><span class="s">undefined</span><span class="se">\n\n</span><span class="s">The</span><span class="nv"> </span><span class="s">error</span><span class="nv"> </span><span class="s">appears</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">be</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">'/Users/jeffkala/Documents/github-clones/ansible-examples/strategies/pb.yml':</span><span class="nv"> </span><span class="s">line</span><span class="nv"> </span><span class="s">8,</span><span class="nv"> </span><span class="s">column</span><span class="nv"> </span><span class="s">7,</span><span class="nv"> </span><span class="s">but</span><span class="nv"> </span><span class="s">may</span><span class="se">\n</span><span class="s">be</span><span class="nv"> </span><span class="s">elsewhere</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">file</span><span class="nv"> </span><span class="s">depending</span><span class="nv"> </span><span class="s">on</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">exact</span><span class="nv"> </span><span class="s">syntax</span><span class="nv"> </span><span class="s">problem.</span><span class="se">\n\n</span><span class="s">The</span><span class="nv"> </span><span class="s">offending</span><span class="nv"> </span><span class="s">line</span><span class="nv"> </span><span class="s">appears</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">be:</span><span class="se">\n\n</span><span class="nv"> </span><span class="s">tasks:</span><span class="se">\n</span><span class="nv"> </span><span class="s">-</span><span class="nv"> </span><span class="s">name:</span><span class="nv"> </span><span class="se">\"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES</span><span class="se">\"\n</span><span class="nv"> </span><span class="s">^</span><span class="nv"> </span><span class="s">here</span><span class="se">\n</span><span class="s">This</span><span class="nv"> </span><span class="s">one</span><span class="nv"> </span><span class="s">looks</span><span class="nv"> </span><span class="s">easy</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">fix.</span><span class="nv"> </span><span class="s">It</span><span class="nv"> </span><span class="s">seems</span><span class="nv"> </span><span class="s">that</span><span class="nv"> </span><span class="s">there</span><span class="nv"> </span><span class="s">is</span><span class="nv"> </span><span class="s">a</span><span class="nv"> </span><span class="s">value</span><span class="nv"> </span><span class="s">started</span><span class="se">\n</span><span class="s">with</span><span class="nv"> </span><span class="s">a</span><span class="nv"> </span><span class="s">quote,</span><span class="nv"> </span><span class="s">and</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">YAML</span><span class="nv"> </span><span class="s">parser</span><span class="nv"> </span><span class="s">is</span><span class="nv"> </span><span class="s">expecting</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">see</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">line</span><span class="nv"> </span><span class="s">ended</span><span class="se">\n</span><span class="s">with</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">same</span><span class="nv"> </span><span class="s">kind</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">quote.</span><span class="nv"> </span><span class="s">For</span><span class="nv"> </span><span class="s">instance:</span><span class="se">\n\n</span><span class="nv"> </span><span class="s">when:</span><span class="nv"> </span><span class="se">\"</span><span class="s">ok</span><span class="se">\"</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">result.stdout</span><span class="se">\n\n</span><span class="s">Could</span><span class="nv"> </span><span class="s">be</span><span class="nv"> </span><span class="s">written</span><span class="nv"> </span><span class="s">as:</span><span class="se">\n\n</span><span class="nv"> </span><span class="s">when:</span><span class="nv"> </span><span class="s">'</span><span class="se">\"</span><span class="s">ok</span><span class="se">\"</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">result.stdout'</span><span class="se">\n\n</span><span class="s">Or</span><span class="nv"> </span><span class="s">equivalently:</span><span class="se">\n\n</span><span class="nv"> </span><span class="s">when:</span><span class="nv"> </span><span class="se">\"</span><span class="s">'ok'</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">result.stdout</span><span class="se">\"\n</span><span class="s">"</span><span class="err">}</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="na">TASK</span><span class="pi">:</span> <span class="na">10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES (debug)&gt; dir()</span> <span class="pi">[</span><span class="s1">'</span><span class="s">host'</span><span class="pi">,</span> <span class="s1">'</span><span class="s">play_context'</span><span class="pi">,</span> <span class="s1">'</span><span class="s">result'</span><span class="pi">,</span> <span class="s1">'</span><span class="s">task'</span><span class="pi">,</span> <span class="s1">'</span><span class="s">task_vars'</span><span class="pi">]</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="na">TASK</span><span class="pi">:</span> <span class="na">10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES (debug)&gt; task_vars.keys()</span> <span class="s">dict_keys(['ansible_network_os', 'inventory_file', 'inventory_dir', 'inventory_hostname', 'inventory_hostname_short', 'group_names', 'ansible_facts', 'playbook_dir', 'ansible_playbook_python', 'ansible_config_file', 'ansible_role_names', 'ansible_play_role_names', 'ansible_dependent_role_names', 'role_names', 'ansible_play_name', 'groups', 'ansible_play_hosts_all', 'ansible_play_hosts', 'ansible_play_batch', 'play_hosts', 'omit', 'ansible_version', 'ansible_check_mode', 'ansible_diff_mode', 'ansible_forks', 'ansible_inventory_sources', 'ansible_skip_tags', 'ansible_run_tags', 'ansible_verbosity', 'hostvars', 'environment', 'vars', 'ansible_current_hosts', 'ansible_failed_hosts'])</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="na">TASK</span><span class="pi">:</span> <span class="na">10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES (debug)&gt; task_vars.get('inventory_hostname')</span> <span class="s1">'</span><span class="s">nxos-spine1'</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="na">TASK</span><span class="pi">:</span> <span class="na">10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES (debug)&gt; exit()</span> </code></pre></div></div> <p>This strategy is extremely helpful. Looking at the play execution, the first task fails on the first host. Ansible raises a <code class="language-plaintext highlighter-rouge">fatal</code> error and provides the message but then drops the user into a <code class="language-plaintext highlighter-rouge">debug</code> shell where standard <code class="language-plaintext highlighter-rouge">Python</code> can be used to troubleshoot. The first step I performed was to run <code class="language-plaintext highlighter-rouge">dir()</code> to see what objects I could look into. For this example, I next looked into the <code class="language-plaintext highlighter-rouge">task_vars.key()</code> to see the valid keys. Once I printed this information I notice that I needed to use <code class="language-plaintext highlighter-rouge">inventory_hostname</code> instead of <code class="language-plaintext highlighter-rouge">inventory_hostnames</code>. Of course, this is a simplified example, but on more complex playbooks it’s a valuable option to identify issues.</p> <h2 id="host-pinned">Host Pinned</h2> <p>Host pinned is a bit more tricky, and in the backend uses serial to determine the batch number used. If <code class="language-plaintext highlighter-rouge">serial</code> is not set, it defaults to all.</p> <p>The playbook:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">strategy</span><span class="pi">:</span> <span class="s2">"</span><span class="s">host_pinned"</span> <span class="na">serial</span><span class="pi">:</span> <span class="m">2</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">inventory_hostname</span><span class="nv"> </span><span class="s">}}"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10015:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">OS"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">}}"</span> <span class="nn">...</span> </code></pre></div></div> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">▶ ansible-playbook -i inventory.cfg pb.yml</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr1"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr3"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="s">PLAY RECAP ******************************************************************************************************************************</span> <span class="na">csr1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> </code></pre></div></div> <p>Pay close attention to each iteration. When <code class="language-plaintext highlighter-rouge">serial</code> is used, it’s looping over the <code class="language-plaintext highlighter-rouge">PLAY</code> itself with the number specified in the <code class="language-plaintext highlighter-rouge">serial</code> keyword. In the above output, the <code class="language-plaintext highlighter-rouge">PLAY</code> was executed six times, each time running through the tasks with two hosts.</p> <p>Now that the main strategies available have been demonstrated, the next few demos will focus on some of the other keywords that can be used to change the execution of the playbook.</p> <h2 id="serial">Serial</h2> <p>Using <code class="language-plaintext highlighter-rouge">serial</code> without tying it to a strategy allows for further flexibility. Running playbooks in batches defined by the administrator can help validate that a playbook is working as expected before executing it against a larger batch of host. Since the previous example showed what <code class="language-plaintext highlighter-rouge">serial</code> does, below I’ll provide a few options when specifying the <code class="language-plaintext highlighter-rouge">serial</code> option.</p> <ol> <li>Using different batches.</li> </ol> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">serial</span><span class="pi">:</span> <span class="pi">-</span> <span class="m">1</span> <span class="pi">-</span> <span class="m">2</span> <span class="pi">-</span> <span class="m">5</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="s">&lt;omitted&gt;</span> <span class="nn">...</span> </code></pre></div></div> <p>This will execute the <code class="language-plaintext highlighter-rouge">PLAY</code> with one host, then the second iteration would run on two host, then 5 host for however many iterations remain.</p> <ol> <li>Using a percentage.</li> </ol> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">serial</span><span class="pi">:</span> <span class="pi">-</span> <span class="s2">"</span><span class="s">30%"</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="s">&lt;omitted&gt;</span> <span class="nn">...</span> </code></pre></div></div> <p>It is also possible to mix and match the different options. The <code class="language-plaintext highlighter-rouge">serial</code> <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.html#setting-the-batch-size-with-serial">Ansible Documentation</a> is excellent.</p> <h2 id="ordered">Ordered</h2> <p>When I use the keyword <code class="language-plaintext highlighter-rouge">order</code> in my play definition and use <code class="language-plaintext highlighter-rouge">sorted</code>, the playbook is run on the host in alphabetical order.</p> <blockquote> <p>Note: I’m back to using the default strategy of <code class="language-plaintext highlighter-rouge">linear</code>.</p> </blockquote> <p>The playbook:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">order</span><span class="pi">:</span> <span class="s2">"</span><span class="s">sorted"</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">inventory_hostname</span><span class="nv"> </span><span class="s">}}"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10015:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">OS"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">}}"</span> <span class="nn">...</span> </code></pre></div></div> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">▶ ansible-playbook -i inventory.cfg pb.yml</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx3"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="s">PLAY RECAP ******************************************************************************************************************************</span> <span class="na">csr1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> </code></pre></div></div> <p>The playbook execution ran the host in alphabetical order. This could be useful for simple human readability or potentially useful if your companies hostname standard can be used to interpret certain device roles.</p> <h2 id="throttle">Throttle</h2> <p>Lastly, I’ll cover the <code class="language-plaintext highlighter-rouge">throttle</code> keyword. I’ve used <code class="language-plaintext highlighter-rouge">throttle</code> quite a bit in network automation when working with network orchestrators or other REST APIs that use rate limiting. If I have a large list of hosts that I need to gather information on from an API and that API rate limits the number of connections per minute, I can setup <code class="language-plaintext highlighter-rouge">throttle</code> to slow down the execution of that task within the playbook.</p> <p>The playbook:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">DEBUG</span><span class="nv"> </span><span class="s">INVENTORY"</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s2">"</span><span class="s">all"</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">False</span> <span class="na">connection</span><span class="pi">:</span> <span class="s2">"</span><span class="s">network_cli"</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10010:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">NAMES"</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">inventory_hostname</span><span class="nv"> </span><span class="s">}}"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10015:</span><span class="nv"> </span><span class="s">GET</span><span class="nv"> </span><span class="s">DEVICE</span><span class="nv"> </span><span class="s">OS"</span> <span class="na">throttle</span><span class="pi">:</span> <span class="m">1</span> <span class="na">debug</span><span class="pi">:</span> <span class="na">msg</span><span class="pi">:</span> <span class="s2">"</span><span class="s">{{</span><span class="nv"> </span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">}}"</span> <span class="nn">...</span> </code></pre></div></div> <p>For this one, unfortunately, showing the output of the file doesn’t demonstrate the value. The way this playbook would execute would be based on the default <code class="language-plaintext highlighter-rouge">forks</code> of 5 on task <code class="language-plaintext highlighter-rouge">10010</code>, the next task <code class="language-plaintext highlighter-rouge">10015</code> being run one host at a time (ignoring the forks).</p> <p>In order to help demonstrate the time it took, I will quickly use <code class="language-plaintext highlighter-rouge">callback</code> to get the timer.</p> <p>In my <code class="language-plaintext highlighter-rouge">ansible.cfg</code> under the <code class="language-plaintext highlighter-rouge">[defaults]</code> I will add:</p> <div class="language-ini highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">callback_whitelist</span> <span class="p">=</span> <span class="s">profile_tasks</span> </code></pre></div></div> <p>The result of the playbook now shows the timer.</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">▶ ansible-playbook -i inventory.cfg pb.yml</span> <span class="s">PLAY [DEBUG INVENTORY] ******************************************************************************************************************</span> <span class="s">TASK [10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES] **********************************************************************************************************</span> <span class="s">Thursday 29 April 2021 11:20:05 -0500 (0:00:00.017) 0:00:00.017 ********</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos-spine2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">vmx2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">csr3"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-leaf2"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine1"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos-spine2"</span> <span class="err">}</span> <span class="s">TASK [10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS] *************************************************************************************************************</span> <span class="s">Thursday 29 April 2021 11:20:09 -0500 (0:00:03.836) 0:00:03.853 ********</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">nxos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nxos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">vmx3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">junos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">csr3</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ios"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-leaf2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine1</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="na">ok</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">eos-spine2</span><span class="pi">]</span> <span class="s">=&gt; {</span> <span class="s">"msg"</span><span class="pi">:</span> <span class="s2">"</span><span class="s">eos"</span> <span class="err">}</span> <span class="s">PLAY RECAP ******************************************************************************************************************************</span> <span class="na">csr1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">csr3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-leaf2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">eos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">nxos-spine2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx1 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx2 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="na">vmx3 </span><span class="pi">:</span> <span class="s">ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span> <span class="s">Thursday 29 April 2021 11:20:22 -0500 (0:00:13.460) 0:00:17.314 ********</span> <span class="s">===============================================================================</span> <span class="na">10015</span><span class="pi">:</span> <span class="s">GET DEVICE OS ------------------------------------------------------------------------------------------------------------ 13.46s</span> <span class="na">10010</span><span class="pi">:</span> <span class="s">GET DEVICE NAMES ---------------------------------------------------------------------------------------------------------- 3.84s</span> </code></pre></div></div> <p>Notice task <code class="language-plaintext highlighter-rouge">10010</code> took only ~4 seconds, while task <code class="language-plaintext highlighter-rouge">10015</code> which was run in <code class="language-plaintext highlighter-rouge">throttle</code> mode, took drastically longer at ~13 seconds.</p> <p>I hope this blog post helps to demonstrate the value of <code class="language-plaintext highlighter-rouge">strategies</code> and other <code class="language-plaintext highlighter-rouge">keyword</code> options that Ansible comes with natively. Each has valid use cases and mixing and matching the options can bring additional stability and flexibility to a playbook. Every user has different concerns with making changes to an environment, but utilizing these options can help ease the natural anxiety that comes with making changes across an entire infrastructure.</p> <p>-Jeff</p>Jeff KalaAdjusting Ansible playbook execution strategies is a use case that has come up several times. The default strategy Ansible uses, is called linear, and it works great a majority of the time, but what if you’re automating a task that need to be done in a specific order or way. Strategies can be used to define how a playbook should be executed. This includes the ability to run in serial, run in batches, and even, in extreme cases, allows the ability to write a custom strategy plugin.Nautobot 1.1.0 Key Feature Overview2021-07-22T00:00:00+00:002021-07-22T00:00:00+00:00https://blog.networktocode.com/post/nautobot-1.1.0-feature-overview<p>Nautobot 1.1.0 is upon us! This release comes with a group of features that will offer time savings, and a customized experience for users. This post will introduce some of those features.</p> <p>Additionally, be sure to check out the <a href="https://youtu.be/3Wj0Jl3ceTk">Nautobot 1.1.0 Key Feature Overview</a> YouTube video, which also covers many of these features in more detail.</p> <h2 id="computed-fields">Computed Fields</h2> <p>Computed fields allow users to create custom <em>read-only</em> fields from data already in the database. This new feature allows the user to specify the content type (object type) to apply the computed field to (Interface, Site, Device, etc.) and then define a <em>Jinja2</em> template to create the computed field for the objects.</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/1-computedfield-1.png" alt="" /></p> <p>As soon as the computed field is defined, it will immediately apply to the specified content type. This example shows a computed field for Interface objects:</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/2-computed-field-2.png" alt="" /></p> <h3 id="custom-jinja2-templates">Custom Jinja2 Templates</h3> <p>For more complex logic schemes, users can now define a custom Jinja2 filter for computed fields and custom links. Here is an example of a custom template with non-trivial logic:</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/2.5-computed-field-3.png" alt="" /></p> <p>Nautobot’s documentation on <a href="https://nautobot.readthedocs.io/en/latest/plugins/development/#including-jinja2-filters">including Jinja2 filters</a> has more information on the custom Jinja2 templates and how to implement them.</p> <h3 id="computed-fields-api">Computed Fields API</h3> <p>The <em>computed fields</em> feature also comes with an API enhancement, allowing the user to retrieve computed fields programmatically using the <code class="language-plaintext highlighter-rouge">include</code> query parameter:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://127.0.0.1:8000/api/dcim/interfaces/?name=Ethernet1%2F1&amp;device=ams01-edge-01&amp;include=computed_fields </code></pre></div></div> <p>Here is a snip of the returned <code class="language-plaintext highlighter-rouge">computed_fields</code> output:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>"computed_fields": { "connection-description": "Ethernet1/1.ams01-edge-01---Ethernet1/1.ams01-edge-02", }, </code></pre></div></div> <p>Computed field data can also be retrieved via GraphQL queries using <code class="language-plaintext highlighter-rouge">cpf_&lt;computed_field_slug&gt;</code>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>query { devices(name:["ams-edge-01"]) { interfaces { cpf_connection_description } } } </code></pre></div></div> <p>Please see the <a href="https://nautobot.readthedocs.io/en/latest/additional-features/computed-fields/#computed-fields">Nautobot documentation on computed fields</a> for more information.</p> <h2 id="config-context-schemas">Config Context Schemas</h2> <p>Nautobot’s <em>config context</em> is an existing feature that allows arbitrary data to be stored in Nautobot. At scale, however, it can be helpful and necessary to apply constraints on the data structure to ensure consistency and avoid data entry errors.</p> <p>In Nautobot 1.1.0, <em>config context JSON schemas</em> can provide such constraints.</p> <p>In the example below, the config context schema applies the following constraints:</p> <ul> <li>List of items</li> <li>Min items = 2</li> <li>Max items = 2</li> <li>String type</li> <li>IPv4 format</li> </ul> <p><img src="../../../static/images/blog_posts/1.1.0-features/3-context-schema-1.png" alt="" /></p> <p>Now, when we define a new config context, we can specify the config context schema we just created. Nautobot will not allow the config context to be created if the data violates the applied schema:</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/4-context-schema-2.png" alt="" /></p> <p>Once the config context conforms to the schema, it can be created.</p> <p>Check out the <a href="https://nautobot.readthedocs.io/en/latest/additional-features/config-contexts/#config-context-schemas">Nautobot documentation on config context schemas</a> for more info.</p> <h2 id="saved-graphql-queries">Saved GraphQL Queries</h2> <p>Nautobot 1.1.0 allows users to save GraphQL queries. This will save time and effort by eliminating the need to constantly re-create frequently-used GraphQL queries.</p> <p>Users can craft the query in Nautobot’s <em>GraphiQL</em> interface. Once the query is properly tuned, creating a saved query is as easy as pasting the query into the <em>Add a new GraphQL query</em> form.</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/5-saved-graphql-1.png" alt="" /></p> <p>Once saved, each query will have its own main page:</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/6-saved-graphql-2.png" alt="" /></p> <p>From the query’s main page, users can:</p> <ul> <li>Edit the query</li> <li>Execute the query</li> <li>Open the query in Nautobot’s GraphiQL interface</li> <li>Clone the query</li> <li>Delete the query</li> </ul> <h3 id="executing-saved-queries-programmatically">Executing Saved Queries Programmatically</h3> <p>To execute a stored query via the REST API, a POST request can be sent to this endpoint:</p> <p><code class="language-plaintext highlighter-rouge">/api/extras/graphql-queries/[slug]/run/</code></p> <blockquote> <p>TIP: the slug is available on the query’s main page</p> </blockquote> <p>Visit the <a href="https://nautobot.readthedocs.io/en/latest/additional-features/graphql/#saved-queries">Nautobot documentation on saved GraphQL queries</a> for details.</p> <h2 id="read-only-jobs">Read-Only Jobs</h2> <p>Jobs can now be flagged as explicitly <strong>read-only</strong> to disallow committing of changes to the database. Developers can set the new <code class="language-plaintext highlighter-rouge">read_only</code> <em>Meta</em> class attribute to <strong>True</strong>, making the Job read-only.</p> <p>Such Jobs have a “Read-only” badge and do not have the <em>Commit changes</em> checkbox, since the Job will hard-coded to prevent any changes.</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/7-ro-jobs-1.png" alt="" /></p> <p>Read-only Jobs also eliminate log messages about database rollbacks, which will remove confusion for users.</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/8-ro-jobs-2.png" alt="" /></p> <p>Users may also feel more confident about executing read-only Jobs because there is no chance of changing any data.</p> <p>Reference <a href="https://nautobot.readthedocs.io/en/latest/additional-features/jobs/">Nautobot’s documentation on Jobs</a> for more info on the <code class="language-plaintext highlighter-rouge">read_only</code> attribute and read-only Jobs.</p> <h2 id="plugin-defined-navigation">Plugin-Defined Navigation</h2> <p>In Nautobot 1.1.0, plugin authors can add tabs, groups, items, and buttons in the top navigation menu. This allows developers to customize the Nautobot navigation menu for their specific users.</p> <p>In the example below, the <em>Nautobot ChatOps</em> <strong>NavMenuGroup</strong> gets promoted to a <strong>NavMenuTab</strong> named <em>ChatOps</em>:</p> <p><img src="../../../static/images/blog_posts/1.1.0-features/9-plugin-nav-1.png" alt="" /></p> <p>Find more info on this capability in <a href="https://nautobot.readthedocs.io/en/latest/development/navigation-menu/">Nautobot’s navigation menu development guide</a>.</p> <h2 id="under-the-covers">Under the Covers</h2> <p>There are a couple of features to call out that will significantly expand the options and capabilities of a Nautobot deployment.</p> <h3 id="background-tasks-use-celery">Background Tasks Use Celery</h3> <p>Celery replaces RQ for background test execution. RQ is deprecated starting in Nautobot 1.1.0.</p> <p>RQ support for custom tasks is still supported to allow users to <a href="https://nautobot.readthedocs.io/en/latest/installation/services/#migrating-to-celery-from-rq">migrate to Celery</a> or temporarily <a href="https://nautobot.readthedocs.io/en/latest/installation/services/#concurrent-celery-and-rq-nautobot-workers">run Celery and RQ concurrently</a> while custom tasks/plugins that use the RQ worker are migrated.</p> <h3 id="support-for-mysql">Support for MySQL</h3> <p>Nautobot now fully supports MySQL 8.x as a back-end database. PostgreSQL is also still fully supported. The user can choose which database option to run <a href="https://nautobot.readthedocs.io/en/latest/installation/ubuntu/#database-setup">during installation</a>.</p> <p>A direct migration from PostgreSQL to MySQL is not supported. The transition requires a fresh start.</p> <p>Please reach out to us at our <strong>#nautobot</strong> channel in our <em>Network to Code</em> <a href="https://slack.networktocode.com/">public Slack workspace</a> if you have any questions about these features!</p> <p>Thank you, and have a great day!</p> <p>-Tim</p>Tim FiolaNautobot 1.1.0 is upon us! This release comes with a group of features that will offer time savings, and a customized experience for users. This post will introduce some of those features.Using Jinja2 Macros as Template Functions2021-07-20T00:00:00+00:002021-07-20T00:00:00+00:00https://blog.networktocode.com/post/using-jinja2-macros-as-template-functions<p>Jinja2 is a popular text templating engine for Python that is also used heavily with Ansible. Many network engineers are familiar with it and with leveraging Jinja2 templates for device configurations. But I’ve found one feature that’s under-utilized by network engineers: Jinja2’s macro support. Macros within Jinja2 are essentially used like functions with Python and are great for repeating configs with identical structures (think interface configurations, ASA object-groups, etc).</p> <h2 id="real-world-situation">Real-World Situation</h2> <p>Let’s start with a basic real-world configuration example. I’ll start with the following IOS interface configuration for a switch:</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>interface FastEthernet0/1 switchport mode access switchport access vlan 10 interface FastEthernet0/2 switchport mode trunk switchport trunk native vlan 20 </code></pre></div></div> <p>I could repeat this for multiple interfaces, but two interfaces will be enough to demonstrate. If I wanted to write an Ansible playbook to generate this configuration, using structured data YAML files for holding the configuration, one way I could build the YAML file <code class="language-plaintext highlighter-rouge">interfaces.yml</code> would be like this:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="na">all_interfaces</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">FastEthernet0/1</span> <span class="na">vlan</span><span class="pi">:</span> <span class="m">10</span> <span class="na">mode</span><span class="pi">:</span> <span class="s2">"</span><span class="s">access"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">FastEthernet0/2</span> <span class="na">vlan</span><span class="pi">:</span> <span class="m">20</span> <span class="na">mode</span><span class="pi">:</span> <span class="s2">"</span><span class="s">trunk"</span> </code></pre></div></div> <p>I could then create the Jinja2 template file <code class="language-plaintext highlighter-rouge">switch01.j2</code> like this:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#jinja2: lstrip_blocks: True <span class="cp">{%</span> <span class="k">for</span> <span class="nv">interface</span> <span class="ow">in</span> <span class="nv">all_interfaces</span> <span class="cp">%}</span> interface <span class="cp">{{</span> <span class="nv">interface.name</span> <span class="cp">}}</span> switchport mode <span class="cp">{{</span> <span class="nv">interface.mode</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="k">if</span> <span class="nv">interface.mode</span> <span class="o">==</span> <span class="s1">'trunk'</span> <span class="cp">%}</span> switchport trunk native vlan <span class="cp">{{</span> <span class="nv">interface.vlan</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="nv">elif</span> <span class="nv">interface.mode</span> <span class="o">==</span> <span class="s1">'access'</span> <span class="cp">%}</span> switchport access vlan <span class="cp">{{</span> <span class="nv">interface.vlan</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span> <span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span> </code></pre></div></div> <p>When generating the configuration text using the above template, I would get the expected output as shown above in the example IOS switch config. This is all well and good, but what happens if I want to configure a second switch?</p> <p>I don’t want to copy the <code class="language-plaintext highlighter-rouge">switch01.j2</code> file and rename it to <code class="language-plaintext highlighter-rouge">switch02.j2</code>. I also don’t want to simply reuse that file. What happens if switch01 gets replaced with an NX-OS switch, or even a switch from another vendor? I would run into issues quickly as my network expanded. This presents the use case for Jinja2 macros.</p> <h2 id="jinja2-macros-example">Jinja2 Macros Example</h2> <p>To create the interfaces Jinja2 template above and convert it into a macro, I can wrap all of the code with <code class="language-plaintext highlighter-rouge">{% macro %}{% endmacro %}</code>. I can then use that code as a function, import it into other templates, and even pass variables into it (just like I would with a regular Python function). For example, I will first create the template <code class="language-plaintext highlighter-rouge">all_interfaces_template.j2</code>:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{%</span> <span class="k">macro</span> <span class="nv">l2_interfaces</span><span class="p">(</span><span class="nv">interfaces</span><span class="p">)</span> <span class="cp">%}</span> <span class="cp">{%</span> <span class="k">for</span> <span class="nv">interface</span> <span class="ow">in</span> <span class="nv">interfaces</span> <span class="cp">%}</span> interface <span class="cp">{{</span> <span class="nv">interface.name</span> <span class="cp">}}</span> switchport mode <span class="cp">{{</span> <span class="nv">interface.mode</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="k">if</span> <span class="nv">interface.mode</span> <span class="o">==</span> <span class="s1">'trunk'</span> <span class="cp">%}</span> switchport trunk native vlan <span class="cp">{{</span> <span class="nv">interface.vlan</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="nv">elif</span> <span class="nv">interface.mode</span> <span class="o">==</span> <span class="s1">'access'</span> <span class="cp">%}</span> switchport access vlan <span class="cp">{{</span> <span class="nv">interface.vlan</span> <span class="cp">}}</span> <span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span> <span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span> <span class="cp">{%</span> <span class="k">endmacro</span> <span class="cp">%}</span> </code></pre></div></div> <p>Line 1 is <code class="language-plaintext highlighter-rouge">{% macro l2_interfaces(interfaces) %}</code>. The first part of the tag is <code class="language-plaintext highlighter-rouge">macro</code>, which simply defines this tag as a tag type of <em>macro</em>. The next part <code class="language-plaintext highlighter-rouge">l2_interfaces()</code> is used to define the name of the function. Lastly, <code class="language-plaintext highlighter-rouge">interfaces</code> is the name of the variable I want to pass into this macro when calling it.</p> <p>If I were to write this out in Python, it would look like:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">l2_interfaces</span><span class="p">(</span><span class="n">interfaces</span><span class="p">):</span> <span class="k">for</span> <span class="n">interface</span> <span class="ow">in</span> <span class="n">interfaces</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"interface </span><span class="si">{</span><span class="n">interface</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span> <span class="k">if</span> <span class="n">interface</span><span class="p">[</span><span class="s">"mode"</span><span class="p">]</span> <span class="o">==</span> <span class="s">"trunk"</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">" switchport trunk native vlan </span><span class="si">{</span><span class="n">interface</span><span class="p">[</span><span class="s">'vlan'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span> <span class="k">elif</span> <span class="n">interface</span><span class="p">[</span><span class="s">"mode"</span><span class="p">]</span> <span class="o">==</span> <span class="s">"access"</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">" switchport access vlan </span><span class="si">{</span><span class="n">interface</span><span class="p">[</span><span class="s">'vlan'</span><span class="p">]</span><span class="si">}</span><span class="s">"</span><span class="p">)</span> </code></pre></div></div> <p>Now I have a macro I can use in other Jinja2 templates. Using the same <code class="language-plaintext highlighter-rouge">interfaces.yml</code> YAML file from before where the interfaces are defined, I will first create a template for switch01, with filename <code class="language-plaintext highlighter-rouge">switch01.j2</code>:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#jinja2: lstrip_blocks: True <span class="cp">{%</span> <span class="k">from</span> <span class="s1">'interfaces_template.j2'</span> <span class="k">import</span> <span class="nv">l2_interfaces</span> <span class="cp">%}</span> <span class="cp">{{</span> <span class="nv">l2_interfaces</span><span class="p">(</span><span class="nv">all_interfaces</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>And that’s it! Both interfaces will be properly generated as expected into the following config (shown at the top of this blog post):</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>interface FastEthernet0/1 switchport mode access switchport access vlan 10 interface FastEthernet0/2 switchport mode trunk switchport trunk native vlan 20 </code></pre></div></div> <p>There are only three lines in the template file <code class="language-plaintext highlighter-rouge">switch01.j2</code>, but I want to explain them in detail.</p> <h3 id="line-1">Line 1</h3> <ul> <li>The first line is <code class="language-plaintext highlighter-rouge">#jinja2: lstrip_blocks: True</code>. This helps to control extra whitespace generated by Jinja2 from the tags. More info on controlling whitespace can be found on a previous <a href="https://blog.networktocode.com/post/whitespace-control-in-jinja-templates/">NTC blog post</a>.</li> </ul> <h3 id="line-2">Line 2</h3> <ul> <li>The second line is <code class="language-plaintext highlighter-rouge">{% from 'interfaces_template.j2' import l2_interfaces %}</code>. You’ll notice there is no closing tag for it, like <code class="language-plaintext highlighter-rouge">{% endfrom %}</code>, as Jinja2 does not require closing this tag.</li> <li>The part <code class="language-plaintext highlighter-rouge">'interfaces_template.j2'</code> references which file to import from. If it were nested in a folder called “switches/”, it would be imported with <code class="language-plaintext highlighter-rouge">switches/interfaces_template.j2</code>.</li> <li>The last part, <code class="language-plaintext highlighter-rouge">import l2_interfaces</code>, tells Jinja2 the name of the macro to import.</li> </ul> <h3 id="line-3">Line 3</h3> <ul> <li>The last line is <code class="language-plaintext highlighter-rouge">{{ l2_interfaces(all_interfaces) }}</code>, which calls the function I just imported on the line above and passes in the variable <code class="language-plaintext highlighter-rouge">all_interfaces</code>, which is obtained from the YAML file <code class="language-plaintext highlighter-rouge">interfaces.yml</code>.</li> </ul> <p>Now I can easily create a template for a second switch, and expand the configuration some more, with Jinja2 template <code class="language-plaintext highlighter-rouge">switch02.j2</code>:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{%</span> <span class="k">from</span> <span class="s1">'interfaces_template.j2'</span> <span class="k">import</span> <span class="nv">l2_interfaces</span> <span class="cp">%}</span> hostname Switch02 ip domain-name networktocode.com ! <span class="cp">{{</span> <span class="nv">l2_interfaces</span><span class="p">(</span><span class="nv">all_interfaces</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <p>Which generates:</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>hostname Switch02 ip domain-name networktocode.com ! interface FastEthernet0/1 switchport mode access switchport access vlan 10 interface FastEthernet0/2 switchport mode trunk switchport trunk native vlan 20 </code></pre></div></div> <p>If I wanted to add interfaces, all I would have to do is add them to <code class="language-plaintext highlighter-rouge">interfaces.yml</code>, then run my Jinja2 template again! While this example is good for demonstration purposes, I would ideally expand it out further in a real-world environment, to allow different interface information on a per-switch basis.</p> <h2 id="importing-functions-abbreviated">Importing Functions, Abbreviated</h2> <p>I can rename the function “l2_interfaces” to “l2i” when imported, <em>just</em> like I can when importing in Python, by adding <code class="language-plaintext highlighter-rouge">as l2i</code> to the end. For example: <code class="language-plaintext highlighter-rouge">{% from 'interfaces_template.j2' import l2_interfaces as l2i %}</code>. This can be handy when importing macros with long names.</p> <p>I would then use the function like so:</p> <div class="language-jinja highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{%</span> <span class="k">from</span> <span class="s1">'interfaces_template.j2'</span> <span class="k">import</span> <span class="nv">l2_interfaces</span> <span class="k">as</span> <span class="nv">l2i</span> <span class="cp">%}</span> <span class="cp">{{</span> <span class="nv">l2i</span><span class="p">(</span><span class="nv">all_interfaces</span><span class="p">)</span> <span class="cp">}}</span> </code></pre></div></div> <h2 id="wrapping-it-up">Wrapping It Up</h2> <p>I hope you’ve found this topic helpful, and possibly learned something entirely new about Jinja2! Feel free to comment below, or reach out to us at Network to Code on our public Slack at <a href="slack.networktocode.com">slack.networktocode.com</a> and ask around.</p> <p>-Matt</p>Matt VitaleJinja2 is a popular text templating engine for Python that is also used heavily with Ansible. Many network engineers are familiar with it and with leveraging Jinja2 templates for device configurations. But I’ve found one feature that’s under-utilized by network engineers: Jinja2’s macro support. Macros within Jinja2 are essentially used like functions with Python and are great for repeating configs with identical structures (think interface configurations, ASA object-groups, etc).Nautobot Relationships2021-07-15T00:00:00+00:002021-07-15T00:00:00+00:00https://blog.networktocode.com/post/nautobot-relationships-part-1<p>The native data model in Nautobot will suffice for the majority of use cases. However, some deployments have additional requirements between objects based on the design of their networks. Nautobot now provides the capability to define new relationships to support a more customized network data model.</p> <p>For example, a VLAN may need to be defined on a per-device basis, rather than the native model of a VLAN per site. This would allow a user to define and model VLANs for a network device that are locally significant.</p> <p>This series of posts will outline the new user-defined relationship feature in Nautobot and explore some use cases where this feature can be applied to help model a network design.</p> <h2 id="relationships-primer">Relationships Primer</h2> <p>One of the columns in a database table will store a primary key, which will uniquely identify each object. This field is used in the object-relational mapping (ORM) when defining relationships between objects. This simply allows objects to build associations with each other.</p> <p>A common example used to explain this concept uses books. A publisher has a relationship with a book as it can publish a book. Each book will have a publisher.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/db-assoc.png" alt="database relationship" title="Database Relationship" /></p> <p>Additional constraints can be introduced to be more specific in the relationship definition. The next sections outline the different types of relationships that can exist between objects.</p> <h3 id="one-to-many">One-to-Many</h3> <p>A <code class="language-plaintext highlighter-rouge">one-to-many</code> type introduces a constraint in that a book can have only one publisher, denoted with the <code class="language-plaintext highlighter-rouge">1</code> on line beside the publisher. A publisher can have many books, denoted with <code class="language-plaintext highlighter-rouge">N</code> for number on the many side.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/rel-one-to-many.png" alt="one-to-many relationship" title="One-to-Many" /></p> <h3 id="many-to-many">Many-to-Many</h3> <p>A <code class="language-plaintext highlighter-rouge">many-to-many</code> relationship has no constraints. A customer can buy many books and a book can be bought by many customers.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/rel-many-to-many.png" alt="many-to-many relationship" title="Many-to-Many" /></p> <h3 id="one-to-one">One-to-One</h3> <p>A <code class="language-plaintext highlighter-rouge">one-to-one</code> relationship has a unique constraint on both sides of the relationship. For example, an order can have only one payment and a payment can be made on one order.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/rel-one-to-one.png" alt="one-to-one relationship" title="One-to-One" /></p> <h2 id="nautobot-model">Nautobot Model</h2> <p>Nautobot is designed as a Source of Truth application aimed at modeling modern networks. Its core data models allow users to define the intended state of a network. This is achieved by providing a native data model, with its inherent relationships between the objects.</p> <p>As an example, a subset of the Nautobot core data model illustrates the relationships between some of the objects in the <code class="language-plaintext highlighter-rouge">nautobot.ipam</code> and <code class="language-plaintext highlighter-rouge">nautobot.circuit</code> applications. The database tables are represented as boxes including their fields. The lines between the tables denote an association or relationship between the tables. The diagram shows how an <strong>IPAddress</strong> can have a <strong>VRF</strong>, a <strong>Prefix</strong> can have a <strong>VRF</strong> and a <strong>VLAN</strong>, and finally a <strong>Circuit</strong> can have a <strong>Provider</strong>. At a database level, we can see that these relationships are represented as foreign keys between the database tables.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/graph_model_incl_fields.png" alt="nautobot data model subset" title="Nautobot Data Model" /></p> <blockquote> <p><em>INFO: This diagram was rendered using the graph models feature in the <code class="language-plaintext highlighter-rouge">django_extensions</code> package.</em></p> </blockquote> <p>The core data model is quite established, based on years of iterative open-source development that began with the NetBox project. However, it’s very difficult to provide a standard framework that can meet the requirements of all networks. There will always be components of the network that will be different. One of the key objectives of Nautobot is to provide maximum data model flexibility. This is where the new Relationships feature can be applied. It allows users to define their own relationships between the objects in the system.</p> <h2 id="use-case-1-ipaddress---circuit">Use Case 1: IPAddress - Circuit</h2> <p>The first use case will focus on modeling an IPv4-based circuit, such as an enterprise router connection to an Internet Service Provider WAN. Each circuit can have two IP addresses, one at each end of the circuit. This type of relationship is not defined in the native data model. To enable it, a new relationship (highlighted in red) can be created between an <strong>IPAddress</strong> and a <strong>Circuit</strong>.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/graph_model_no_fields_new_rel.png" alt="Circuit - IP Address Model" title="Circuit - IP Address Model" /></p> <h3 id="new-relationship-ipaddress---circuit">New Relationship: IPAddress - Circuit</h3> <p>To create a new relationship, navigate to <strong>Extensibility -&gt; Relationships</strong> on the Nautobot web user interface. The use case in question supports one circuit having two IPs. Any model that can have more than one object in the association is defined as a <strong>many</strong> in the relationship. Enter a custom name to identify the new relationship and set the relationship type as <strong>One to Many</strong>.</p> <p>The <strong>Source type</strong> is <strong>circuit | circuit</strong> as this is the <strong>one</strong> side of the relationship and it is defined as the source. The <strong>Source Label</strong> will be used to display the other end of the relationship, on the source page. For this use case the <strong>Source Label</strong> is <strong>IP Address</strong>. The <strong>Destination</strong> fields are the reverse of the <strong>Source</strong> fields. Both source and destination filters can be left blank.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/circuit-ipaddress-relationship.png" alt="Circuit - IP Address Relationship" title="Circuit - IP Address Relationship" /></p> <h3 id="update-circuit">Update Circuit</h3> <p>Once a relationship has been created, it will be visible when configuring one of the model objects. A new <strong>Relationship</strong> panel will be available to select an existing object of the defined source/destination type.</p> <p>In the example, circuit <code class="language-plaintext highlighter-rouge">A123456789</code> has two IP addresses associated with it. Multiple IPs can be can be selected since <strong>IPAddress</strong> is on the <strong>many</strong> side of the new relationship.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/edit_circuit_ip_relationship.png" alt="Edit Circuit" title="Edit Circuit" /></p> <h3 id="update-ip-address">Update IP Address</h3> <p>For the IP addresses, one circuit can be selected in each IP address object. The selection drop-down list will only allow one object to be configured, as <strong>Circuit</strong> is on the <strong>one</strong> side of the relationship.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/edit_ipaddress_circuit_relationship.png" alt="Edit IP Address" title="Edit IP Address" /></p> <p>Defining a new relationship allows network design use cases to be managed through the web user interface by creating new associations between existing objects. This reduces the need to manage additional layers of data by adding custom fields to objects and maintaining the custom field data synchronization through scripts or playbooks. Relationships can also be defined using the REST API.</p> <p>An alternative solution for this use case might involve creating a <code class="language-plaintext highlighter-rouge">one-to-one</code> relationship between an IP Address and Circuit. In this case, a circuit would be associated with one IP Address. Creating two of these associations would model the circuit end-to-end.</p> <h2 id="use-case-2-vlan---vlan-group">Use Case 2: VLAN - VLAN Group</h2> <p>Another, perhaps more advanced, use case would be to create a relationship between models that already have a native relationship. For example, a VLAN group and VLAN have an inherent <strong>One to Many</strong> relationship, whereby a VLAN Group can have many VLANs but a VLAN can only be a member of only one VLAN group. Some networks may have a need to have VLANs in multiple VLAN groups. VLANs could be members of a management group for a site, while also being part of a group that requires access to cloud-based Operational Support Systems (OSS). This use case could be managed with custom tags or custom fields, but relationships provide a way to overlay a new association between the models and link existing objects together.</p> <p>The diagram below illustrates how the new relationship (highlighted in red) would be modeled, alongside the existing relationship.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/graph_model_no_fields_vlans.png" alt="VLAN - VLAN Groups Model" title="VLAN - VLAN Groups Model" /></p> <h3 id="new-relationship-vlan---vlan-group">New Relationship: VLAN - VLAN Group</h3> <p>A <strong>Many to Many</strong> relationship is required to support this use case. Following on from the guidelines in the previous use case, with the addition of a filter to restrict the relationship to VLANs with the role of leaf, filtering is defined using JSON data format to identify the model fields and their values.</p> <p>In the example, the new relationship will be applied only on VLANs that have a role of <code class="language-plaintext highlighter-rouge">leaf</code>. Roles can be applied to VLANs to assist with network design modeling e.g., define leaf-switch VLANs.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/vlan_edit_multiple_groups_relationship.png" alt="VLAN Relationship" title="VLAN Relationship" /></p> <p>For demonstration purposes, a short list of VLANs shows some <strong>Leaf</strong> and <strong>Core</strong> roles to illustrate the filtering capability.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/vlan_list_relationship.png" alt="VLAN List" title="VLAN List" /></p> <h3 id="update-vlan">Update VLAN</h3> <p>Editing a VLAN allows multiple VLAN groups to be associated with a single VLAN through the newly defined relationship. A key point to note is that the new relationship is managed through the <strong>Relationships</strong> panel and not through the native <strong>Group</strong> field, which is left blank.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/edit_vlan_relationship.png" alt="VLAN Edit" title="VLAN Edit" /></p> <h3 id="update-vlan-group">Update VLAN Group</h3> <p>For VLAN group(s) configured in the relationship, apply the new VLANs. Due to the filtering applied on the VLAN side of the relationship, only VLANs with a role of <strong>Leaf</strong> can be selected.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/vlan_group_edit_filter_vlan_relationship.png" alt="VLAN Filter" title="VLAN Filter" /></p> <p>Management of the new VLANs is through the <strong>Relationships</strong> panel on the VLAN Group page. For this advanced use case, the native VLAN view (right-hand side) on the VLAN group page is not populated.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/vlan_group_manage_vlan.png" alt="VLAN Group Management" title="VLAN Group Management" /></p> <p>Selecting the <strong>VLANs</strong> hyperlink displays all of the VLANs through the relationship-association table.</p> <p><img src="../../../static/images/blog_posts/nautobot-relationships/vlan_group_relationship_assoc.png" alt="Relationship Association" title="Relationship Association" /></p> <h2 id="conclusion">Conclusion</h2> <p>Defining new relationships is a very powerful feature in Nautobot. It supports flexible data modeling to assist with specific use cases of network design representation in a Source of Truth application.</p> <p>This guide focused on how to define the relationships in the web user interface. In a future post, we’ll look at using REST and GraphQL based APIs to interact with the new relationships in Nautobot.</p> <p>-Paddy</p> <h2 id="resources">Resources</h2> <ul> <li><a href="https://github.com/nautobot">Nautobot</a></li> <li><a href="https://nautobot.readthedocs.io/en/latest/models/extras/relationship/">Nautobot Relationships</a></li> <li><a href="https://django-extensions.readthedocs.io/en/latest/graph_models.html">Graph Models</a></li> </ul>Paddy KellyThe native data model in Nautobot will suffice for the majority of use cases. However, some deployments have additional requirements between objects based on the design of their networks. Nautobot now provides the capability to define new relationships to support a more customized network data model.Learning Salt with Ansible References2021-07-13T00:00:00+00:002021-07-13T00:00:00+00:00https://blog.networktocode.com/post/learn-salt-with-ansible-references<p>Have you seen or taken the <a href="https://dgarros.github.io/netdevops-survey/reports/2020">2020 NetDevOps</a> survey? If not, don’t worry, it is held yearly from Sept 30th to Oct 31st, so you will get your chance. To quickly recap, the survey’s intention is to understand which tools are most commonly used by Network Operators and Engineers to automate their day-to-day jobs. It is a very interesting survey, and I encourage you to read it.</p> <p>If we take a look at this data, particularly the questions about tools used in automation, we will see that Ansible is the most popular one. At the same time, there are other tools that are used, but not so much—like SaltStack (aka Salt). If I would speculate why SaltStack is not so popular among Network Engineers, I would say that’s because Salt is more friendly with managing the servers and not so much the networking devices. So I thought, if I write a blog about Salt with references to Ansible it might encourage engineers to learn this tool a bit more.</p> <blockquote> <p><strong>NOTE</strong> <br /> I will do my best to address situations where features available to Salt do not have direct counterparts in Ansible.</p> </blockquote> <h2 id="salt-terminologies">Salt Terminologies</h2> <p>Before you start using Salt, you need to get familiar with its terminologies. The below table summarizes the ones that I will use in this blog:</p> <table> <thead> <tr> <th>SaltStack Name</th> <th>Description</th> <th>Ansible Name</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td>Master</td> <td>Server that commands-and-controls the minions.</td> <td>Server</td> <td>The machine where Ansible is installed.</td> </tr> <tr> <td>Minion</td> <td>Managed end-hosts where the commands are executed.</td> <td>Hosts</td> <td>The devices that are managed by the Ansible server.</td> </tr> <tr> <td>Proxy Minion</td> <td>End-hosts that cannot run the <code class="language-plaintext highlighter-rouge">salt-minion</code> service and can be controlled using proxy-minion setup. <strong><em>These hosts are mostly networking or IoT devices.</em></strong></td> <td>N/A</td> <td>N/A</td> </tr> <tr> <td>Pillars</td> <td>Information (e.g., username, passwords, configuration parameters, variables, etc.) that needs to be defined and associated with one or more minion(s) in order to interact with or distribute to the end-host.</td> <td>Inventory and Static variables</td> <td>The Ansible Inventory and variables that need to be associated with the managed host(s).</td> </tr> <tr> <td>Grains</td> <td>Information (e.g., IP address, OS type, memory utilization etc.) that is obtained from the managed minion. Grains also can be statically assigned to the minions.</td> <td>Fact</td> <td>Information obtained from the host with the <code class="language-plaintext highlighter-rouge">gather_facts</code> or similar task operation.</td> </tr> <tr> <td>Modules</td> <td>Commands that are executed locally on the master or on the minion.</td> <td>Modules</td> <td>Commands that are executed locally on the server or on the end-host.</td> </tr> <tr> <td>SLS Files</td> <td>(S)a(l)t (S)tate files, also known as formulas, are the representation of the state in which the end-hosts should be. SLS files are mainly written in YAML, but other languages are also supported.</td> <td>Playbook/Tasks</td> <td>The file where one or more tasks that need to be executed on the end-host are defined.</td> </tr> </tbody> </table> <h2 id="salt-and-ansible-playground">Salt and Ansible Playground</h2> <p>Let’s take a look at my playground. These are the hosts and related information involved in this blog:</p> <p><strong>Salt</strong></p> <ul> <li>Role: Hosts Salt’s master and proxy-minion configuration.</li> <li>OS: Ubuntu 18.04.3</li> <li>Salt: Version 3003.1</li> <li>IP: 172.29.224.100</li> </ul> <p><strong>Ansible</strong></p> <ul> <li>Role: Hosts Ansible execution environment</li> <li>OS: Ubuntu 18.04.3</li> <li>Ansible: Version 2.10.7</li> <li>IP: 172.29.224.99</li> </ul> <p><strong>IOS-XE</strong></p> <ul> <li>Role: End host for manipulation</li> <li>OS: 16.12.3</li> <li>IP: 172.29.224.101</li> </ul> <p><strong>NX-OS</strong></p> <ul> <li>Role: End host for manipulation</li> <li>OS: 7.0.9</li> <li>IP: 172.29.224.102</li> </ul> <p><strong>Junos</strong></p> <ul> <li>Role: End host for manipulation</li> <li>OS: 21.2R1.10</li> <li>IP: 172.29.224.103</li> </ul> <p><img src="../../../static/images/blog_posts/saltstack_with_ansible/01.png" alt="Lab Diagram" /></p> <h2 id="salt-and-ansible-installation">Salt and Ansible Installation</h2> <p>The Salt installation can vary based on the OS that you are using, so I would suggest that you read the <a href="https://repo.saltproject.io/">installation instructions</a> that match your OS. At the end, you should have <code class="language-plaintext highlighter-rouge">salt-master</code> and <code class="language-plaintext highlighter-rouge">salt-minion</code> installed on your machine.</p> <p>The Ansible installation is a bit more straightforward; use the <code class="language-plaintext highlighter-rouge">pip install ansible==2.10.7</code> command to install the same version that I use.</p> <h2 id="salt-environment-configuration">Salt Environment Configuration</h2> <p>The time has come to configure the Salt environment. Confirm that <code class="language-plaintext highlighter-rouge">salt-master</code> service is active. If not, start it with <code class="language-plaintext highlighter-rouge">sudo salt-master -d</code> command.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>service salt-master status ● salt-master.service - The Salt Master Server Loaded: loaded <span class="o">(</span>/lib/systemd/system/salt-master.service<span class="p">;</span> enabled<span class="p">;</span> vendor preset: enabled<span class="o">)</span> Active: active <span class="o">(</span>running<span class="o">)</span> since Tue 2021-07-06 23:29:24 UTC<span class="p">;</span> 7min ago <span class="o">&gt;&gt;&gt;</span> Trimmed <span class="k">for </span>brevity <span class="o">&lt;&lt;&lt;</span> </code></pre></div></div> <p>Using your favorite text editor, open the <code class="language-plaintext highlighter-rouge">/etc/salt/master</code> configuration file and append the below text to it:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">file_roots</span><span class="pi">:</span> <span class="c1"># Allows minions and proxy-minions to access SLS files with salt:// prefix</span> <span class="na">base</span><span class="pi">:</span> <span class="c1"># Name of the environment</span> <span class="pi">-</span> <span class="s">/opt/salt/base/root/</span> <span class="c1"># Full path where state and related files will be stored</span> <span class="na">pillar_roots</span><span class="pi">:</span> <span class="c1"># Instructs master to look for pillar files at the specified location(s)</span> <span class="na">base</span><span class="pi">:</span> <span class="c1"># Name of the environment</span> <span class="pi">-</span> <span class="s">/opt/salt/base/pillar</span> <span class="c1"># Full path where pillar files will be stored</span> </code></pre></div></div> <p>Next, configure the <code class="language-plaintext highlighter-rouge">proxy</code> file in the <code class="language-plaintext highlighter-rouge">/etc/salt/master</code> directory and point it to the Salt master’s IP or FQDN, like so:</p> <blockquote> <p><strong>NOTE</strong> <br /> In my setup, I am running the proxy service on the same machine where the <code class="language-plaintext highlighter-rouge">salt-master</code> is configured, so I will use the <code class="language-plaintext highlighter-rouge">localhost</code>.</p> </blockquote> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> /etc/salt/proxy master: localhost </code></pre></div></div> <p>Save the changes and restart the <code class="language-plaintext highlighter-rouge">salt-master</code> service using <code class="language-plaintext highlighter-rouge">sudo service salt-master restart</code> command. After the restart, you will need to create necessary folders. For that, use the <code class="language-plaintext highlighter-rouge">sudo mkdir -p /opt/salt/base/{root,pillar}</code> command.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo mkdir</span> <span class="nt">-p</span> /opt/salt/base/<span class="o">{</span>root,pillar<span class="o">}</span> <span class="nv">$ </span>ll /opt/salt/base/ total 16 drwxr-xr-x 4 root root 4096 Jul 7 00:17 ./ drwxr-xr-x 3 root root 4096 Jul 7 00:17 ../ drwxr-xr-x 2 root root 4096 Jul 7 00:17 pillar/ drwxr-xr-x 2 root root 4096 Jul 7 00:17 root/ </code></pre></div></div> <p>Great, the <code class="language-plaintext highlighter-rouge">salt-master</code> is ready. Now you can configure the pillar files for the <code class="language-plaintext highlighter-rouge">proxy-minion</code> service. This will contain instructions on how to communicate with the networking devices. Create <code class="language-plaintext highlighter-rouge">junos-pillar.sls</code> file in the <code class="language-plaintext highlighter-rouge">/opt/salt/base/pillar/</code> folder and populate with the following data:</p> <blockquote> <p><strong>NOTE</strong> <br /> Some vendors, such as Juniper and Cisco, have their own proxy types. Cisco currently has proxy module only for NXOS platform. For IOS, you would need to use either netmiko or NAPALM. To avoid complications, we will stick with netmiko for all three platforms.</p> </blockquote> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">proxy</span><span class="pi">:</span> <span class="na">proxytype</span><span class="pi">:</span> <span class="s">netmiko</span> <span class="c1"># Proxy type</span> <span class="na">device_type</span><span class="pi">:</span> <span class="s">juniper_junos</span> <span class="c1"># A setting required by the netmiko</span> <span class="na">ip</span><span class="pi">:</span> <span class="s">172.29.224.103</span> <span class="c1"># IP address of the host</span> <span class="na">username</span><span class="pi">:</span> <span class="s">admin</span> <span class="na">password</span><span class="pi">:</span> <span class="s">admin123</span> </code></pre></div></div> <p>Next, create the <code class="language-plaintext highlighter-rouge">nxos-pillar.sls</code> file in the same folder and populate with following data:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">proxy</span><span class="pi">:</span> <span class="na">proxytype</span><span class="pi">:</span> <span class="s">netmiko</span> <span class="na">device_type</span><span class="pi">:</span> <span class="s">cisco_nxos</span> <span class="na">ip</span><span class="pi">:</span> <span class="s">172.29.224.102</span> <span class="na">username</span><span class="pi">:</span> <span class="s">admin</span> <span class="na">password</span><span class="pi">:</span> <span class="s">admin123</span> </code></pre></div></div> <p>Finally create the <code class="language-plaintext highlighter-rouge">ios-pillar.sls</code>:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">proxy</span><span class="pi">:</span> <span class="na">proxytype</span><span class="pi">:</span> <span class="s">netmiko</span> <span class="na">device_type</span><span class="pi">:</span> <span class="s">cisco_xe</span> <span class="na">ip</span><span class="pi">:</span> <span class="s">172.29.224.101</span> <span class="na">username</span><span class="pi">:</span> <span class="s">admin</span> <span class="na">password</span><span class="pi">:</span> <span class="s">admin123</span> </code></pre></div></div> <blockquote> <p><strong>NOTE</strong> <br /> You can create other pillar files for the same devices and use different <code class="language-plaintext highlighter-rouge">proxytype</code>s. For Juniper devices you can use <a href="https://docs.saltproject.io/en/latest/ref/proxy/all/salt.proxy.junos.html"><code class="language-plaintext highlighter-rouge">junos</code></a> proxy and for Cisco NXOS devices you can use <a href="https://docs.saltproject.io/en/latest/ref/proxy/all/salt.proxy.nxos.html"><code class="language-plaintext highlighter-rouge">nxos</code></a> or <a href="https://docs.saltproject.io/en/latest/ref/proxy/all/salt.proxy.nxos_api.html"><code class="language-plaintext highlighter-rouge">nxos_api</code></a> proxies.</p> </blockquote> <p>To compare with Ansible, this is similar to the inventory file where you need to define the <code class="language-plaintext highlighter-rouge">ansible_host</code>, <code class="language-plaintext highlighter-rouge">ansible_user</code>, <code class="language-plaintext highlighter-rouge">ansible_password</code> and <code class="language-plaintext highlighter-rouge">ansible_network_os</code> variables.</p> <div class="language-ini highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">[base]</span> <span class="err">IOS-XE</span> <span class="py">ansible_host</span><span class="p">=</span><span class="s">172.29.224.101 ansible_user=admin ansible_password=admin123 ansible_network_os=ios ansible_connection=network_cli</span> <span class="err">NX-OS</span> <span class="py">ansible_host</span><span class="p">=</span><span class="s">172.29.224.102 ansible_user=admin ansible_password=admin123 ansible_network_os=nxos ansible_connection=network_cli</span> <span class="err">Junos</span> <span class="py">ansible_host</span><span class="p">=</span><span class="s">172.29.224.103 ansible_user=admin ansible_password=admin123 ansible_network_os=junos ansible_connection=netconf</span> </code></pre></div></div> <p>Okay, let’s take a quick break to understand how these files were constructed. The most important one is the <code class="language-plaintext highlighter-rouge">proxy</code> key. Salt will look for this key, and the information within it, to set up the the <code class="language-plaintext highlighter-rouge">proxy-minion</code> service. The rest of the key value pairs were defined based on the official documentation that can be found <a href="https://docs.saltproject.io/en/latest/ref/proxy/all/index.html">here</a>.</p> <p>Now you need to create the pillar <code class="language-plaintext highlighter-rouge">top.sls</code> file (main file) where each pillar file will be associated with a respective proxy-minion name. The <code class="language-plaintext highlighter-rouge">top.sls</code> file should be located in the <code class="language-plaintext highlighter-rouge">/opt/salt/base/pillar/</code> folder and should contain the following data:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">base</span><span class="pi">:</span> <span class="c1"># The name of the environment</span> <span class="s1">'</span><span class="s">Junos'</span><span class="pi">:</span> <span class="c1"># The name of the proxy-minion. Can be any name, but it is a good idea to put the device's hostname.</span> <span class="pi">-</span> <span class="s">junos-pillar</span> <span class="c1"># This is the name of the previously created pillar file. Note the `.sls` suffix is not specified.</span> <span class="s1">'</span><span class="s">NX-OS'</span><span class="pi">:</span> <span class="pi">-</span> <span class="s">nxos-pillar</span> <span class="s1">'</span><span class="s">IOS-XE'</span><span class="pi">:</span> <span class="pi">-</span> <span class="s">ios-pillar</span> </code></pre></div></div> <p>It is time to start the <code class="language-plaintext highlighter-rouge">proxy-minion</code> service for each host. For that, use the following commands:</p> <blockquote> <p><strong>NOTE</strong> <br /> The value passed to the <code class="language-plaintext highlighter-rouge">--proxyid</code> argument should match the name of the proxy-minion defined in the <code class="language-plaintext highlighter-rouge">top.sls</code> file.</p> </blockquote> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt-proxy <span class="nt">--proxyid</span><span class="o">=</span>Junos <span class="nt">-d</span> <span class="nv">$ </span><span class="nb">sudo </span>salt-proxy <span class="nt">--proxyid</span><span class="o">=</span>NX-OS <span class="nt">-d</span> <span class="nv">$ </span><span class="nb">sudo </span>salt-proxy <span class="nt">--proxyid</span><span class="o">=</span>IOS-XE <span class="nt">-d</span> </code></pre></div></div> <p>Finally, if everything was done properly, you should see three unaccepted keys after you issue the <code class="language-plaintext highlighter-rouge">sudo salt-key --list-all</code> command. These are the requests that the proxy services made to register the proxy-minions against the master node.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt-key <span class="nt">--list-all</span> Accepted Keys: Denied Keys: Unaccepted Keys: IOS-XE Junos NX-OS Rejected Keys: </code></pre></div></div> <p>To accept all keys at once, issue <code class="language-plaintext highlighter-rouge">sudo salt-key --accept-all</code> command. When prompted press <code class="language-plaintext highlighter-rouge">y</code>.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt-key <span class="nt">--accept-all</span> The following keys are going to be accepted: Unaccepted Keys: IOS-XE Junos NX-OS Proceed? <span class="o">[</span>n/Y] y Key <span class="k">for </span>minion IOS-XE accepted. Key <span class="k">for </span>minion Junos accepted. Key <span class="k">for </span>minion NX-OS accepted. </code></pre></div></div> <h2 id="saltstack-and-ansible-ad-hoc-commands">SaltStack and Ansible Ad Hoc Commands</h2> <p>You made it to the fun part of the blog. Here you will start executing some ad hoc commands against the networking devices. To start, you will need to install some Python packages on both Salt and Ansible hosts. Using <code class="language-plaintext highlighter-rouge">sudo pip install netmiko junos-eznc jxmlease yamlordereddictloader napalm</code> command install the packages.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>pip <span class="nb">install </span>netmiko junos-eznc jxmlease yamlordereddictloader napalm Collecting napalm Downloading napalm-3.3.1-py2.py3-none-any.whl <span class="o">(</span>256 kB<span class="o">)</span> |████████████████████████████████| 256 kB 91 kB/s Collecting junos-eznc Downloading junos_eznc-2.6.1-py2.py3-none-any.whl <span class="o">(</span>195 kB<span class="o">)</span> |████████████████████████████████| 195 kB 113 kB/s <span class="o">&gt;&gt;&gt;</span> Trimmed <span class="k">for </span>brevity <span class="o">&lt;&lt;&lt;</span> </code></pre></div></div> <p>Now you can run <code class="language-plaintext highlighter-rouge">sudo salt "*" test.ping</code> command on the <code class="language-plaintext highlighter-rouge">salt-master</code></p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt <span class="s2">"*"</span> test.ping IOS-XE: True Junos: True NX-OS: True </code></pre></div></div> <p>The Ansible counterpart command will be: <code class="language-plaintext highlighter-rouge">ansible -i inv.ini base -m ping</code>.</p> <p>Now let’s take a look at the grains that salt master gathers from the devices. The <code class="language-plaintext highlighter-rouge">sudo salt "*" grains.items</code> command will reveal the below information. Similarly, you can do the same thing from the Ansible host using <code class="language-plaintext highlighter-rouge">ansible -m ios_facts -i inv.ini IOS-XE</code>, <code class="language-plaintext highlighter-rouge">ansible -m nxos_facts -i inv.ini NX-OS</code>, and <code class="language-plaintext highlighter-rouge">ansible -m junos_facts -i inv.ini Junos</code> commands.</p> <blockquote> <p><strong>NOTE</strong> <br /> Unfortunately, <code class="language-plaintext highlighter-rouge">netmiko</code> does not pull grain information from the devices, but hopefully in the future this gets fixed.</p> </blockquote> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Junos: <span class="nt">----------</span> cpuarch: x86_64 cwd: / dns: <span class="nt">----------</span> domain: ip4_nameservers: - 192.168.65.5 ip6_nameservers: nameservers: - 192.168.65.5 options: search: sortlist: fqdns: gpus: hwaddr_interfaces: <span class="nt">----------</span> <span class="nb">id</span>: Junos <span class="o">&lt;&lt;&lt;</span> TRIMMED <span class="o">&gt;&gt;&gt;</span> NX-OS: <span class="nt">----------</span> cpuarch: x86_64 cwd: / dns: <span class="nt">----------</span> domain: ip4_nameservers: - 192.168.65.5 ip6_nameservers: nameservers: - 192.168.65.5 options: search: sortlist: fqdns: gpus: hwaddr_interfaces: <span class="nt">----------</span> <span class="nb">id</span>: NX-OS <span class="o">&lt;&lt;&lt;</span> TRIMMED <span class="o">&gt;&gt;&gt;</span> IOS-XE: <span class="nt">----------</span> cpuarch: x86_64 cwd: / dns: <span class="nt">----------</span> domain: ip4_nameservers: - 192.168.65.5 ip6_nameservers: nameservers: - 192.168.65.5 options: search: sortlist: fqdns: gpus: hwaddr_interfaces: <span class="nt">----------</span> <span class="nb">id</span>: IOS-XE <span class="o">&lt;&lt;&lt;</span> TRIMMED <span class="o">&gt;&gt;&gt;</span> </code></pre></div></div> <p>Below data were gathered to demonstrate what grains will look like if different <code class="language-plaintext highlighter-rouge">proxytype</code>s were used:</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt <span class="s2">"IOS-XE"</span> napalm_net.facts IOS-XE: <span class="nt">----------</span> comment: out: <span class="nt">----------</span> fqdn: IOS-XE.local.host <span class="nb">hostname</span>: IOS-XE interface_list: - GigabitEthernet1 - GigabitEthernet2 - GigabitEthernet3 - GigabitEthernet4 model: CSR1000V os_version: Virtual XE Software <span class="o">(</span>X86_64_LINUX_IOSD-UNIVERSALK9-M<span class="o">)</span>, Version 16.12.3, RELEASE SOFTWARE <span class="o">(</span>fc5<span class="o">)</span> serial_number: ABC123ABC123 <span class="nb">uptime</span>: 5820 vendor: Cisco result: True <span class="nv">$ </span><span class="nb">sudo </span>salt <span class="s2">"NX-OS"</span> nxos.grains NX-OS: <span class="nt">----------</span> hardware: <span class="nt">----------</span> Device name: NX-OS bootflash: 4287040 kB plugins: - Core Plugin - Ethernet Plugin software: <span class="nt">----------</span> BIOS: version BIOS compile <span class="nb">time</span>: NXOS compile <span class="nb">time</span>: 12/22/2019 2:00:00 <span class="o">[</span>12/22/2019 14:00:37] NXOS image file is: bootflash:///nxos.9.3.3.bin <span class="nv">$ </span><span class="nb">sudo </span>salt <span class="s2">"Junos"</span> junos.facts Junos: <span class="nt">----------</span> facts: <span class="nt">----------</span> 2RE: False HOME: /var/home/admin RE0: <span class="nt">----------</span> last_reboot_reason: Router rebooted after a normal shutdown. mastership_state: master model: VSRX RE status: Testing up_time: 9 minutes, 20 seconds RE1: None RE_hw_mi: False current_re: - master - fpc0 - node - fwdd - member - pfem - re0 - fpc0.pic0 domain: None fqdn: Junos <span class="nb">hostname</span>: Junos <span class="o">&gt;&gt;&gt;</span> Trimmed <span class="k">for </span>brevity <span class="o">&lt;&lt;&lt;</span> </code></pre></div></div> <p>You have probably noticed that for the example data, I used different modules like <code class="language-plaintext highlighter-rouge">junos.facts</code> or <code class="language-plaintext highlighter-rouge">nxos.grains</code>. So why can’t we use the vendor-specific modules? This is due to the <code class="language-plaintext highlighter-rouge">proxytype</code> assigned to the device. Even though the <code class="language-plaintext highlighter-rouge">salt "Junos" junos.facts</code> is a correct command, Salt will check whether it is associated with <code class="language-plaintext highlighter-rouge">proxytype: netmiko</code> and, if not, it will report that it is not available for use.</p> <p>To understand the syntax of the <code class="language-plaintext highlighter-rouge">salt</code> shell command, I would refer to the image below. The <code class="language-plaintext highlighter-rouge">salt</code> allows for the commands to be executed on remote systems (minions) in parallel. The documentation for all supported salt modules can be found <a href="https://docs.saltproject.io/en/latest/ref/modules/all/index.html">here</a>.</p> <p><img src="../../../static/images/blog_posts/saltstack_with_ansible/02.png" alt="Salt Command Syntax" /></p> <p>Here are some more commands for you to try:</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>salt <span class="s2">"Junos"</span> netmiko.send_command <span class="s2">"show version"</span> <span class="nb">sudo </span>salt <span class="nt">--out</span> json <span class="s2">"IOS-XE"</span> netmiko.send_command <span class="s2">"show ip interface brief"</span> <span class="nv">use_textfsm</span><span class="o">=</span>True <span class="nb">sudo </span>salt <span class="s2">"NX-OS"</span> netmiko.send_config <span class="nv">config_commands</span><span class="o">=</span><span class="s2">"['hostname NX-OS-SALT']"</span> </code></pre></div></div> <h2 id="saltstack-state-files-and-ansible-playbooks">SaltStack State Files and Ansible Playbooks</h2> <p>Ad hoc commands are very useful for checking or obtaining information from multiple devices, but they are not practical because they can’t act upon that information the way SaltState files (formulas) can. Similarly, the Ansible Playbooks are used.</p> <p>You can start by creating a new pillar file, which should contain NTP server information. The file should be placed in the <code class="language-plaintext highlighter-rouge">/opt/salt/base/pillar/</code> folder, named as <code class="language-plaintext highlighter-rouge">ntp_servers.sls</code> and contain the following data:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">ntp_servers</span><span class="pi">:</span> <span class="pi">-</span> <span class="s">172.29.224.1</span> </code></pre></div></div> <p>Next, the <code class="language-plaintext highlighter-rouge">top.sls</code> file needs to be updated with instructions permitting all proxy-minions to use the information contained in the <code class="language-plaintext highlighter-rouge">ntp_servers.sls</code> file.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> /opt/salt/base/pillar/top.sls base: <span class="c"># Environment name</span> <span class="s1">'Junos'</span>: <span class="c"># proxy-minion name</span> - junos-pillar <span class="c"># pillar file</span> <span class="s1">'IOS-XE'</span>: - ios-pillar <span class="s1">'NX-OS'</span>: - nxos-pillar <span class="s1">'*'</span>: <span class="c"># All hosts will match.</span> - ntp_servers </code></pre></div></div> <p>Now you need to refresh the pillar information by using <code class="language-plaintext highlighter-rouge">sudo salt "*" saltutil.refresh_pillar</code> command.</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IOS-XE: True Junos: True NX-OS: True </code></pre></div></div> <p>Confirm that information is available to all three hosts with <code class="language-plaintext highlighter-rouge">sudo salt "*" pillar.data</code> or <code class="language-plaintext highlighter-rouge">sudo salt "*" pillar.items</code> commands:</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo </span>salt <span class="s2">"*"</span> pillar.data IOS-XE: <span class="nt">----------</span> ntp_servers: - 172.29.224.1 proxy: <span class="nt">----------</span> driver: ios host: 172.29.224.101 password: admin123 provider: napalm_base proxytype: napalm username: admin NX-OS: <span class="nt">----------</span> ntp_servers: - 172.29.224.1 proxy: <span class="nt">----------</span> connection: ssh host: 172.29.224.102 key_accept: True password: admin123 prompt_name: NX-OS proxytype: nxos username: admin Junos: <span class="nt">----------</span> ntp_servers: - 172.29.224.1 proxy: <span class="nt">----------</span> host: 172.29.224.103 password: admin123 port: 830 proxytype: junos username: admin </code></pre></div></div> <p>Next, you need to create a Jinja template for NTP configuration. The <code class="language-plaintext highlighter-rouge">IOS-XE</code> and <code class="language-plaintext highlighter-rouge">NX-OS</code> devices share the same configuration syntax, so all you need to do is to check the ID value stored in <code class="language-plaintext highlighter-rouge">grains</code> and identify the <code class="language-plaintext highlighter-rouge">Junos</code> device and, based on that, construct proper configuration syntax for it. Name the file as <code class="language-plaintext highlighter-rouge">ntp_config.set</code> and store it in the <code class="language-plaintext highlighter-rouge">/opt/salt/base/root/</code> folder. The contents of the file should look like this:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">{</span><span class="err">%</span> <span class="nv">set ntp = pillar</span><span class="pi">[</span><span class="s1">'</span><span class="s">ntp_servers'</span><span class="pi">][</span><span class="nv">0</span><span class="pi">]</span> <span class="err">%</span><span class="pi">}</span> <span class="pi">{</span><span class="err">%</span> <span class="nv">set device_type = grains</span><span class="pi">[</span><span class="s1">'</span><span class="s">id'</span><span class="pi">]</span> <span class="err">%</span><span class="pi">}</span> <span class="pi">{</span><span class="err">%</span> <span class="nv">if device_type == "Junos" %</span><span class="pi">}</span> <span class="s">set system ntp server {{ ntp }}</span> <span class="pi">{</span><span class="err">%</span> <span class="nv">else %</span><span class="pi">}</span> <span class="s">ntp server {{ ntp }}</span> <span class="pi">{</span><span class="err">%</span> <span class="nv">endif %</span><span class="pi">}</span> </code></pre></div></div> <p>Finally, you need to create the Salt State file (aka formula) and specify what it should do. The file needs to be located in the <code class="language-plaintext highlighter-rouge">/opt/salt/base/root/</code> folder and named as <code class="language-plaintext highlighter-rouge">update_ntp.sls</code> with the following contents:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">UPDATE NTP CONFIGURATION</span><span class="pi">:</span> <span class="c1"># The ID name of the state. Should be unique if multiple IDs are present in the formula.</span> <span class="s">module.run</span><span class="pi">:</span> <span class="c1"># state function to run module functions</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">netmiko.send_config</span> <span class="c1"># Name of the module function</span> <span class="pi">-</span> <span class="na">config_file</span><span class="pi">:</span> <span class="s">salt://ntp_config.set</span> <span class="c1"># Argument that needs to be passed to the module function. The salt:// will convert to `/opt/salt/base/root/` path.</span> </code></pre></div></div> <p>You might ask: “Why did we use <code class="language-plaintext highlighter-rouge">module.run</code> to run <code class="language-plaintext highlighter-rouge">netmiko.send_config</code>?” That’s a great question! SaltStack separates modules by their purposes, and <a href="https://docs.saltproject.io/en/latest/ref/modules/all/index.html">execution modules</a> cannot be used to maintain a state on the minion. For that, the <a href="https://docs.saltproject.io/en/latest/ref/states/all/index.html">state modules</a> exist. Since <code class="language-plaintext highlighter-rouge">netmiko</code> does not have a state module, the <a href="https://docs.saltproject.io/en/latest/ref/states/all/salt.states.module.html"><code class="language-plaintext highlighter-rouge">module.run</code></a> allows you to run execution modules in the Salt State file.</p> <p>Okay, let’s compare the state file with an Ansible playbook. This is how a playbook that would do the same thing would look:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">UPDATE NTP CONFIGURATION</span> <span class="na">hosts</span><span class="pi">:</span> <span class="s">base</span> <span class="na">gather_facts</span><span class="pi">:</span> <span class="s">no</span> <span class="na">vars</span><span class="pi">:</span> <span class="na">pillar</span><span class="pi">:</span> <span class="na">ntp_servers</span><span class="pi">:</span> <span class="pi">-</span> <span class="s">172.29.224.1</span> <span class="na">tasks</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">5 - UPDATE NTP CONFIGURATION FOR JUNOS</span> <span class="na">junos_config</span><span class="pi">:</span> <span class="na">src</span><span class="pi">:</span> <span class="s">ntp_config.set</span> <span class="na">vars</span><span class="pi">:</span> <span class="na">grains</span><span class="pi">:</span> <span class="na">id</span><span class="pi">:</span> <span class="s">Junos</span> <span class="na">when</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">==</span><span class="nv"> </span><span class="s">'junos'"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">10 - UPDATE NTP CONFIGURATION FOR NX-OS</span> <span class="na">nxos_config</span><span class="pi">:</span> <span class="na">src</span><span class="pi">:</span> <span class="s">ntp_config.set</span> <span class="na">vars</span><span class="pi">:</span> <span class="na">grains</span><span class="pi">:</span> <span class="na">id</span><span class="pi">:</span> <span class="s">NX-OS</span> <span class="na">when</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">==</span><span class="nv"> </span><span class="s">'nxos'"</span> <span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">15 - UPDATE NTP CONFIGURATION FOR IOS</span> <span class="na">ios_config</span><span class="pi">:</span> <span class="na">src</span><span class="pi">:</span> <span class="s">ntp_config.set</span> <span class="na">vars</span><span class="pi">:</span> <span class="na">grains</span><span class="pi">:</span> <span class="na">id</span><span class="pi">:</span> <span class="s">IOS-XE</span> <span class="na">when</span><span class="pi">:</span> <span class="s2">"</span><span class="s">ansible_network_os</span><span class="nv"> </span><span class="s">==</span><span class="nv"> </span><span class="s">'ios'"</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">salt "*" state.apply update_ntp</code> command will apply the configuration:</p> <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>salt <span class="s2">"*"</span> state.apply update_ntp IOS: <span class="nt">----------</span> ID: UPDATE NTP CONFIGURATION Function: module.run Name: netmiko.send_config Result: True Comment: Module <span class="k">function </span>netmiko.send_config executed Started: 02:26:17.021615 Duration: 858.925 ms Changes: <span class="nt">----------</span> ret: configure terminal Enter configuration commands, one per line. End with CNTL/Z. IOS-XE<span class="o">(</span>config<span class="o">)</span><span class="c">#ntp server 172.29.224.1</span> IOS-XE<span class="o">(</span>config<span class="o">)</span><span class="c">#end</span> IOS-XE# Summary <span class="k">for </span>IOS <span class="nt">------------</span> Succeeded: 1 <span class="o">(</span><span class="nv">changed</span><span class="o">=</span>1<span class="o">)</span> Failed: 0 <span class="nt">------------</span> Total states run: 1 Total run <span class="nb">time</span>: 858.925 ms NXOS: <span class="nt">----------</span> ID: UPDATE NTP CONFIGURATION Function: module.run Name: netmiko.send_config Result: True Comment: Module <span class="k">function </span>netmiko.send_config executed Started: 02:26:17.074178 Duration: 5612.827 ms Changes: <span class="nt">----------</span> ret: configure terminal Enter configuration commands, one per line. End with CNTL/Z. NX-OS<span class="o">(</span>config<span class="o">)</span><span class="c"># ntp server 172.29.224.1</span> NX-OS<span class="o">(</span>config<span class="o">)</span><span class="c"># end</span> NX-OS# Summary <span class="k">for </span>NXOS <span class="nt">------------</span> Succeeded: 1 <span class="o">(</span><span class="nv">changed</span><span class="o">=</span>1<span class="o">)</span> Failed: 0 <span class="nt">------------</span> Total states run: 1 Total run <span class="nb">time</span>: 5.613 s SRX: <span class="nt">----------</span> ID: UPDATE NTP CONFIGURATION Function: module.run Name: netmiko.send_config Result: True Comment: Module <span class="k">function </span>netmiko.send_config executed Started: 02:26:17.083703 Duration: 11961.207 ms Changes: <span class="nt">----------</span> ret: configure Entering configuration mode <span class="o">[</span>edit] admin@Junos# <span class="nb">set </span>system ntp server 172.29.224.1 <span class="o">[</span>edit] admin@Junos# <span class="nb">exit </span>configuration-mode Exiting configuration mode admin@Junos&gt; Summary <span class="k">for </span>SRX <span class="nt">------------</span> Succeeded: 1 <span class="o">(</span><span class="nv">changed</span><span class="o">=</span>1<span class="o">)</span> Failed: 0 <span class="nt">------------</span> Total states run: 1 Total run <span class="nb">time</span>: 11.961 s </code></pre></div></div> <p>SaltStack is a very powerful tool, and covering all of its aspects, such as reactors and syslog engines, here would be impossible. But I hope that the information presented above was enough for you to get started learning it. If you have any questions, feel free to leave a comment below, and I will do my best to answer them!</p> <p>-Armen</p>Armen MartirosyanHave you seen or taken the 2020 NetDevOps survey? If not, don’t worry, it is held yearly from Sept 30th to Oct 31st, so you will get your chance. To quickly recap, the survey’s intention is to understand which tools are most commonly used by Network Operators and Engineers to automate their day-to-day jobs. It is a very interesting survey, and I encourage you to read it.Contributing to the Nautobot Documentation2021-07-08T00:00:00+00:002021-07-08T00:00:00+00:00https://blog.networktocode.com/post/contributing-to-nautobot-docs<p>Useful documentation is an important component of Nautobot’s usability. To that point, this week’s blog will focus on how the NTC community can contribute new content to the docs or submit corrections for the docs when they find a flaw.</p> <p>Contributing to the Nautobot project via code or documentation has many benefits:</p> <ul> <li>It shows leadership</li> <li>It is a public example showing your commitment to putting in your own time and effort to see that things are done right</li> <li>The community will appreciate your help in making Nautobot more usable</li> <li>It saves others time by correcting the docs or by adding helpful new modules to the docs</li> <li>It makes a nice entry on a resumé (just sayin’ . . .)</li> </ul> <h2 id="create-an-issue">Create an Issue</h2> <p>The <a href="https://nautobot.readthedocs.io/en/latest/development/getting-started/#submitting-pull-requests">development docs state this clearly</a>, but it bears repeating here:</p> <blockquote> <p><em>Pull requests are entertained only for accepted issues. If an issue you want to work on hasn’t been approved by a maintainer yet, it’s best to avoid risking your time and effort on a change that might not be accepted.</em></p> </blockquote> <p>If you are considering submitting an addition to the docs and/or a correction, be sure to open an issue for the change prior to investing the time to create a PR.</p> <p>You can open an issue <a href="https://github.com/nautobot/nautobot/issues/new/choose">here</a>:</p> <ul> <li>Select <code class="language-plaintext highlighter-rouge">Bug Report</code> if you have found an error in the docs</li> <li>Select <code class="language-plaintext highlighter-rouge">Feature Request</code> if you would like to expand the docs by adding a section or otherwise add content</li> </ul> <p>In the issue, be sure to describe the edits/adds/deletions you propose and describe why those changes are needed and how they are helpful.</p> <p>See the Nautobot wiki page on <a href="https://github.com/nautobot/nautobot/wiki/Work-Intake-&amp;-Issue-Management">Work Intake &amp; Issue Management</a> for information on what to expect after submitting an issue.</p> <h2 id="create-your-fork">Create Your Fork</h2> <p>Once your issue is accepted, it’s time to set up your dev environment. You will need to set up a fork of the Nautobot repository, if you don’t have one already.</p> <p>On the <a href="https://github.com/nautobot/nautobot">main page for the Nautobot repository</a>, look in the top-right corner for the <code class="language-plaintext highlighter-rouge">Fork</code> button and click on it. Select which of your accounts you want to place the fork in.</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/11-fork-it-fork-it-good.png" alt="" /></p> <h2 id="clone-your-repository">Clone Your Repository</h2> <p>Next, go to your repository page to access the options for how to clone your repository locally. The example below shows selection of the <code class="language-plaintext highlighter-rouge">ssh</code> method to retrieve the repository.</p> <ol> <li>Click on the <code class="language-plaintext highlighter-rouge">Code</code> button</li> <li>Select how you want to clone the respository (SSH option shown)</li> </ol> <p><img src="../../../static/images/blog_posts/contrib_to_docs/12-get_repo_link.png" alt="" /></p> <blockquote> <p>TIP: Instructions to set up SSH keys in your GitHub account, which enables use of the <code class="language-plaintext highlighter-rouge">ssh</code> clone method, can be found <a href="https://docs.github.com/en/enterprise-server@3.0/github/authenticating-to-github/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account">here</a>.</p> </blockquote> <p>With your fork ready, go to the CLI (or your favorite IDE) and clone your Nautobot repository fork. The example below shows use of the <code class="language-plaintext highlighter-rouge">ssh</code> resource we copied in the picture above.</p> <blockquote> <p>NOTE: This requires <code class="language-plaintext highlighter-rouge">git</code> to be installed on your system</p> </blockquote> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% % git clone git@github.com:tim-fiola/nautobot.git Cloning into 'nautobot'... remote: Enumerating objects: 65038, done. remote: Counting objects: 100% (330/330), done. remote: Compressing objects: 100% (170/170), done. remote: Total 65038 (delta 217), reused 262 (delta 160), pack-reused 64708 Receiving objects: 100% (65038/65038), 38.94 MiB | 24.31 MiB/s, done. Resolving deltas: 100% (51959/51959), done. % </code></pre></div></div> <p>Once the download is complete, change to the <code class="language-plaintext highlighter-rouge">nautobot</code> directory. Do a <code class="language-plaintext highlighter-rouge">git remote -v</code> to verify your origin.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd nautobot % git remote -v origin git@github.com:tim-fiola/nautobot.git (fetch) origin git@github.com:tim-fiola/nautobot.git (push) % % </code></pre></div></div> <p>You will also want to track the official Nautobot repository, so you can pull in any changes.</p> <p>To do this, use the <code class="language-plaintext highlighter-rouge">git remote add upstream git@github.com:nautobot/nautobot.git</code> command. This will add the Nautobot respository as the remote upstream.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git remote add upstream git@github.com:nautobot/nautobot.git % % git remote -v origin git@github.com:tim-fiola/nautobot.git (fetch) origin git@github.com:tim-fiola/nautobot.git (push) upstream git@github.com:nautobot/nautobot.git (fetch) upstream git@github.com:nautobot/nautobot.git (push) % </code></pre></div></div> <p>Switch to the <code class="language-plaintext highlighter-rouge">develop</code> branch and then pull from the upstream’s <code class="language-plaintext highlighter-rouge">develop</code> branch. This will ensure your local <code class="language-plaintext highlighter-rouge">develop</code> branch is up to date and lessen the chances of merge conflicts.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% % git checkout develop Already on 'develop' Your branch is up to date with 'origin/develop'. % % git pull upstream develop . . . ----- &lt; snip &gt; ----- . . . scripts/cibuild.sh | 5 +++++ 40 files changed, 626 insertions(+), 156 deletions(-) create mode 100644 examples/okta/README.md create mode 100644 examples/okta/group_sync.py delete mode 100644 nautobot/project-static/jquery-ui-1.12.1/package.json % </code></pre></div></div> <h2 id="create-your-branch">Create Your Branch</h2> <p>Now create your local branch where you will perform your edits.</p> <blockquote> <p>TIP: The command shown below will create a new branch in your local installation. This new branch will need to be pushed to your origin for tracking; this guide will show that step when the time comes.</p> </blockquote> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% % git checkout -b tim-fiola-doc-blog Switched to a new branch 'tim-fiola-doc-blog' % </code></pre></div></div> <p>Verify your branch is clean:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git status On branch tim-fiola-doc-blog nothing to commit, working tree clean % </code></pre></div></div> <p>In our example, we created the new branch locally and so the branch does not exist in our origin yet.</p> <p>We will use <code class="language-plaintext highlighter-rouge">git push --set-upstream origin &lt;local-branch-name&gt;</code> to add the branch to our origin. Our local branch will then track this upstream branch.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git push --set-upstream origin tim-fiola-doc-blog Enumerating objects: 204, done. Counting objects: 100% (180/180), done. Delta compression using up to 12 threads Compressing objects: 100% (63/63), done. Writing objects: 100% (120/120), 22.95 KiB | 11.47 MiB/s, done. Total 120 (delta 91), reused 83 (delta 55), pack-reused 0 remote: Resolving deltas: 100% (91/91), completed with 37 local objects. remote: remote: Create a pull request for 'tim-fiola-doc-blog' on GitHub by visiting: remote: https://github.com/tim-fiola/nautobot/pull/new/tim-fiola-doc-blog remote: To github.com:tim-fiola/nautobot.git * [new branch] tim-fiola-doc-blog -&gt; tim-fiola-doc-blog Branch 'tim-fiola-doc-blog' set up to track remote branch 'tim-fiola-doc-blog' from 'origin'. % </code></pre></div></div> <h2 id="install-poetry">Install Poetry</h2> <p>Nautobot uses Poetry, which will transparently create a virtualenv for you, automatically install all dependencies required for Nautobot to operate, and also install the <code class="language-plaintext highlighter-rouge">nautobot-server</code> CLI command that you will utilize to interact with Nautobot from here on out. More info on Poetry can be found in the <a href="https://nautobot.readthedocs.io/en/latest/development/getting-started/#working-with-poetry">Nautobot docs</a>.</p> <p>Use the following command to install Poetry: <code class="language-plaintext highlighter-rouge">curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -</code></p> <blockquote> <p>TIP: you may have to use <code class="language-plaintext highlighter-rouge">| python3 -</code> instead of <code class="language-plaintext highlighter-rouge">| python -</code> in that curl command, depending on your Python setup.</p> </blockquote> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3 - Retrieving Poetry metadata # Welcome to Poetry! This will download and install the latest version of Poetry, a dependency and package manager for Python. It will add the `poetry` command to Poetry's bin directory, located at: /home/tim/.local/bin You can uninstall at any time by executing this script with the --uninstall option, and these changes will be reverted. You are installing 1.1.7. When using the current installer, this version does not support updating using the 'self update' command. Please use 1.2.0a1 or later. Installing Poetry (1.1.7): Done Poetry (1.1.7) is installed now. Great! To get started you need Poetry's bin directory (/home/tim/.local/bin) in your `PATH` environment variable. Add `export PATH="/home/tim/.local/bin:$PATH"` to your shell configuration file. Alternatively, you can call Poetry explicitly with `/home/tim/.local/bin/poetry`. You can test that everything is set up by executing: `poetry --version` $ </code></pre></div></div> <blockquote> <p>NOTE: As of the writing of this article, Poetry had just switched to a new install script (the one used above). I discovered this when I installed Poetry on my local machine using the now deprecated curl install script (<code class="language-plaintext highlighter-rouge">curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -</code>) and saw the deprecation message. More info on this change can be found <a href="https://python-poetry.org/blog/announcing-poetry-1.2.0a1/">here, on the Poetry webpage</a>.</p> </blockquote> <h2 id="use-poetry-to-install-nautobot-dependencies">Use Poetry to Install Nautobot Dependencies</h2> <p>The repository you cloned has a <code class="language-plaintext highlighter-rouge">nautobot</code> directory. Open a <strong>new</strong> shell, and change to that directory.</p> <p>Execute <code class="language-plaintext highlighter-rouge">poetry install</code> from the CLI to install dependencies.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd nautobot % poetry install Installing dependencies from lock file Package operations: 102 installs, 0 updates, 0 removals • Installing six (1.16.0) • Installing certifi (2021.5.30) • Installing chardet (4.0.0) . . . ----- &lt; snip &gt; ----- . . . Installing the current project: nautobot (1.0.3-beta.1) % </code></pre></div></div> <h2 id="serve-the-docs">Serve the Docs</h2> <p>Your environment is now set up to allow you to edit the documentation. We highly recommend serving the documents locally while you edit. Serving the docs locally allows you to see the fully rendered documents and what your changes will look like.</p> <p>First, activate Poetry’s virtual environment (venv) by running <code class="language-plaintext highlighter-rouge">poetry shell</code> in the <code class="language-plaintext highlighter-rouge">nautobot</code> directory:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% pwd (tim-fiola-doc-blog)nautobot /Users/timothyfiola/nautobot_blog/nautobot % poetry shell (tim-fiola-doc-blog)nautobot Spawning shell within /Users/timothyfiola/Library/Caches/pypoetry/virtualenvs/nautobot-yNEcye0N-py3.8 Restored session: Thu Jul 1 15:39:50 MDT 2021 % . /Users/timothyfiola/Library/Caches/pypoetry/virtualenvs/nautobot-yNEcye0N-py3.8/bin/activate (tim-fiola-doc-blog)nautobot (nautobot-yNEcye0N-py3.8) % </code></pre></div></div> <p>Now, run the <code class="language-plaintext highlighter-rouge">mkdocs serve</code> command from within the virtual environment:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(nautobot-yNEcye0N-py3.9) % mkdocs serve INFO - Building documentation... WARNING - Config value: 'python'. Warning: Unrecognised configuration name: python INFO - Cleaning site directory INFO - The following pages exist in the docs directory, but are not included in the "nav" configuration: - installation/centos.md - installation/selinux-troubleshooting.md - installation/ubuntu.md - models/circuits/circuit.md - models/circuits/circuittermination.md . . . ----- &lt; snip &gt; ----- . . . - release-notes/index.md WARNING - Documentation file 'additional-features/jobs.md' contains a link to '../media/admin_ui_run_permission.png' which is not found in the documentation files. WARNING - Documentation file 'configuration/required-settings.md' contains a link to '.' which is not found in the documentation files. WARNING - Documentation file 'core-functionality/power.md' contains a link to '../media/power_distribution.png' which is not found in the documentation files. INFO - Documentation built in 2.67 seconds [I 210621 14:32:43 server:335] Serving on http://127.0.0.1:8001 INFO - Serving on http://127.0.0.1:8001 [I 210621 14:32:43 handlers:62] Start watching changes INFO - Start watching changes [I 210621 14:32:43 handlers:64] Start detecting changes INFO - Start detecting changes </code></pre></div></div> <blockquote> <p>Notice the <code class="language-plaintext highlighter-rouge">INFO - Start detecting changes</code> message in the terminal. This indicates that the server will detect changes and update the served docs as the changes are detected.</p> </blockquote> <p><strong>Keep this command running in the terminal.</strong> Go to <code class="language-plaintext highlighter-rouge">http://127.0.0.1:8001</code> to view the docs:</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/1-docs_being_served.png" alt="" /></p> <h2 id="example-doc-contribution">Example Doc Contribution</h2> <p>This example will show a fix for a documentation bug for a <a href="https://nautobot.readthedocs.io/en/latest/development/getting-started/#using-docker-with-invoke">section in the Development Environment</a> that needs a correction.</p> <blockquote> <p>NOTE: The Nautobot issue for this correction is <a href="https://github.com/nautobot/nautobot/issues/600">#600</a>.</p> </blockquote> <p>A screenshot of the docs prior to the fix is below, where it mentions <code class="language-plaintext highlighter-rouge">invoke createsuperuser</code>. It should also mention that, in a fresh install, it’s necessary to run <code class="language-plaintext highlighter-rouge">invoke migrate</code> <strong>prior</strong> to <code class="language-plaintext highlighter-rouge">invoke createsuperuser</code>.</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/2-fix-needed-here.png" alt="" /></p> <h2 id="locate-the-files-to-modify">Locate the File(s) to Modify</h2> <p>To help locate the file that should be modified, notice the URL for the page we want to modify has a path that ends with <code class="language-plaintext highlighter-rouge">development/getting-started/</code></p> <p><code class="language-plaintext highlighter-rouge">https://nautobot.readthedocs.io/en/latest/development/getting-started/</code></p> <p>Open another terminal window and go to the <code class="language-plaintext highlighter-rouge">nautobot</code> directory (be sure to keep the doc serve process running).</p> <p>To help see where we can access the page we want to modify, check out the <code class="language-plaintext highlighter-rouge">mkdocs.yml</code> page in the <code class="language-plaintext highlighter-rouge">nautobot</code> directory:</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/6-mkdocs-file.png" alt="" /></p> <p>View the <code class="language-plaintext highlighter-rouge">mkdocs.yml file</code>; there will be a top-tier section called <code class="language-plaintext highlighter-rouge">nav</code>. Within <code class="language-plaintext highlighter-rouge">nav</code> is a section called <code class="language-plaintext highlighter-rouge">Development</code>, and beneath that, <code class="language-plaintext highlighter-rouge">Getting Started</code>.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> % cat mkdocs.yml dev_addr: '127.0.0.1:8001' . . . ----- &lt; snip &gt; ----- . . . nav: - Introduction: 'index.md' . . . ----- &lt; snip &gt; ----- . . . - Development: - Introduction: 'development/index.md' - Getting Started: 'development/getting-started.md' - Best Practices: 'development/best-practices.md' - Style Guide: 'development/style-guide.md' - Extending Models: 'development/extending-models.md' - Application Registry: 'development/application-registry.md' - User Preferences: 'development/user-preferences.md' - Release Checklist: 'development/release-checklist.md' - Release Notes: - Version 1.0: 'release-notes/version-1.0.md' % </code></pre></div></div> <p>This matches closely to the <code class="language-plaintext highlighter-rouge">development/getting-started/</code> URL path and is likely where we’d want to get started.</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/3-navigate-to-file.png" alt="" /></p> <p>Navigate to the directory <code class="language-plaintext highlighter-rouge">docs/development</code>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% pwd (tim-fiola-doc-blog)nautobot /Users/timothyfiola/nautobot_blog/nautobot % ls -l total 392 -rw-r--r-- 1 timothyfiola staff 119 Jun 18 13:15 CHANGELOG.md -rw-r--r-- 1 timothyfiola staff 120 Jun 18 13:15 CONTRIBUTING.md -rw-r--r-- 1 timothyfiola staff 10174 Jun 18 13:15 LICENSE.txt -rw-r--r-- 1 timothyfiola staff 267 Jun 18 13:15 NOTICE -rw-r--r-- 1 timothyfiola staff 2475 Jun 18 13:15 README.md drwxr-xr-x 8 timothyfiola staff 256 Jun 18 13:15 development drwxr-xr-x 6 timothyfiola staff 192 Jun 18 13:15 docker lrwxr-xr-x 1 timothyfiola staff 13 Jun 18 13:15 docs -&gt; nautobot/docs drwxr-xr-x 5 timothyfiola staff 160 Jun 18 13:18 examples -rwxr-xr-x 1 timothyfiola staff 4160 Jun 18 13:15 install.sh -rw-r--r-- 1 timothyfiola staff 453 Jun 18 13:15 invoke.yml.example -rwxr-xr-x 1 timothyfiola staff 111 Jun 18 13:15 manage.py -rw-r--r-- 1 timothyfiola staff 4315 Jun 18 13:18 mkdocs.yml drwxr-xr-x 16 timothyfiola staff 512 Jun 18 13:28 nautobot -rw-r--r-- 1 timothyfiola staff 617 Jun 18 13:15 nautobot.code-workspace -rw-r--r-- 1 timothyfiola staff 117557 Jun 18 13:15 poetry.lock -rw-r--r-- 1 timothyfiola staff 3997 Jun 18 13:15 pyproject.toml drwxr-xr-x 4 timothyfiola staff 128 Jun 18 13:18 scripts -rw-r--r-- 1 timothyfiola staff 17630 Jun 18 13:15 tasks.py % cd docs/development/ docs/development % </code></pre></div></div> <h2 id="make-your-edits">Make Your Edits</h2> <p>We’ll edit the <code class="language-plaintext highlighter-rouge">nautobot/docs/development/getting-started.md</code> file, adding a <code class="language-plaintext highlighter-rouge">tip</code> admonishment that notifies the user of the requirement:</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/4-make-fix-in-file.png" alt="" /></p> <p>Save the file then go back to your browser, where you are serving the docs locally; you should see the change:</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/5-fix-rendered-locally.png" alt="" /></p> <blockquote> <p>NOTE: You will see the change rendered automatically only if you kept the process that was rendering the docs running in the terminal.</p> </blockquote> <p>If you look back at your terminal window where you initiated the docs to be served, you’ll see this message in the log after you make the change:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[I 210621 15:09:03 handlers:92] Reload 1 waiters: /Users/timothyfiola/nautobot_blog/nautobot/docs/development/getting-started.md INFO - Reload 1 waiters: /Users/timothyfiola/nautobot_blog/nautobot/docs/development/getting-started.md </code></pre></div></div> <p>This notifies you that the <code class="language-plaintext highlighter-rouge">mkdocs</code> server noticed a change in the repository, and so it automatically re-rendered the documentation.</p> <h2 id="make-the-commit">Make the Commit</h2> <p>Add the <code class="language-plaintext highlighter-rouge">getting-started.md</code> file to your git staging, then make the commit for your change:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git status On branch tim-fiola-doc-blog Changes not staged for commit: (use "git add &lt;file&gt;..." to update what will be committed) (use "git restore &lt;file&gt;..." to discard changes in working directory) modified: nautobot/docs/development/getting-started.md no changes added to commit (use "git add" and/or "git commit -a") % git add nautobot/docs/development/getting-started.md % % git status On branch tim-fiola-doc-blog Changes to be committed: (use "git restore --staged &lt;file&gt;..." to unstage) modified: nautobot/docs/development/getting-started.md % git commit -m 'made correction in docs' (tim-fiola-doc-blog)nautobot [tim-fiola-doc-blog 090cc0c1] made correction in docs 1 file changed, 3 insertions(+) % </code></pre></div></div> <h2 id="push-the-change">Push the Change</h2> <p>Now push your change to your origin branch:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git push Enumerating objects: 11, done. Counting objects: 100% (11/11), done. Delta compression using up to 12 threads Compressing objects: 100% (6/6), done. Writing objects: 100% (6/6), 592 bytes | 592.00 KiB/s, done. Total 6 (delta 5), reused 0 (delta 0), pack-reused 0 remote: Resolving deltas: 100% (5/5), completed with 5 local objects. To github.com:tim-fiola/nautobot.git a9ca3a2b..090cc0c1 tim-fiola-doc-blog -&gt; tim-fiola-doc-blog % </code></pre></div></div> <h2 id="create-a-pr">Create a PR</h2> <p>The commit in the prior section updated the branch in our forked repository. Now it’s time to submit a pull request (PR).</p> <ol> <li>Navigate to your forked Nautobot repository</li> <li>Click on the <em>branches</em> link</li> <li>On the branches screen, click on <code class="language-plaintext highlighter-rouge">Yours</code></li> </ol> <p><img src="../../../static/images/blog_posts/contrib_to_docs/10-navigate-to-branch.png" alt="" /></p> <p>Next, find the branch with your changes in the list and click on the <code class="language-plaintext highlighter-rouge">New pull request</code> button next to the branch name.</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/7-find-branch-and-pr-button.png" alt="" /></p> <p>Fill in the PR form. Be sure to link to the approved issue that your PR is fixing. You can do this by simply adding the issue number after the <code class="language-plaintext highlighter-rouge">### Fixes:#</code> text in the PR form. You will know that the issue reference succeeds when the auto-link shows, as in the picture below:</p> <p><img src="../../../static/images/blog_posts/contrib_to_docs/8-open-the-pr.png" alt="" /></p> <p>When you have filled out the PR form completely, submit the PR.</p> <h2 id="what-to-expect">What to Expect</h2> <p>Once you submit your PR, you can expect that there will be some conversation within it, including some suggestions from Nautobot’s core Developers. Keep your eyes open for updates to the PR so you can respond accordingly.</p> <p>Once everything is in order, the PR will be staged for a merge in the next release. The Nautobot Developers, Developer Advocates, and the community will appreciate your help in making Nautobot more usable!</p> <p>Thank you and have a great day!</p> <p>-Tim</p>Tim FiolaUseful documentation is an important component of Nautobot’s usability. To that point, this week’s blog will focus on how the NTC community can contribute new content to the docs or submit corrections for the docs when they find a flaw.Understanding the Ansible Project Packaging2021-07-06T00:00:00+00:002021-07-06T00:00:00+00:00https://blog.networktocode.com/post/understanding-the-ansible-project-packaging<p>Ansible released 3.x in February 2021, which continues the project’s recent announcement that the code base will be splitting. As the project continues the transition, it’s important to understand some of the new terminology and understand how it might affect your environment.</p> <p>The main difference is the philosophy on installation, i.e., installation separation of <code class="language-plaintext highlighter-rouge">core</code> and <code class="language-plaintext highlighter-rouge">community packages</code>. Ansible 3.0.0 is a <code class="language-plaintext highlighter-rouge">community package</code> that relies on <code class="language-plaintext highlighter-rouge">ansible-base</code> but comes natively with a subset of collections.</p> <p>On top of the project split there is also going to be a change in the Ansible core engine naming.</p> <ul> <li>Ansible core engine is called <code class="language-plaintext highlighter-rouge">ansible-base</code> which is new for version 2.10, and <code class="language-plaintext highlighter-rouge">ansible-core</code> with 2.11+.</li> </ul> <h2 id="ansible-3">Ansible 3</h2> <p>As mentioned previously, the installation of Ansible 3.x is considered a <code class="language-plaintext highlighter-rouge">community package</code> which will install <code class="language-plaintext highlighter-rouge">ansible-base</code> and <code class="language-plaintext highlighter-rouge">ansible 3.x</code>. The package list will no longer just show <code class="language-plaintext highlighter-rouge">ansible</code> as it has in the past.</p> <p>In a fresh virtual environment Ansible is installed.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-3) ~/ansible-testing ▶ pip install ansible Collecting ansible Downloading ansible-3.2.0.tar.gz (31.3 MB) |████████████████████████████████| 31.3 MB 25.7 MB/s Collecting ansible-base&lt;2.11,&gt;=2.10.7 Using cached ansible-base-2.10.7.tar.gz (5.7 MB) ---- output omitted ---- Successfully installed MarkupSafe-1.1.1 PyYAML-5.4.1 ansible-3.2.0 ansible-base-2.10.7 cffi-1.14.5 cryptography-3.4.7 jinja2-2.11.3 packaging-20.9 pycparser-2.20 pyparsing-2.4.7 </code></pre></div></div> <p>If we check out packages with <code class="language-plaintext highlighter-rouge">pip freeze | grep ansible</code>, the project split is evident. We have ansible 3.2.0 with its dependency of <code class="language-plaintext highlighter-rouge">ansible-base</code> pinned to specific versions <code class="language-plaintext highlighter-rouge">&gt;=2.10.6,&lt;2.11</code>.</p> <blockquote> <p>Note: 2.11 is when <code class="language-plaintext highlighter-rouge">ansible-base</code> is renamed to <code class="language-plaintext highlighter-rouge">ansible-core</code>.</p> </blockquote> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>▶ pip freeze | grep ansible ansible==3.2.0 ansible-base==2.10.7 ▶ pip show ansible | grep Requires Requires: ansible-base </code></pre></div></div> <p>Since Ansible 3.x is the community package to see the native collections, execute <code class="language-plaintext highlighter-rouge">ansible-galaxy collection list</code>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-3) ~/ansible-testing ▶ ansible-galaxy collection list # /Users/ntc/ansible-testing/ansible-3/lib/python3.9/site-packages/ansible_collections Collection Version ----------------------------- ------- amazon.aws 1.4.1 ansible.netcommon 1.5.0 ansible.posix 1.2.0 ansible.utils 2.0.2 ansible.windows 1.4.0 arista.eos 1.3.0 ---- omitted ---- vyos.vyos 1.1.1 wti.remote 1.0.1 </code></pre></div></div> <h2 id="ansible-4">Ansible 4</h2> <p>Ansible 4.0.0 was released in May 2021. It is similar to Ansible 3.x, i.e., it will be a <code class="language-plaintext highlighter-rouge">community package</code> that includes a subset of collections, the main difference will be Ansible 4.x dependency will be <code class="language-plaintext highlighter-rouge">ansible-core</code> which is <code class="language-plaintext highlighter-rouge">&gt;=2.11,&lt;2.12</code>.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-4) ~/ansible-testing ▶ pip install ansible Collecting ansible Downloading ansible-4.1.0.tar.gz (34.0 MB) |████████████████████████████████| 34.0 MB 14.3 MB/s Collecting ansible-core&lt;2.12,&gt;=2.11.1 Downloading ansible-core-2.11.1.tar.gz (6.1 MB) |████████████████████████████████| 6.1 MB 9.5 MB/s ---- output omitted ---- </code></pre></div></div> <p>Checking PIP freeze we see <code class="language-plaintext highlighter-rouge">ansible-core</code> was installed.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>▶ pip freeze | grep ansible ansible==4.1.0 ansible-core==2.11.1 ▶ pip show ansible | grep Requires Requires: ansible-core </code></pre></div></div> <p>After the install, running <code class="language-plaintext highlighter-rouge">ansible-galaxy collection list</code> shows the collections that came natively with the <code class="language-plaintext highlighter-rouge">community package</code>. This is similar to Ansible 3.x.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-4) ~/ansible-testing ▶ ansible-galaxy collection list # /Users/ntc/ansible-testing/ansible-4/lib/python3.9/site-packages/ansible_collections Collection Version ----------------------------- ------- amazon.aws 1.5.0 ansible.netcommon 2.1.0 ansible.posix 1.2.0 ansible.utils 2.2.0 ansible.windows 1.6.0 arista.eos 2.1.2 ---- omitted ---- vyos.vyos 2.3.0 wti.remote 1.0.1 </code></pre></div></div> <h2 id="collection-flexibility">Collection Flexibility</h2> <p>For anyone that has worked in or works in a locked down environment, the Ansible project split offers additional flexibility that allows a user to install <code class="language-plaintext highlighter-rouge">ansible-base</code>(3.0) or <code class="language-plaintext highlighter-rouge">ansible-core</code>(4.0) and then <strong>only</strong> install the collections that are needed. For example, perhaps the environment only has Cisco IOS—it is possible to run the core Ansible and then install only the Cisco IOS collection.</p> <p>To demonstrate that functionality, we will install <code class="language-plaintext highlighter-rouge">ansible-base</code> and then install the Cisco IOS collection.</p> <p>Install Ansible Base:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-base-only) ~/ansible-testing ▶ pip install ansible-base==2.10.7 Collecting ansible-base==2.10.7 Using cached ansible-base-2.10.7.tar.gz (5.7 MB) ---- omitted ---- Successfully installed MarkupSafe-1.1.1 PyYAML-5.4.1 ansible-base-2.10.7 cffi-1.14.5 cryptography-3.4.7 jinja2-2.11.3 packaging-20.9 pycparser-2.20 pyparsing-2.4.7 </code></pre></div></div> <p>PIP shows only <code class="language-plaintext highlighter-rouge">ansible-base</code> is installed.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(ansible-base-only) ~/ansible-testing ▶ pip freeze | grep ansible ansible-base==2.10.7 </code></pre></div></div> <p>Checking the collection list shows <strong>no</strong> collections installed.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>▶ ansible-galaxy collection list </code></pre></div></div> <p>Next, install the Cisco IOS collection.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>▶ ansible-galaxy collection install cisco.ios Starting galaxy collection install process Process install dependency map Starting collection install process Installing 'cisco.ios:2.0.1' to '/Users/ntc/.ansible/collections/ansible_collections/cisco/ios' Downloading https://galaxy.ansible.com/download/cisco-ios-2.0.1.tar.gz to /Users/ntc/.ansible/tmp/ansible-local-333579nqv2gur/tmp_96gsuu5 cisco.ios (2.0.1) was installed successfully Installing 'ansible.netcommon:2.0.1' to '/Users/ntc/.ansible/collections/ansible_collections/ansible/netcommon' Downloading https://galaxy.ansible.com/download/ansible-netcommon-2.0.1.tar.gz to /Users/ntc/.ansible/tmp/ansible-local-333579nqv2gur/tmp_96gsuu5 ansible.netcommon (2.0.1) was installed successfully Installing 'ansible.utils:2.0.2' to '/Users/ntc/.ansible/collections/ansible_collections/ansible/utils' Downloading https://galaxy.ansible.com/download/ansible-utils-2.0.2.tar.gz to /Users/ntc/.ansible/tmp/ansible-local-333579nqv2gur/tmp_96gsuu5 ansible.utils (2.0.2) was installed successfully </code></pre></div></div> <p>When the Cisco IOS collection is installed it will execute a dependency check and install required collections. In the case of Cisco IOS, the Ansible Netcommon and Ansible Utils collections are installed.</p> <p>Listing the collections shows only the 3 collections versus the 85+ in the <code class="language-plaintext highlighter-rouge">community package</code>.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>▶ ansible-galaxy collection list # /Users/ntc/.ansible/collections/ansible_collections Collection Version --------------------- ------- ansible.netcommon 2.0.1 ansible.utils 2.0.2 cisco.ios 2.0.1 </code></pre></div></div> <h2 id="summary">Summary</h2> <p>As the Ansible project continues to evolve the code base split changes the way Ansible is managed in an environment. It becomes more flexible, but also has some additional dependencies that you need to be aware of. While troubleshooting new Ansible installs, a few helpful commands include:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ansible --version </code></pre></div></div> <blockquote> <p>Note: The Ansible version will show the core/base version number NOT the community package version.</p> </blockquote> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ansible-galaxy collection list </code></pre></div></div> <p>These commands show which collections are installed and the path to the collections respectively.</p> <p>Finally, the <code class="language-plaintext highlighter-rouge">ansible-galaxy</code> command for <code class="language-plaintext highlighter-rouge">collections</code> doesn’t currently support an <code class="language-plaintext highlighter-rouge">uninstall</code> option. To delete a collection, use <code class="language-plaintext highlighter-rouge">rm -rf /path/to/collection</code>. To force an update of a collection use the <code class="language-plaintext highlighter-rouge">--force</code> command line argument.</p> <p>-Jeff</p>Jeff KalaAnsible released 3.x in February 2021, which continues the project’s recent announcement that the code base will be splitting. As the project continues the transition, it’s important to understand some of the new terminology and understand how it might affect your environment.