The limited candidate pool makes it more crucial than ever for organizations to provide early-career professionals with opportunities to develop their skills and grow their knowledge of the network automation industry.
Recognizing the need to find and keep the right talent, Network to Code launched its own talent development program, NTC University, designed to propel the network engineers of today into the network automation engineers of the future.
So, what makes NTC University the go-to development program for aspiring network engineers and Network to Code the employer of choice?
Let’s find out!
The job market for network engineers is growing rapidly, with a projected 6% increase in employment from 2016 to 2026. With the increasing reliance on technology, the need for companies to maintain and secure their networks is more important than ever.
Network engineers are one of the most wanted professionals in the US. Despite the growing demand for network engineers, the industry is experiencing a need for more qualified professionals in the field. This shortage is due to the rapidly changing nature of technology, which requires constant learning and adaptation. Network engineers have not been offered career growth opportunities in the industry.
NTC University is a trailblazing new corporate development program from Network to Code for early-career network engineers. The ten-week immersive program will provide the necessary technical education and consulting skill set for a select group of network engineering professionals. With acceptance into this program, individuals will join Network to Code as full-time Associate Engineers before the start of the program in the fall of 2023.
Participating in this talent development program means you will join our team as a full-time employee from day one and represent our core values in all you do. We are looking for results-driven individuals and creative problem solvers who love to do it differently and want to make an impact on the future of Network Automation in real time.
After completing the ten-week program, participants will join a Network Automation team at Network to Code, starting to work on customer projects delivering solutions for clients, with job shadow and mentorship support for an entire year.
And the benefits are invaluable.
NTC University offers a comprehensive technical education that covers a wide range of topics. From the fundamentals of networking to advanced automation techniques, network engineers have access to a wealth of knowledge that can help them excel in their careers at Network to Code and beyond. The curriculum is designed to be hands-on, allowing participants to gain practical experience to apply in real-world scenarios at Network to Code after completing the program.
In addition to technical education, NTC University also offers training in consulting skills, including communication, project management, and problem-solving skills. With NTC University, network engineers can develop a well-rounded skill set to help them succeed in their new role at Network to Code.
NTC University uniquely provides network engineers with real-world experience from a network automation leader named to the Inc. Best Workplaces list along with special recognition in the Prosperous and Thriving category. Participants will have access to dedicated instructors, shadow engineers, and mentors to set them up for success in their new position at Network to Code. The instructors are experts in network automation and have years of experience working with some of the world’s largest and most complex networks.
Through NTCU, Network to Code aims to create a clear career path to propel today’s network engineers into the future as network automation engineers.
In the ten-week program, eligible candidates who are located in the United States and have at least two years of network engineering experience, a basic understanding of Python and data structures, and a passion for network automation will learn everything from Python & Django Fundamentals and Network Automation with Python & Ansible to Network Source of Truth with Nautobot and Nautobot App Development.
Network to Code will host an informational session on NTC University for interested network engineers to learn more about the program. The next session will be on May 16th at 1 p.m. ET. So save your seat and learn more about this trailblazing new corporate development program from Network to Code.
To learn more and apply for NTC University, visit: https://go.networktocode.com/NTCU.
-Ken
]]>Bear in mind that Services or Tasks are running in specific subnets of a VPC, therefore you have to define at which subnets the Fargate components will run. Also, in order to load balance the traffic between containers, an Application or Network Load Balancer is going to be needed. Furthermore, Nautobot requires a database back end (PostgreSQL or MySQL) and a Redis in-memory data store to operate. Since we are deploying in AWS, managed services will be used: Amazon RDS for PostgreSQL and Amazon ElastiCache for Redis. Deployment of those two components is out of the scope of this post, but remember to create a database (named nautobot
) before moving on with this post.
As for Nautobot itself, we would need three Services and the respective Task definitions: Nautobot Application, Nautobot Worker, and Nautobot Scheduler. At this point we should mention that you should deploy only one instance of Nautobot Scheduler and one or more of the other Services (based on your traffic), as shown in the diagram below.
You will notice that there is an Nginx container alongside the Nautobot application. As a best practice, you should have a proper web server in front of the uWSGI server. For more detailed information about the Nautobot architecture, take a look at the official documentation.
The first item you need to deploy in this topology is the Application Load Balancer that will serve the requests to the back end. In order to do that, deploy a Load Balancer with a target group type IP
with no targets for now. There are a lot of resources out there on how to do that, so we won’t enter into much detail.
As for the Fargate deployment, we will start by deploying the ECS cluster.
nautobot-cluster
) and choose the VPC and subnets that will host your cluster.Before we move forward, Nginx needs a configuration file to operate. You have two solutions around that: use either an external volume to mount the config file or build a new pre-configured Container Image as presented here.
The configuration of Nginx should just pass traffic to localhost on port 8080, and we will expose port 80 (http) for the Nginx container. A simple basic example may be found below for the nginx.conf
and the Dockerfile
.
# nginx.conf
events {
worker_connections 1024;
}
http {
resolver 127.0.0.1;
upstream docker-nautobot-http {
server 127.0.0.1:8080;
}
server {
listen 80;
access_log off;
location / {
proxy_pass http://docker-nautobot-http;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
}
# Dockerfile for Nginx
FROM nginx:stable
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
If you are using a private Amazon Elastic Container Registry (ECR), be sure to log in.
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region>.amazonaws.com
Then we build, tag, and push the container image to a private ECR repository.
docker build -t <aws_account_id>.dkr.ecr.<region>.amazonaws.com/nginx:nautobot .
docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/nginx:nautobot
Next, back in the ECS console, go to Task definitions on the left in order to create the three task definitions that are needed (Nautobot App, Nautobot Worker, and Nautobot Scheduler).
For the Nautobot App task definition:
nautobot-app-definition
).nautobot
and networktocode/nautobot:1.5-py3.9
as the image URI. Please check the Nautobot documentation for versions, etc. Also, keep checking for the Nautobot 2.0 release. ;)Then we should add the environmental variables that Nautobot needs; in order to simplify the process, you can take them from a file (stored in an S3 bucket) as we did here, which allowed us to use Add environment file
. The sensitive ones (e.g., passwords) may be retrieved from AWS Secrets Manager or AWS Systems Manager by using the ValueFrom
option. More information can be found in ECS developer guides for passing environment variables and retrieving secrets. As a minimum, the below values are essential. Make sure to change these values to match your infra!
# General settings
NAUTOBOT_ALLOWED_HOSTS=*
NAUTOBOT_BANNER_TOP="Local Production"
NAUTOBOT_SECRET_KEY=<random_key>
NAUTOBOT_LOG_LEVEL=INFO
NAUTOBOT_METRICS_ENABLED=True
# Database settings
NAUTOBOT_DB_NAME=nautobot
NAUTOBOT_DB_PASSWORD=<db_password>
NAUTOBOT_DB_HOST=<rds_host_endpoint>
# Redis settings
NAUTOBOT_REDIS_HOST=<elasticache_host_endpoint>
NAUTOBOT_REDIS_PORT=6379
# Napalm settings
NAUTOBOT_NAPALM_USERNAME=<napalm_username>
NAUTOBOT_NAPALM_PASSWORD=<napalm_password>
NAUTOBOT_NAPALM_TIMEOUT=5
# Superuser settings
NAUTOBOT_CREATE_SUPERUSER=true
NAUTOBOT_SUPERUSER_NAME=admin
NAUTOBOT_SUPERUSER_EMAIL=admin@example.com
NAUTOBOT_SUPERUSER_PASSWORD=admin
NAUTOBOT_SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567
Command: CMD-SHELL, curl -f http://localhost/health || exit 1
Interval: 30
Timeout: 10
StartPeriod: 60
Retries: 5
Linux/x86_64
; and give 2vCPU and 4 GB of RAM as size.ecsTaskExecutionRole
) based on your environment (more details here).Because we are a Network Automation company, seriously it would be a shame to go through all these steps. Instead, you could have just clicked “Create a new task definition from JSON” in step 1, copy paste the below JSON, change the <variables>
, and click the big orange button. This is the approach that we will take for the subsequent tasks.
{
"requiresCompatibilities": [
"FARGATE"
],
"family": "nautobot-app-definition",
"containerDefinitions": [
{
"name": "nautobot",
"image": "networktocode/nautobot:1.5-py3.9",
"portMappings": [
{
"name": "nautobot-8080-tcp",
"containerPort": 8080,
"hostPort": 8080,
"protocol": "tcp",
"appProtocol": "http"
}
],
"essential": true,
"environmentFiles": [
{
"value": "arn:aws:s3:::<s3_bucket_name>/<env_file.env>",
"type": "s3"
}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-create-group": "true",
"awslogs-group": "/ecs/nautobot-app-definition",
"awslogs-region": "<region>",
"awslogs-stream-prefix": "ecs"
}
},
"healthCheck": {
"command": [
"CMD-SHELL",
"curl -f http://localhost/health || exit 1"
],
"interval": 30,
"timeout": 10,
"retries": 5,
"startPeriod": 60
}
},
{
"name": "nginx",
"image": "<aws_account_id>.dkr.ecr.<region>.amazonaws.com/nginx:nautobot",
"portMappings": [
{
"name": "nginx-80-tcp",
"containerPort": 80,
"hostPort": 80,
"protocol": "tcp",
"appProtocol": "http"
}
],
"essential": true,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-create-group": "true",
"awslogs-group": "/ecs/nautobot-app-definition",
"awslogs-region": "<region>",
"awslogs-stream-prefix": "ecs"
}
}
}
],
"networkMode": "awsvpc",
"cpu": "2048",
"memory": "4096",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"executionRoleArn": "arn:aws:iam::<account_id>:role/ecsTaskExecutionRole"
}
For the Nautobot Worker definition, repeat the same steps as above (either from console or by using JSON) to create another Task definition, with the below changes:
nautobot-worker-definition
).nautobot-worker
, ignore/remove the Nginx one.nautobot-server,celery,worker
; this is the comma-separated representation of equivalent nautobot-server celery worker
. If you are using JSON, it should look like the following, nested under containerDefinitions
.
"command": [
"nautobot-server",
"celery",
"worker"
],
"healthCheck": {
"command": ["CMD", "bash", "-c", "nautobot-server celery inspect ping --destination celery@$HOSTNAME"],
"interval": 30,
"timeout": 10,
"retries": 5,
"startPeriod": 60
}
For the Nautobot Scheduler definition, the config will be the same as Nautobot Worker with some changes:
nautobot-scheduler-definition
).nautobot-server,celery,beat
. If you are using JSON, it should look like the following.
"command": [
"nautobot-server",
"celery",
"beat"
],
All the other config should be the same as the worker.
Now for the fun part. Let’s create services in order to run all those tasks we defined previously. As you can imagine, we need three services, one for each task.
First, let’s create the Nautobot App service.
nautobot-app-service
).Load balancing
options. You will notice that you can create a new one, or use an existing one if you already provisioned it. For the Container to load balance
, choose Nginx with port 80. And on Target Group
parameters, set the “Health check grace period” to 60s and the path to /health/
.Again, instead of jumping through all these clickety hoops, you could change the <variables>
and use the below JSON file.
{
"serviceName": "nautobot-app-service",
"cluster": "nautobot-cluster",
"loadBalancers": [
{
"targetGroupArn": "<lb_target_group_arn>",
"containerName": "nginx",
"containerPort": 80
}
],
"healthCheckGracePeriodSeconds": 60,
"desiredCount": 1,
"launchType": "FARGATE",
"platformVersion": "LATEST",
"taskDefinition": "nautobot-app-definition",
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"<subnet-x>",
"<subnet-y>",
"<subnet-z>"
],
"securityGroups": [
"<security_group_id>"
],
"assignPublicIp": "ENABLED"
}
},
"schedulingStrategy": "REPLICA",
"deploymentController": {
"type": "ECS"
}
}
Save the file locally and run.
aws ecs create-service --cli-input-json file:///tmp/nautobot-app-service.json
Repeat the above steps for the Nautobot Worker and Nautobot Scheduler services (nautobot-worker-service
, nautobot-scheduler-service
), without configuring a Load Balancer this time. Nautobot Worker and Scheduler communicate directly with the database and Redis to execute tasks. Remember, Nautobot Scheduler must have only one replica (instance). For the other two, it depends on your traffic/load. We would suggest exploring Auto-scaling based on CPU and memory load and the usage of the application.
At this point (or 5 to 10 minutes later), if everything has gone according to the plan, you should be able to see this in your ECS Cluster:
ECS Cluster Services
ECS Cluster Tasks
In order to verify that Nautobot works end-to-end, go back to the Load Balancer, copy the DNS name URL, and open it in your favorite browser. With just a little bit of luck, you should be able to see Nautobot home view.
Note: In the context of things, this whole deployment ideally would be part of an Infrastructure as Code approach. Classical candidates for this would be Terraform and Cloudformation. If, on the other hand, you prefer to have the full power of a programming language, CDKTF or Pulumi are your friends. ;)
In case something is not working as expected, don’t fret—there are a lot prerequisites in achieving this deployment. We would suggest to consult with your SREs and/or cloud engineers—the usual suspects are Security Groups and IAM. :) Feel free to ping us in NTC Slack for anything you may need. Cheers!
-Gerasimos
]]>Another common issue?
Unauthorized network changes often go unnoticed for months or years, especially when NOC personnel deviate from established configuration standards for the sake of business continuity or change requests are improperly implemented.
While most organizations aim to maintain compliance with configuration standards, achieving a high level of adherence through manual methods is often unattainable. This is because, in order to achieve rigorous conformity, dozens of engineers would need to be dedicated to manually reviewing network configurations. However, this approach is usually impractical due to its high costs and limited effectiveness.
But there is a better way to maintain and manage networks in 2023.
That’s where NetDevOps-enabled configuration management and more specifically, configuration compliance comes in.
Configuration compliance guarantees that network devices are configured to established standards, thus improving the reliability, predictability and security of a network. For example, interfaces that are configured and maintained to a standard are less likely to exhibit unexpected behaviors and are easier to troubleshoot.
This is accomplished by defining configuration standards and then auditing the network for deviations from those standards. The standards are defined through a combination of configuration models and data. Understanding the factors that cause variances in those standards, such as the role of a device or its location, are important. Fortunately, Nautobot excels in this capacity, allowing for dynamic standards that are automatically generated. Deviations from a standard are measured by comparing the standard against actual device configurations.
When discrepancies are found, they’re brought to the attention of network administrators. They can then decide whether it’s appropriate to realign the configuration with the standard. In some instances, as in the case of a break-fix, it may be desirable to keep a non-standard configuration in place temporarily. While the temporary solution is in place, a proper solution ensures that administrators are aware of the deviation. Once a permanent solution has been discovered, the standard can be updated to reflect the change.
In order to fully embrace proper configuration compliance, automation and a source of truth are essential.
Would you like to achieve a stable and highly predictable network? Data-driven configuration compliance makes this coveted state obtainable.
Fewer incidents mean that networks are less noisy and fewer resources are dedicated to remediating those incidents. When incidents do occur, administrators are troubleshooting familiar configurations, expediting resolution times. End users enjoy an enhanced experience because interruptions are less frequent and shorter in duration.
The governance and implementation of changes also become much easier. Network administrators can be confident the pre-change state of the network is predictably configured. Change managers benefit because projects are completed on schedule without implementation disruptions. NOC Managers can rest easy knowing that both the intended state and actual state of the network are backed up in a version-controlled repository.
Reducing the complexity of network configurations enhances monitoring operations. Administrators simply contend a familiar solution as opposed to numerous one-offs, so less time is spent addressing unique implementations. This reduces the effort to set up and maintain alerts, so your team can focus on optimizing monitoring performance.
Transitions to new device models and vendors are streamlined. Because configurations are an output of intent, one merely updates Nautobot to reflect the hardware change. This causes a new intended configuration to be generated which reflects the change in hardware.
By making networks more reliable with configuration compliance, enterprises are able to deliver the resiliency that core business operations require.
In order to help organizations navigate configuration management challenges with a renewed clarity that invites reliability, resiliency, and security, Network to Code has developed a number of solutions powered by Nautobot to ensure networks operate in a highly predictable, scalable, and reliable manner, including:
With Nautobot Single Source of Truth, your organization can aggregate and source authoritative data from multiple internal systems, including ServiceNow, Infoblox, and Nautobot, into a single location. This approach provides a single repository of standardized up-to-date and accessible data, which can then be leveraged to build configurations.
Version-controlled backup configurations are critical for organizations when sites go down, data is lost or connectivity is upended. With Network to Code’s free and open source Golden Configuration app, there’s instant access to a Git repository with configuration backups.
With this information, it’s simple to compare the rendered configuration with the backup configuration to determine compliance, by leveraging an existing solution (Rancid, Ansible, Oxidized, etc) or the Nautobot platform.
Nautobot generates intended configurations by leveraging Jinja templates and data aggregated and stored in Nautobot. Then Nautobot stores the intended configurations in a Git repository, which makes tracking changes to the intended configuration easy.
Lastly, the Nautobot Golden Config App can schedule comparisons of the current network configuration to an intended state configuration. By alerting administrators to deviations and errors, a lack of compliance can easily be spotted—and fixed. Even better? Single-touch or zero-touch resolutions are possible.
It’s time to mitigate the unnecessary network outages and deviations that lead to revenue loss.
Ready to streamline your approach to configuration management with a NetDevOps-first mindset?
Download the NTC Fast Track Solutions eBook to learn more about configuration management and other solutions today.
-Chris M.
]]>The complexity, scale, and rapid evolution of modern networks have created a demanding environment in which administrators are expected to provide a highly available network that sustains core business operations.
In today’s ever-changing network environments, network automation is becoming less of a suggestion and more of a requirement. This is because, without it, your team will continue reactively managing outages, performance degradations, and security vulnerabilities. And in a fast-paced, competitive world, this operational drag acts as an impediment that puts your business at a disadvantage.
It’s time to take a deep dive into the benefits of network automation to fully understand the impact it has on network teams and why it has become more of a requirement than a suggestion.
Network automation has many benefits for enterprises of all sizes. Let’s dig into some of the top benefits and how they may impact your organization.
By simplifying operations and programmatically executing tasks, engineers can focus on project work and strategic business priorities, fully leveraging their creativity and ingenuity to do what they do best: imagine and revolutionize the way things are done with game-changing ideas.
Automated networks are more reliable. The leading cause of network outages is human error. Whether that’s due to a misconfigured technology or an improperly applied maintenance window, these problems are completely avoidable with programmatic execution. NetDevOps solutions not only execute predictably and layer fail-safes throughout, but they also catch the mistakes humans make.
The ability to deploy services and operationalize plans quickly is not only a key benefit of network automation, but it is also key to maintaining a competitive edge. NetDevOps solutions condense the timelines and reduce the capital requirements of new projects, extending their functional life. Automation can speed up the implementation of the service, which allows the benefits of the service to be realized much sooner—potentially weeks or months in advance.
With cyberattacks expected to rise sharply in coming years, it’s absolutely critical that organizations harden their security posture. Luckily, the benefits of network automation reduce the likelihood of security attacks. Automation assists organizations throughout the world to maintain security best practices. Leveraging automated solutions means that your infrastructure is always running the recommended software version and your configurations are maintained and monitored to ensure they conform to a standard. Network automation also provides a framework that enables organizations to isolate security threats more quickly by programmatically retrieving information for administrators.
Understanding the current state of the network is crucial for effective management. Administrators of NetDevOps-inspired networks enjoy a nuanced, dynamic and detailed view of the network, augmented by Source of Truth data. By enriching the collection process with the latest technologies, they enjoy access to a larger data lake. Rich dashboards are intelligently designed to present the information you need, where you need it; enabling organizations to make business decisions informed by empirical data.
Now that we’ve explored the benefits of network automation, it’s important to realize that it takes the right people, technology, and processes to create a great network automation program.
Many teams don’t realize the type of heavy lifting that’s required to establish automation for your networks. It’s difficult for companies to figure out which tools they need because of the lack of architectural and solution experts in this field.
Network to Code recognized that many network teams saw the benefits of network automation but were facing obstacles and various challenges when it came to choosing the right network automation solution for their team.
NTC’s Fast Track Solutions are solutions that are tried-and-true, have been deployed numerous times into production environments, and are rooted in a NetDevOps and data-driven approach to network automation. Fast Track Solutions represent the most common use cases and challenges network organizations are facing today.
Don’t let the challenges that come with choosing the right network automation solution prevent you from reaping the benefits of network automation.
Download our eBook to explore the significant benefits of network automation and acquaint yourself with NTC’s newly announced prebuilt Fast Track Solutions that can jump-start your network automation journey.
-Tim S.
]]>The automation engine is flexible and can be built on top of many different languages and frameworks. A few examples are: Python, Golang, Rust; they can be further broken down into specific frameworks such as: Ansible, Salt, Nornir, NAPALM, Netmiko which is the scope of Python, Terraform for cloud, Scrapligo in the cases of Golang, or even validation focused tools like Batfish. The automation engine attempts to achieve tasks such as: configuration backups/rendering/compliance/provisioning, Zero Touch Provisioning (ZTP), as well as NetDevOps principles such as Continuous Integration/Continuous Delivery (CI/CD).
The automation engine is the component that manages the network state and performs network tasks. This component will actively make changes to the network state and the connectivity between the automation engine and in-scope devices should be permitted by security policy.
There are some challenges that the automation engine has to solve; interacting with network devices is complicated. Command line interfaces (CLIs) have been the main interface for network engineers to modify and manage network equipment. More recent trends involve an API to interact with specific devices, e.g. Arista eAPI. Alternatively, some vendors are moving toward element managers that offer APIs and handle device connections within their own frameworks. Finally, YANG via NETCONF/RESTCONF/gNMI was developed to attempt to solve vendor independent automation, but is still working towards gaining mass adoption.
CLIs were not built for automation; but over the last many years there have been many projects that have been built and open sourced to help solve these problems. Some of these were mentioned in the introduction; however for the sake of clarity, Nornir, Scrapli(go), NAPALM, Netmiko are all examples that provide frameworks to interact with CLIs and automate these tasks.
These projects generally require a few pieces of metadata:
Note: These are the bare minimum attributes, and they should be stored within the Source of Truth (SoT) component. The automation engine should have a method to query for the information.
While APIs have helped aid the adoption of automation and made the interaction with these devices simpler, each vendors API is implemented differently. The automation engine must provide a flexible interface that is capable of manipulating parameters and reading multiple returned data formats E.g (XML, JSON).
While considering configuration management and the automation engine in general some of the key challenges are listed below. This is not an exhaustive list.
extra
and what is missing
. (As an example, this is solved in Nautobot Golden Config App.) It’s a completely different challenge to remediate those configurations.For some of the more advanced topics mentioned above the next section serves to provide addition details and considerations.
Let’s deep dive into several of nuances of some of these topics.
As you can see from the challenges above, there are many questions you must answer. Once these questions are answered, it becomes much easier to try to choose an automation engine that fits your organization’s goals.
One of the biggest challenges with the Automation Engine component of this architecture is picking the right tool(s) for the job. There is no shortage of open source tools that fit this component of the architecture; furthermore, there is an ever-expanding catalog of closed source / vendor specific tools that aim to accomplish the tasks.
This is an interesting topic. Throughout the years, NTC has engaged with many customers. Even customers at the most basic entry into their network automation journey are already using the automation engine element. A simple one-off script that goes and collects data off of a device fits this component. Since this component in most cases is one of the first to be selected, it’s not always easy convincing a client that other options exist.
For these and many other reasons, we’ve found that most of the automation engine options available can achieve great results if you have the rest of the automation architecture in place. Selecting the right engine for your business comes down to skill set, previous adoption, willingness to learn, and in some cases having product support, which many large enterprises rely on today.
Regardless of the application/framework in use, the automation engine communicates with network devices. And as mentioned in Network Automation Architecture - The Components, it’s important to understand the automation engine not as an isolated component, but as the final executor of the outcome of the other components.
Furthermore; there will be situations where a single automation engine does not meet the business requirements, in these circumstances multiple automation engines can be used, but a level of effort should be exhausted to keep the number of different automation engines to a minium; otherwise, the learning curve and skill set to operate/maintain this component gets too complex and leads to slowed adoption.
Some of the characteristics to consider are mentioned below:
One of the biggest challenges related to the automation engine is the connectivity conundrum that exists in enterprises. The security of networks continues to grow in complexity; the management control plane of network devices is no different. In many cases centralized applications aren’t allowed to connect to network devices. Whether that is due to DMZ design, Geo location issues, or mergers and acquisitions, the automation engine must be flexible enough to run inside those pods.
Here are some of the existing solutions to this problem.
Automation Engine | Solution |
---|---|
Ansible | Execution Environments |
Python | Celery, Redis (RQ), Taskmaster |
Saltstack | Master/Minion |
To close out this blog, I want to show what a release process with validation steps might look like in a high-level diagram, this diagram came directly from one of the Webinars Ken Celenza and I did Community Webinar: Using Batfish for Network & Routing Verification.
Keep an eye out for the remaining parts of this series!
Cheers, -Jeff
]]>During one of our latest webinars, Network to Code’s John Anderson, Principal Consultant, and Christian Adell, Principal Architect, took a deep dive into the current state of enterprise network automation and the world of Nautobot Single Source of Truth (SSoT).
Read on to learn more about the challenges enterprises face with network automation and how Nautobot Single Source of Truth can help with network data management.
Network automation continues to evolve as organizations find innovative ways to deploy, manage, and consume their networks. Most network teams have adopted some form of scripting, but the shift to common operating models and more advanced management platforms is still not a reality for many organizations.
The way to move the enterprise network automation industry forward is focusing on data. Many organizations have already adopted sources of truth for inventory management, IP address management, configurations, and many other data requirements. In fact, EMA Research found that 63% of enterprises have multiple authoritative data repositories for different classes of information.
There’s no doubt that network Sources of Truth (SoTs) are now abundant. By 2023, Gartner predicts that 40% of enterprises will maintain an accurate Network Source of Truth within operations to enable automation initiatives. This requires a centralized view of network intent that utilizes all of these distributed sources of truth to drive network automation.
At the enterprise level, managing data is both a technical and business problem. Organizations have many different systems which deal with their network data, and this data needs to flow between these systems to support automation. From a business perspective, however, data ownership and governance is complex because of the need for regulatory compliance, audit trails, and other operational requirements.
A single source of truth (SSoT) approach is the foundation of enterprise data management because it can provide a centralized view of network intent while maintaining the distributed management of data sources. An SSoT platform can take numerous Sources of Truth—which are authoritative sources of data for particular domains—and aggregate this data into a single view. This is the key to overcoming both the technical and business challenges involved with scaling enterprise network automation.
Nautobot is a single source of truth and network automation platform that defines the intended state of the network. By adopting Nautobot for network automation, enterprise organizations can deploy a trusted source of data that facilitates good data hygiene.
More specifically, Nautobot allows organizations to define custom fields and their own unique relationships between data stored in the platform. It’s also an extensible platform that enables organizations to build their own custom apps to meet more specific business requirements using REST APIs, GraphQL, webhooks, or a plugin system.
The Single Source of Truth framework is the way the Nautobot platform facilitates integration and data synchronization between various source of truth (SoT) systems. There are many individual apps built atop this SSoT framework for seamlessly connecting with Infoblox, IP Fabric, ServiceNow, and other enterprise platforms. This framework approach is designed to make it easy for users to integrate Nautobot into their own unique environments to serve as a true single source of truth (SSoT).
In short, Nautobot streamlines enterprise network data management by unifying disparate data sources and creating a centralized single source of truth. This can help enterprise organizations adopt more advanced network automation platforms and processes that transform network operations, while still ensuring individual domains retain control of their data.
Want to learn more about enterprise network data management with Nautobot? Watch the full webinar here.
-Tim S
]]>The first day kicked off with an announcement of Network to Code becoming an official Cisco Select Advisor Partner, enabling NTC to provide Cisco customers with greater network automation support.
But the excitement didn’t stop there.
For those who weren’t able to attend, or just want a quick refresher, below are the top trends and takeaways from CLEMEA 2023 from Network to Code.
Regardless of the industry or the organization, each entity faces similar challenges when it comes to improving the overall reliability, management, performance, and security of their networks. Whether an organization is in the finance industry, education, or is a service provider, most of the daily business operations require a reliable network.
Network automation solutions can have a vast range of benefits, from increased network stability to improved security, and each solution can be tailored and built upon to solve an organization’s unique needs. On top of that, a network automation solution can greatly reduce the downtime caused by human errors or manual diagnosis and fixing the issue, while making operations more scalable and uniform.
Simplicity was a key theme at the event all around, and that carried over into the questions and conversations had at the Network to Code booth.
It is easy for any organization to see the value of implementing a network automation solution, but many are still unsure of how and where to start that journey. The best answer to this question is that the “tool” is the last bit of concern when defining the solution. It mostly depends on what problem the organization is trying to solve or the task the organization is trying to automate.
During the event, we had the pleasure of hosting and participating in various working sessions and demonstrations.
We joined the IP Fabric team for two working sessions: Building a Network Automation Solution with a Source of Truth and Automation Assurance Engine and Open Source Contributions for Network Automation.
The team also hosted two DevNet Workshops: Take Advantage of Your Telemetry Stack! How to Retrieve Data Programmatically to Aid in Your Network Operations and Managing Source of Truth Network Data with DiffSync.
Interested in learning more about any of these sessions or topics? Contact us at info@networktocode.com.
Network to Code offers a variety of solutions and services designed to help organizations jump-start their network automation journey. Network to Code helps enterprises by offering Network Automation as a Service to provide a fully managed network automation platform built on top of Nautobot, the most open and extensible Network Source of Truth. We also just launched Fast Track Solutions, a catalog of prebuilt network automation workflows, that can be added to the NAaaS platform!
Any trends we missed or takeaways you’d like to share? Drop us a note at info@networktocode.com or check out some of our latest blogs to learn more about network automation.
-Tim
]]>print()
for debugging your code! Perhaps it’s still the case today? Stare at the traceback, find the faulty line number, and insert a print statement just above it hoping to shed some light on the error. Although that’s a simple method, it has never been a very efficient one: the print()
statement has to be moved to the next line… and the next one… and the next one… with no option to move around the code in an interactive way or play around with the imported libraries or functions. What about flooding your code in frustration with thousands of prints? There must be a better way to do it, right?
Fortunately, the community has come to our rescue with an amazing library called pdb — The Python Debugger. While you can use pdb as a regular library where you pass arguments—for an example, look at pdb.post_mortem()—we are mainly interested in the interactive debugging mode.
Let’s take a basic example using the NTC open source library—jdiff:
from jdiff import CheckType, extract_data_from_json
def pre_post_change_result(reference, comparison):
"""Evaluate pre and post network change."""
path = "result[*].interfaces.*.[$name$,interfaceStatus]"
reference_value = extract_data_from_json(reference, path)
comparison_value = extract_data_from_json(comparison, path)
my_check = CheckType.create(check_type="exact ")
return my_check.evaluate(reference_value, comparison_value)
if __name__ == "__main__":
reference = {
"result": [
{
"interfaces": {
"Management1": {
"name": "Management1",
"interfaceStatus": "connected",
}
}
}
]
}
comparison = {
"result": [
{
"interfaces": {
"Management1": {
"name": "Management1",
"interfaceStatus": "down",
}
}
}
]
}
pre_post_change_result(reference, comparison)
When I run the above code, however, I get a NotImplementedError
:
Traceback (most recent call last):
File "/Users/olivierif/Desktop/test.py", line 38, in <module>
print(pre_post_change_result(reference,comparison))
File "/Users/olivierif/Desktop/test.py", line 8, in pre_post_change_result
my_check = CheckType.create(check_type="exact")
File "/usr/local/lib/python3.10/site-packages/jdiff/check_types.py", line 29, in create
raise NotImplementedError
NotImplementedError
Let’s see how we can debug the above code using pdb. My favorite way is to insert a breakpoint()
line in the code, enter in debug mode, and move around from there.
New in version 3.7: The built-in breakpoint(), when called with defaults, can be used instead of import pdb; pdb.set_trace().
def pre_post_change_result(reference, comparison):
"""Evaluate pre and post network change."""
breakpoint()
path = "result[*].interfaces.*.[$name$,interfaceStatus]"
reference_value = extract_data_from_json(reference, path)
comparison_value = extract_data_from_json(comparison, path)
my_check = CheckType.create(check_type="exact")
return my_check.evaluate(reference_value, comparison_value)
As soon as I run the code, the execution pauses and I am dropped into Python interpreter where the breakpoint()
line was added. As we can see from the below output, pdb returns the code filename and directory path, the line and line number just below breakpoint()
. I can now move around the code and start debugging…
> /Users/olivierif/Desktop/test.py(6)pre_post_change_result()
-> path = "result[*].interfaces.*.[$name$,interfaceStatus]"
Let’s move closer to the line number returned by the traceback. Typing n
as next, will move pdb to the next line—line number 7.
(Pdb) n
> /Users/olivierif/Desktop/test.py(7)pre_post_change_result()
-> reference_value = extract_data_from_json(reference, path)
What if we want to print, for example, one of the function arguments or a variable? Just type the argument or variable name… Be aware, though, that the terminal must have passed the line where your variable is defined. pdb knows about only the code that has been through already.
(Pdb) reference
{'result': [{'interfaces': {'Management1': {'name': 'Management1', 'interfaceStatus': 'connected'}}}]}
(Pdb) my_check
*** NameError: name 'my_check' is not defined
(Pdb)
Let’s now use j
to jump to the fault code line. Before doing that, let’s see where we are in the code with l
as list.
(Pdb) l
2
3 def pre_post_change_result(reference, comparison):
4 """Evaluate pre and post network change."""
5 breakpoint()
6 path = "result[*].interfaces.*.[$name$,interfaceStatus]"
7 -> reference_value = extract_data_from_json(reference, path)
8 comparison_value = extract_data_from_json(comparison, path)
9 my_check = CheckType.create(check_type="exact")
10
11 return my_check.evaluate(reference_value, comparison_value)
12
(Pdb) j 9
> /Users/olivierif/Desktop/test.py(9)pre_post_change_result()
-> my_check = CheckType.create(check_type="exact")
Note that from line 7 I was able to move directly to line 9 with j 9
where 9
is the line number that I want pdb to move to.
Now the cool bit: In the code above, I am using the evaluate
method to build my check type. If you remember the traceback, that was the line that gave me the error. While I am in pdb terminal I can s
—as step—into that method and move around it:
(Pdb) s
--Call--
> /usr/local/lib/python3.10/site-packages/jdiff/check_types.py(11)create()
-> @staticmethod
(Pdb) l
6
7 # pylint: disable=arguments-differ
8 class CheckType(ABC):
9 """Check Type Base Abstract Class."""
10
11 -> @staticmethod
12 def create(check_type: str):
13 """Factory pattern to get the appropriate CheckType implementation.
14
15 Args:
16 check_type: String to define the type of check.
(Pdb) n
> /usr/local/lib/python3.10/site-packages/jdiff/check_types.py(18)create()
-> if check_type == "exact_match":
Wait… what was the argument passed to this method? Can’t really remember. Let’s type a
as for args.
Pdb) a
check_type = 'exact_matches'
(Pdb)
…here we are! The method accepts exact_match
string as check type, not exact
!
Good, let’s now move pdb until we hit a return or raise line—with r
key—so we can see our NotImplementedError
line.
(Pdb) r
--Return--
> /usr/local/lib/python3.10/site-packages/jdiff/check_types.py(29)create()->None
-> raise NotImplementedError
As you can see, using pdb is a way more efficient way to debug the code. There are tons of useful functions that can be used in interactive mode, and you can also use it to add useful verbosity to your code. I do invite you to spend some time around docs and play with it. Once you get acquainted with the library, you won’t have any more frustration in debugging your code.
-Federico
]]>Fundamentally, a code review serves to answer three questions, loosely in descending order of importance:
I want to especially draw your attention to the nuances of that first question. Code is never perfected, and expecting perfection in a code review can be counterproductive and frustrating to the contributor. The goal of a code review is not perfect code, but better code—the code can always be further improved in the future if it proves necessary to do so. Keeping that in mind can make for more efficient reviewing.
As a maintainer of a software project, you should prioritize setting contributors up for successful code reviews. The details may vary depending on the size of the project and its pool of potential contributors, but ideas that generally apply include:
help wanted
or good first issue
as a way to highlight issues that would be well-suited for a new contributor to take on, as one example.)CONTRIBUTING.md
, as well as by providing well-written developer documentation in general.As a contributor to a software project, key points to keep in mind include:
As a reviewer of a code review, you should:
I like to think of different approaches to a code review as a series of distinct frames of mind, or “hats” that I might “wear”. You can also think of “wearing a hat” as assuming a different persona as a reviewer, then focusing on areas that are important to that persona. In practice, there are no firm dividing lines between these, and I’ll often “wear” many “hats” at once as I’m doing a code review, but it can be a useful checklist to keep in mind for thoroughness.
The hats that I’ll discuss here are:
This involves an approach often labeled as “beginner’s mind” in other contexts. Fundamentally, the goal is to approach the code without preconceptions or assumptions, being unafraid to ask questions and seek clarification. The key skill here is curiosity. For example, you might ask:
When wearing this hat, you focus on the experience of the user of this software. This could be a human user, but could also be another piece of software that interacts with this project via an API of some sort. Example questions to ask as a user might include:
This hat is what many of us may think of first when we think of approaches to code review, and it absolutely is a useful and important one at that. This approach focuses heavily on the details of the code and implementation, asking questions like:
This hat is my personal favorite, as I started my career as a software tester. The tester’s goal is to think of what might go wrong, as well as what needs to go right. You might ask:
… there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. – Sir Tony Hoare
This is another fun hat (or maybe I just like finding problems?). The attacker’s hat is closely related to the tester’s hat, but takes on a more devious frame of mind. Here you should ask questions like:
This is the big-picture counterpart to the detail-focused developer’s hat. Here you should be asking questions like:
I hope that you have learned something new from this overview of code reviewing approaches and best practices. While many of us may take more pleasure from writing our own code than reviewing that of others, it’s always worth remembering that a code review represents both a learning and a teaching opportunity for everyone involved, and that ultimately, the goal of every code review is to help make a better software project. I urge you to make code reviews a priority!
Many others have also written about the art and science of effective code reviews. Here are a few that I found particularly useful:
-Glenn
]]>As a Network Engineering Manager during the age of the great resignation, are you finding it difficult to retain your employees (not to mention also trying to find ways to gain greater insight and network control and stay innovative all at the same time)? The industry is growing, and there are more and more demands on managers to keep the company ship steering toward its goal.
One way to check several of these management boxes is to invest in your employees’ career development. Since the pandemic began in 2020, research has shown that companies can combat the effects of this resignation movement with a culture of learning. This decision will, in turn, ensure success for your other management goals. This is not only a move that will benefit the employees, but it’s also a way to achieve the business goals of the company, by training for improved job performance.
With network automation and other new technologies changing the network industry, there is an impending need to train top talent with the skills they need to meet the future of the workforce. And research also shows that employers that invest in their employees’ career development keep their employees longer. Those employees see their company’s dedication to their own upward mobility and will choose to stay at that company rather than search for other positions at other companies (that ultimately WILL invest in them). According to performance-management platform 15Five’s 2022 Workplace Report, nearly half of American workers say clear career growth is one of the most important factors for them to remain at a company. More than three-quarters (76%) of employees say they work harder for an employer that shows it cares about their growth as a professional than one that doesn’t.
Now that you are ready to start investing in your employees with training, _where does one start with team training and enablement these days in the network engineering/automation space? _In an industry that is constantly evolving, with the myriad of offerings available, from completely online and self-paced, to onsite, in-person options, it is hard to know which will be the most effective for your team. We have established that employers need to focus on upskilling their company’s talent, but they must also do this in ways that are manageable and fiscally feasible, while also making sure that the training program supports the company’s goals. Strategic alignment is key to developing training that will improve job performance (which, in turn, will benefit the employees).
Here are a few pieces of advice to get started:
What does “done” look like after the training is over? Have a plan and set measurable, attainable goals for your engineers. Ask yourself what specific job tasks related to network automation will each engineer do on a daily/weekly basis following the training program? Then, before the start of training, make sure those engineers know what they need to do in their jobs once they are done with the training. When employees know the bigger picture, and they see how the training will directly relate to their own job performance, they will be more invested in the learning opportunities provided.
Before any training is delivered, the organization should administer an assessment that provides leaders with an understanding of where their employees rank in network automation technologies, knowledge, and skills. This will help uncover additional knowledge gaps, better evaluate student comprehension of the course material, and track engineers’ skills progress over time. At NTC, we start our diagnostics with a self-ranked assessment by the engineers, ranking their knowledge and skills from 1 (No knowledge) to 4 (Advanced- Subject Matter Expert). The organization should do an assessment before and after the training to better understand and track progress in job performance. The assessment not only provides meaningful data for you as a manager but also provides valuable information to the training entity or company to better understand which training programs (which technologies and skills) each of your engineers should start with and which training type or medium would best fit the needs of those teams (live, self-paced, or a hybrid approach).
In addition to the self-ranked assessment, it is important to include an evaluation of the training from the engineers where they have to apply their knowledge and skills, like an exam, a graded lab challenge, or if it fits with the program, a culminating hackathon—where they can apply their training in practical application. Not only does this give the managers additional data, but it allows for greater knowledge retention with the employees, especially if there are supplementary resources available for them to refer back to, like short on-demand videos, or a knowledge base, to cement the learned concepts. Practice makes perfect.
Managers must ensure there is time during the workday for training, learning, and practicing skills. It is not feasible to set the precedent that your employees learn and practice network automation while also managing their regular network engineering workload—at least not at the start of their automation journey. How can you as a manager ensure there are dedicated times where they can attend a virtual workshop, complete a challenge assignment, watch self-paced modules, or attend a five-day training and also complete their job tasks? Remember, with network automation training specifically, this training investment will result in more workplace efficiency and less human error, along with other gains, so the benefits of scheduling this time in the short term will pay off in the long run.
At NTC, our Network Automation Academy provides a hybrid approach to learning for the busy network engineer in a delivery format that allows for greater retention of skills—skills that will improve job performance.
We take the time to strategically align the training program to your team’s needs. We will hold interviews with your team(s) and uncover gaps in knowledge that the right type of enablement will rectify. For example, in our Strategic Architecture and Design Analysis process with our Professional Services offering, we evaluate where enablement would support and reinforce the use of new technologies and workflows and use that enablement to increase the adoption of network automation at your company.
We offer flexible/remote learning; our experienced instructors can go onsite to one location to teach our formal training, or stay virtual if your employees are scattered across the globe. Either way, students will receive our signature 50% lecture/50% lab format. With self-paced learning options and graded challenge assignments, the engineers won’t quickly forget what they learned in that formal course. We also started a Training Credits program in 2022, allowing managers to send their engineers to our public training courses in waves so they aren’t all away from their job tasks at the same time.
Lastly, for the busy engineering team that is unable to attend even a three- to five-day network automation course, NTC Academy will build custom self-paced learning modules diving deep into your company’s unique automation environment to educate all levels of automation users (users, contributors, developers) on the specific information they need to know to succeed in your organization. Complete with knowledge quizzes and challenge assignments to measure learning and comprehension success, these modules provide that sought-after flexibility to fit into the days of busy engineer workloads. With a live guided discussion held every other week, your employees will still have the opportunity for those instructor touchpoints to keep them moving forward on their automation journey.
If you’d like to learn more about the NTC Academy and how we can skyrocket your company’s network automation journey through enablement and adoption, please visit networktocode.com/academy or email us at training@networktocode.com.
-Elizabeth
]]>