Python Multiprocessing - Netmiko

netmiko vs paramiko
netmiko connect to multiple devices
python multithreading
netmiko python

I really a newbie to python and network automation; I am trying out multiprocessing with Python and netmiko, but haven't successful; the code keeps being executed sequentially per device.

Below is my code and results:

========================

import datetime
from netmiko import ConnectHandler
import threading
from time import time
import multiprocessing
from multiprocessing import Process, Lock


starting_time = time()

def newthread():
    with open('routers.txt', 'r') as devices:
        for line in devices:
            deviceip = line.strip()
            host = {
                'device_type': 'cisco_ios',
                'ip': deviceip,
                'username': 'cisco',
                'password': 'cisco',
                'secret': 'cisco'
            }

            try:
                connection = ConnectHandler(**host)
                print('Trying router', deviceip)
                print('Connection Established to Host:', deviceip)
                connection.enable()
                sendcommand = connection.send_command('sh run | i hostname')
                print(sendcommand)
            except:
                print('Connection Failed to host', deviceip)


threadtask = Process(target=newthread)
threadtask.start()
threadtask.join()

print('Time Elaspsed:', time() - starting_time)


====Result===
Trying router 10.10.32.2
Connection Established to Host: 10.10.32.2
hostname R1
Trying router 10.10.32.3
Connection Established to Host: 10.10.32.3
hostname R2
Trying router 10.10.32.4
Connection Established to Host: 10.10.32.4
hostname R4
Trying router 10.10.32.5
Connection Established to Host: 10.10.32.5
hostname R3
Time Elaspsed: 26.788068771362305

Process finished with exit code 0

What could i be doing wrong? am kinda stuck. Thank You.

__ Regards Desmon K

Use concurrent.futures inbuilt module. It provides high level API's to execute tasks asynchronously.

https://docs.python.org/3/library/concurrent.futures.html

Below is modified code. Hope it helps.

import time
import concurrent.futures
from netmiko import ConnectHandler

hosts_info = []
with open('routers.txt', 'r') as devices:
    for line in devices:
        deviceip = line.strip()
        host = {
            'device_type': 'cisco_ios',
            'ip': deviceip,
            'username': 'cisco',
            'password': 'cisco',
            'secret': 'cisco'
        }
        hosts_info.append(host)

starting_time = time.perf_counter()

def open_connection(host):
    try:
        connection = ConnectHandler(**host)

        print('Trying router', host['ip'])
        print('Connection Established to Host:', host['ip'])
        connection.enable()
        sendcommand = connection.send_command('sh run | i hostname')
        return sendcommand
    except:
        print('Connection Failed to host', host['ip'])


with concurrent.futures.ProcessPoolExecutor() as executor:
    results = executor.map(open_connection, hosts_info)

    for result in results:
        print(result)

finish = time.perf_counter()
print('Time Elapsed:', finish - starting_time)

[python]Multiprocessing with Netmiko – cyruslab, The purpose is to record how multiprocessing can be done together with netmiko. The configuration required are: Set up a banner motd; Set the� [python]Multiprocessing with Netmiko. cyruslab Python, Scripting October 12, 2019 October 12, 2019 5 Minutes. Network diagram. What I want to achieve

After following @Sandeep's code i edited a few things and finally achieved the primary goal;

The final working code is here

import time
import concurrent.futures
from netmiko import ConnectHandler

hosts_info = []

starting_time = time.perf_counter()

with open('routers.txt', 'r') as devices:
    for line in devices:
        deviceip = line.strip()
        host = {
            'device_type': 'cisco_ios',
            'ip': deviceip,
            'username': 'cisco',
            'password': 'cisco',
            'secret': 'cisco'
        }
        hosts_info.append(host)

def open_connection(host):
    try:
        connection = ConnectHandler(**host)
        print('Connection Established to Host:', host['ip'])
        connection.enable()
        sendcommand = connection.send_command('sh run | i hostname')
        return sendcommand + ' ' + 'For Device:' + host['ip']
    except:
        print('Connection Failed to host', host['ip'])


if __name__ == '__main__':
    with concurrent.futures.ProcessPoolExecutor() as executor:
        results = executor.map(open_connection, hosts_info)
        for result in results:
            print(result)

    finish = time.perf_counter()
    print('Time Elapsed:', finish - starting_time)

Output:

Connection Established to Host: 10.10.32.2
Connection Established to Host: 10.10.32.4
Connection Established to Host: 10.10.32.5
Connection Established to Host: 10.10.32.3
hostname R1 For Device:10.10.32.2
hostname R2 For Device:10.10.32.3
hostname R4 For Device:10.10.32.4
hostname R3 For Device:10.10.32.5
Time Elapsed: 7.035125793000589

Please note that ip is supposed to be in quotes as 'ip'. Output is not that Fancy, but this will do for now. Thank you @ https://stackoverflow.com/users/4087758/sandeep-nagendra

Connecting to Multiple Devices with Netmiko Using Python Threads , Ultimately I landed on threading instead of multiprocessing because when you're connecting to devices/APIs over the network, you're typically� I really a newbie to python and network automation; I am trying out multiprocessing with Python and netmiko, but haven't successful; the code keeps being executed sequentially per device.

Ensure that Hyper threading settings/Multi core support is enabled in BIOS.

Multiprocessing with more than a few (read thousands of) switches , Hello All, maybe i am too new to python and am missing something obvious, but if i test the multiprocessing example, i basically end with not� In order to speed up my scripts and connect to multiple devices at once (using Netmiko, for example), the path to that is through queues, and threading/multiprocessing. Ultimately I landed on threading instead of multiprocessing because when you’re connecting to devices/APIs over the network, you’re typically waiting for a remote host to

Using Python Multi-Processing for Networking–Part 1 – The Networker, Using Python Multi-Processing for Networking–Part 1 that connect to router and execute command on it using netmiko library and we want to� The following are 30 code examples for showing how to use netmiko.ConnectHandler().These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example.

Multiprocessing with Netmiko libraries – GANOS Internetworking, Netmiko is an easy to use Multi-vendor python library to simplify Paramiko SSH connections to network devices. Original source code can be� I show you how to run Python Netmiko scripts at the same time by using threading My threading example: https://github.com/reillychase/Simple-Netmiko-Threadin

Multiprocessing with netmiko : Cisco, This is a demonstration on using python with netmiko module to push configuration over to multiple cisco routers. In this video I demonstrate the difference in performance between Python Netmiko scripts that use sequential backups of Cisco device configs versus using threading and threading pools. To scale

Comments
  • I’m not familiar with netmiko, but as far as I can tell you’re only creating a single process. Have you read the documentation, looked at any guides or tutorials?
  • Hi Sandeep, Thanks for this direction, i applied the config, though am trying to see why am getting this error: NameError: name 'ip' is not defined
  • Fixed that; ip is supposed to be in quotes. see final code in my final answer
  • @Desmon Could you please accept the answer if the solution was helpful.